Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5,

2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1
- initial changeset that introduced the fix for 
Bug#989 and follow up fixes for all test suite failures
introduced in the initial changeset. 
------------------------------------------------------------
revno: 2617.31.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Fri 2009-03-06 19:17:00 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking

Currently the MySQL server does not keep metadata locks on
schema objects for the duration of a transaction, thus failing
to guarantee the integrity of the schema objects being used
during the transaction and to protect then from concurrent
DDL operations. This also poses a problem for replication as
a DDL operation might be replicated even thought there are
active transactions using the object being modified.

The solution is to defer the release of metadata locks until
a active transaction is either committed or rolled back. This
prevents other statements from modifying the table for the
entire duration of the transaction. This provides commitment
ordering for guaranteeing serializability across multiple
transactions.

- Incompatible change:

If MySQL's metadata locking system encounters a lock conflict,
the usual schema is to use the try and back-off technique to
avoid deadlocks -- this schema consists in releasing all locks
and trying to acquire them all in one go.

But in a transactional context this algorithm can't be utilized
as its not possible to release locks acquired during the course
of the transaction without breaking the transaction commitments.
To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be
returned if a lock conflict is encountered during a transaction.

Let's consider an example:

A transaction has two statements that modify table t1, then table
t2, and then commits. The first statement of the transaction will
acquire a shared metadata lock on table t1, and it will be kept
utill COMMIT to ensure serializability.

At the moment when the second statement attempts to acquire a
shared metadata lock on t2, a concurrent ALTER or DROP statement
might have locked t2 exclusively. The prescription of the current
locking protocol is that the acquirer of the shared lock backs off
-- gives up all his current locks and retries. This implies that
the entire multi-statement transaction has to be rolled back.

- Incompatible change:

FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ
LOCK won't cause locked tables to be implicitly unlocked anymore.

mysql-test/t/flush_block_commit.test

@@ -6,7 +6,7 @@
 # And it requires InnoDB
 --source include/have_innodb.inc
 
-# Save the initial number of concurrent sessions
+--echo # Save the initial number of concurrent sessions
 --source include/count_sessions.inc
 
 --echo # Establish connection con1 (user=root)
@@ -29,19 +29,26 @@ BEGIN;
 INSERT INTO t1 VALUES(1);
 --echo # Switch to connection con2
 connection con2;
-FLUSH TABLES WITH READ LOCK;
-SELECT * FROM t1;
+--send FLUSH TABLES WITH READ LOCK
 --echo # Switch to connection con1
 connection con1;
-send COMMIT; # blocked by con2
-sleep 1;
+--echo # Sending:
+COMMIT;
 --echo # Switch to connection con2
 connection con2;
-SELECT * FROM t1; # verify con1 was blocked and data did not move
+--reap
+--echo # Wait until COMMIT gets blocked.
+#let $wait_condition=
+#  select count(*) = 1 from information_schema.processlist
+#  where state = "Waiting for release of readlock" and info = "COMMIT";
+#--source include/wait_condition.inc
+--echo # Verify that 'con1' was blocked and data did not move.
+SELECT * FROM t1;
 UNLOCK TABLES;
 --echo # Switch to connection con1
 connection con1;
-reap;
+#--echo # Reaping COMMIT
+#--reap
 
 # No deadlock ?
 
@@ -63,6 +70,7 @@ COMMIT; # should not be blocked by con3
 --echo # Switch to connection con2
 connection con2;
 reap;
+COMMIT;
 --echo # Switch to connection con3
 connection con3;
 reap;
@@ -79,8 +87,6 @@ connection con1;
 BEGIN;
 INSERT INTO t1 VALUES(10);
 FLUSH TABLES WITH READ LOCK;
-COMMIT;
-UNLOCK TABLES;
 --echo # Switch to connection con2
 connection con2;
 FLUSH TABLES WITH READ LOCK; # bug caused hang here
@@ -91,19 +97,21 @@ UNLOCK TABLES;
 BEGIN;
 SELECT * FROM t1;
 SHOW CREATE DATABASE test;
-
-DROP TABLE t1;
+COMMIT;
 
 
-# Cleanup
+--echo # Cleanup
 --echo # Switch to connection default and close connections con1, con2, con3
 connection default;
 disconnect con1;
 disconnect con2;
 disconnect con3;
 
-# End of 4.1 tests
+--echo # We commit open transactions when we disconnect: only then we can
+--echo # drop the table.
+DROP TABLE t1;
+--echo # End of 4.1 tests
 
-# Wait till all disconnects are completed
+--echo # Wait till all disconnects are completed
 --source include/wait_until_count_sessions.inc