mirror of
https://github.com/MariaDB/server.git
synced 2025-05-27 01:57:48 +03:00
0228c98936
This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it. mysql-test/include/handler.inc: After introduction of basic deadlock detector for metadata locks it became necessary to change parts of test for HANDLER statements which covered some of scenarios in which ER_LOCK_DEADLOCK error was detected in absence of real deadlock (with new deadlock detector this no longer happens). Also adjusted test to the fact that HANDLER READ for the table no longer will be blocked by ALTER TABLE for the same table which awaits for metadata lock upgrade (this is due to removal of mysql_lock_abort() from wait_while_table_is_used()). mysql-test/r/handler_innodb.result: After introduction of basic deadlock detector for metadata locks it became necessary to change parts of test for HANDLER statements which covered some of scenarios in which ER_LOCK_DEADLOCK error was detected in absence of real deadlock (with new deadlock detector this no longer happens). Also adjusted test to the fact that HANDLER READ for the table no longer will be blocked by ALTER TABLE for the same table which awaits for metadata lock upgrade (this is due to removal of mysql_lock_abort() from wait_while_table_is_used()). mysql-test/r/handler_myisam.result: After introduction of basic deadlock detector for metadata locks it became necessary to change parts of test for HANDLER statements which covered some of scenarios in which ER_LOCK_DEADLOCK error was detected in absence of real deadlock (with new deadlock detector this no longer happens). Also adjusted test to the fact that HANDLER READ for the table no longer will be blocked by ALTER TABLE for the same table which awaits for metadata lock upgrade (this is due to removal of mysql_lock_abort() from wait_while_table_is_used()). mysql-test/r/mdl_sync.result: Added test coverage for basic deadlock detection in metadata locking subsystem and for bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". mysql-test/r/sp-lock.result: Adjusted test coverage for metadata locking for stored routines since after introduction of basic deadlock detector for metadata locks number of scenarios in which ER_LOCK_DEADLOCK error in absence of deadlock has decreased. mysql-test/t/mdl_sync.test: Added test coverage for basic deadlock detection in metadata locking subsystem and for bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". mysql-test/t/sp-lock.test: Adjusted test coverage for metadata locking for stored routines since after introduction of basic deadlock detector for metadata locks number of scenarios in which ER_LOCK_DEADLOCK error in absence of deadlock has decreased. sql/log_event_old.cc: close_tables_for_reopen() now takes one more argument which specifies at which point it should stop releasing metadata locks acquired by this connection. sql/mdl.cc: Changed metadata locking subsystem to support basic deadlock detection with a help of the following simple empiric -- we assume that there is a deadlock if there is a connection which has to wait for a metadata lock which is currently acquired by some connection which is itself waiting to be able to acquire some shared metadata lock. To implement this change: - Added MDL_context::can_wait_lead_to_deadlock()/_impl() methods which allow to find out if there is someone waiting for metadata lock which is held by the connection and therefore deadlocks are possible if this connection is going to wait for some metadata lock. To do this added version of MDL_ticket::has_pending_conflicting_lock() method which assumes that its caller already owns LOCK_mdl mutex. - Changed MDL_context::wait_for_locks() to use one of the above methods to check if somebody is waiting for metadata lock owned by this context (and therefore deadlock is possible) and emit ER_LOCK_DEADLOCK error in this case. Also now we mark context of connections waiting inside of this method by setting MDL_context::m_is_waiting_in_mdl member. Thanks to this such connection could be waken up if some other connection starts waiting for one of its metadata locks and so a deadlock can occur. - Adjusted notify_shared_lock() to wake up connections which wait inside MDL_context::wait_for_locks() while holding shared metadata lock. - Changed MDL_ticket::upgrade_shared_lock_to_exclusive() to add temporary ticket for exclusive lock to MDL_lock::waiting queue, so request for metadata lock upgrade can be properly detected by our empiric. Also now this method invokes a callback which forces transactions holding shared metadata lock on the table to call MDL_context:: can_wait_lead_to_deadlock() method even if they don't need any new metadata locks. Thanks to this such transactions can detect deadlocks/ livelocks between MDL and table-level locks. Also reduced timeouts between calls to notify_shared_lock() in MDL_ticket::upgrade_shared_lock_to_exclusive() and MDL_context::acquire_exclusive_locks(). This was necessary to get rid of call to mysql_lock_abort() in wait_while_table_is_used(). (Now we instead rely on notify_shared_lock() timely calling mysql_lock_abort_for_thread() for the table on which lock is being upgraded/acquired). sql/mdl.h: - Added a version of MDL_ticket::has_pending_conflicting_lock() method to be used in situations when caller already has acquired LOCK_mdl mutex. - Added MDL_context::can_wait_lead_to_deadlock()/_impl() methods which allow to find out if there is someone waiting for metadata lock which is held by this connection and thus deadlocks are possible if this connections will start waiting for some metadata lock. - Added MDL_context::m_is_waiting_in_mdl member to mark connections waiting in MDL_context::wait_for_locks() method of metadata locking subsystem. Added getter method for this private member to make it accessible in notify_shared_lock() auxiliary so we can wake-up such connections if they hold shared metadata locks. - Finally, added mysql_abort_transactions_with_shared_lock() callback to be able force transactions which don't need any new metadata locks still call MDL_context::can_wait_lead_to_deadlock() and detect some of deadlocks between metadata locks and table-level locks. sql/mysql_priv.h: close_tables_for_reopen() now takes one more argument which specifies at which point it should stop releasing metadata locks acquired by this connection. sql/sql_base.cc: Changed approach to metadata locking for multi-statement transactions. We no longer fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting locks to go away by calling MDL_context::wait_for_locks() method. To avoid deadlocks the latter implements simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. To implement the change described above: - Introduced Open_table_context::m_start_of_statement_svp member to store state of metadata locks at the start of the statement. - Changed Open_table_context::request_backoff_action() not to fail with ER_LOCK_DEADLOCK immediately if back-off is requested due to conflicting metadata lock. - Added new argument for close_tables_for_reopen() procedure which allows to specify subset of metadata locks to be released. - Changed open_tables() not to release all metadata locks acquired by current transaction when metadata lock conflict is discovered. Instead we release only locks acquired by current statement. - Changed open_ltable() and open_and_lock_tables_derived() not to emit ER_LOCK_DEADLOCK error when mysql_lock_tables() is aborted in multi-statement transaction when somebody tries to acquire exclusive metadata lock on the table. Instead we release metadata locks acquired by current statement and try to wait until they can be re-acquired. - Adjusted tdc_wait_for_old_versions() to check if there is someone waiting for one of metadata locks held by this connection and run deadlock detection in order to avoid deadlocks in some situations. - Added mysql_abort_transactions_with_shared_lock() callback which allows to force transactions holding shared metadata lock on the table to call MDL_context::can_wait_lead_to_deadlock() even if they don't need any new metadata locks so they can detect potential deadlocks between metadata locking subsystem and table-level locks. - Adjusted wait_while_table_is_used() not to set TABLE::version to 0 as it is now done only when necessary by the above-mentioned callback. Also removed unnecessary call to mysql_lock_abort(). Instead we rely on code performing metadata lock upgrade aborting waits on the table-level lock for this table by calling mysql_lock_abort_for_thread() (invoked by mysql_notify_thread_having_shared_lock()). In future this should allow to reduce number of scenarios in which we produce ER_LOCK_DEADLOCK error even though no real deadlock exists. sql/sql_class.h: Introduced Open_table_context::m_start_of_statement_svp member to store state of metadata locks at the start of the statement. Replaced Open_table_context::m_can_deadlock member with m_has_locks member to reflect the fact that we no longer unconditionally emit ER_LOCK_DEADLOCK error for transaction having some metadata locks when conflicting metadata lock is discovered. sql/sql_insert.cc: close_tables_for_reopen() now takes one more argument which specifies at which point it should stop releasing metadata locks acquired by this connection. sql/sql_plist.h: Made I_P_List_iterator<T, B> usable with const lists. sql/sql_show.cc: close_tables_for_reopen() now takes one more argument which specifies at which point it should stop releasing metadata locks acquired by this connection. sql/sql_update.cc: Changed UPDATE and MULTI-UPDATE code not to release all metadata locks when calls to mysql_lock_tables() are aborted. Instead we release only locks which are acquired by this statement and then try to reacquire them by calling open_tables(). This solves bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER".
This directory contains a test suite for the MySQL daemon. To run the currently existing test cases, simply execute ./mysql-test-run in this directory. It will fire up the newly built mysqld and test it. Note that you do not have to have to do "make install", and you could actually have a co-existing MySQL installation. The tests will not conflict with it. All tests must pass. If one or more of them fail on your system, please read the following manual section for instructions on how to report the problem: http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html If you want to use an already running MySQL server for specific tests, use the --extern option to mysql-test-run. Please note that in this mode, the test suite expects you to provide the names of the tests to run. For example, here is the command to run the "alias" and "analyze" tests with an external server: mysql-test-run --extern alias analyze To match your setup, you might also need to provide --socket, --user, and other relevant options. With no test cases named on the command line, mysql-test-run falls back to the normal "non-extern" behavior. The reason for this is that some tests cannot run with an external server. You can create your own test cases. To create a test case, create a new file in the t subdirectory using a text editor. The file should have a .test extension. For example: xemacs t/test_case_name.test In the file, put a set of SQL statements that create some tables, load test data, and run some queries to manipulate it. We would appreciate it if you name your test tables t1, t2, t3 ... (to not conflict too much with existing tables). Your test should begin by dropping the tables you are going to create and end by dropping them again. This ensures that you can run the test over and over again. If you are using mysqltest commands (like result file names) in your test case, you should create the result file as follows: mysql-test-run --record test_case_name or mysqltest --record < t/test_case_name.test If you only have a simple test cases consisting of SQL statements and comments, you can create the test case in one of the following ways: mysql-test-run --record test_case_name mysql test < t/test_case_name.test > r/test_case_name.result mysqltest --record --record-file=r/test_case_name.result < t/test_case_name.test When this is done, take a look at r/test_case_name.result - If the result is incorrect, you have found a bug. In this case, you should edit the test result to the correct results so that we can verify that the bug is corrected in future releases. To submit your test case, put your .test file and .result file(s) into a tar.gz archive, add a README that explains the problem, ftp the archive to ftp://support.mysql.com/pub/mysql/secret/ and send a mail to bugs@lists.mysql.com