mirror of
https://github.com/MariaDB/server.git
synced 2025-05-16 03:24:47 +03:00
111 Commits
Author | SHA1 | Message | Date | |
---|---|---|---|---|
|
1b62f0f58c | Merge maria-5.5-mwl248 -> 10.0-base. | ||
|
7760efad74 | Merge mariadb-5.5 -> 10.0-base. | ||
|
a06224bd15 |
Addressed all remaining issues from the review of the patch
that introduced engine independent persistent statistics. In particular: - added an enumeration type for possible values of the system variable use_stat_tables - renamed KEY::real_rec_per_key to KEY::actual_rec_per_key - optimized the collection of statistical data for any primary key defined only on one column. |
||
|
6e8c4d696a |
MDEV-452 Add full support for auto-initialized/updated timestamp and datetime
Post-review changes according to Monty's review from 28/11/2012. |
||
|
65820439bd |
Fixed bug mdev-3891.
If a query referenced some system statistical tables, but not all of them, then executing an ANALYZE command simultaneously with this query could lead to a deadlock. The fix prohibited reading statistics from system statistical tables for such queries. Removed the function unlock_tables_n_open_system_tables_for_write() as not used anymore. Performed some minor refactoring of the code in sql_statistics.cc. |
||
|
4235e46ea2 |
Bug #15954872 "MAKE MDL SUBSYSTEM AND TABLE DEFINITION CACHE
ROBUST AGAINST BUGS IN CALLERS". Both MDL subsystems and Table Definition Cache code assume that callers ensure that names of objects passed to them are not longer than NAME_LEN bytes. Unfortunately due to bugs in callers this assumption might be broken in some cases. As result we get nasty bugs causing buffer overruns when we construct MDL key or TDC key from object names. This patch makes MDL and TDC code more robust against such bugs by ensuring that we always checking size of result buffer when constructing MDL and TDC keys. This doesn't free its callers from ensuring that both db and table names are shorter than NAME_LEN bytes. But at least these steps prevents buffer overruns in case of bug in caller, replacing them with less harmful behavior. This is 5.5-only version of patch. Changed code of MDL_key::mdl_key_init() to take into account size of buffer for the key. Introduced new version of create_table_def_key() helper function which constructs TDC key without risk of result buffer overrun. Places in code that construct TDC keys were changed to use this function. Also changed rm_temporary_table() and open_new_frm() functions to avoid use of "unsafe" strmov() and strxmov() functions and use safer strnxmov() instead. |
||
|
2447bc4c81 |
Addressed the following issue from the review of the patch for
engine-independent statistics. When the primary key was dropped or changed statistics on secondary indexes for the prefixes that included components of the primary key was not removed from the table mysql.index_stats. Also fixed: in the some cases when a column was changed statistics on the indexes that included this column was not removed from the table mysql.index_stats. Also disabled the test mdev-504 for --ps-protocol. |
||
|
33f3a11e2d |
Implemented MDEV-3941: CREATE TABLE xxx IF NOT EXISTS should not block if table exists.
- Added option to check_if_table_exists() to quickly check if table exists (either SHARE or .FRM) - Extended lock_table_names() to not wait for meta data locks if CREATE IF NOT EXISTS is used. mysql-test/r/create.result: New test case mysql-test/t/create.test: New test case sql/sql_base.cc: Added option to check_if_table_exists() to quickly check if table exists (either SHARE or .FRM) Extended lock_table_names() to not wait for meta data locks if CREATE IF NOT EXISTS is used. sql/sql_base.h: Updated prototype sql/sql_db.cc: Added extra argument to call to check_if_table_exists() |
||
|
f8bfb65b13 | Merge 5.5->mwl248 | ||
|
a48a91d90f | 5.3->5.5 merge | ||
|
bc4a456758 |
MDEV-452 Add full support for auto-initialized/updated timestamp and datetime
Generalized support for auto-updated and/or auto-initialized timestamp and datetime columns. This patch is a reimplementation of MySQL's "WL#5874: CURRENT_TIMESTAMP as DEFAULT for DATETIME columns". In order to ease future merges, this implementation reused few function and variable names from MySQL's patch, however the implementation is quite different. TODO: The only unresolved problem in this patch is the semantics of LOAD DATA for TIMESTAMP and DATETIME columns in the cases when there are missing or NULL columns. I couldn't fully comprehend the logic behind MySQL's behavior and its relationship with their own documentation, so I left the results to be more consistent with all other LOAD cases. The problematic test cases can be seen by running the test file function_defaults, and observing the test case differences. Those were left on purpose for discussion. |
||
|
245298f25d | MDEV-506 Cassandra dynamic columns access | ||
|
b3f09e8aa0 |
Fixed bug mdev-504.
Opening system statistical tables and reading statistical data from them for a regular table should be done after opening and locking this regular table. No test case is provided with this patch. |
||
|
8c499274da |
Performed re-factoring and re-structuring of the code for mwl#248:
- Moved the definitions of the classes to store data from persistent statistical tables into statistics.h, leaving in other internal data structures only references to the corresponding objects. - Defined class Column_statistics_collected derived from the class Column_statistics. This is a helper class to collect statistics on columns. - Moved references to read statistics to TABLE SHARE, leaving the the reference to the collected statistics in TABLE. - Added a new clone method for the class Field allowing to clone fields attached to table shares. It was was used to create fields for min/max values in the memory of the table share. A lso: - Added procedures to allocate memory for statistical data in the table share memory and in table memory. Also: - Added a test case demonstrating how ANALYZE could work in parallel to collect statistics on different indexes of the same table. - Added a test two demonstrate how two connections working simultaneously could allocate memory for statistical data in the table share memory. |
||
|
47fae7f08f |
Added procedures to delete records by keys from statistical tables.
Now when a table is dropped the statistics on the table is removed from the statistical tables. If the table is altered in such a way that a column is dropped or the type of the column is changed then statistics on the column is removed from the table column_stat. It also triggers removal of the statistics on the indexes who use this column as its component. Added procedures that changes the names of the tables or columns in the statistical tables for. These procedures are used when tables/columns are renamed. Also partly re-factored the code that introduced the persistent statistical tables. Added test cases into statistics.test to cover the new code. |
||
|
d48b4a83a2 | Merge. | ||
|
6d8e329c9a |
Fixed bug LP:973039 - Assertion `share->in_trans == 0' failed in maria_close on DROP TABLE under LOCK
- 5.5 was missing calls to ha_extra(HA_PREPARE_FOR_DROP | HA_PREPARE_FOR_RENAME); Lost in merge 5.3 -> 5.5 sql/sql_admin.cc: Updated arguments for close_all_tables_for_name sql/sql_base.h: Updated arguments for close_all_tables_for_name sql/sql_partition.cc: Updated arguments for close_all_tables_for_name sql/sql_table.cc: Updated arguments for close_all_tables_for_name Removed test of kill, as we have already called 'ha_extra(HA_PREPARE_FOR_DROP)' and the table may be inconsistent. sql/sql_trigger.cc: Updated arguments for close_all_tables_for_name sql/sql_truncate.cc: For truncate that is done with drop + recreate, signal that the table will be dropped. |
||
|
1c0a89afcc |
The pilot implementation of mwl#250: Use the statistics from persistent
statistical tables instead of the statistics provided by engine. |
||
|
ff3d16fba8 | Merge maria-5.3-mwl248 -> 5.5 = maria-5.5-mwl248. | ||
|
76f0b94bb0 |
merge with 5.3
sql/sql_insert.cc: CREATE ... IF NOT EXISTS may do nothing, but it is still not a failure. don't forget to my_ok it. ****** CREATE ... IF NOT EXISTS may do nothing, but it is still not a failure. don't forget to my_ok it. sql/sql_table.cc: small cleanup ****** small cleanup |
||
|
9809f05199 | 5.5-merge | ||
|
bafe24035d |
Bug #11764779 (former 57649)
FLUSH TABLES under FLUSH TABLES <list> WITH READ LOCK leads to assert failure. This assert was triggered if a statement tried up upgrade a metadata lock with an active FLUSH TABLE <list> WITH READ LOCK. The assert checks that the connection already holds a global intention exclusive metadata lock. However, FLUSH TABLE <list> WITH READ LOCK does not acquire this lock in order to be compatible with FLUSH TABLES WITH READ LOCK. Therefore any metadata lock upgrade caused the assert to be triggered. This patch fixes the problem by preventing metadata lock upgrade if the connection has an active FLUSH TABLE <list> WITH READ LOCK. ER_TABLE_NOT_LOCKED_FOR_WRITE will instead be reported to the client. Test case added to flush.test. |
||
|
65ca700def |
merge.
checkpoint. does not compile. |
||
|
6bf6272fda |
Patch that refactors global read lock implementation and fixes
bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations. mysql-test/include/check_ftwrl_compatible.inc: Added helper script which allows to check that a statement is compatible with FLUSH TABLES WITH READ LOCK. mysql-test/include/check_ftwrl_incompatible.inc: Added helper script which allows to check that a statement is incompatible with FLUSH TABLES WITH READ LOCK. mysql-test/include/handler.inc: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/include/wait_show_condition.inc: Fixed small error in the timeout message. The correct name of variable used as parameter for this script is "$condition" and not "$wait_condition". mysql-test/r/delayed.result: Added test coverage for scenario which triggered assert in metadata locking subsystem. mysql-test/r/events_2.result: Updated test results after prohibiting event DDL operations under LOCK TABLES. mysql-test/r/flush.result: Added test coverage for bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". mysql-test/r/flush_read_lock.result: Added test coverage for various aspects of FLUSH TABLES WITH READ LOCK functionality. mysql-test/r/flush_read_lock_kill.result: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Use new debug_sync point. Do not disable concurrent inserts as now InnoDB we always use InnoDB table. mysql-test/r/handler_innodb.result: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/r/handler_myisam.result: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/r/mdl_sync.result: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Replaced usage of GRL-specific debug_sync's with appropriate sync points in MDL subsystem. mysql-test/suite/perfschema/r/dml_setup_instruments.result: Updated test results after removing global COND_global_read_lock condition variable. mysql-test/suite/perfschema/r/func_file_io.result: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/r/func_mutex.result: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/r/global_read_lock.result: Adjusted test case to take into account that new GRL implementation is based on MDL. mysql-test/suite/perfschema/r/server_init.result: Adjusted test case after replacing custom global read lock implementation with one based on MDL and replacing LOCK_event_metadata mutex with metadata lock. mysql-test/suite/perfschema/t/func_file_io.test: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/t/func_mutex.test: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/t/global_read_lock.test: Adjusted test case to take into account that new GRL implementation is based on MDL. mysql-test/suite/perfschema/t/server_init.test: Adjusted test case after replacing custom global read lock implementation with one based on MDL and replacing LOCK_event_metadata mutex with metadata lock. mysql-test/suite/rpl/r/rpl_tmp_table_and_DDL.result: Updated test results after prohibiting event DDL under LOCK TABLES. mysql-test/t/delayed.test: Added test coverage for scenario which triggered assert in metadata locking subsystem. mysql-test/t/events_2.test: Updated test case after prohibiting event DDL operations under LOCK TABLES. mysql-test/t/flush.test: Added test coverage for bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". mysql-test/t/flush_block_commit.test: Adjusted test case after changing thread state name which is used when COMMIT waits for FLUSH TABLES WITH READ LOCK from "Waiting for release of readlock" to "Waiting for commit lock". mysql-test/t/flush_block_commit_notembedded.test: Adjusted test case after changing thread state name which is used when DML waits for FLUSH TABLES WITH READ LOCK. Now we use "Waiting for global read lock" in this case. mysql-test/t/flush_read_lock.test: Added test coverage for various aspects of FLUSH TABLES WITH READ LOCK functionality. mysql-test/t/flush_read_lock_kill-master.opt: We no longer need to use make_global_read_lock_block_commit_loop debug tag in this test. Instead we rely on an appropriate debug_sync point in MDL code. mysql-test/t/flush_read_lock_kill.test: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Use new debug_sync point. Do not disable concurrent inserts as now InnoDB we always use InnoDB table. mysql-test/t/lock_multi.test: Adjusted test case after changing thread state names which are used when DML or DDL waits for FLUSH TABLES WITH READ LOCK to "Waiting for global read lock". mysql-test/t/mdl_sync.test: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Replaced usage of GRL-specific debug_sync's with appropriate sync points in MDL subsystem. Updated thread state names which are used when DDL waits for FTWRL. mysql-test/t/trigger_notembedded.test: Adjusted test case after changing thread state names which are used when DML or DDL waits for FLUSH TABLES WITH READ LOCK to "Waiting for global read lock". sql/event_data_objects.cc: Removed Event_queue_element::status/last_executed_changed members and Event_queue_element::update_timing_fields() method. We no longer use this class for updating mysql.events once event is chosen for execution. Accesses to instances of this class in scheduler thread require protection by Event_queue::LOCK_event_queue mutex and we try to avoid updating table while holding this lock. sql/event_data_objects.h: Removed Event_queue_element::status/last_executed_changed members and Event_queue_element::update_timing_fields() method. We no longer use this class for updating mysql.events once event is chosen for execution. Accesses to instances of this class in scheduler thread require protection by Event_queue::LOCK_event_queue mutex and we try to avoid updating table while holding this lock. sql/event_db_repository.cc: - Changed Event_db_repository methods to not release all metadata locks once they are done updating mysql.events table. This allows to keep metadata lock protecting against GRL and lock protecting particular event around until corresponding DDL statement is written to the binary log. - Removed logic for conditional update of "status" and "last_executed" fields from update_timing_fields_for_event() method. In the only case when this method is called now "last_executed" is always modified and tracking change of "status" is too much hassle. sql/event_db_repository.h: Removed logic for conditional update of "status" and "last_executed" fields from Event_db_repository:: update_timing_fields_for_event() method. In the only case when this method is called now "last_executed" is always modified and tracking change of "status" field is too much hassle. sql/event_queue.cc: Changed event scheduler code not to update mysql.events table while holding Event_queue::LOCK_event_queue mutex. Doing so led to a deadlock with a new GRL implementation. This deadlock didn't occur with old implementation due to fact that code acquiring protection against GRL ignored pending GRL requests (which lead to GRL starvation). One of goals of new implementation is to disallow GRL starvation and so we have to solve problem with this deadlock in a different way. sql/events.cc: Changed methods of Events class to acquire protection against GRL while perfoming DDL statement and keep it until statement is written to the binary log. Unfortunately this step together with new GRL implementation exposed deadlock involving Events::LOCK_event_metadata and GRL. To solve it Events::LOCK_event_metadata mutex was replaced with a metadata lock on event. As a side-effect events DDL has to be prohibited under LOCK TABLES even in cases when mysql.events table was explicitly locked for write. sql/events.h: Replaced Events::LOCK_event_metadata mutex with a metadata lock on event. sql/ha_ndbcluster.cc: Updated code after replacing custom global read lock implementation with one based on MDL. Since MDL subsystem should now be able to detect deadlocks involving metadata locks and GRL there is no need for special handling of active GRL. sql/handler.cc: Replaced custom implementation of global read lock with one based on metadata locks. Consequently when doing commit instead of calling method of Global_read_lock class to acquire protection against GRL we simply acquire IX in COMMIT namespace. sql/lock.cc: Replaced custom implementation of global read lock with one based on metadata locks. This step allows to expose wait for GRL to deadlock detector of MDL subsystem and thus succesfully resolve deadlocks similar to one behind bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". It also solves problem with GRL starvation described in bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'" since metadata locks used by GRL give preference to FTWRL statement instead of DML statements (if needed in future this can be changed to fair scheduling). Similar to old implementation of acquisition of GRL is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. To support this change: - Global_read_lock::lock/unlock_global_read_lock and make_global_read_lock_block_commit methods were changed accordingly. - Global_read_lock::wait_if_global_read_lock() and start_waiting_global_read_lock() methods were dropped. It is now responsibility of code acquiring metadata locks opening tables to acquire protection against GRL by explicitly taking global IX lock with statement duration. - Global variables, mutex and condition variable used by old implementation was removed. - lock_routine_name() was changed to use statement duration for its global IX lock. It was also renamed to lock_object_name() as it now also used to take metadata locks on events. - Global_read_lock::set_explicit_lock_duration() was added which allows not to release locks used for GRL when leaving prelocked mode. sql/lock.h: - Renamed lock_routine_name() to lock_object_name() and changed its signature to allow its usage for events. - Removed broadcast_refresh() function. It is no longer needed with new GRL implementation. sql/log_event.cc: Release metadata locks with statement duration at the end of processing legacy event for LOAD DATA. This ensures that replication thread processing such event properly releases its protection against global read lock. sql/mdl.cc: Changed MDL subsystem to support new MDL-based implementation of global read lock. Added COMMIT and EVENTS namespaces for metadata locks. Changed thread state name for GLOBAL namespace to "Waiting for global read lock". Optimized MDL_map::find_or_insert() method to avoid taking m_mutex mutex when looking up MDL_lock objects for GLOBAL or COMMIT namespaces. We keep pre-created MDL_lock objects for these namespaces around and simply return pointers to these global objects when needed. Changed MDL_lock/MDL_scoped_lock to properly handle notification of insert delayed handler threads when FTWRL takes global S lock. Introduced concept of lock duration. In addition to locks with transaction duration which work in the way which is similar to how locks worked before (i.e. they are released at the end of transaction), locks with statement and explicit duration were introduced. Locks with statement duration are automatically released at the end of statement. Locks with explicit duration require explicit release and obsolete concept of transactional sentinel. * Changed MDL_request and MDL_ticket classes to support notion of duration. * Changed MDL_context to keep locks with different duration in different lists. Changed code handling ticket list to take this into account. * Changed methods responsible for releasing locks to take into account duration of tickets. Particularly public MDL_context::release_lock() method now only can release tickets with explicit duration (there is still internal method which allows to specify duration). To release locks with statement or transaction duration one have to use release_statement/transactional_locks() methods. * Concept of savepoint for MDL subsystem now has to take into account locks with statement duration. Consequently MDL_savepoint class was introduced and methods working with savepoints were updated accordingly. * Added methods which allow to set duration for one or all locks in the context. sql/mdl.h: Changed MDL subsystem to support new MDL-based implementation of global read lock. Added COMMIT and EVENTS namespaces for metadata locks. Introduced concept of lock duration. In addition to locks with transaction duration which work in the way which is similar to how locks worked before (i.e. they are released at the end of transaction), locks with statement and explicit duration were introduced. Locks with statement duration are automatically released at the end of statement. Locks with explicit duration require explicit release and obsolete concept of transactional sentinel. * Changed MDL_request and MDL_ticket classes to support notion of duration. * Changed MDL_context to keep locks with different duration in different lists. Changed code handling ticket list to take this into account. * Changed methods responsible for releasing locks to take into account duration of tickets. Particularly public MDL_context::release_lock() method now only can release tickets with explicit duration (there is still internal method which allows to specify duration). To release locks with statement or transaction duration one have to use release_statement/transactional_locks() methods. * Concept of savepoint for MDL subsystem now has to take into account locks with statement duration. Consequently MDL_savepoint class was introduced and methods working with savepoints were updated accordingly. * Added methods which allow to set duration for one or all locks in the context. sql/mysqld.cc: Removed global mutex and condition variables which were used by old implementation of GRL. Also we no longer need to initialize Events::LOCK_event_metadata mutex as it was replaced with metadata locks on events. sql/mysqld.h: Removed global variable, mutex and condition variables which were used by old implementation of GRL. sql/rpl_rli.cc: When slave thread closes tables which were open for handling of RBR events ensure that it releases global IX lock which was acquired as protection against GRL. sql/sp.cc: Adjusted code to the new signature of lock_object/routine_name(), to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sp_head.cc: Ensure that statements in stored procedures release statement metadata locks and thus release their protectiong against GRL in proper moment in time. Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_admin.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_base.cc: - Implemented support for new approach to acquiring protection against global read lock. We no longer acquire such protection explicitly on the basis of statement flags. Instead we always rely on code which is responsible for acquiring metadata locks on object to be changed acquiring this protection. This is achieved by acquiring global IX metadata lock with statement duration. Code doing this also responsible for checking that current connection has no active GRL by calling an Global_read_lock::can_acquire_protection() method. Changed code in open_table() and lock_table_names() accordingly. Note that as result of this change DDL and DML on temporary tables is always compatible with GRL (before it was incompatible in some cases and compatible in other cases). - To speed-up code acquiring protection against GRL introduced m_has_protection_against_grl member in Open_table_context class. It indicates that protection was already acquired sometime during open_tables() execution and new attempts can be skipped. - Thanks to new GRL implementation calls to broadcast_refresh() became unnecessary and were removed. - Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_base.h: Adjusted code to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. Also introduced Open_table_context::m_has_protection_against_grl member which allows to avoid acquiring protection against GRL while opening tables if such protection was already acquired. sql/sql_class.cc: Changed THD::leave_locked_tables_mode() after transactional sentinel for metadata locks was obsoleted by introduction of locks with explicit duration. sql/sql_class.h: - Adjusted code to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. - Changed Global_read_lock class according to changes in global read lock implementation: * wait_if_global_read_lock and start_waiting_global_read_lock are now gone. Instead code needing protection against GRL has to acquire global IX metadata lock with statement duration itself. To help it new can_acquire_protection() was introduced. Also as result of the above change m_protection_count member is gone too. * Added m_mdl_blocks_commits_lock member to store metadata lock blocking commits. * Adjusted code to the fact that concept of transactional sentinel was obsoleted by concept of lock duration. - Removed CF_PROTECT_AGAINST_GRL flag as it is no longer necessary. New GRL implementation acquires protection against global read lock automagically when statement acquires metadata locks on tables or other objects it is going to change. sql/sql_db.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_handler.cc: Removed call to broadcast_refresh() function. It is no longer needed with new GRL implementation. Adjusted code after introducing duration concept for metadata locks. Particularly to the fact transactional sentinel was replaced with explicit duration. sql/sql_handler.h: Renamed mysql_ha_move_tickets_after_trans_sentinel() to mysql_ha_set_explicit_lock_duration() after transactional sentinel was obsoleted by locks with explicit duration. sql/sql_insert.cc: Adjusted code handling delaying inserts after switching to new GRL implementation. Now connection thread initiating delayed insert has to acquire global IX lock in addition to metadata lock on table being inserted into. This IX lock protects against GRL and similarly to SW lock on table being inserted into has to be passed to handler thread in order to avoid deadlocks. sql/sql_lex.cc: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/sql_lex.h: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/sql_parse.cc: - Implemented support for new approach to acquiring protection against global read lock. We no longer acquire such protection explicitly on the basis of statement flags. Instead we always rely on code which is responsible for acquiring metadata locks on object to be changed acquiring this protection. This is achieved by acquiring global IX metadata lock with statement duration. This lock is automatically released at the end of statement execution. - Changed implementation of CREATE/DROP PROCEDURE/FUNCTION not to release metadata locks and thus protection against of GRL in the middle of statement execution. - Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_prepare.cc: Adjusted code to the to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_rename.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before renaming tables. This happens automatically in code which acquires metadata locks on tables being renamed. sql/sql_show.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_table.cc: - With new GRL implementation there is no need to explicitly acquire protection against GRL before dropping tables. This happens automatically in code which acquires metadata locks on tables being dropped. - Changed mysql_alter_table() not to release lock on new table name explicitly and to rely on automatic release of locks at the end of statement instead. This was necessary since now MDL_context::release_lock() is supported only for locks for explicit duration. sql/sql_trigger.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before changing table triggers. This happens automatically in code which acquires metadata locks on tables which triggers are to be changed. sql/sql_update.cc: Fix bug exposed by GRL testing. During prepare phase acquire only S metadata locks instead of SW locks to keep prepare of multi-UPDATE compatible with concurrent LOCK TABLES WRITE and global read lock. sql/sql_view.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before creating view. This happens automatically in code which acquires metadata lock on view to be created. sql/sql_yacc.yy: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/table.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/table.h: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/transaction.cc: Replaced custom implementation of global read lock with one based on metadata locks. Consequently when doing commit instead of calling method of Global_read_lock class to acquire protection against GRL we simply acquire IX in COMMIT namespace. Also adjusted code to the fact that MDL savepoint is now represented by MDL_savepoint class. |
||
|
881da26b83 | Remove a dead declaration. | ||
|
476939cb45 |
Reverted a temporary workaround for bug #56405 "Deadlock
in the MDL deadlock detector". It is no longer needed as a better fix for this bug has been pushed. |
||
|
bf55d1fcf0 | Merge from mysql-5.5-bugteam to mysql-5.5-runtime | ||
|
e86bbbda55 |
Fix for bug #56251 "Deadlock with INSERT DELAYED and MERGE
tables". Attempting to issue an INSERT DELAYED statement for a MERGE table might have caused a deadlock if it happened as part of a transaction or under LOCK TABLES, and there was a concurrent DDL or LOCK TABLES ... WRITE statement which tried to lock one of its underlying tables. The problem occurred when a delayed insert handler thread tried to open a MERGE table and discovered that to do this it had also to open all underlying tables and hence acquire metadata locks on them. Since metadata locks on the underlying tables were not pre-acquired by the connection thread executing INSERT DELAYED, attempts to do so might lead to waiting. In this case the connection thread had to wait for the delayed insert thread. If the thread which was preventing the lock on the underlying table from being acquired had to wait for the connection thread (due to this or other metadata locks), a deadlock occurred. This deadlock was not detected by the MDL deadlock detector since waiting for the handler thread by the connection thread is not represented in the wait-for graph. This patch solves the problem by ensuring that the delayed insert handler thread never tries to open underlying tables of a MERGE table. Instead open_tables() is aborted right after the parent table is opened and a ER_DELAYED_NOT_SUPPORTED error is emitted (which is passed to the connection thread and ultimately to the user). mysql-test/r/merge.result: Added test for bug #56251 "Deadlock with INSERT DELAYED and MERGE tables". mysql-test/t/merge.test: Added test for bug #56251 "Deadlock with INSERT DELAYED and MERGE tables". sql/sql_base.cc: Changed open_n_lock_single_table() to take prelocking strategy as an argument instead of always using DML_prelocking_strategy. sql/sql_base.h: Changed open_n_lock_single_table() to take prelocking strategy as an argument instead of always using DML_prelocking_strategy. Added a version of this function which is compatible with old signature. sql/sql_insert.cc: When opening MERGE table in delayed insert thread stop and emit ER_DELAYED_NOT_SUPPORTED right after opening main table and before opening underlying tables. This ensures that we won't try to acquire metadata lock on underlying tables which might lead to a deadlock. This is achieved by using special prelocking strategy which abort open_tables() process as soon as we discover that we have opened table with engine which doesn't support delayed inserts. |
||
|
65a438d87a |
Fix for bug #55273 "FLUSH TABLE tm WITH READ LOCK for Merge
table causes assert failure". Attempting to use FLUSH TABLE table_list WITH READ LOCK statement for a MERGE table led to an assertion failure if one of its children was not present in the list of tables to be flushed. The problem was not visible in non-debug builds. The assertion failure was caused by the fact that in such situations FLUSH TABLES table_list WITH READ LOCK implementation tried to use (e.g. lock) such child tables without acquiring metadata lock on them. This happened because when opening tables we assumed metadata locks on all tables were already acquired earlier during statement execution and a such assumption was false for MERGE children. This patch fixes the problem by ensuring at open_tables() time that we try to acquire metadata locks on all tables to be opened. For normal tables such requests are satisfied instantly since locks are already acquired for them. For MERGE children metadata locks are acquired in normal fashion. Note that FLUSH TABLES merge_table WITH READ LOCK will lock for read both the MERGE table and its children but will flush only the MERGE table. To flush children one has to mention them in table list explicitly. This is expected behavior and it is consistent with usage patterns for this statement (e.g. in mysqlhotcopy script). mysql-test/r/flush.result: Added test case for bug #55273 "FLUSH TABLE tm WITH READ LOCK for Merge table causes assert failure". mysql-test/t/flush.test: Added test case for bug #55273 "FLUSH TABLE tm WITH READ LOCK for Merge table causes assert failure". sql/sql_base.cc: Changed lock_table_names() to support newly introduced MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag. sql/sql_base.h: Introduced MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag for open_tables() and lock_table_names() which allows to skip acquiring of global and schema-scope locks when SNW, SNRW or X metadata locks are acquired. sql/sql_reload.cc: Changed "FLUSH TABLES table_list WITH READ LOCK" code not to cause assert about missing metadata locks when MERGE table is flushed without one of its underlying tables. To achieve this we no longer call open_and_lock_tables() with MYSQL_OPEN_HAS_MDL_LOCK flag so this function automatically acquires metadata locks on MERGE children if such lock has not been already acquired at earlier stage. Instead we call this function with MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag to suppress acquiring of global IX lock in order to keep FLUSH TABLES table_list WITH READ LOCK compatible with FLUSH TABLE WITH READ LOCK. Also changed implementation to use lock_table_names() function for pre-acquiring of metadata locks instead of custom code. To implement this change moved setting of open_type member for table list elements to parser. sql/sql_yacc.yy: Now we set acceptable type of table for FLUSH TABLES table_list WITH READ LOCK at parsing time instead of execution time. |
||
|
f052fa3aad |
A temporary workaround for bug #56405 "Deadlock in the
MDL deadlock detector". Deadlock could have occurred when workload containing mix of DML, DDL and FLUSH TABLES statements affecting same set of tables was executed in heavily concurrent environment. This deadlock occurred when several connections tried to perform deadlock detection in metadata locking subsystem. The first connection started traversing wait-for graph, encountered sub-graph representing wait for flush, acquired LOCK_open and dived into sub-graph inspection. When it has encounterd sub-graph corresponding to wait for metadata lock and blocked while trying to acquire rd-lock on MDL_lock::m_rwlock (*) protecting this subgraph, since some other thread had wr-lock on it. When this wr-lock was released it could have happened (if there was other pending wr-lock against this rwlock) that rd-lock from the first connection was left unsatisfied but at the same time new rd-lock request from the second connection sneaked in and was satisfied (for this to be possible second rd- request should come exactly after wr-lock is released but before pending wr-lock manages to grab rwlock, which is possible both on Linux and in our own rwlock implementation). If this second connection continued traversing wait-for graph and encountered sub-graph representing wait for flush it tried to acquire LOCK_open and thus deadlock was created. This patch tries to workaround this problem but not allowing deadlock detector to lock LOCK_open mutex if some other thread doing deadlock detection already owns it and current search depth is greater than 0. Instead deadlock is reported. Other possible solutions are either known to have negative effects on performance or require much more time for proper implementation and testing. No test case is provided as this bug is very hard to repeat in MTR environment but is repeatable with the help of RQG tests. sql/mdl.cc: Moved Deadlock_detection_visitor::m_current_search_depth to parent class to make it available in TABLE_SHARE::visit_subgraph(). Added MDL_wait_for_graph_visitor::abort_traversal() method which allows to abort traversal of a wait-for graph and report a deadlock. sql/mdl.h: Moved Deadlock_detection_visitor::m_current_search_depth to parent class to make it available in TABLE_SHARE::visit_subgraph(). Added MDL_wait_for_graph_visitor::abort_traversal() method which allows to abort traversal of a wait-for graph and report a deadlock. sql/sql_base.cc: Added dd_owns_lock_open counter and mutex protecting it to track number of connections which do deadlock detection and own or try to acquire LOCK_open. sql/sql_base.h: Added dd_owns_lock_open counter and mutex protecting it to track number of connections which do deadlock detection and own or try to acquire LOCK_open. sql/table.cc: Workaround bug #56405 but not allowing MDL deadlock detector to lock LOCK_open mutex if some other thread doing deadlock detection already owns it and current search depth is greater than 0. Instead report deadlock. |
||
|
cc0925f5d9 |
Bug#27480 (Extend CREATE TEMPORARY TABLES privilege
to allow temp table operations) -- prerequisite patch #3. Rename open_temporary_table() to open_table_uncached(). open_temporary_table() will be introduced in following patches to open temporary tables for a statement. |
||
|
b4badf8013 |
Remove unused enum (enum open_table_mode).
It was added by mistake during backport from 6.0. |
||
|
65c5e8dcbf |
Bug#27480 (Extend CREATE TEMPORARY TABLES privilege
to allow temp table operations) -- prerequisite patch #2. Introduce a new form of find_temporary_table() function: find_temporary_table() by a table key. It will be used in further patches. Replace find_temporary_table(table_list->db, table_list->name) by more appropiate find_temporary_table(table_list) across the codebase. |
||
|
02863b4180 | Merge from mysql-5.5-runtime to mysql-5.5-bugfixing. | ||
|
ac6026ce27 |
BUG#53452 Inconsistent behavior of binlog_direct_non_transactional_updates with
temp table This patch introduces two key changes in the replication's behavior. Firstly, it reverts part of BUG#51894 which puts any update to temporary tables into the trx-cache. Now, updates to temporary tables are handled according to the type of their engines as a regular table. Secondly, an unsafe mixed statement, (i.e. a statement that access transactional table as well non-transactional or temporary table, and writes to any of them), are written into the trx-cache in order to minimize errors in the execution when the statement logging format is in use. Such changes has a direct impact on which statements are classified as unsafe statements and thus part of BUG#53259 is reverted. |
||
|
41caa7f483 |
Bug #56085 Embedded server tests fails with assert in
check_if_table_exists() This assert was triggered when the server tried to load plugins while running in embedded server mode. In embedded server mode, check_if_table_exists() was used to check if mysql.plugin existed so that ER_NO_SUCH_TABLE could be silently ignored. The problem was that this check was done without acquiring a metadata lock on mysql.plugin first. This triggered the assert. This patch fixes the problem by removing the call to check_if_table_exists() from plugin_load(). Instead an error handler which traps ER_NO_SUCH_TABLE is installed before trying to open mysql.plugin when running in embedded server mode. No test coverage added since this assert was triggered by existing tests running in embedded server mode. sql/sql_base.cc: Renamed Prelock_error_handler to No_such_table_error_handler and moved the declaration to sql_base.h to make it usable in plugin_load(). sql/sql_base.h: Renamed Prelock_error_handler to No_such_table_error_handler and moved the declaration to sql_base.h to make it usable in plugin_load(). sql/sql_plugin.cc: Removed call to check_if_table_exists() used to check for mysql.plugin in plugin_load() for embedded server. Instead install error handler which traps ER_NO_SUCH_TABLE during open_and_lock_tables(). |
||
|
eb498cce4d | Manual merge from mysql-5.5-bugfixing to mysql-5.5-runtime. | ||
|
8ce6e98aa3 |
Extract reload_acl_and_cache() and flush_tables_with_read_lock()
into an own implementation file. |
||
|
daf0e6b725 |
Bug#53676: Unexpected errors and possible table
corruption on ADD PARTITION and LOCK TABLE Bug#53770: Server crash at handler.cc:2076 on LOAD DATA after timed out COALESCE PARTITION 5.5 fix for: Bug#51042: REORGANIZE PARTITION can leave table in an inconsistent state in case of crash Needs to be back-ported to 5.1 5.5 fix for: Bug#50418: DROP PARTITION does not interact with transactions Main problem was non-persistent operations done before meta-data lock was taken (53770+53676). And 53676 needed to keep the table/partitions opened and locked while copying the data to the new partitions. Also added thorough tests to spot some additional bugs in the ddl_log code, which could result in bad state between the .frm and partitions. Collapsed patch, includes all fixes required from the reviewers. mysql-test/r/partition_innodb.result: updated result with new test mysql-test/suite/parts/inc/partition_crash.inc: crash test include file mysql-test/suite/parts/inc/partition_crash_add.inc: test all states in fast_alter_partition_table ADD PARTITION branch mysql-test/suite/parts/inc/partition_crash_change.inc: test all states in fast_alter_partition_table CHANGE PARTITION branch mysql-test/suite/parts/inc/partition_crash_drop.inc: test all states in fast_alter_partition_table DROP PARTITION branch mysql-test/suite/parts/inc/partition_fail.inc: recovery test including injecting errors mysql-test/suite/parts/inc/partition_fail_add.inc: test all states in fast_alter_partition_table ADD PARTITION branch mysql-test/suite/parts/inc/partition_fail_change.inc: test all states in fast_alter_partition_table CHANGE PARTITION branch mysql-test/suite/parts/inc/partition_fail_drop.inc: test all states in fast_alter_partition_table DROP PARTITION branch mysql-test/suite/parts/inc/partition_mgm_crash.inc: include file that runs all crash and failure injection tests. mysql-test/suite/parts/r/partition_debug_innodb.result: new test result file mysql-test/suite/parts/r/partition_debug_myisam.result: new test result file mysql-test/suite/parts/r/partition_special_innodb.result: updated result mysql-test/suite/parts/r/partition_special_myisam.result: updated result mysql-test/suite/parts/t/partition_debug_innodb-master.opt: opt file for using with crashing tests of partitioned innodb mysql-test/suite/parts/t/partition_debug_innodb.test: partitioned innodb test that require debug builds mysql-test/suite/parts/t/partition_debug_myisam-master.opt: opt file for using with crashing tests of partitioned myisam mysql-test/suite/parts/t/partition_debug_myisam.test: partitioned myisam test that require debug builds mysql-test/suite/parts/t/partition_special_innodb-master.opt: added innodb-file-per-table to easier verify partition status on disk mysql-test/suite/parts/t/partition_special_innodb.test: added test case mysql-test/suite/parts/t/partition_special_myisam.test: added test case mysql-test/t/partition_innodb.test: added test case sql/sql_base.cc: Moved alter_close_tables to sql_partition.cc sql/sql_base.h: removed some non existing and duplicated functions. sql/sql_partition.cc: fast_alter_partition_table: Spletted abort_and_upgrad_lock_and_close_table to its parts (wait_while_table_is_used and alter_close_tables) and always have wait_while_table_is_used before any persistent operations (including logs, which will be executed on failure) and alter_close_tables after create/read/write operations and before drop operations. moved alter_close_tables here from sql_base.cc Added error injections for better test coverage. write_log_final_change_partition: fixed a log_entry linking bug (delete_frm was not linked to change/drop partition) and drop partition must be executed before change partition (change partition can rename a partition to an old name, like REORG p1 INTO (p1,p2). write_log_add_change_partition: need to use drop_frm first, and relinking that entry and reusing its execute entry. sql/sql_table.cc: added initialization of next_active_log_entry. sql/table.h: removed a duplicate declaration. |
||
|
82759db0cc |
Remove dead code.
sql/sql_base.h: Remove an unused parameter. sql/sql_servers.cc: Remove an unused parameter. |
||
|
29c4873ad5 |
Commit on behalf of Dmitry Lenev.
Merge his patch for Bug#52044 into 5.5, and apply review comments. |
||
|
523066987d |
A fix for Bug#41158 "DROP TABLE holds LOCK_open during unlink()".
Remove acquisition of LOCK_open around file system operations, since such operations are now protected by metadata locks. Rework table discovery algorithm to not require LOCK_open. No new tests added since all MDL locking operations are covered in lock.test and mdl_sync.test, and as long as these tests pass despite the increased concurrency, consistency must be unaffected. mysql-test/t/disabled.def: Disable NDB tests due to Bug#55799. sql/datadict.cc: No longer necessary to protect ha_create_table() with LOCK_open. Serial execution is now ensured by metadata locks. sql/ha_ndbcluster.cc: Do not manipulate with LOCK_open in cluster code. sql/ha_ndbcluster_binlog.cc: Do not manipulate with LOCK_open in cluster code. sql/ha_ndbcluster_binlog.h: Update function signature. sql/handler.cc: Implement ha_check_if_table_exists(). @todo: some engines provide ha_table_exists_in_engine() handlerton call, for those we perhaps shouldn't call ha_discover(), to be more efficient. Since currently it's only NDB, postpone till integration with NDB. sql/handler.h: Declare ha_check_if_table_exists() function. sql/mdl.cc: Remove an obsolete comment. sql/sql_base.cc: Update to a new signature of close_cached_tables(): from now on we always call it without LOCK_open. Update comments. Remove get_table_share_with_create(), we should not attempt to create a table under LOCK_open. Introduce get_table_share_with_discover() instead, which would request a back off action if the table exists in engine. Remove acquisition of LOCK_open for data dictionary operations, such as check_if_table_exists(). Do not use get_table_share_with_create/discover for views, where it's not needed. Make tdc_remove_table() optionally acquire LOCK_open to simplify usage of this function. Use the right mutex in the partitioning code when manipulating with thd->open_tables. sql/sql_base.h: Update signatures of changes functions. sql/sql_insert.cc: Do not wrap quick_rm_table() with LOCK_open acquisition, this is unnecessary. sql/sql_parse.cc: Update to the new calling convention of tdc_remove_table(). Update to the new signature of close_cached_tables(). Update comments. sql/sql_rename.cc: Update to the new calling convention of tdc_remove_table(). Remove acquisition of LOCK_open around filesystem operations. sql/sql_show.cc: Remove get_trigger_table_impl(). Do not acquire LOCK_open for a dirty read of the trigger file. sql/sql_table.cc: Do not acquire LOCK_open for filesystem operations. sql/sql_trigger.cc: Do not require LOCK_open for trigger file I/O. sql/sql_truncate.cc: Update to the new signature of tdc_remove_table(). sql/sql_view.cc: Do not require LOCK_open for view I/O. Use tdc_remove_table() to expel view share. Update comments. sql/sys_vars.cc: Update to the new signature of close_cached_tables(). |
||
|
a91a5ee3bf |
Cleanup: remove unused declarations from sql_base.h.
sql/sql_base.cc: Update a comment to not refer to a non-existing function. sql/sql_base.h: Remove unused declarations (bad merge with the header files split worklog). sql/sql_test.cc: Cleanup: remove unused declarations. |
||
|
d85c628180 | Merge trunk-bugfixing -> trunk-runtime. | ||
|
00496b7acd |
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH
TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush. mysql-test/include/handler.inc: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/r/flush.result: Added test which checks that "flush tables <list> with read lock" is compatible with active "flush tables with read lock" but not vice-versa. This test also covers bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". mysql-test/r/mdl_sync.result: Added scenarios in which wait for table to be flushed causes deadlocks to the coverage of MDL deadlock detector. mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after removal of COND_refresh condition variable. mysql-test/suite/perfschema/r/server_init.result: Adjusted test and its results after removal of COND_refresh condition variable. mysql-test/suite/perfschema/t/server_init.test: Adjusted test and its results after removal of COND_refresh condition variable. mysql-test/t/flush.test: Added test which checks that "flush tables <list> with read lock" is compatible with active "flush tables with read lock" but not vice-versa. This test also covers bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". mysql-test/t/kill.test: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/t/lock_multi.test: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/t/mdl_sync.test: Added scenarios in which wait for table to be flushed causes deadlocks to the coverage of MDL deadlock detector. sql/ha_ndbcluster.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/ha_ndbcluster_binlog.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/lock.cc: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/mdl.cc: Now MDL deadlock detector takes into account information about waits for table flushes when searching for deadlock. To implement this change: - Declaration of enum_deadlock_weight and Deadlock_detection_visitor were moved to mdl.h header to make them available to the code in table.cc which implements deadlock detector traversal through edges of waiters graph representing waiting for flush. - Since now MDL_context may wait not only for metadata lock but also for table to be flushed an abstract Wait_for_edge class was introduced. Its descendants MDL_ticket and Flush_ticket incapsulate specifics of inspecting waiters graph when following through edge representing wait of particular type. We no longer require global IX metadata lock when acquiring SNW or SNRW locks. Such locks are needed only when metadata locks of these types are upgraded to X locks. This allows to use SNW locks in FLUSH TABLES <list> WITH READ LOCK implementation and keep the latter compatible with global read lock. sql/mdl.h: Now MDL deadlock detector takes into account information about waits for table flushes when searching for deadlock. To implement this change: - Declaration of enum_deadlock_weight and Deadlock_detection_visitor were moved to mdl.h header to make them available to the code in table.cc which implements deadlock detector traversal through edges of waiters graph representing waiting for flush. - Since now MDL_context may wait not only for metadata lock but also for table to be flushed an abstract Wait_for_edge class was introduced. Its descendants MDL_ticket and Flush_ticket incapsulate specifics of inspecting waiters graph when following through edge representing wait of particular type. - Deadlock_detection_visitor now has m_table_shares_visited member which allows to support recursive locking for LOCK_open. This is required when deadlock detector inspects waiters graph which contains several edges representing waits for flushes or needs to come through the such edge more than once. sql/mysqld.cc: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/mysqld.h: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/sql_base.cc: Changed approach to how threads are waiting for table to be flushed. Now thread that wants to wait for old table to go away subscribes for notification by adding Flush_ticket to table's share and waits using MDL_context::m_wait object. Once table gets flushed (i.e. all tables are closed and table share is ready to be destroyed) all such waiters are notified individually. Thanks to this change MDL deadlock detector can take such waits into account. To implement this/as result of this change: - tdc_wait_for_old_versions() was replaced with tdc_wait_for_old_version() which waits for individual old share to go away and which is called by open_table() after finding out that share is outdated. We don't need to perform back-off before such waiting thanks to the fact that deadlock detector now sees such waits. - As result Open_table_ctx::m_mdl_requests became unnecessary and was removed. We no longer allocate copies of MDL_request objects on MEM_ROOT when MYSQL_OPEN_FORCE_SHARED/SHARED_HIGH_PRIO flags are in effect. - close_cached_tables() and tdc_wait_for_old_version() share code which implements waiting for share to be flushed - the both use TABLE_SHARE::wait_until_flush() method. Thanks to this close_cached_tables() supports timeouts and has extra parameter for this. - Open_table_context::OT_MDL_CONFLICT enum element was renamed to OT_CONFLICT as it is now also used in cases when back-off is required to resolve deadlock caused by waiting for flush and not metadata lock. - In cases when we discover that current connection tries to open tables from different generation we now simply back-off and restart process of opening tables. To support this Open_table_context::OT_REOPEN_TABLES enum element was added. - COND_refresh condition variable became unnecessary and was removed. - mysql_notify_thread_having_shared_lock() no longer wakes up connections waiting for flush as all such connections can be waken up by deadlock detector if necessary. sql/sql_base.h: - close_cached_tables() now has one more parameter - timeout for waiting for table to be flushed. - Open_table_context::OT_MDL_CONFLICT enum element was renamed to OT_CONFLICT as it is now also used in cases when back-off is required to resolve deadlock caused by waiting for flush and not metadata lock. Added new OT_REOPEN_TABLES enum element to be used in cases when we need to restart open tables process even in the middle of transaction. - Open_table_ctx::m_mdl_requests became unnecessary and was removed. sql/sql_class.h: Added assert ensuring that we won't use LOCK_open mutex with THD::enter_cond(). Otherwise deadlocks can arise in MDL deadlock detector. sql/sql_parse.cc: Changed FLUSH TABLES <list> WITH READ LOCK to take SNW metadata locks instead of X locks on tables to be flushed. Since we no longer require global IX lock to be taken when SNW locks are taken this makes this statement compatible with FLUSH TABLES WITH READ LOCK statement. Since SNW locks allow other connections to have table opened FLUSH TABLES <list> WITH READ LOCK now has to wait during open_tables() for old version to go away. Such waits can lead to deadlocks which will be detected by MDL deadlock detector which now takes waits for table to be flushed into account. Also adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/sql_yacc.yy: FLUSH TABLES <list> WITH READ LOCK now needs only SNW metadata locks on tables. sql/sys_vars.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/table.cc: Implemented new approach to how threads are waiting for table to be flushed. Now thread that wants to wait for old table to go away subscribes for notification by adding Flush_ticket to table's share and waits using MDL_context::m_wait object. Once table gets flushed (i.e. all tables are closed and table share is ready to be destroyed) all such waiters are notified individually. This change allows to make such waits visible inside of MDL deadlock detector. To do it: - Added list of waiters/Flush_tickets to TABLE_SHARE class. - Changed free_table_share() to postpone freeing of share memory until last waiter goes away and to wake up subscribed waiters. - Added TABLE_SHARE::wait_until_flushed() method which implements subscription to the list of waiters for table to be flushed and waiting for this event. Implemented interface which allows to expose waits for flushes to MDL deadlock detector: - Introduced Flush_ticket class a descendant of Wait_for_edge class. - Added TABLE_SHARE::find_deadlock() method which allows deadlock detector to find out what contexts are still using old version of table in question (i.e. to find out what contexts are waited for by owner of Flush_ticket). sql/table.h: In order to support new strategy of waiting for table flush (see comment for table.cc for details) added list of waiters/Flush_tickets to TABLE_SHARE class. Implemented interface which allows to expose waits for flushes to MDL deadlock detector: - Introduced Flush_ticket class a descendant of Wait_for_edge class. - Added TABLE_SHARE::find_deadlock() method which allows deadlock detector to find out what contexts are still using old version of table in question (i.e. to find out what contexts are waited for by owner of Flush_ticket). |
||
|
36290c0923 |
A pre-requisite patch for the fix for Bug#52044.
This patch also fixes Bug#55452 "SET PASSWORD is replicated twice in RBR mode". The goal of this patch is to remove the release of metadata locks from close_thread_tables(). This is necessary to not mistakenly release the locks in the course of a multi-step operation that involves multiple close_thread_tables() or close_tables_for_reopen(). On the same token, move statement commit outside close_thread_tables(). Other cleanups: Cleanup COM_FIELD_LIST. Don't call close_thread_tables() in COM_SHUTDOWN -- there are no open tables there that can be closed (we leave the locked tables mode in THD destructor, and this close_thread_tables() won't leave it anyway). Make open_and_lock_tables() and open_and_lock_tables_derived() call close_thread_tables() upon failure. Remove the calls to close_thread_tables() that are now unnecessary. Simplify the back off condition in Open_table_context. Streamline metadata lock handling in LOCK TABLES implementation. Add asserts to ensure correct life cycle of statement transaction in a session. Remove a piece of dead code that has also become redundant after the fix for Bug 37521. mysql-test/r/variables.result: Update results: set @@autocommit and statement transaction/ prelocked mode. mysql-test/r/view.result: A harmless change in CHECK TABLE <view> status for a broken view. If previously a failure to prelock all functions used in a view would leave the connection in LTM_PRELOCKED mode, now we call close_thread_tables() from open_and_lock_tables() and leave prelocked mode, thus some check in mysql_admin_table() that works only in prelocked/locked tables mode is no longer activated. mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result: Fixed Bug#55452 "SET PASSWORD is replicated twice in RBR mode": extra binlog events are gone from the binary log. mysql-test/t/variables.test: Add a test case: set autocommit and statement transaction/prelocked mode. sql/event_data_objects.cc: Simplify code in Event_job_data::execute(). Move sp_head memory management to lex_end(). sql/event_db_repository.cc: Move the release of metadata locks outside close_thread_tables(). Make sure we call close_thread_tables() when open_and_lock_tables() fails and remove extra code from the events data dictionary. Use close_mysql_tables(), a new internal function to properly close mysql.* tables in the data dictionary. Contract Event_db_repository::drop_events_by_field, drop_schema_events into one function. When dropping all events in a schema, make sure we don't mistakenly release all locks acquired by DROP DATABASE. These include locks on the database name and the global intention exclusive metadata lock. sql/event_db_repository.h: Function open_event_table() does not require an instance of Event_db_repository. sql/events.cc: Use close_mysql_tables() instead of close_thread_tables() to bootstrap events, since the latter no longer releases metadata locks. sql/ha_ndbcluster.cc: - mysql_rm_table_part2 no longer releases acquired metadata locks. Do it in the caller. sql/ha_ndbcluster_binlog.cc: Deploy the new protocol for closing thread tables in run_query() and ndb_binlog_index code. sql/handler.cc: Assert that we never call ha_commit_trans/ ha_rollback_trans in sub-statement, which is now the case. sql/handler.h: Add an accessor to check whether THD_TRANS object is empty (has no transaction started). sql/log.cc: Update a comment. sql/log_event.cc: Since now we commit/rollback statement transaction in mysql_execute_command(), we need a mechanism to communicate from Query_log_event::do_apply_event() to mysql_execute_command() that the statement transaction should be rolled back, not committed. Ideally it would be a virtual method of THD. I hesitate to make THD a virtual base class in this already large patch. Use a thd->variables.option_bits for now. Remove a call to close_thread_tables() from the slave IO thread. It doesn't open any tables, and the protocol for closing thread tables is more complicated now. Make sure we properly close thread tables, however, in Load_data_log_event, which doesn't follow the standard server execution procedure with mysql_execute_command(). @todo: this piece should use Server_runnable framework instead. Remove an unnecessary call to mysql_unlock_tables(). sql/rpl_rli.cc: Update Relay_log_info::slave_close_thread_tables() to follow the new close protocol. sql/set_var.cc: Remove an unused header. sql/slave.cc: Remove an unnecessary call to close_thread_tables(). sql/sp.cc: Remove unnecessary calls to close_thread_tables() from SP DDL implementation. The tables will be closed by the caller, in mysql_execute_command(). When dropping all routines in a database, make sure to not mistakenly drop all metadata locks acquired so far, they include the scoped lock on the schema. sql/sp_head.cc: Correct the protocol that closes thread tables in an SP instruction. Clear lex->sphead before cleaning up lex with lex_end to make sure that we don't delete the sphead twice. It's considered to be "cleaner" and more in line with future changes than calling delete lex->sphead in other places that cleanup the lex. sql/sp_head.h: When destroying m_lex_keeper of an instruction, don't delete the sphead that all lex objects share. @todo: don't store a reference to routine's sp_head instance in instruction's lex. sql/sql_acl.cc: Don't call close_thread_tables() where the caller will do that for us. Fix Bug#55452 "SET PASSWORD is replicated twice in RBR mode" by disabling RBR replication in change_password() function. Use close_mysql_tables() in bootstrap and ACL reload code to make sure we release all metadata locks. sql/sql_base.cc: This is the main part of the patch: - remove manipulation with thd->transaction and thd->mdl_context from close_thread_tables(). Now this function is only responsible for closing tables, nothing else. This is necessary to be able to easily use close_thread_tables() in procedures, that involve multiple open/close tables, which all need to be protected continuously by metadata locks. Add asserts ensuring that TABLE object is only used when is protected by a metadata lock. Simplify the back off condition of Open_table_context, we no longer need to look at the autocommit mode. Make open_and_lock_tables() and open_normal_and_derived_tables() close thread tables and release metadata locks acquired so-far upon failure. This simplifies their usage. Implement close_mysql_tables(). sql/sql_base.h: Add declaration for close_mysql_tables(). sql/sql_class.cc: Remove a piece of dead code that has also become redundant after the fix for Bug 37521. The code became dead when my_eof() was made a non-protocol method, but a method that merely modifies the diagnostics area. The code became redundant with the fix for Bug#37521, when we started to cal close_thread_tables() before Protocol::end_statement(). sql/sql_do.cc: Do nothing in DO if inside a substatement (the assert moved out of trans_rollback_stmt). sql/sql_handler.cc: Add comments. sql/sql_insert.cc: Remove dead code. Release metadata locks explicitly at the end of the delayed insert thread. sql/sql_lex.cc: Add destruction of lex->sphead to lex_end(), lex "reset" method called at the end of each statement. sql/sql_parse.cc: Move close_thread_tables() and other related cleanups to mysql_execute_command() from dispatch_command(). This has become possible after the fix for Bug#37521. Mark federated SERVER statements as DDL. Next step: make sure that we don't store eof packet in the query cache, and move the query cache code outside mysql_parse. Brush up the code of COM_FIELD_LIST. Remove unnecessary calls to close_thread_tables(). When killing a query, don't report "OK" if it was a suicide. sql/sql_parse.h: Remove declaration of a function that is now static. sql/sql_partition.cc: Remove an unnecessary call to close_thread_tables(). sql/sql_plugin.cc: open_and_lock_tables() will clean up after itself after a failure. Move close_thread_tables() above end: label, and replace with close_mysql_tables(), which will also release the metadata lock on mysql.plugin. sql/sql_prepare.cc: Now that we no longer release locks in close_thread_tables() statement prepare code has become more straightforward. Remove the now redundant check for thd->killed() (used only by the backup project) from Execute_server_runnable. Reorder code to take into account that now mysql_execute_command() performs lex->unit.cleanup() and close_thread_tables(). sql/sql_priv.h: Add a new option to server options to interact between the slave SQL thread and execution framework (hack). @todo: use a virtual method of class THD instead. sql/sql_servers.cc: Due to Bug 25705 replication of DROP/CREATE/ALTER SERVER is broken. Make sure at least we do not attempt to replicate these statements using RBR, as this violates the assert in close_mysql_tables(). sql/sql_table.cc: Do not release metadata locks in mysql_rm_table_part2, this is done by the caller. Do not call close_thread_tables() in mysql_create_table(), this is done by the caller. Fix a bug in DROP TABLE under LOCK TABLES when, upon error in wait_while_table_is_used() we would mistakenly release the metadata lock on a non-dropped table. Explicitly release metadata locks when doing an implicit commit. sql/sql_trigger.cc: Now that we delete lex->sphead in lex_end(), zero the trigger's sphead in lex after loading the trigger, to avoid double deletion. sql/sql_udf.cc: Use close_mysql_tables() instead of close_thread_tables(). sql/sys_vars.cc: Remove code added in scope of WL#4284 which would break when we perform set @@session.autocommit along with setting other variables and using tables or functions. A test case added to variables.test. sql/transaction.cc: Add asserts. sql/tztime.cc: Use close_mysql_tables() rather than close_thread_tables(). |
||
|
9a714c935f |
BUG#55387: binlog.binlog_tmp_table crashes the server
sporadically There are two problems: 1. When closing temporary tables, during the THD clean up - and after the session connection was already closed, there is a chance we can push an error into the THD diagnostics area, if the writing of the implicit DROP event to the binary log fails for some reason. As a consequence an assertion can be triggered, because at that point the diagnostics area is already set. 2. Using push_warning with MYSQL_ERROR::WARN_LEVEL_ERROR is a bug. Given that close_temporary_tables is mostly called from THD::cleanup - ie, with the session already closed, we fix problem #1 by allowing the diagnostics area to be overwritten. There is one other place in the code that calls close_temporary_tables - while applying Start_log_event_v3. To cover that case, we make close_temporary_tables to return the error, thus, propagating upwards in the stack. To fix problem #2, we replace push_warning with sql_print_error. sql/log_event.cc: Added handling of error returned by close_temporary_tables to Start_log_event_v3::do_apply_event. sql/sql_base.cc: Three changes to close_temporary_tables: 1. it returns a boolean now (instead of void) 2. it uses sql_print_error instead of push_warning when writing to binary log fails 3. we set can_overwrite_status before writing to the binary log, thence not risking triggering an assertion by any other push into diagnostics area happening inside mysql_bin_log.write. sql/sql_base.h: Changed the interface of close_temporary_tables so that it returns bool instead of void. |
||
|
10582e2abe |
A follow-up for 5.5 version of fix for bug#54360 "Deadlock
DROP/ALTER/CREATE DATABASE with open HANDLER". Remove wait_for_condition() which became unused after database locks were replaced with MDL scoped locks. If one needs functionality provided by this call one can always use THD::enter_cond()/exit_cond() methods. Also removed an unused include from sql_db.cc and updated comment describing one of used includes to reflect current situation. |
||
|
9ff272fbbd |
A 5.5 version of the fix for Bug #54360 "Deadlock DROP/ALTER/CREATE
DATABASE with open HANDLER" Remove LOCK_create_db, database name locks, and use metadata locks instead. This exposes CREATE/DROP/ALTER DATABASE statements to the graph-based deadlock detector in MDL, and paves the way for a safe, deadlock-free implementation of RENAME DATABASE. Database DDL statements will now take exclusive metadata locks on the database name, while table/view/routine DDL statements take intention exclusive locks on the database name. This prevents race conditions between database DDL and table/view/routine DDL. (e.g. DROP DATABASE with concurrent CREATE/ALTER/DROP TABLE) By adding database name locks, this patch implements WL#4450 "DDL locking: CREATE/DROP DATABASE must use database locks" and WL#4985 "DDL locking: namespace/hierarchical locks". The patch also changes code to use init_one_table() where appropriate. The new lock_table_names() function requires TABLE_LIST::db_length to be set correctly, and this is taken care of by init_one_table(). This patch also adds a simple template to help work with the mysys HASH data structure. Most of the patch was written by Konstantin Osipov. |
||
|
33c57e2f9e |
WL#5419 "LOCK_open scalability: make tdc_refresh_version
an atomic counter" Split the large LOCK_open section in open_table(). Do not call open_table_from_share() under LOCK_open. Remove thd->version. This fixes Bug#50589 "Server hang on a query evaluated using a temporary table" Bug#51557 "LOCK_open and kernel_mutex are not happy together" Bug#49463 "LOCK_table and innodb are not nice when handler instances are created". This patch has effect on storage engines that rely on ha_open() PSEA method being called under LOCK_open. In particular: 1) NDB is broken and left unfixed. NDB relies on LOCK_open being kept as part of ha_open(), since it uses auto-discovery. While previously the NDB open code was race-prone, now it simply fails on asserts. 2) HEAP engine had a race in ha_heap::open() when a share for the same table could be added twice to the list of shares, or a dangling reference to a share stored in HEAP handler. This patch aims to address this problem by 'pinning' the newly created share in the internal HEAP engine share list until at least one handler instance is created using that share. include/heap.h: Add members to HP_CREATE_INFO. Declare heap_release_share(). sql/lock.cc: Remove thd->version, use thd->open_tables->s->version instead. sql/repl_failsafe.cc: Remove thd->version. sql/sql_base.cc: - close_thread_table(): move handler cleanup code outside the critical section protected by LOCK_open. - remove thd->version - split the large critical section in open_table() that opens a new table from share and is protected by LOCK_open into 2 critical sections, thus reducing the critical path. - make check_if_table_exists() acquire LOCK_open internally. - use thd->open_tables->s->version instead of thd->refresh_version to make sure that all tables in thd->open_tables are in the same refresh series. sql/sql_base.h: Add declaration for check_if_table_exists(). sql/sql_class.cc: Remove init_open_tables_state(), it's now equal to reset_open_tables_state(). sql/sql_class.h: Remove thd->version, THD::init_open_tables_state(). sql/sql_plugin.cc: Use table->m_needs_reopen to mark the table as stale rather than manipulate with thd->version, which is no more. sql/sql_udf.cc: Use table->m_needs_reopen to mark the table as stale rather than manipulate with thd->version, which is no more. sql/table.h: Remove an unused variable. sql/tztime.cc: Use table->m_needs_reopen to mark the table as stale rather than manipulate with thd->version, which is no more. storage/heap/CMakeLists.txt: Add heap tests to cmake build files. storage/heap/ha_heap.cc: Fix a race when ha_heap::ha_open() could insert two HP_SHARE objects into the internal share list or store a dangling reference to a share in ha_heap instance, or wrongly set implicit_emptied. storage/heap/hp_create.c: Optionally pin a newly created share in the list of shares by increasing its open_count. This is necessary to make sure that a newly created share doesn't disappear while a HP_INFO object is being created to reference it. storage/heap/hp_open.c: When adding a new HP_INFO object to the list of objects in the heap share, make sure the open_count is not increased twice. storage/heap/hp_test1.c: Adjust the test to new function signatures. storage/heap/hp_test2.c: Adjust the test to new function signatures. |