This writes a warning on stderr if one uses mutex in different order,
like if one in one case would lock mutex in the order A,B and in another case
would lock mutex in the order B,A
This is inspired by and loosely based on the LOCKDEP patch by Jonas
Wrong mutex order is either fixed or mutex are marked with MYF_NO_DEADLOCK_DETECTION
if used inconsistently (need to be fixed by server team)
KNOWN_BUGS.txt:
Added information that one need to dump and restore Maria tables
include/hash.h:
Added prototype function for walking over all elements in a hash
include/my_pthread.h:
Added my_pthread_mutex_init() and my_pthread_mutex_lock(); These should be used if one wants to disable mutex order checking.
Changed names of the nonposix mutex_init functions to not conflict with my_phread_mutex_init()
Added and extended structures for mutex deadlock detection.
New arguments to sage_mutex_init() and safe_mutex_lock() to allow one to disable mutex order checking.
Added variable 'safe_mutex_deadlock_detector' to enable/disable deadlock detection for all pthread_mutex_init()
mysys/Makefile.am:
Added cleaning of test files
Added test_thr_mutex
mysys/hash.c:
Added hash_iterate() to iterate over all elements in a hash
More comments
mysys/my_init.c:
Added calls to destory all mutex uses by mysys()
Added waiting for threads to end before calling TERMINATE() to list not freed memory
mysys/my_pthread.c:
Changed names to free my_pthread_mutex_init() for mutex-lock-order-checking
mysys/my_sleep.c:
Fixed too long wait if using 1000000L as argument
mysys/my_thr_init.c:
Mark THR_LOCK_threads and THR_LOCK_malloc to not have mutex deadlock detection.
(We can't have it enabled for this as these are internal mutex used by the detector
Call my_thread_init() early as we need thread specific variables enabled for the following pthread_mutex_init()
Move code to wait for threads to end to my_wait_for_other_threads_to_die()
Don't destroy mutex and conditions unless all threads have died
Added my_thread_destroy_mutex() to destroy all mutex used by the mysys thread system
Name the thread specific mutex as "mysys_var->mutex"
Added my_thread_var_mutex_in_use() to return pointer to mutex in use or 0 if thread variables are not initialized
mysys/mysys_priv.h:
Added prototypes for functions used internally with mutex-wrong-usage detection
mysys/thr_mutex.c:
Added runtime detection of mutex used in conflicting order
See WL#3262 or test_thr_mutex.c for examples
The base idea is for each mutex have two hashes:
- mutex->locked_mutex points to all mutex used after this one
- mutex->used_mutex points to all mutex which has this mutex in it's mutex->locked_mutex
There is a wrong mutex order if any mutex currently locked before this mutex is in the mutex->locked_mutex hash
sql/event_queue.cc:
Mark mutex used inconsistently (need to be fixed by server team)
sql/event_scheduler.cc:
Declare the right order to take the mutex
sql/events.cc:
Mark mutex used inconsistently (need to be fixed by server team)
sql/ha_ndbcluster_binlog.cc:
Mark mutex used inconsistently (need to be fixed by server team)
sql/log.cc:
Mark mutex used inconsistently (need to be fixed by server team)
sql/mysqld.cc:
Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread
This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex.
Added call to free sp cache.
Add destruction of LOCK_server_started and COND_server_started.
Added register_mutex_order() function to register in which order mutex should be taken
(to initiailize mutex_deadlock_detector).
Added option to turn off safe_mutex_deadlock_detector
sql/protocol.cc:
Fixed wrong argument to DBUG_PRINT (found by valgrind)
sql/rpl_mi.cc:
Mark mutex used inconsistently (need to be fixed by server team)
sql/set_var.cc:
Remove wrong locking of LOCK_global_system_variables when reading and setting log variables
(would cause inconsistent mutex order).
Update global variables outside of logger.unlock() as LOCK_global_system_variables has to be taken before logger locks
Reviewed by gluh
sql/sp_cache.cc:
Added function to destroy mutex used by sp cache
sql/sp_cache.h:
Added function to destroy mutex used by sp cache
sql/sql_class.cc:
Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread
This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex.
Register order in which LOCK_delete and mysys_var->mutex is taken
sql/sql_insert.cc:
Give a name for Delayed_insert::mutex
Mark mutex used inconsistently (need to be fixed by server team)
Move closing of tables outside of di->mutex (to avoid wrong mutex order)
sql/sql_show.cc:
Don't keep LOCK_global_system_variables locked over value->show_type() as this leads to wrong mutex order
storage/innobase/handler/ha_innodb.cc:
Disable safe_muted_deadlock_detector for innobase intern mutex (to speed up page cache initialization)
storage/maria/ha_maria.cc:
Added flag to ha_maria::info() to signal if we need to lock table share or not.
This is needed to avoid locking mutex in wrong order
storage/maria/ha_maria.h:
Added flag to ha_maria::info() to signal if we need to lock table share or not.
storage/maria/ma_close.c:
Destroy key_del_lock
Simplify freeing ftparser_param
storage/maria/ma_key.c:
Better comment
storage/maria/ma_loghandler.c:
Mark mutex used inconsistently (need to be fixed by sanja)
storage/maria/ma_state.c:
More comments
storage/maria/ma_test1.c:
Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety)
storage/maria/ma_test2.c:
Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety)
If delayed insert fails to upgrade the lock it was not
freeing the temporary memory storage used to keep
newly constructed blob values in memory.
Fixed by iterating over the remaining rows in the delayed
insert rowset and freeing the blob storage for each row.
No test suite because it involves concurrent delayed inserts
on a table and cannot easily be made deterministic.
Added a correct valgrind suppression for Fedora 9.
mysql-test/valgrind.supp:
Added a vagrind suppression for Fedora 9
sql/sql_insert.cc:
Bug #38693: free the blobs temp storage on error.
If delayed insert fails to upgrade the lock it was not
freeing the temporary memory storage used to keep
newly constructed blob values in memory.
Fixed by iterating over the remaining rows in the delayed
insert rowset and freeing the blob storage for each row.
No test suite because it involves concurrent delayed inserts
on a table and cannot easily be made deterministic.
Added a correct valgrind suppression for Fedora 9.
The failure was caused by executing a CREATE-SELECT statement that creates a
table in another database than the current one. In row-based logging, the
CREATE statement was written to the binary log without the database, hence
creating the table in the wrong database, causing the following inserts to
fail since the table didn't exist in the given database.
Fixed the bug by adding a parameter to store_create_info() that will make
the function print the database name before the table name and used that
in the calls that write the CREATE statement to the binary log. The database
name is only printed if it is different than the currently selected database.
The output of SHOW CREATE TABLE has not changed and is still printed without
the database name.
mysql-test/suite/rpl/t/rpl_row_create_table.test:
Added test to check that CREATE-SELECT into another database than the
current one replicates.
sql/sql_insert.cc:
Adding parameter to calls to store_create_info().
sql/sql_show.cc:
Adding parameter to calls to store_create_info().
Extending store_create_info() with parameter 'show_database' that will cause
the database to be written before the table name.
sql/sql_show.h:
Adding parameter to call to store_create_info() to tell if the database should be shown or not.
sql/sql_table.cc:
Adding parameter to calls to store_create_info().
The failure was caused by executing a CREATE-SELECT statement that creates a
table in another database than the current one. In row-based logging, the
CREATE statement was written to the binary log without the database, hence
creating the table in the wrong database, causing the following inserts to
fail since the table didn't exist in the given database.
Fixed the bug by adding a parameter to store_create_info() that will make
the function print the database name before the table name and used that
in the calls that write the CREATE statement to the binary log. The database
name is only printed if it is different than the currently selected database.
The output of SHOW CREATE TABLE has not changed and is still printed without
the database name.
Concurrent inserts produce valgrind error messages.
The reason is that the query cache is invalidated after the target table object
is closed.
Since the delayed insert thread already takes care of invalidating the query
cache there is no need to try to synchronize an extra cache invalidation call.
The fix is to remove the query_cache_invalidate3 call altogether.
sql/sql_insert.cc:
When end_delayed_insert is called, the table_list items will be invalidated
by the concurrent insert thread. Further more there is no need to call
query_cache_invalidate here since the delayed insert thread takes care of
this already.
Concurrent inserts produce valgrind error messages.
The reason is that the query cache is invalidated after the target table object
is closed.
Since the delayed insert thread already takes care of invalidating the query
cache there is no need to try to synchronize an extra cache invalidation call.
The fix is to remove the query_cache_invalidate3 call altogether.
The bug was that prepared statements didn't downgrade TL_WRITE_CONCURRENT properly
mysql-test/r/maria.result:
Added test case
mysql-test/t/maria.test:
Added test case
sql/mysql_priv.h:
Make upgrade_lock_type() global
sql/sql_base.cc:
Fixed indentation
sql/sql_insert.cc:
Make upgrade_lock_type() global
sql/sql_prepare.cc:
Call upgrade_lock_type_for_insert() to get right lock to use
sql/sql_view.cc:
Indentation fix
Fix the write_record function to record auto increment
values in a consistent way.
mysql-test/r/auto_increment.result:
Updated the test result file with the output of the
new test case added to verify this bug.
mysql-test/t/auto_increment.test:
Added a new test case to verify this bug.
sql/sql_insert.cc:
The algorithm for the write_record function
in sql_insert.cc is (more emphasis given to
the parts that deal with the autogenerated values)
1) If a write fails
1.1) save the autogenerated value to avoid
thd->insert_id_for_cur_row to become 0.
1.2) <logic to handle INSERT ON DUPLICATE KEY
UPDATE and REPLACE>
2) record the first successful insert id.
explanation of the failure
--------------------------
As long as 1.1) was executed 2) worked fine.
1.1) was always executed when REPLACE worked
with the last row update optimization, but
in cases where 1.1) was not executed 2)
would fail and would result in the autogenerated
value not being saved.
solution
--------
repeat a check for thd->insert_id_for_cur_row
being zero similar to 1.1) before 2) and ensure
that the correct value is saved.
If a delayed insert thread was aborted by a concurrent 'truncate table'
statement, then the diagnostic area would fail with an assert in a debug build
because no actual error message was pushed on the stack despite a thread
being killed.
This patch adds an error message to the stack.
sql/sql_insert.cc:
* Changed sql_print_error() to my_error() to avoid assertion in the DA
* Added assertion in "should never happen" branch.
If a delayed insert thread was aborted by a concurrent 'truncate table'
statement, then the diagnostic area would fail with an assert in a debug build
because no actual error message was pushed on the stack despite a thread
being killed.
This patch adds an error message to the stack.
in open_table()
Problem: repeating "CREATE... ( AUTOINCREMENT) ... SELECT" may lead to
an assertion failure.
Fix: reset table->auto_increment_field_not_null after each record
writing.
mysql-test/r/create.result:
Fix for bug#38821: Assert table->auto_increment_field_not_null failed
in open_table()
- test result.
mysql-test/t/create.test:
Fix for bug#38821: Assert table->auto_increment_field_not_null failed
in open_table()
- test case.
sql/sql_insert.cc:
Fix for bug#38821: Assert table->auto_increment_field_not_null failed
in open_table()
- reset table->auto_increment_field_not_null after writing a record
for "{CREATE, INSERT}..SELECT".
in open_table()
Problem: repeating "CREATE... ( AUTOINCREMENT) ... SELECT" may lead to
an assertion failure.
Fix: reset table->auto_increment_field_not_null after each record
writing.
The assert is about binlogging must have been activated, but it was
not actually according to the reported how-to-repeat instuctions.
Analysis revealed that binlog_start_trans_and_stmt() was called
without prior testing if binlogging is ON.
Fixed with avoing entering binlog_start_trans_and_stmt() if binlog is
not activated.
mysql-test/r/skip_log_bin.result:
new results.
mysql-test/t/skip_log_bin-master.opt:
the option to deactivate binlogging.
mysql-test/t/skip_log_bin.test:
regression test for the bug.
sql/sql_insert.cc:
avoing entering binlog_start_trans_and_stmt() if binlog is not activated.
The assert is about binlogging must have been activated, but it was
not actually according to the reported how-to-repeat instuctions.
Analysis revealed that binlog_start_trans_and_stmt() was called
without prior testing if binlogging is ON.
Fixed with avoing entering binlog_start_trans_and_stmt() if binlog is
not activated.
In order to handle CHAR() fields, 8 bits were reserved for
the size of the CHAR field. However, instead of denoting the
number of characters in the field, field_length was used which
denotes the number of bytes in the field.
Since UTF-8 fields can have three bytes per character (and
has been extended to have four bytes per character in 6.0),
an extra two bits have been encoded in the field metadata
work for fields of type Field_string (i.e., CHAR fields).
Since the metadata word is filled, the extra bits have been
encoded in the upper 4 bits of the real type (the most
significant byte of the metadata word) by computing the
bitwise xor of the extra two bits. Since the upper 4 bits
of the real type always is 1111 for Field_string, this
means that for fields of length <256, the encoding is
identical to the encoding used in pre-5.1.26 servers, but
for lengths of 256 or more, an unrecognized type is formed,
causing an old slave (that does not handle lengths of 256
or more) to stop.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Adding test cases for replicating UTF-8 fields of lengths
of 256 or more (bytes).
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Result file change.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Adding tests to trigger check that an error is generated when replicating from a
5.1.25 server for tables with a CHAR(128) but not when replicating a table with a
CHAR(63). Although the bug indicates that the limit is 83, we elected to use CHAR(63)
since 6.0 uses 4-byte UTF-8, and anything exceeding 63 would then cause the test to fail
when the patch is merged to 6.0.
mysql-test/suite/bugs/combinations:
Adding combinations file to run all bug reports in all binlog modes (where
applicable).
mysql-test/suite/bugs/r/rpl_bug37426.result:
Result file change.
mysql-test/suite/bugs/t/rpl_bug37426.test:
Added test for reported bug.
mysql-test/suite/rpl/r/rpl_row_basic_2myisam.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
Result file change.
sql/field.cc:
Encoding an extra two bits in the most significant nibble (4 bits)
of the metadata word. Adding assertions to ensure that no attempt
is made to use lengths longer than supported.
Extending compatible_field_size() function with an extra parameter
holding a Relay_log_instace for error reporting.
Field_string::compatible_field_size() now reports an error if field
size for a CHAR is >255.
sql/field.h:
Field length is now computed from most significant 4 bits
of metadata word, or is equal to the row pack length if
there is no metadata.
Extending compatible_field_size() function with an extra parameter
holding a Relay_log_instace for error reporting.
sql/rpl_utility.cc:
Adding relay log parameter to compatible_field_size().
Minor refactoring to eliminate duplicate code.
sql/slave.cc:
Extending rpl_master_has_bug() with a single-argument predicate function and
a parameter to the predicate function. The predicate function can be used to
test for extra conditions for the bug before writing an error message.
sql/slave.h:
Extending rpl_master_has_bug() with a single-argument predicate function and
a parameter to the predicate function. The predicate function can be used to
test for extra conditions for the bug before writing an error message.
Also removing gratuitous default argument.
sql/sql_insert.cc:
Changing calls to rpl_master_has_bug() to adapt to changed signature.
In order to handle CHAR() fields, 8 bits were reserved for
the size of the CHAR field. However, instead of denoting the
number of characters in the field, field_length was used which
denotes the number of bytes in the field.
Since UTF-8 fields can have three bytes per character (and
has been extended to have four bytes per character in 6.0),
an extra two bits have been encoded in the field metadata
work for fields of type Field_string (i.e., CHAR fields).
Since the metadata word is filled, the extra bits have been
encoded in the upper 4 bits of the real type (the most
significant byte of the metadata word) by computing the
bitwise xor of the extra two bits. Since the upper 4 bits
of the real type always is 1111 for Field_string, this
means that for fields of length <256, the encoding is
identical to the encoding used in pre-5.1.26 servers, but
for lengths of 256 or more, an unrecognized type is formed,
causing an old slave (that does not handle lengths of 256
or more) to stop.
mysql-test/r/maria.result:
result; before the bugfix it would be "TRANSACTIONAL=1 transactional=1"
mysql-test/t/maria.test:
test for BUG#36104 "INFORMATION_SCHEMA.TABLES shows TRANSACTIONAL=1 twice in CREATE_OPTIONS"
sql-bench/example:
doblewrite->doublewrite
sql/mysqld.cc:
fix of a wrong 5.1->maria merge of the past
sql/sql_insert.cc:
removing my old idea of disabling transactionality in CREATE SELECT:
1) it caused bugs because re-enabling (ha_enable_transaction()) causes implicit commit, so in complex cases
like "CREATE SELECT some_func())", where some_func() would want to insert two rows in another table, and fail on the second row, the implicit commit would commit the inserted row, while it should roll back.
2) it's not needed anymore, because CREATE SELECT uses bulk insert, and Maria has transactionality disabled by
bulk insert.
sql/sql_show.cc:
This was duplicate code, causing BUG#36104 "INFORMATION_SCHEMA.TABLES shows TRANSACTIONAL=1 twice in CREATE_OPTIONS"
