It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
So, in this patch we introduce the option
- "--binlog-direct-non-transactional-updates" that can be used to bypass
the current behavior in order to write directly to binary log statements
that change non-transactional tables.
Besides, it is used to enable the WL#2687 which is disabled by default.
mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache. It also fixes comments.
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
The STMT mode is unsafe when mixed-statements are executed thus making slaves to go out of sync. For that reason, it checks consistency if not in STMT mode.
mysql-test/include/default_mysqld.cnf:
Makes binlog-direct-non-transactional-updates "TRUE" by default in the test
cases.
mysql-test/r/mysqld--help-notwin.result:
Updates the result file with the new option.
mysql-test/r/mysqld--help-win.result:
Updates the result file with the new option.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/binlog/r/binlog_switch_inside_trans.result:
Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
within a transaction or a procedure/function/trigger.
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam-master.opt:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing
the test case and its result file.
mysql-test/suite/binlog/t/binlog_switch_inside_trans.test:
Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
within a transaction or a procedure/function/trigger.
mysql-test/suite/ndb/r/ndb_binlog_format.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
Updates the result file because non-trx-changes are written ahead of the
transaction
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_stm_start_stop_slave.result:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_begin_commit_rollback.test:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_start_stop_slave.test:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_stop_middle_group.test:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
sql/log.cc:
Verifies if changes should be written to either the trx-cache or non-trx-cache through the use of the function use_trans_cache(). It also organizes the code.
sql/log.h:
Changes the signature of some functions by adding the modifier "const" to the thd parameter. Specifically, the following functions are changed:
bool trans_has_updated_trans_table(const THD* thd);
bool stmt_has_updated_trans_table(const THD *thd);
bool use_trans_cache(const THD*, bool is_transactional);
sql/share/errmsg-utf8.txt:
Creates error messages to report when an user tries to change the new option
binlog_direct_non_transactional_updates within a transaction or a procedure/
function/trigger.
sql/share/errmsg.txt:
Creates error messages to report when an user tries to change the new option
binlog_direct_non_transactional_updates within a transaction or a procedure/
function/trigger.
sql/sql_class.h:
Adds the new option binlog_direct_non_transactional_updates.
sql/sys_vars.cc:
Adds the new option binlog_direct_non_transactional_updates.
support-files/my-small.cnf.sh:
Adds binlog-direct-non-transactional-updates to the example file. By default
the option is disabled.
On Unix systems TZ can be set to change the time zone for specific processes only.
But on Windows this does not fully work. It changes some aspects of time zones
in the system but not others (notably localtime() vs. file system time stamps).
This causes test failures in Windows in certain time zones (but not all),
where PURGE BEFORE DATE statements get the wrong files when TZ is set to
+03:00 by default.
Fix by only setting TZ in the small number of tests that really need it, and leave
it untouched in the rest.
Thanks to Alex Budovski for helping with this.
mysql-test/include/ps_conv.inc:
Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it.
mysql-test/mysql-test-run.pl:
Do not set TZ by default, as it causes problems on Windows.
mysql-test/suite/binlog/r/binlog_unsafe.result:
Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it.
mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result:
Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it.
mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test:
Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it.
mysql-test/t/mysqlbinlog2-master.opt:
mysql-test-run.pl no longer sets TZ by default, so set it explicitly for this particular test.
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_servers.cc
Text conflict in sql/sql_update.cc
Text conflict in support-files/mysql.spec.sh
Manually deleteing one or more entries from 'master-bin.index', will
cause master infinitely loop to send one binlog file.
When starting a dump session, master opens index file and search the binlog file
which is being requested by the slave. The position of the binlog file in the
index file is recorded. it will be used to find the next binlog file when current
binlog file has dumped completely. As only the position is used, it may
not get the correct file if some entries has been removed manually from the index file.
the master will reopen the current binlog file which has been dump completely
and redump it if it can not get the next binlog file's name from index file.
It obviously is a logical error.
Even though it is allowed to manually change index file,
but it is not recommended. so after this patch, master
sends a fatal error to slave and close the dump session if a new binlog file
has been generated and master can not get it from the index file.
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/extra/rpl_tests/rpl_loaddata.test
Text conflict in mysql-test/r/mysqlbinlog2.result
Text conflict in mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
Text conflict in mysql-test/suite/binlog/r/binlog_unsafe.result
Text conflict in mysql-test/suite/rpl/r/rpl_insert_id.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result
Text conflict in mysql-test/suite/rpl/r/rpl_udf.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Text conflict in sql/field.h
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysql_priv.h
Text conflict in sql/share/errmsg.txt
Text conflict in sql/sp.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.h
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_insert.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_lex.h
Text conflict in sql/sql_load.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_update.cc
Text conflict in sql/sql_view.cc
Conflict adding files to storage/innobase. Created directory.
Conflict because storage/innobase is not versioned, but has versioned children. Versioned directory.
Conflict adding file storage/innobase. Moved existing file to storage/innobase.moved.
Conflict adding files to storage/innobase/handler. Created directory.
Conflict because storage/innobase/handler is not versioned, but has versioned children. Versioned directory.
Contents conflict in storage/innobase/handler/ha_innodb.cc
in mysqld--help test.
When mysql is compiled with different options,columns
might have different size (this depends on parameter
names and some parameters might only be available
when some option is switched on)
This change is supposed to reduce number of ER_LOCK_DEADLOCK
errors which occur when multi-statement transaction encounters
conflicting metadata lock in cases when waiting is possible.
The idea is not to fail ER_LOCK_DEADLOCK error immediately when
we encounter conflicting metadata lock. Instead we release all
metadata locks acquired by current statement and start to wait
until conflicting lock go away. To avoid deadlocks we use simple
empiric which aborts waiting with ER_LOCK_DEADLOCK error if it
turns out that somebody is waiting for metadata locks owned by
this transaction.
This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989
is not fully fixed in case of ALTER".
The bug was that concurrent execution of UPDATE or MULTI-UPDATE
statement as a part of multi-statement transaction that already
has used table being updated and ALTER TABLE statement might have
resulted of loss of isolation between this transaction and ALTER
TABLE statement, which manifested itself as changes performed by
ALTER TABLE becoming visible in transaction and wrong binary log
order as a consequence.
This problem occurred when UPDATE or MULTI-UPDATE's wait in
mysql_lock_tables() call was aborted due to metadata lock
upgrade performed by concurrent ALTER TABLE. After such abort all
metadata locks held by transaction were released but transaction
silently continued to be executed as if nothing has happened.
We solve this problem by changing our code not to release all
locks in such case. Instead we release only locks which were
acquired by current statement and then try to reacquire them
by restarting open/lock tables process. We piggyback on simple
deadlock detector implementation since this change has to be
done anyway for it.
mysql-test/include/handler.inc:
After introduction of basic deadlock detector for metadata locks
it became necessary to change parts of test for HANDLER statements
which covered some of scenarios in which ER_LOCK_DEADLOCK error
was detected in absence of real deadlock (with new deadlock detector
this no longer happens).
Also adjusted test to the fact that HANDLER READ for the table no
longer will be blocked by ALTER TABLE for the same table which awaits
for metadata lock upgrade (this is due to removal of mysql_lock_abort()
from wait_while_table_is_used()).
mysql-test/r/handler_innodb.result:
After introduction of basic deadlock detector for metadata locks
it became necessary to change parts of test for HANDLER statements
which covered some of scenarios in which ER_LOCK_DEADLOCK error
was detected in absence of real deadlock (with new deadlock detector
this no longer happens).
Also adjusted test to the fact that HANDLER READ for the table no
longer will be blocked by ALTER TABLE for the same table which awaits
for metadata lock upgrade (this is due to removal of mysql_lock_abort()
from wait_while_table_is_used()).
mysql-test/r/handler_myisam.result:
After introduction of basic deadlock detector for metadata locks
it became necessary to change parts of test for HANDLER statements
which covered some of scenarios in which ER_LOCK_DEADLOCK error
was detected in absence of real deadlock (with new deadlock detector
this no longer happens).
Also adjusted test to the fact that HANDLER READ for the table no
longer will be blocked by ALTER TABLE for the same table which awaits
for metadata lock upgrade (this is due to removal of mysql_lock_abort()
from wait_while_table_is_used()).
mysql-test/r/mdl_sync.result:
Added test coverage for basic deadlock detection in metadata
locking subsystem and for bug #46273 "MySQL 5.4.4 new MDL:
Bug#989 is not fully fixed in case of ALTER".
mysql-test/r/sp-lock.result:
Adjusted test coverage for metadata locking for stored routines
since after introduction of basic deadlock detector for metadata
locks number of scenarios in which ER_LOCK_DEADLOCK error in
absence of deadlock has decreased.
mysql-test/t/mdl_sync.test:
Added test coverage for basic deadlock detection in metadata
locking subsystem and for bug #46273 "MySQL 5.4.4 new MDL:
Bug#989 is not fully fixed in case of ALTER".
mysql-test/t/sp-lock.test:
Adjusted test coverage for metadata locking for stored routines
since after introduction of basic deadlock detector for metadata
locks number of scenarios in which ER_LOCK_DEADLOCK error in
absence of deadlock has decreased.
sql/log_event_old.cc:
close_tables_for_reopen() now takes one more argument which
specifies at which point it should stop releasing metadata
locks acquired by this connection.
sql/mdl.cc:
Changed metadata locking subsystem to support basic deadlock detection
with a help of the following simple empiric -- we assume that there is
a deadlock if there is a connection which has to wait for a metadata
lock which is currently acquired by some connection which is itself
waiting to be able to acquire some shared metadata lock.
To implement this change:
- Added MDL_context::can_wait_lead_to_deadlock()/_impl() methods
which allow to find out if there is someone waiting for metadata
lock which is held by the connection and therefore deadlocks are
possible if this connection is going to wait for some metadata lock.
To do this added version of MDL_ticket::has_pending_conflicting_lock()
method which assumes that its caller already owns LOCK_mdl mutex.
- Changed MDL_context::wait_for_locks() to use one of the above methods
to check if somebody is waiting for metadata lock owned by this
context (and therefore deadlock is possible) and emit ER_LOCK_DEADLOCK
error in this case. Also now we mark context of connections waiting
inside of this method by setting MDL_context::m_is_waiting_in_mdl
member. Thanks to this such connection could be waken up if some
other connection starts waiting for one of its metadata locks and
so a deadlock can occur.
- Adjusted notify_shared_lock() to wake up connections which wait inside
MDL_context::wait_for_locks() while holding shared metadata lock.
- Changed MDL_ticket::upgrade_shared_lock_to_exclusive() to add
temporary ticket for exclusive lock to MDL_lock::waiting queue, so
request for metadata lock upgrade can be properly detected by our
empiric.
Also now this method invokes a callback which forces transactions
holding shared metadata lock on the table to call MDL_context::
can_wait_lead_to_deadlock() method even if they don't need any new
metadata locks. Thanks to this such transactions can detect deadlocks/
livelocks between MDL and table-level locks.
Also reduced timeouts between calls to notify_shared_lock()
in MDL_ticket::upgrade_shared_lock_to_exclusive() and
MDL_context::acquire_exclusive_locks(). This was necessary
to get rid of call to mysql_lock_abort() in wait_while_table_is_used().
(Now we instead rely on notify_shared_lock() timely calling
mysql_lock_abort_for_thread() for the table on which lock
is being upgraded/acquired).
sql/mdl.h:
- Added a version of MDL_ticket::has_pending_conflicting_lock() method
to be used in situations when caller already has acquired LOCK_mdl
mutex.
- Added MDL_context::can_wait_lead_to_deadlock()/_impl() methods
which allow to find out if there is someone waiting for metadata lock
which is held by this connection and thus deadlocks are possible if
this connections will start waiting for some metadata lock.
- Added MDL_context::m_is_waiting_in_mdl member to mark connections
waiting in MDL_context::wait_for_locks() method of metadata locking
subsystem. Added getter method for this private member to make it
accessible in notify_shared_lock() auxiliary so we can wake-up such
connections if they hold shared metadata locks.
- Finally, added mysql_abort_transactions_with_shared_lock() callback
to be able force transactions which don't need any new metadata
locks still call MDL_context::can_wait_lead_to_deadlock() and detect
some of deadlocks between metadata locks and table-level locks.
sql/mysql_priv.h:
close_tables_for_reopen() now takes one more argument which
specifies at which point it should stop releasing metadata
locks acquired by this connection.
sql/sql_base.cc:
Changed approach to metadata locking for multi-statement transactions.
We no longer fail ER_LOCK_DEADLOCK error immediately when we encounter
conflicting metadata lock. Instead we release all metadata locks
acquired by current statement and start to wait until conflicting
locks to go away by calling MDL_context::wait_for_locks() method.
To avoid deadlocks the latter implements simple empiric which aborts
waiting with ER_LOCK_DEADLOCK error if it turns out that somebody
is waiting for metadata locks owned by this transaction.
To implement the change described above:
- Introduced Open_table_context::m_start_of_statement_svp member to
store state of metadata locks at the start of the statement.
- Changed Open_table_context::request_backoff_action() not to
fail with ER_LOCK_DEADLOCK immediately if back-off is requested
due to conflicting metadata lock.
- Added new argument for close_tables_for_reopen() procedure which
allows to specify subset of metadata locks to be released.
- Changed open_tables() not to release all metadata locks acquired
by current transaction when metadata lock conflict is discovered.
Instead we release only locks acquired by current statement.
- Changed open_ltable() and open_and_lock_tables_derived() not to emit
ER_LOCK_DEADLOCK error when mysql_lock_tables() is aborted in
multi-statement transaction when somebody tries to acquire exclusive
metadata lock on the table. Instead we release metadata locks acquired
by current statement and try to wait until they can be re-acquired.
- Adjusted tdc_wait_for_old_versions() to check if there is someone
waiting for one of metadata locks held by this connection and run
deadlock detection in order to avoid deadlocks in some
situations.
- Added mysql_abort_transactions_with_shared_lock() callback which
allows to force transactions holding shared metadata lock on the
table to call MDL_context::can_wait_lead_to_deadlock() even if they
don't need any new metadata locks so they can detect potential
deadlocks between metadata locking subsystem and table-level locks.
- Adjusted wait_while_table_is_used() not to set TABLE::version to
0 as it is now done only when necessary by the above-mentioned
callback. Also removed unnecessary call to mysql_lock_abort().
Instead we rely on code performing metadata lock upgrade aborting
waits on the table-level lock for this table by calling
mysql_lock_abort_for_thread() (invoked by
mysql_notify_thread_having_shared_lock()). In future this should
allow to reduce number of scenarios in which we produce
ER_LOCK_DEADLOCK error even though no real deadlock exists.
sql/sql_class.h:
Introduced Open_table_context::m_start_of_statement_svp member to
store state of metadata locks at the start of the statement.
Replaced Open_table_context::m_can_deadlock member with m_has_locks
member to reflect the fact that we no longer unconditionally emit
ER_LOCK_DEADLOCK error for transaction having some metadata locks
when conflicting metadata lock is discovered.
sql/sql_insert.cc:
close_tables_for_reopen() now takes one more argument which
specifies at which point it should stop releasing metadata
locks acquired by this connection.
sql/sql_plist.h:
Made I_P_List_iterator<T, B> usable with const lists.
sql/sql_show.cc:
close_tables_for_reopen() now takes one more argument which
specifies at which point it should stop releasing metadata
locks acquired by this connection.
sql/sql_update.cc:
Changed UPDATE and MULTI-UPDATE code not to release all metadata
locks when calls to mysql_lock_tables() are aborted. Instead we
release only locks which are acquired by this statement and then
try to reacquire them by calling open_tables(). This solves
bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in
case of ALTER".
Simplify testing of needed characterset
Remove ndb from --with-plugins=max build
mysqlbug now sends email to maria-developers@lists.launchpad.net
client/mysqltest.cc:
SKIP now expands variables (for better error messages)
mysql-test/include/have_big5.inc:
Simplify by using have_collation.inc
mysql-test/include/have_collation.inc:
Test if '$collation' is supported
mysql-test/include/have_cp1250_ch.inc:
Simplify by using have_collation.inc
mysql-test/include/have_cp1251.inc:
Simplify by using have_collation.inc
mysql-test/include/have_cp866.inc:
Simplify by using have_collation.inc
mysql-test/include/have_cp932.inc:
Simplify by using have_collation.inc
mysql-test/include/have_eucjpms.inc:
Simplify by using have_collation.inc
mysql-test/include/have_euckr.inc:
Simplify by using have_collation.inc
mysql-test/include/have_gb2312.inc:
Simplify by using have_collation.inc
mysql-test/include/have_gbk.inc:
Simplify by using have_collation.inc
mysql-test/include/have_koi8r.inc:
Simplify by using have_collation.inc
mysql-test/include/have_latin2_ch.inc:
Simplify by using have_collation.inc
mysql-test/include/have_sjis.inc:
Simplify by using have_collation.inc
mysql-test/include/have_tis620.inc:
Simplify by using have_collation.inc
mysql-test/include/have_ucs2.inc:
Simplify by using have_collation.inc
mysql-test/include/have_ujis.inc:
Simplify by using have_collation.inc
mysql-test/include/have_utf8.inc:
Simplify by using have_collation.inc
mysql-test/r/create-uca.result:
Create tests that uses unicode
mysql-test/r/create.result:
Move test with unicode to create-uca.test
mysql-test/r/have_big5.require:
Not needed anymore
mysql-test/r/have_cp1250_ch.require:
Not needed anymore
mysql-test/r/have_cp1251.require:
Not needed anymore
mysql-test/r/have_cp866.require:
Not needed anymore
mysql-test/r/have_cp932.require:
Not needed anymore
mysql-test/r/have_eucjpms.require:
Not needed anymore
mysql-test/r/have_euckr.require:
Not needed anymore
mysql-test/r/have_gb2312.require:
Not needed anymore
mysql-test/r/have_gbk.require:
Not needed anymore
mysql-test/r/have_koi8r.require:
Not needed anymore
mysql-test/r/have_latin2_ch.require:
Not needed anymore
mysql-test/r/have_sjis.require:
Not needed anymore
mysql-test/r/have_tis620.require:
Not needed anymore
mysql-test/r/have_ucs2.require:
Not needed anymore
mysql-test/r/have_ujis.require:
Not needed anymore
mysql-test/r/have_utf8.require:
Not needed anymore
mysql-test/r/innodb.result:
Move tests that depends on unicode to innodb_utf8.test
mysql-test/r/innodb_utf8.result:
Test moved from innodb.test
mysql-test/suite/rpl/t/rpl_ignore_table.test:
Test for required collations
mysql-test/t/create-uca.test:
Create tests that uses unicode
mysql-test/t/create.test:
Move test with unicode to create-uca.test
mysql-test/t/ctype_utf8.test:
Test that require unicode
mysql-test/t/ddl_i18n_koi8r.test:
Test for required collations
mysql-test/t/ddl_i18n_utf8.test:
Test for required collations
mysql-test/t/fulltext.test:
Test for required collations
mysql-test/t/fulltext2.test:
Test for required collations
mysql-test/t/innodb.test:
Move tests that depends on unicode to innodb_utf8.test
mysql-test/t/innodb_utf8.test:
Tests that uses unicode
mysql-test/t/query_cache_ps_no_prot.test:
Test for required collations
mysql-test/t/query_cache_ps_ps_prot.test:
Test for required collations
scripts/mysqlbug.sh:
Send emails to maria-developers@lists.launchpad.net
storage/ndb/plug.in:
Don't include ndb in 'max' builds
"HANDLER statements within a transaction might lead to deadlocks".
Introduce a notion of a sentinel to MDL_context. A sentinel
is a ticket that separates all tickets in the context into two
groups: before and after it. Currently we can have (and need) only
one designated sentinel -- it separates all locks taken by LOCK
TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK
and all other locks, which must be released at COMMIT or ROLLBACK.
The tricky part is maintaining the sentinel up to date when
someone release its corresponding ticket. This can happen, e.g.
if someone issues DROP TABLE under LOCK TABLES (generally,
see all calls to release_all_locks_for_name()).
MDL_context::release_ticket() is modified to take care of it.
******
A fix and a test case for Bug#46224 "HANDLER statements within a
transaction might lead to deadlocks".
An attempt to mix HANDLER SQL statements, which are transaction-
agnostic, an open multi-statement transaction,
and DDL against the involved tables (in a concurrent connection)
could lead to a deadlock. The deadlock would occur when
HANDLER OPEN or HANDLER READ would have to wait on a conflicting
metadata lock. If the connection that issued HANDLER statement
also had other metadata locks (say, acquired in scope of a
transaction), a classical deadlock situation of mutual wait
could occur.
Incompatible change: entering LOCK TABLES mode automatically
closes all open HANDLERs in the current connection.
Incompatible change: previously an attempt to wait on a lock
in a connection that has an open HANDLER statement could wait
indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK
is produced.
The idea of the fix is to merge thd->handler_mdl_context
with the main mdl_context of the connection, used for transactional
locks. This makes deadlock detection possible, since all waits
with locks are "visible" and available to analysis in a single
MDL context of the connection.
Since HANDLER locks and transactional locks have a different life
cycle -- HANDLERs are explicitly open and closed, and so
are HANDLER locks, explicitly acquired and released, whereas
transactional locks "accumulate" till the end of a transaction
and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT,
a concept of "sentinel" was introduced to MDL_context.
All locks, HANDLER and others, reside in the same linked list.
However, a selected element of the list separates locks with
different life cycle. HANDLER locks always reside at the
end of the list, after the sentinel. Transactional locks are
prepended to the beginning of the list, before the sentinel.
Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only
release those locks that reside before the sentinel. HANDLER locks
must be released explicitly as part of HANDLER CLOSE statement,
or an implicit close.
The same approach with sentinel
is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES
statement has never worked together, the implementation is
made simple and only maintains one sentinel, which is used either
for HANDLER locks, or for LOCK TABLES locks.
mysql-test/include/handler.inc:
Add test coverage for Bug#46224 "HANDLER statements within a
transaction might lead to deadlocks".
Extended HANDLER coverage to cover a mix of HANDLER, transactions
and DDL statements.
mysql-test/r/handler_innodb.result:
Update results (Bug#46224).
mysql-test/r/handler_myisam.result:
Update results (Bug#46224).
sql/lock.cc:
Remove thd->some_tables_deleted, it's never used.
sql/log_event.cc:
No need to check for thd->locked_tables_mode,
it's done inside release_transactional_locks().
sql/mdl.cc:
Implement the concept of HANDLER and LOCK TABLES "sentinel".
Implement a method to clone an acquired ticket.
Do not return tickets beyond the sentinel when acquiring
locks, create a copy.
Remove methods to merge and backup MDL_context, they are now
not used (Hurra!). This opens a path to a proper constructor
and destructor of class MDL_context (to be done in a separate
patch).
Modify find_ticket() to provide information about where
the ticket position is with regard to the sentinel.
sql/mdl.h:
Add declarations necessary for the implementation of the concept
of "sentinel", a dedicated ticket separating transactional and
non-transactional locks.
sql/mysql_priv.h:
Add mark_tmp_table_for_reuse() declaration,
a function to "close" a single session (temporary) table.
sql/sql_base.cc:
Remove thd->some_tables_deleted.
Modify deadlock-prevention asserts and deadlock detection
heuristics to take into account that from now on HANDLER locks
reside in the same locking context.
Add broadcast_refresh() to mysql_notify_thread_having_shared_lock():
this is necessary for the case when a thread having a shared lock
is asleep in tdc_wait_for_old_versions(). This situation is only
possible with HANDLER t1 OPEN; FLUSH TABLE (since all over code paths
that lead to tdc_wait_for_old_versions() always have an
empty MDL_context). Previously the server would simply deadlock
in this situation.
sql/sql_class.cc:
Remove now unused member "THD::some_tables_deleted".
Move mysql_ha_cleanup() a few lines above in THD::cleanup()
to make sure that all handlers are closed when it's time to
destroy the MDL_context of this connection.
Remove handler_mdl_context and handler_tables.
sql/sql_class.h:
Remove THD::handler_tables, THD::handler_mdl_context,
THD::some_tables_deleted.
sql/sql_handler.cc:
Remove thd->handler_tables.
Remove thd->handler_mdl_context.
Rewrite mysql_ha_open() to have no special provision for MERGE
tables, now that we don't have to manipulate with thd->handler_tables
it's easy to do.
Remove dead code.
Fix a bug in mysql_ha_flush() when we would always flush
a temporary HANDLER when mysql_ha_flush() is called (actually
mysql_ha_flush() never needs to flush temporary tables).
sql/sql_insert.cc:
Update a comment, no more thd->some_tables_deleted.
sql/sql_parse.cc:
Implement an incompatible change: entering LOCK TABLES closes
active HANDLERs, if any.
Now that we have a sentinel, we don't need to check
for thd->locked_tables_mode when releasing metadata locks in
COMMIT/ROLLBACK.
sql/sql_plist.h:
Add new (now necessary) methods to the list class.
sql/sql_prepare.cc:
Make sure we don't release HANDLER locks when rollback to a
savepoint, set to not keep locks taken at PREPARE.
sql/sql_servers.cc:
Update to a new signature of MDL_context::release_all_locks().
sql/sql_table.cc:
Remove thd->some_tables_deleted.
sql/transaction.cc:
Add comments.
Make sure rollback to (MDL) savepoint works under LOCK TABLES and
with HANDLER tables.
- Make index condition pushdown be controlled by an @@optimizer_switch flag,
not by @@engine_condition_pushdown
- Make MRR buffer size be controlled by @@mrr_buffer_size, not
by @@read_rnd_buffer_size
- Move parts of code to separate files
- Code cleanup
- Add --sorted_result to some SELECTs in tests.
Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#44797 plugins w/o command-line options have no disabling option in --help
Bug#46314 string system variables don't support expressions
Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number
Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds
Bug#49417 some complaints about mysqld --help --verbose output
Bug#49540 DEFAULT value of binlog_format isn't the default value
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
Bug#49646 mysql --show-warnings crashes when server dies
CMakeLists.txt:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
client/mysql.cc:
don't crash with --show-warnings when mysqld dies
config/ac-macros/plugins.m4:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
include/my_getopt.h:
comments
include/my_pthread.h:
fix double #define
mysql-test/mysql-test-run.pl:
run sys_vars suite by default
properly recognize envirinment variables (e.g. MTR_MAX_SAVE_CORE) set to 0
escape gdb command line arguments
mysql-test/suite/sys_vars/r/rpl_init_slave_func.result:
init_slave+utf8 bug
mysql-test/suite/sys_vars/t/rpl_init_slave_func.test:
init_slave+utf8 bug
mysys/my_getopt.c:
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
mysys/typelib.c:
support for flagset
sql/ha_ndbcluster.cc:
backport from telco tree
sql/item_func.cc:
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
sql/sql_builtin.cc.in:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
sql/sql_plugin.cc:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#32902 plugin variables don't know their names
Bug#44797 plugins w/o command-line options have no disabling option in --help
sql/sys_vars.cc:
all server variables are defined here
storage/myisam/ft_parser.c:
remove unnecessary updates of param->quot
storage/myisam/ha_myisam.cc:
myisam_* variables belong here
strings/my_vsnprintf.c:
%o and %llx
unittest/mysys/my_vsnprintf-t.c:
%o and %llx tests
vio/viosocket.c:
bugfix: fix @@wait_timeout to work with socket timeouts (vs. alarm thread)
