mirror of
https://github.com/MariaDB/server.git
synced 2025-07-29 05:21:33 +03:00
BUG#46364 MyISAM transbuffer problems (NTM problem)
It is well-known that due to concurrency issues, a slave can become inconsistent when a transaction contains updates to both transaction and non-transactional tables. In a nutshell, the current code-base tries to preserve causality among the statements by writing non-transactional statements to the txn-cache which is flushed upon commit. However, modifications done to non-transactional tables on behalf of a transaction become immediately visible to other connections but may not immediately get into the binary log and therefore consistency may be broken. In general, it is impossible to automatically detect causality/dependency among statements by just analyzing the statements sent to the server. This happen because dependency may be hidden in the application code and it is necessary to know a priori all the statements processed in the context of a transaction such as in a procedure. Moreover, even for the few cases that we could automatically address in the server, the computation effort required could make the approach infeasible. So, in this patch we introduce the option - "--binlog-direct-non-transactional-updates" that can be used to bypass the current behavior in order to write directly to binary log statements that change non-transactional tables. Besides, it is used to enable the WL#2687 which is disabled by default. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error. mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test: Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache. It also fixes comments. mysql-test/extra/rpl_tests/rpl_mixing_engines.test: The STMT mode is unsafe when mixed-statements are executed thus making slaves to go out of sync. For that reason, it checks consistency if not in STMT mode. mysql-test/include/default_mysqld.cnf: Makes binlog-direct-non-transactional-updates "TRUE" by default in the test cases. mysql-test/r/mysqld--help-notwin.result: Updates the result file with the new option. mysql-test/r/mysqld--help-win.result: Updates the result file with the new option. mysql-test/suite/binlog/r/binlog_multi_engine.result: Updates the result file because non-trx-changes are written ahead of the transaction. mysql-test/suite/binlog/r/binlog_switch_inside_trans.result: Verifies if the user cannot change the opion binlog_direct_non_transactional_updates within a transaction or a procedure/function/trigger. mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam-master.opt: Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file. mysql-test/suite/binlog/t/binlog_switch_inside_trans.test: Verifies if the user cannot change the opion binlog_direct_non_transactional_updates within a transaction or a procedure/function/trigger. mysql-test/suite/ndb/r/ndb_binlog_format.result: Updates the result file because non-trx-changes are written ahead of the transaction. mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result: Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file. mysql-test/suite/rpl/r/rpl_concurrency_error.result: Updates the result file because non-trx-changes are written ahead of the transaction mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result: Updates the result file because non-trx-changes are written ahead of the transaction. mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result: Updates the result file because non-trx-changes are written ahead of the transaction. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error. mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result: Updates the result file because non-trx-changes are written ahead of the transaction. mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result: Updates the result file because non-trx-changes are written ahead of the transaction. mysql-test/suite/rpl/r/rpl_stm_start_stop_slave.result: Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file. mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result: Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file. mysql-test/suite/rpl/t/rpl_begin_commit_rollback.test: Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file. mysql-test/suite/rpl/t/rpl_stm_start_stop_slave.test: Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file. mysql-test/suite/rpl/t/rpl_stm_stop_middle_group.test: Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file. sql/log.cc: Verifies if changes should be written to either the trx-cache or non-trx-cache through the use of the function use_trans_cache(). It also organizes the code. sql/log.h: Changes the signature of some functions by adding the modifier "const" to the thd parameter. Specifically, the following functions are changed: bool trans_has_updated_trans_table(const THD* thd); bool stmt_has_updated_trans_table(const THD *thd); bool use_trans_cache(const THD*, bool is_transactional); sql/share/errmsg-utf8.txt: Creates error messages to report when an user tries to change the new option binlog_direct_non_transactional_updates within a transaction or a procedure/ function/trigger. sql/share/errmsg.txt: Creates error messages to report when an user tries to change the new option binlog_direct_non_transactional_updates within a transaction or a procedure/ function/trigger. sql/sql_class.h: Adds the new option binlog_direct_non_transactional_updates. sql/sys_vars.cc: Adds the new option binlog_direct_non_transactional_updates. support-files/my-small.cnf.sh: Adds binlog-direct-non-transactional-updates to the example file. By default the option is disabled.
This commit is contained in:
@ -6,6 +6,7 @@ drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9;
|
||||
start slave;
|
||||
call mtr.add_suppression("Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT");
|
||||
call mtr.add_suppression("Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT");
|
||||
SET @@session.binlog_direct_non_transactional_updates= FALSE;
|
||||
DROP DATABASE IF EXISTS db1;
|
||||
CREATE DATABASE db1;
|
||||
use db1;
|
||||
|
@ -38,12 +38,14 @@ Warning 1196 Some non-transactional changed tables couldn't be rolled back
|
||||
show binlog events from <binlog_start>;
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO n VALUES (now(),"brown")
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; UPDATE t SET f = 'yellow 2' WHERE i = 3
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; UPDATE t SET f = 'magenta 2' WHERE f = 'red'
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO t VALUES (5 + (2 * 10),"brown")
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO n VALUES (now(),"brown")
|
||||
master-bin.000001 # Query # # ROLLBACK
|
||||
SET AUTOCOMMIT = 1;
|
||||
BEGIN;
|
||||
@ -63,12 +65,14 @@ COMMIT;
|
||||
show binlog events from <binlog_start>;
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO n VALUES (now(),"brown")
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; UPDATE t SET f = 'gray 2' WHERE i = 3
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; UPDATE t SET f = 'dark blue 2' WHERE f = 'red'
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO t VALUES (6 + (2 * 10),"brown")
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO n VALUES (now(),"brown")
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
SET AUTOCOMMIT = 0;
|
||||
UPDATE t SET f = 'yellow 1' WHERE i = 3;
|
||||
@ -88,12 +92,14 @@ Warning 1196 Some non-transactional changed tables couldn't be rolled back
|
||||
show binlog events from <binlog_start>;
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO n VALUES (now(),"brown")
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; UPDATE t SET f = 'yellow 1' WHERE i = 3
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; UPDATE t SET f = 'magenta 1' WHERE f = 'red'
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO t VALUES (5 + (1 * 10),"brown")
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO n VALUES (now(),"brown")
|
||||
master-bin.000001 # Query # # ROLLBACK
|
||||
SET AUTOCOMMIT = 0;
|
||||
UPDATE t SET f = 'gray 1' WHERE i = 3;
|
||||
@ -111,12 +117,14 @@ COMMIT;
|
||||
show binlog events from <binlog_start>;
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO n VALUES (now(),"brown")
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; UPDATE t SET f = 'gray 1' WHERE i = 3
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; UPDATE t SET f = 'dark blue 1' WHERE f = 'red'
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO t VALUES (6 + (1 * 10),"brown")
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO n VALUES (now(),"brown")
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
source include/diff_master_slave.inc;
|
||||
source include/diff_master_slave.inc;
|
||||
|
@ -393,8 +393,10 @@ CREATE TEMPORARY TABLE tt_xx (a int);
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO tt_1(ddl_case) VALUES (11)
|
||||
master-bin.000001 # Query # # use `test`; CREATE TEMPORARY TABLE tt_xx (a int)
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO tt_1(ddl_case) VALUES (11)
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
|
||||
|
@ -11868,6 +11868,9 @@ Log_name Pos Event_type Server_id End_log_pos Info
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> CT << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
CREATE TEMPORARY TABLE tt_xx_11 (a int) engine=Innodb;;
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; CREATE TEMPORARY TABLE tt_xx_11 (a int) engine=Innodb
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> CT << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> R << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
ROLLBACK;
|
||||
@ -11875,6 +11878,9 @@ Log_name Pos Event_type Server_id End_log_pos Info
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> R << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> B T CT R << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; CREATE TEMPORARY TABLE tt_xx_11 (a int) engine=Innodb
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> B T CT R << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> drop-CT << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
@ -11906,6 +11912,9 @@ Log_name Pos Event_type Server_id End_log_pos Info
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> CT << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
CREATE TEMPORARY TABLE tt_xx_12 (a int) engine=Innodb;;
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; CREATE TEMPORARY TABLE tt_xx_12 (a int) engine=Innodb
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> CT << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> R1 << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
ROLLBACK TO s1;
|
||||
@ -11919,6 +11928,9 @@ Log_name Pos Event_type Server_id End_log_pos Info
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> R << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> B T S1 T CT R1 R << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; CREATE TEMPORARY TABLE tt_xx_12 (a int) engine=Innodb
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> B T S1 T CT R1 R << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> drop-CT << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
@ -11942,6 +11954,9 @@ Log_name Pos Event_type Server_id End_log_pos Info
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> CT << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
CREATE TEMPORARY TABLE tt_xx_13 (a int) engine=Innodb;;
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; CREATE TEMPORARY TABLE tt_xx_13 (a int) engine=Innodb
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> CT << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> T << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
INSERT INTO tt_1(trans_id, stmt_id) VALUES (355, 5);
|
||||
@ -11953,6 +11968,9 @@ Log_name Pos Event_type Server_id End_log_pos Info
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> R << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> B T CT T R << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; CREATE TEMPORARY TABLE tt_xx_13 (a int) engine=Innodb
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> B T CT T R << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> drop-CT << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
|
@ -130,8 +130,8 @@ Note 1592 Unsafe statement binlogged in statement format since BINLOG_FORMAT = S
|
||||
Got one of the listed errors
|
||||
Got one of the listed errors
|
||||
Got one of the listed errors
|
||||
Got one of the listed errors
|
||||
Got one of the listed errors
|
||||
Warnings:
|
||||
Note 1592 Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT. Reason for unsafeness: Non-transactional reads or writes are unsafe if they occur after transactional reads or writes inside a transaction.
|
||||
Warnings:
|
||||
Note 1592 Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT. Reason for unsafeness: Non-transactional reads or writes are unsafe if they occur after transactional reads or writes inside a transaction.
|
||||
COMMIT;
|
||||
@ -139,10 +139,7 @@ BEGIN;
|
||||
Warnings:
|
||||
Note 1592 Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT. Reason for unsafeness: Non-transactional reads or writes are unsafe if they occur after transactional reads or writes inside a transaction.
|
||||
Got one of the listed errors
|
||||
Got one of the listed errors
|
||||
COMMIT;
|
||||
SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
|
||||
START SLAVE SQL_THREAD;
|
||||
source include/diff_master_slave.inc;
|
||||
########################################################################################
|
||||
# CLEAN
|
||||
|
@ -393,8 +393,10 @@ CREATE TEMPORARY TABLE tt_xx (a int);
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO tt_1(ddl_case) VALUES (11)
|
||||
master-bin.000001 # Query # # use `test`; CREATE TEMPORARY TABLE tt_xx (a int)
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO tt_1(ddl_case) VALUES (11)
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
|
||||
@ -429,8 +431,10 @@ DROP TEMPORARY TABLE IF EXISTS new_tt_xx;
|
||||
-b-b-b-b-b-b-b-b-b-b-b- >> << -b-b-b-b-b-b-b-b-b-b-b-
|
||||
Log_name Pos Event_type Server_id End_log_pos Info
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO tt_1(ddl_case) VALUES (8)
|
||||
master-bin.000001 # Query # # use `test`; DROP TEMPORARY TABLE IF EXISTS new_tt_xx
|
||||
master-bin.000001 # Query # # COMMIT
|
||||
master-bin.000001 # Query # # BEGIN
|
||||
master-bin.000001 # Query # # use `test`; INSERT INTO tt_1(ddl_case) VALUES (8)
|
||||
master-bin.000001 # Xid # # COMMIT /* XID */
|
||||
-e-e-e-e-e-e-e-e-e-e-e- >> << -e-e-e-e-e-e-e-e-e-e-e-
|
||||
|
||||
|
File diff suppressed because it is too large
Load Diff
@ -4,6 +4,7 @@ reset master;
|
||||
reset slave;
|
||||
drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9;
|
||||
start slave;
|
||||
SET @@session.binlog_direct_non_transactional_updates= FALSE;
|
||||
call mtr.add_suppression("Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT");
|
||||
call mtr.add_suppression("Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT");
|
||||
create table t1(n int);
|
||||
|
@ -4,6 +4,7 @@ reset master;
|
||||
reset slave;
|
||||
drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9;
|
||||
start slave;
|
||||
SET @@session.binlog_direct_non_transactional_updates= FALSE;
|
||||
call mtr.add_suppression("Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT");
|
||||
create table tm (a int auto_increment primary key) engine=myisam;
|
||||
create table ti (a int auto_increment primary key) engine=innodb;
|
||||
|
@ -8,6 +8,8 @@ call mtr.add_suppression("Unsafe statement binlogged in statement format since B
|
||||
connection master;
|
||||
call mtr.add_suppression("Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT");
|
||||
|
||||
SET @@session.binlog_direct_non_transactional_updates= FALSE;
|
||||
|
||||
disable_warnings;
|
||||
DROP DATABASE IF EXISTS db1;
|
||||
enable_warnings;
|
||||
|
@ -5,4 +5,5 @@
|
||||
--source include/have_binlog_format_statement.inc
|
||||
--source include/have_innodb.inc
|
||||
|
||||
SET @@session.binlog_direct_non_transactional_updates= FALSE;
|
||||
--source ./extra/rpl_tests/rpl_start_stop_slave.test
|
||||
|
@ -6,4 +6,5 @@
|
||||
-- source include/have_innodb.inc
|
||||
-- source include/have_binlog_format_statement.inc
|
||||
|
||||
SET @@session.binlog_direct_non_transactional_updates= FALSE;
|
||||
-- source extra/rpl_tests/rpl_stop_middle_group.test
|
||||
|
Reference in New Issue
Block a user