BUG#39934: Slave stops for engine that only support row-based logging

General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.

Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
 - binlog_format=STATEMENT for a row injection
 - binlog_format=STATEMENT for an unsafe statement
 - binlog_format=STATEMENT for an engine only supporting row logging
 - binlog_format=ROW for an engine only supporting statement logging
 - statement is unsafe and engine does not support row logging
 - row injection in a table that does not support statement logging
 - statement modifies one table that does not support row logging and
   one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()

Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().

Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.

Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.

mysql-test/suite/rpl_ndb/t/rpl_ndb_2innodb.test

@@ -8,6 +8,9 @@
 # test and to have control over the tests.
 ##############################################################
 -- source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/master-slave.inc
 
 -- connection slave

mysql-test/suite/rpl_ndb/t/rpl_ndb_2myisam.test

@@ -8,6 +8,9 @@
 # test and to have control over the tests.
 ##############################################################
 -- source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/master-slave.inc
 
 -- connection slave

mysql-test/suite/rpl_ndb/t/rpl_ndb_basic.test

@@ -1,4 +1,7 @@
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 --source include/ndb_master-slave.inc
 
 

mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors-master.opt

@@ -0,0 +1 @@
+--innodb $EXAMPLE_PLUGIN_OPT

mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors-slave.opt

@@ -0,0 +1 @@
+--innodb $EXAMPLE_PLUGIN_OPT

mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test

@@ -0,0 +1,193 @@
+# ==== Purpose ====
+#
+# Verify that errors or warnings are issued for all error conditions
+# related to deciding the binlog format of a statement.  The possible
+# errors are listed in a comment above decide_logging_format() in
+# sql_base.cc.
+#
+# ==== Method ====
+#
+# Each error condition is executed; we verify that there is an error.
+#
+# ==== Related bugs ====
+#
+# BUG#39934: Slave stops for engine that only support row-based logging
+# BUG#42829: binlogging enabled for all schemas regardless of binlog-db-db / binlog-ignore-db
+#
+# ==== Related test cases ====
+#
+# binlog.binlog_unsafe verifies more thoroughly that a warning is
+# given for the case when an unsafe statement is executed and
+# binlog_format = STATEMENT.
+
+--source include/have_debug.inc
+# The test changes binlog_format, so there is no reason to run it
+# under more than one binlog format.
+--source include/have_binlog_format_row.inc
+--source include/have_ndb.inc
+--source include/have_innodb.inc
+--source include/ndb_master-slave.inc
+
+
+--echo ==== Initialize ====
+
+--echo [on slave]
+--connection slave
+
+SET @old_binlog_format= @@global.binlog_format;
+INSTALL PLUGIN example SONAME 'ha_example.so';
+
+--echo [on master]
+--connection master
+
+SET @old_binlog_format= @@global.binlog_format;
+INSTALL PLUGIN example SONAME 'ha_example.so';
+
+CREATE TABLE t (a VARCHAR(100)) ENGINE = MYISAM;
+CREATE TABLE t_self_logging (a VARCHAR(100)) ENGINE = NDB;
+CREATE TABLE t_row (a VARCHAR(100)) ENGINE = INNODB;
+CREATE TABLE t_stmt (a VARCHAR(100)) ENGINE = EXAMPLE;
+CREATE TABLE t_slave_stmt (a VARCHAR(100)) ENGINE = MYISAM;
+# This makes the innodb table row-only
+SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
+
+--echo [on slave]
+--sync_slave_with_master
+
+DROP TABLE t_slave_stmt;
+CREATE TABLE t_slave_stmt (a INT) ENGINE = EXAMPLE;
+
+--echo [on master]
+--connection master
+
+
+--echo ==== Test ====
+
+--echo ---- binlog_format=row ----
+
+--echo * Modify tables of more than one engine, one of which is self-logging
+--eval CREATE TRIGGER trig_1 AFTER INSERT ON t_self_logging FOR EACH ROW BEGIN INSERT INTO t VALUES (1); END
+--error ER_BINLOG_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE
+INSERT INTO t_self_logging VALUES (1);
+DROP trigger trig_1;
+
+--echo * Modify both row-only and stmt-only table
+--eval CREATE TRIGGER trig_2 AFTER INSERT ON t_stmt FOR EACH ROW BEGIN INSERT INTO t_row VALUES(1); END
+--error ER_BINLOG_ROW_ENGINE_AND_STMT_ENGINE
+INSERT INTO t_stmt VALUES (1);
+DROP TRIGGER trig_2;
+
+--echo * Stmt-only table and binlog_format=row
+--error ER_BINLOG_ROW_MODE_AND_STMT_ENGINE
+INSERT INTO t_stmt VALUES (1);
+
+--echo * Row injection and stmt-only table: in slave sql thread
+INSERT INTO t_slave_stmt VALUES (1);
+--echo [on slave]
+--connection slave
+# 1640 = ER_BINLOG_ROW_INJECTION_AND_STMT_ENGINE
+--let $slave_sql_errno= 1640
+--let $show_sql_error= 1
+--source include/wait_for_slave_sql_error_and_skip.inc
+
+--echo * Row injection and stmt-only table: use BINLOG statement
+# This is a format description event
+BINLOG '
+1gRVSg8BAAAAZgAAAGoAAAABAAQANS4xLjM2LWRlYnVnLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAA
+AAAAAAAAAAAAAAAAAADWBFVKEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
+';
+# This is a Table_map_event and a Write_rows_event. Together, they are
+# equivalent to 'INSERT INTO t_stmt VALUES (1)'
+--error ER_BINLOG_ROW_INJECTION_AND_STMT_ENGINE
+BINLOG '
+1gRVShMBAAAALwAAAEABAAAAABcAAAAAAAAABHRlc3QABnRfc3RtdAABDwJkAAE=
+1gRVShcBAAAAIAAAAGABAAAQABcAAAAAAAEAAf/+ATE=
+';
+
+
+--echo ---- binlog_format=mixed ----
+
+--echo [on slave]
+--sync_slave_with_master
+--source include/stop_slave.inc
+SET @@global.binlog_format = MIXED;
+--source include/start_slave.inc
+--echo [on master]
+--connection master
+SET @@global.binlog_format = MIXED;
+SET @@session.binlog_format = MIXED;
+
+--echo * Unsafe statement and stmt-only engine
+# This will give a warning.
+INSERT INTO t_stmt VALUES (UUID());
+
+
+--echo ---- binlog_format=statement ----
+
+--echo [on slave]
+--sync_slave_with_master
+--source include/stop_slave.inc
+SET @@global.binlog_format = STATEMENT;
+--source include/start_slave.inc
+--echo [on master]
+--connection master
+SET @@global.binlog_format = STATEMENT;
+SET @@session.binlog_format = STATEMENT;
+
+--echo * Row-only engine and binlog_format=statement: innodb-specific message
+--error ER_BINLOG_STMT_MODE_AND_ROW_ENGINE
+INSERT INTO t_row VALUES (1);
+
+--echo * Row-only engine and binlog_format=statement: generic message
+SET @@session.debug= "+d,no_innodb_binlog_errors";
+--error ER_BINLOG_STMT_MODE_AND_ROW_ENGINE
+INSERT INTO t_row VALUES (1);
+
+--echo * Row injection and binlog_format=statement: BINLOG statement
+# This is a format description event
+BINLOG '
+b9pVSg8BAAAAZgAAAGoAAAABAAQANS4xLjM2LWRlYnVnLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAA
+AAAAAAAAAAAAAAAAAABv2lVKEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
+';
+# This is a Table_map_event and a Write_rows_event. Together, they are
+# equivalent to 'INSERT INTO t VALUES (1)'.
+--error ER_BINLOG_ROW_INJECTION_AND_STMT_MODE
+BINLOG '
+cNpVShMBAAAAKgAAADYBAAAAABcAAAAAAAAABHRlc3QAAXQAAQ8CZAAB
+cNpVShcBAAAAIAAAAFYBAAAQABcAAAAAAAEAAf/+ATE=
+';
+
+--echo * Unsafe statement and binlog_format=statement
+# This will give a warning.
+INSERT INTO t VALUES (UUID());
+
+
+--echo ---- master: binlog_format=mixed, slave: binlog_format=statement ----
+
+SET @@global.binlog_format = MIXED;
+SET @@session.binlog_format = MIXED;
+
+--echo * Row injection and binlog_format=statement: in slave sql thread
+INSERT INTO t VALUES (UUID());
+--echo [on slave]
+--connection slave
+# 1642 = ER_BINLOG_ROW_INJECTION_AND_STMT_MODE
+--let $slave_sql_errno= 1642
+--let $show_sql_error= 1
+--source include/wait_for_slave_sql_error_and_skip.inc
+
+--echo [on master]
+--connection master
+
+
+--echo ==== Clean up ====
+
+DROP TABLE t, t_self_logging, t_row, t_stmt, t_slave_stmt;
+SET @@global.binlog_format = @old_binlog_format;
+SET @@session.binlog_format = @old_binlog_format;
+UNINSTALL PLUGIN example;
+--echo [on slave]
+--sync_slave_with_master
+SET @@global.binlog_format = @old_binlog_format;
+SET @@session.binlog_format = @old_binlog_format;
+UNINSTALL PLUGIN example;

mysql-test/suite/rpl_ndb/t/rpl_ndb_blob.test

@@ -1,4 +1,7 @@
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 --source include/ndb_master-slave.inc
 
 #

mysql-test/suite/rpl_ndb/t/rpl_ndb_blob2.test

@@ -3,6 +3,11 @@
 # Using wrapper to share test   #
 # code between engine tests     #
 #################################
+#
+# Since the master generates row-based events, the slave may not be in
+# STATEMENT mode to accept the events.
+#
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/have_ndb.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDBCLUSTER;

mysql-test/suite/rpl_ndb/t/rpl_ndb_circular.test

@@ -1,4 +1,7 @@
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 --source include/ndb_master-slave.inc
 
 # set up circular replication

mysql-test/suite/rpl_ndb/t/rpl_ndb_circular_simplex.test

@@ -1,4 +1,7 @@
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 --source include/ndb_master-slave.inc
 
 connection master;
@@ -78,3 +81,7 @@ SELECT * FROM t1 ORDER BY a;
 --replace_result $MASTER_MYPORT MASTER_PORT
 --replace_column 1 # 7 # 8 # 9 # 22 # 23 # 33 # 35 # 36 #
 query_vertical SHOW SLAVE STATUS;
+
+connection master;
+DROP TABLE t1;
+sync_slave_with_master;

mysql-test/suite/rpl_ndb/t/rpl_ndb_commit_afterflush.test

@@ -4,6 +4,10 @@
 # Different engines                 #
 # By JBM 2004-02-15                 #
 #####################################
+
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/have_ndb.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDB;

mysql-test/suite/rpl_ndb/t/rpl_ndb_ctype_ucs2_def.test

@@ -1,5 +1,8 @@
 --source include/have_ucs2.inc
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 --source include/ndb_master-slave.inc
 
 #

mysql-test/suite/rpl_ndb/t/rpl_ndb_delete_nowhere.test

@@ -3,6 +3,9 @@
 # Share test code between engine tests  #
 #########################################
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDB;
 -- source extra/rpl_tests/rpl_delete_no_where.test

mysql-test/suite/rpl_ndb/t/rpl_ndb_do_db.test

@@ -6,6 +6,9 @@
 ##########################################################
 
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 --source include/ndb_master-slave.inc
 
 --disable_warnings

mysql-test/suite/rpl_ndb/t/rpl_ndb_do_table.test

@@ -6,6 +6,9 @@
 ##########################################################
 
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 --source include/ndb_master-slave.inc
 
 --disable_warnings

mysql-test/suite/rpl_ndb/t/rpl_ndb_func003.test

@@ -7,6 +7,9 @@
 # reduce test case code           #
 ###################################
 -- source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDB;
 -- source extra/rpl_tests/rpl_row_func003.test

mysql-test/suite/rpl_ndb/t/rpl_ndb_innodb_trans.test

@@ -2,6 +2,9 @@
 
 -- source include/have_ndb.inc
 -- source include/have_innodb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 
 create table t1 (a int, unique(a)) engine=ndbcluster;

mysql-test/suite/rpl_ndb/t/rpl_ndb_insert_ignore.test

@@ -2,6 +2,9 @@
 # Wrapper for rpl_insert_ignore.test#
 #####################################
 -- source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDB;
 let $engine_type2=myisam;

mysql-test/suite/rpl_ndb/t/rpl_ndb_mixed_engines_transactions.test

@@ -23,6 +23,9 @@
 
 
 source include/have_ndb.inc;
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+source include/have_binlog_format_mixed_or_row.inc;
 source include/ndb_master-slave.inc;
 source include/have_innodb.inc;
 

mysql-test/suite/rpl_ndb/t/rpl_ndb_multi_update3.test

@@ -3,6 +3,9 @@
 # to reuse test code between engine runs                   #
 ############################################################
 -- source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDB;
 -- source extra/rpl_tests/rpl_multi_update3.test

mysql-test/suite/rpl_ndb/t/rpl_ndb_rep_ignore.test

@@ -7,6 +7,9 @@
 ##########################################################
 
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 --source include/ndb_master-slave.inc
 
 --disable_warnings

mysql-test/suite/rpl_ndb/t/rpl_ndb_row_001.test

@@ -2,6 +2,9 @@
 # By JBM 2005-02-15 Wrapped to allow reuse of test code#
 ########################################################
 --source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+--source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDB;
 -- source extra/rpl_tests/rpl_row_001.test

mysql-test/suite/rpl_ndb/t/rpl_ndb_sp003.test

@@ -5,6 +5,9 @@
 # For different engines         #
 #################################
 -- source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDBCLUSTER;
 -- source extra/rpl_tests/rpl_row_sp003.test

mysql-test/suite/rpl_ndb/t/rpl_ndb_sp006.test

@@ -5,6 +5,9 @@
 # For different engines         #
 #################################
 -- source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDBCLUSTER;
 -- source extra/rpl_tests/rpl_row_sp006.test

mysql-test/suite/rpl_ndb/t/rpl_ndb_trig004.test

@@ -8,6 +8,9 @@
 
 # Includes
 -- source include/have_ndb.inc
+# Since the master generates row-based events, the slave must be in
+# ROW or MIXED mode to accept the events.
+-- source include/have_binlog_format_mixed_or_row.inc
 -- source include/ndb_master-slave.inc
 let $engine_type=NDB;
 -- source extra/rpl_tests/rpl_trig004.test