BUG#39325 Server crash inside MYSQL_LOG::purge_first_log halts replicaiton
The patch reverses the order of the purging and updating events for log and relay-log.info/index files respectively.
This solves the problem of having holes caused by crashes happening between updating info/index files and purging logs.
NOTE: This is a combined patch for BUG#38826 and BUG#39325. This patch is based on bugteam tree and takes into account reviewers suggestions.
BUG#39325 Server crash inside MYSQL_LOG::purge_first_log halts replicaiton
The patch reverses the order of the purging and updating events for log and relay-log.info/index files respectively.
This solves the problem of having holes caused by crashes happening between updating info/index files and purging logs.
NOTE: This is a combined patch for BUG#38826 and BUG#39325. This patch is based on bugteam tree and takes into account reviewers suggestions.
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
mysql-test/r/xa.result:
Add test case result for Bug#28323
mysql-test/t/xa.test:
Add test case for Bug#28323
sql/handler.cc:
Reset XID only at the end of the global transaction.
sql/share/errmsg.txt:
Add new error codes.
sql/sql_class.h:
Remember the error reported by the Resource Manager.
sql/sql_parse.cc:
Rollback the transaction if the Resource Manager reported
a error and rolled back its branch of the transaction.
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
mysql-test/r/xa.result:
Add test case result for Bug#28323
mysql-test/t/xa.test:
Add test case for Bug#28323
sql/handler.cc:
Reset XID only at the end of the global transaction.
sql/share/errmsg.txt:
Add new error codes.
sql/sql_class.h:
Remember the error reported by the Resource Manager.
sql/sql_parse.cc:
Rollback the transaction if the Resource Manager reported
a error and rolled back its branch of the transaction.
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
When running Stored Routines the Status Variable "Questions" was wrongly
incremented. According to the manual it should contain the "number of
statements that clients have sent to the server"
Introduced a new status variable 'questions' to replace the query_id
variable which currently corresponds badly with the number of statements
sent by the client.
The new behavior is ment to be backward compatible with 4.0 and at the
same time work with new features in a similar way.
This is a backport from 6.0
mysql-test/r/status2.result:
Added test case
mysql-test/t/status2.test:
Added test case
sql/mysqld.cc:
Introduced a new status variable 'questions' to replace the query_id
variable which currently corresponds badly with the number of statements
sent by the client.
sql/sql_class.h:
Introduced a new status variable 'questions' to replace the query_id
variable which currently corresponds badly with the number of statements
sent by the client.
sql/sql_parse.cc:
To be backward compatible with 4.0 and at the same time extend the
interpretation of the Question variable, it should be increased on
all COM-commands but COM_STATISTICS, COM_PING, COM_STMT_PREPARE,
COM_STMT_CLOSE and COM_STMT_RESET.
Since COM_QUERY can process multiple statements, there has to be an
extra increase there as well.
sql/sql_show.cc:
Removed deprecated SHOW_QUESTION status code.
sql/structs.h:
Removed deprecated SHOW_QUESTION status code.
When running Stored Routines the Status Variable "Questions" was wrongly
incremented. According to the manual it should contain the "number of
statements that clients have sent to the server"
Introduced a new status variable 'questions' to replace the query_id
variable which currently corresponds badly with the number of statements
sent by the client.
The new behavior is ment to be backward compatible with 4.0 and at the
same time work with new features in a similar way.
This is a backport from 6.0
The failure was caused by executing a CREATE-SELECT statement that creates a
table in another database than the current one. In row-based logging, the
CREATE statement was written to the binary log without the database, hence
creating the table in the wrong database, causing the following inserts to
fail since the table didn't exist in the given database.
Fixed the bug by adding a parameter to store_create_info() that will make
the function print the database name before the table name and used that
in the calls that write the CREATE statement to the binary log. The database
name is only printed if it is different than the currently selected database.
The output of SHOW CREATE TABLE has not changed and is still printed without
the database name.
mysql-test/suite/rpl/t/rpl_row_create_table.test:
Added test to check that CREATE-SELECT into another database than the
current one replicates.
sql/sql_insert.cc:
Adding parameter to calls to store_create_info().
sql/sql_show.cc:
Adding parameter to calls to store_create_info().
Extending store_create_info() with parameter 'show_database' that will cause
the database to be written before the table name.
sql/sql_show.h:
Adding parameter to call to store_create_info() to tell if the database should be shown or not.
sql/sql_table.cc:
Adding parameter to calls to store_create_info().
The failure was caused by executing a CREATE-SELECT statement that creates a
table in another database than the current one. In row-based logging, the
CREATE statement was written to the binary log without the database, hence
creating the table in the wrong database, causing the following inserts to
fail since the table didn't exist in the given database.
Fixed the bug by adding a parameter to store_create_info() that will make
the function print the database name before the table name and used that
in the calls that write the CREATE statement to the binary log. The database
name is only printed if it is different than the currently selected database.
The output of SHOW CREATE TABLE has not changed and is still printed without
the database name.
"Trigger fired multiple times leads to gaps in auto_increment sequence".
The bug was that if a trigger fired multiple times inside a top
statement (for example top-statement is a multi-row INSERT,
and trigger is ON INSERT), and that trigger inserted into an auto_increment
column, then gaps could be observed in the auto_increment sequence,
even if there were no other users of the database (no concurrency).
It was wrong usage of THD::auto_inc_intervals_in_cur_stmt_for_binlog.
Note that the fix changes "class handler", I'll tell the Storage Engine API team.
mysql-test/r/trigger-trans.result:
result; before the bugfix, the sequence was 1,2,4,6,8,10,12...
mysql-test/t/trigger-trans.test:
test for BUG#31612
sql/handler.cc:
See revision comment of handler.h.
As THD::auto_inc_intervals_in_cur_stmt_for_binlog is cumulative
over all trigger invokations by the top statement, the
second invokation of the trigger arrived in handler::update_auto_increment()
with already one interval in
THD::auto_inc_intervals_in_cur_stmt_for_binlog. The method thus
believed it had already reserved one interval for that invokation,
thus reserved a twice larger interval (heuristic when we don't know
how large the interval should be: we grow by powers of two). InnoDB
thus increased its internal per-table auto_increment counter by 2
while only one row was to be inserted. Hence a gap in the sequence.
The fix is to use the new handler::auto_inc_intervals_count.
Note that the trigger's statement knows how many rows it is going
to insert, but provides estimation_rows_to_insert == 0 (see comments
in sql_insert.cc why triggers don't call handler::ha_start_bulk_insert()).
* removing white space at end of line
* we don't need to maintain THD::auto_inc_intervals_in_cur_stmt_for_binlog
if no binlogging or if row-based binlogging. Using auto_inc_intervals_count in
the heuristic makes the heuristic independent of binary logging, which is good.
sql/handler.h:
THD::auto_inc_intervals_in_cur_stmt_for_binlog served
- for binlogging
- as a heuristic when we have no estimation of how many records the
statement will insert.
But the first goal needs to be cumulative over all statements which
form a binlog event, while the second one needs to be attached to each
statement. THD::auto_inc_intervals_in_cur_stmt_for_binlog is cumulative,
leading to BUG#31612. So we introduce handler::auto_inc_intervals_count
for the second goal. See the revision comment of handler.cc.
A smaller issue was that, even when the binlog event was only one
statement (no triggers, no stored functions),
THD::auto_inc_intervals_in_cur_stmt.nb_elements() could be lower than
the number of reserved intervals (fooling the heuristic), because its
append() method collapses two contiguous intervals in one.
Note that as auto_inc_intervals_count is in class 'handler' and not
in class 'THD', it does not need to be handled in
THD::reset|restore_sub_statement_state().
sql/log.cc:
Comment is wrong: if auto_increment is second, in handler::update_auto_increment()
'append' is false and so auto_inc_intervals_in_cur_stmt_for_binlog
is empty, we do not come here.
sql/sql_class.h:
comment
"Trigger fired multiple times leads to gaps in auto_increment sequence".
The bug was that if a trigger fired multiple times inside a top
statement (for example top-statement is a multi-row INSERT,
and trigger is ON INSERT), and that trigger inserted into an auto_increment
column, then gaps could be observed in the auto_increment sequence,
even if there were no other users of the database (no concurrency).
It was wrong usage of THD::auto_inc_intervals_in_cur_stmt_for_binlog.
Note that the fix changes "class handler", I'll tell the Storage Engine API team.
NO_BACKSLASH_ESCAPES was not heeded in LOAD DATA INFILE
and SELECT INTO OUTFILE. It is now.
mysql-test/r/loaddata.result:
Show that SQL-mode NO_BACKSLASH_ESCAPES is heeded in
INFILE/OUTFILE, and that dump/restore cycles work!
mysql-test/t/loaddata.test:
Show that SQL-mode NO_BACKSLASH_ESCAPES is heeded in
INFILE/OUTFILE, and that dump/restore cycles work!
sql/sql_class.cc:
Add function to enquire whether ESCAPED BY was given.
When doing SELECT...OUTFILE, use ESCAPED BY if specifically
given; otherwise use sensible default value depending on
SQL-mode features NO_BACKSLASH_ESCAPES.
sql/sql_class.h:
Add function to enquire whether ESCAPED BY was given.
sql/sql_load.cc:
When doing LOAD DATA INFILE, use ESCAPED BY if specifically
given; otherwise use sensible default value depending on
SQL-mode features NO_BACKSLASH_ESCAPES.
The assertion indicates that some data was left in the transaction
cache when the server was shut down, which means that a previous
statement did not commit or rollback correctly.
What happened was that a bug in the rollback of a transactional
table caused the transaction cache to be emptied, but not reset.
The error can be triggered by having a failing UPDATE or INSERT,
on a transactional table, causing an implicit rollback.
Fixed by always flushing the pending event to reset the state
properly.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Testing that a failed update (that writes some rows to the
transaction cache) does not cause the transaction cache to
hold on to the data or forget to reset the transaction cache.
sql/log.cc:
Added call to remove pending event when the transaction cache
is emptied instead of written to binary log. The call will also
clear the outstanding table map count so that the cache is not
left it in a state of "empty but not reset".
Added function MYSQL_BIN_LOG::remove_pending_rows_event().
sql/log.h:
Added function MYSQL_BIN_LOG::remove_pending_rows_event().
sql/sql_class.cc:
Adding function THD::binlog_remove_pending_rows_event().
sql/sql_class.h:
Adding function THD::binlog_remove_pending_rows_event().
The assertion indicates that some data was left in the transaction
cache when the server was shut down, which means that a previous
statement did not commit or rollback correctly.
What happened was that a bug in the rollback of a transactional
table caused the transaction cache to be emptied, but not reset.
The error can be triggered by having a failing UPDATE or INSERT,
on a transactional table, causing an implicit rollback.
Fixed by always flushing the pending event to reset the state
properly.
