General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.
Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
- binlog_format=STATEMENT for a row injection
- binlog_format=STATEMENT for an unsafe statement
- binlog_format=STATEMENT for an engine only supporting row logging
- binlog_format=ROW for an engine only supporting statement logging
- statement is unsafe and engine does not support row logging
- row injection in a table that does not support statement logging
- statement modifies one table that does not support row logging and
one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()
Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().
Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.
Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.
There is an inconsistency with DROP DATABASE|TABLE|EVENT IF EXISTS and
CREATE DATABASE|TABLE|EVENT IF NOT EXISTS. DROP IF EXISTS statements are
binlogged even if either the DB, TABLE or EVENT does not exist. In
contrast, Only the CREATE EVENT IF NOT EXISTS is binlogged when the EVENT
exists.
This patch fixes the following cases for all the replication formats:
CREATE DATABASE IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS,
CREATE TABLE IF NOT EXISTS ... LIKE,
CREAET TABLE IF NOT EXISTS ... SELECT.
sql/sql_insert.cc:
Part of the code was moved from the create_table_from_items to select_create::prepare.
When replication is row based, CREATE TABLE IF NOT EXISTS.. SELECT is binlogged if the table exists.
There is an inconsistency with DROP DATABASE|TABLE|EVENT IF EXISTS and
CREATE DATABASE|TABLE|EVENT IF NOT EXISTS. DROP IF EXISTS statements are
binlogged even if either the DB, TABLE or EVENT does not exist. In
contrast, Only the CREATE EVENT IF NOT EXISTS is binlogged when the EVENT
exists.
This patch fixes the following cases for all the replication formats:
CREATE DATABASE IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS,
CREATE TABLE IF NOT EXISTS ... LIKE,
CREAET TABLE IF NOT EXISTS ... SELECT.
enabled message storing into error message list
for 'drop table' command
mysql-test/r/warnings.result:
test result
mysql-test/t/warnings.test:
test case
sql/sql_table.cc:
We should skip error sending then we should return
warnings to client as some functions may send its
own errors, so we should set no_warnings_for_error= 0
only in case of warning.
The fix is to enable message storing into error message
list for 'drop table' command(only for error case).
tests/mysql_client_test.c:
test fix
The problem: described in the bug report.
The fix:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
client/mysql.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/ha_ndbcluster.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/ha_ndbcluster_binlog.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/handler.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/log.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/mysqld.cc:
removed unnecessary line
sql/parse_file.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_acl.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_base.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_db.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_delete.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_partition.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_rename.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_show.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_table.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_view.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
The problem: described in the bug report.
The fix:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
Manually merge some InnoDB changes into XtraDB.
Fix ALTER TABLE bug in XtraDB with wrong comparison of row type.
mysql-test/include/varchar.inc:
Fix in test case that which of several duplicate keys triggers an error is not
deterministic.
mysql-test/mysql-test-run.pl:
InnoDB does not bother to free resources individually during shutdown, but due to using
its own memory tracking it nevertheless can free everything at exit. But XtraDB adds an
option, on by default, to skip this extra tracking. This causes lots of Valgrind
warnings, so needs to be disabled for Valgrind testing.
mysql-test/r/innodb.result:
Fix in test case that which of several duplicate keys triggers an error is not
deterministic.
mysql-test/t/innodb-use-sys-malloc.test:
InnoDB does not bother to free resources individually during shutdown, but due to using
its own memory tracking it nevertheless can free everything at exit. But XtraDB adds an
option, on by default, to skip this extra tracking. This causes lots of Valgrind
warnings, so needs to be disabled for Valgrind testing.
sql/sql_table.cc:
Add some useful DBUG while debugging alter table.
storage/xtradb/handler/ha_innodb.cc:
Fix that check_if_incompatible_data did not realise that ROW_TYPE_DEFAULT is identical
to the default row format ROW_TYPE_COMPACT, causing excessive table copying in
ALTER TABLE
Add some useful DBUG while debugging alter table.
Manually merge into XtraDB a few small changes for InnoDB from upstream MySQL.
storage/xtradb/include/pars0pars.h:
Manually merge into XtraDB a few small changes for InnoDB from upstream MySQL.
storage/xtradb/include/univ.i:
Manually merge a MariaDB fix in InnoDB into XtraDB.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
or database name in logs
Problem was that InnoDB used filenam_to_tablename,
which do not handle partitions (due to the '#' in
the filename).
Solution is to add a new function for explaining
what the filename means: explain_filename.
It expands the database, table, partition and subpartition
parts and uses errmsg.txt for localization.
It also converts from my_charset_filename to system_charset_info
(i.e. human readable form for non ascii characters).
http://lists.mysql.com/commits/70370
2773 Mattias Jonsson 2009-03-25
It has three different output styles.
NOTE: This is the server side ONLY part (introducing the explain_filename
function). There will be a patch for InnoDB using this function to solve
the bug.
sql/mysql_priv.h:
Bug#32430:'show innodb status' causes errors Invalid (old?) table
or database name in logs
Added EXPLAIN_FILENAME_MAX_EXTRA_LENGTH, enum_explain_filename_mode
and explain_filename.
sql/share/errmsg.txt:
Bug#32430:'show innodb status' causes errors Invalid (old?) table
or database name in logs
Added localization names for Database, Table, Partition, Subpartition
Temporary and Renamed.
sql/sql_table.cc:
Bug#32430:'show innodb status' causes errors Invalid (old?) table
or database name in logs
Added explain_filename function for giving better information
to the user about a specific table/partitions file.
or database name in logs
Problem was that InnoDB used filenam_to_tablename,
which do not handle partitions (due to the '#' in
the filename).
Solution is to add a new function for explaining
what the filename means: explain_filename.
It expands the database, table, partition and subpartition
parts and uses errmsg.txt for localization.
It also converts from my_charset_filename to system_charset_info
(i.e. human readable form for non ascii characters).
http://lists.mysql.com/commits/70370
2773 Mattias Jonsson 2009-03-25
It has three different output styles.
NOTE: This is the server side ONLY part (introducing the explain_filename
function). There will be a patch for InnoDB using this function to solve
the bug.
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
sql/log_event.cc:
Changed constructors of Query_log_event and Execute_load_log_event to accept the error code argument instead of figuring it out by itself
sql/log_event.h:
Changed constructors of Query_log_event and Execute_load_log_event to accept the error code argument
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
When doing ALTER TABLE, we forgot to point out that we actually have
ROW_FORMAT information (from the original table), so we dropped to
"sensible defaults". This affects both ALTER TABLE and OPTIMIZE TABLE
which may fall back on ALTER TABLE for InnoDB.
We now flag that we do indeed know the row-type, thereby preserving
compression-type etc.
No .test in 5.1 since we'd need a reasonable new plugin from InnoDB to
show this properly; in higher versions, maria can demonstrate this.
sql/sql_table.cc:
In mysql_alter_table() flag that we have row-type info
from old table. In compare_tables(), we must explicitly
check whether row-type has changed (rather than rely on
the flag which will always be set at this point now).
When doing ALTER TABLE, we forgot to point out that we actually have
ROW_FORMAT information (from the original table), so we dropped to
"sensible defaults". This affects both ALTER TABLE and OPTIMIZE TABLE
which may fall back on ALTER TABLE for InnoDB.
We now flag that we do indeed know the row-type, thereby preserving
compression-type etc.
No .test in 5.1 since we'd need a reasonable new plugin from InnoDB to
show this properly; in higher versions, maria can demonstrate this.
Problem: executing queries like "ALTER TABLE view1;" we don't
check new view's name (which is not specified),
that leads to server crash.
Fix: do nothing (to be consistent with the behaviour for tables)
in such cases.
mysql-test/r/view.result:
Fix for bug#44860: ALTER TABLE on view crashes server
- test result.
mysql-test/t/view.test:
Fix for bug#44860: ALTER TABLE on view crashes server
- test case.
sql/sql_rename.cc:
Fix for bug#44860: ALTER TABLE on view crashes server
- do_rename(): new view/table name must be specified, ASSERT() added.
sql/sql_table.cc:
Fix for bug#44860: ALTER TABLE on view crashes server
- mysql_alter_table(): renaming a view, check if new
view name is specified.
Problem: executing queries like "ALTER TABLE view1;" we don't
check new view's name (which is not specified),
that leads to server crash.
Fix: do nothing (to be consistent with the behaviour for tables)
in such cases.
When the thread executing a DDL was killed after finished its
execution but before writing the binlog event, the error code in
the binlog event could be set wrongly to ER_SERVER_SHUTDOWN or
ER_QUERY_INTERRUPTED.
This patch fixed the problem by ignoring the kill status when
constructing the event for DDL statements.
This patch also included the following changes in order to
provide the test case.
1) modified mysqltest to support variable for connection command
2) modified mysql-test-run.pl, add new variable MYSQL_SLAVE to
run mysql client against the slave mysqld.
When the thread executing a DDL was killed after finished its
execution but before writing the binlog event, the error code in
the binlog event could be set wrongly to ER_SERVER_SHUTDOWN or
ER_QUERY_INTERRUPTED.
This patch fixed the problem by ignoring the kill status when
constructing the event for DDL statements.
This patch also included the following changes in order to
provide the test case.
1) modified mysqltest to support variable for connection command
2) modified mysql-test-run.pl, add new variable MYSQL_SLAVE to
run mysql client against the slave mysqld.
