Fix for bug #46947 "Embedded SELECT without FOR UPDATE is
causing a lock", with after-review fixes.
SELECT statements with subqueries referencing InnoDB tables
were acquiring shared locks on rows in these tables when they
were executed in REPEATABLE-READ mode and with statement or
mixed mode binary logging turned on.
This was a regression which were introduced when fixing
bug 39843.
The problem was that for tables belonging to subqueries
parser set TL_READ_DEFAULT as a lock type. In cases when
statement/mixed binary logging at open_tables() time this
type of lock was converted to TL_READ_NO_INSERT lock at
open_tables() time and caused InnoDB engine to acquire
shared locks on reads from these tables. Although in some
cases such behavior was correct (e.g. for subqueries in
DELETE) in case of SELECT it has caused unnecessary locking.
This patch tries to solve this problem by rethinking our
approach to how we handle locking for SELECT and subqueries.
Now we always set TL_READ_DEFAULT lock type for all cases
when we read data. When at open_tables() time this lock
is interpreted as TL_READ_NO_INSERT or TL_READ depending
on whether this statement as a whole or call to function
which uses particular table should be written to the
binary log or not (if yes then statement should be properly
serialized with concurrent statements and stronger lock
should be acquired).
Test coverage is added for both InnoDB and MyISAM.
This patch introduces an "incompatible" change in locking
scheme for subqueries used in SELECT ... FOR UPDATE and
SELECT .. IN SHARE MODE.
In 4.1 the server would use a snapshot InnoDB read for
subqueries in SELECT FOR UPDATE and SELECT .. IN SHARE MODE
statements, regardless of whether the binary log is on or off.
If the user required a different type of read (i.e. locking read),
he/she could request so explicitly by providing FOR UPDATE/IN SHARE MODE
clause for each individual subquery.
On of the patches for 5.0 broke this behaviour (which was not documented
or tested), and started to use locking reads fora all subqueries in SELECT ...
FOR UPDATE/IN SHARE MODE. This patch restored 4.1 behaviour.
mysql-test/include/check_concurrent_insert.inc:
Added auxiliary script which allows to check if statement
reading table allows concurrent inserts in it.
mysql-test/include/check_no_concurrent_insert.inc:
Added auxiliary script which allows to check that statement
reading table doesn't allow concurrent inserts in it.
mysql-test/include/check_no_row_lock.inc:
Added auxiliary script which allows to check if statement
reading table doesn't take locks on its rows.
mysql-test/include/check_shared_row_lock.inc:
Added auxiliary script which allows to check if statement
reading table takes shared locks on some of its rows.
mysql-test/r/bug39022.result:
After bug #46947 'Embedded SELECT without FOR UPDATE is
causing a lock' was fixed test case for bug 39022 has to
be adjusted in order to trigger execution path on which
original problem was encountered.
mysql-test/r/innodb_mysql_lock2.result:
Added coverage for handling of locking in various cases when
we read data from InnoDB tables (includes test case for
bug #46947 'Embedded SELECT without FOR UPDATE is causing a
lock').
mysql-test/r/lock_sync.result:
Added coverage for handling of locking in various cases when
we read data from MyISAM tables.
mysql-test/t/bug39022.test:
After bug #46947 'Embedded SELECT without FOR UPDATE is
causing a lock' was fixed test case for bug 39022 has to
be adjusted in order to trigger execution path on which
original problem was encountered.
mysql-test/t/innodb_mysql_lock2.test:
Added coverage for handling of locking in various cases when
we read data from InnoDB tables (includes test case for
bug #46947 'Embedded SELECT without FOR UPDATE is causing a
lock').
mysql-test/t/lock_sync.test:
Added coverage for handling of locking in various cases when
we read data from MyISAM tables.
sql/log_event.cc:
Since LEX::lock_option member was removed we no longer can
rely on its value in Load_log_event::print_query() to
determine that log event correponds to LOAD DATA CONCURRENT
statement (this was not correct in all situations anyway).
A new Load_log_event's member was introduced as a replacement.
It is initialized at event object construction time and
explicitly indicates whether LOAD DATA was concurrent.
sql/log_event.h:
Since LEX::lock_option member was removed we no longer can
rely on its value in Load_log_event::print_query() to
determine that log event correponds to LOAD DATA CONCURRENT
statement (this was not correct in all situations anyway).
A new Load_log_event's member was introduced as a replacement.
It is initialized at event object construction time and
explicitly indicates whether LOAD DATA was concurrent.
sql/sp_head.cc:
sp_head::reset_lex():
Before parsing substatement reset part of parser state
which needs this (e.g. set Yacc_state::m_lock_type to
default value).
sql/sql_acl.cc:
Since LEX::reset_n_backup_query_tables_list() now also
resets LEX::sql_command member (as it became part of
Query_tables_list class) we have to restore it in cases
when while working with proxy Query_table_list we assume
that LEX::sql_command still corresponds to original SQL
command being executed (for example, when we are logging
statement to the binary log while having Query_tables_list
reset and backed up).
sql/sql_base.cc:
Changed read_lock_type_for_table() to return a weak TL_READ
type of lock in cases when we are executing statement which
won't update tables directly and table doesn't belong to
statement's prelocking list and thus can't be used by a
stored function. It is OK to do so since in this case table
won't be used by statement or function call which will be
written to the binary log, so serializability requirements
for it can be relaxed.
One of results from this change is that SELECTs on InnoDB
tables no longer takes shared row locks for tables which
are used in subqueries (i.e. bug #46947 is fixed).
Another result is that for similar SELECTs on MyISAM tables
concurrent inserts are allowed.
In order to implement this change signature of
read_lock_type_for_table() function was changed to take
pointers to Query_tables_list and TABLE_LIST objects.
sql/sql_base.h:
- Function read_lock_type_for_table() now takes pointers
to Query_tables_list and TABLE_LIST elements as its
arguments since to correctly determine lock type it needs
to know what statement is being performed and whether table
element for which lock type to be determined belongs to
prelocking list.
sql/sql_lex.cc:
- Removed LEX::lock_option and st_select_lex::lock_option
members. Places in parser that were using them now use
Yacc_state::m_lock_type instead.
- To emphasize that LEX::sql_command member is used during
process of opening and locking of tables it was moved to
Query_tables_list class. It is now reset by
Query_tables_list::reset_query_tables_list() method.
sql/sql_lex.h:
- Removed st_select_lex::lock_option member as there is no
real need for per-SELECT lock type (HIGH_PRIORITY option
should apply to the whole statement. FOR UPDATE/LOCK IN
SHARE MODE clauses can be handled without this member).
The main effect which was achieved by introduction of this
member, i.e. using TL_READ_DEFAULT lock type for
subqueries, is now achieved by setting LEX::lock_option
(or rather its replacement - Yacc_state::m_lock_type) to
TL_READ_DEFAULT in almost all cases.
- To emphasize that LEX::sql_command member is used during
process of opening and locking of tables it was moved to
Query_tables_list class.
- Replaced LEX::lock_option with Yacc_state::m_lock_type
in order to emphasize that this value is relevant only
during parsing. Unlike for LEX::lock_option the default
value for Yacc_state::m_lock_type is TL_READ_DEFAULT.
Note that for cases when it is OK to take a "weak" read
lock (e.g. simple SELECT) this lock type will be converted
to TL_READ at open_tables() time. So this change won't
cause negative change in behavior for such statements.
OTOH this change ensures that, for example, for SELECTs
which are used in stored functions TL_READ_NO_INSERT lock
is taken when necessary and as result calls to such stored
functions can be written to the binary log with correct
serialization.
sql/sql_load.cc:
Load_log_event constructor now requires a parameter that
indicates whether LOAD DATA is concurrent.
sql/sql_parse.cc:
LEX::lock_option was replaced with Yacc_state::m_lock_type.
And instead of resetting the latter implicitly in
mysql_init_multi_delete() we do it explicitly in the
places in parser which call this function.
sql/sql_priv.h:
- To be able more easily distinguish high-priority SELECTs
in st_select_lex::print() method added flag for
HIGH_PRIORITY option.
sql/sql_select.cc:
Changed code not to rely on LEX::lock_option to determine
that it is high-priority SELECT. It was replaced with
Yacc_state::m_lock_type which is accessible only at
parse time. So instead of LEX::lock_option we now rely
on a newly introduced flag for st_select_lex::options -
SELECT_HIGH_PRIORITY.
sql/sql_show.cc:
Since LEX::reset_n_backup_query_tables_list() now also
resets LEX::sql_command member (as it became part of
Query_tables_list class) we have to restore it in cases
when while working with proxy Query_table_list we assume
that LEX::sql_command still corresponds to original SQL
command being executed.
sql/sql_table.cc:
Since LEX::reset_query_tables_list() now also resets
LEX::sql_command member (as it became part of
Query_tables_list class) we have to restore value of this
member when this method is called by mysql_admin_table(),
to make this code safe for re-execution.
sql/sql_trigger.cc:
Since LEX::reset_n_backup_query_tables_list() now also
resets LEX::sql_command member (as it became part of
Query_tables_list class) we have to restore it in cases
when while working with proxy Query_table_list we assume
that LEX::sql_command still corresponds to original SQL
command being executed (for example, when we are logging
statement to the binary log while having Query_tables_list
reset and backed up).
sql/sql_update.cc:
Function read_lock_type_for_table() now takes pointers
to Query_tables_list and TABLE_LIST elements as its
arguments since to correctly determine lock type it needs
to know what statement is being performed and whether table
element for which lock type to be determined belongs to
prelocking list.
sql/sql_yacc.yy:
- Removed st_select_lex::lock_option member as there is no
real need for per-SELECT lock type (HIGH_PRIORITY option
should apply to the whole statement. FOR UPDATE/LOCK IN
SHARE MODE clauses can be handled without this member).
The main effect which was achieved by introduction of this
member, i.e. using TL_READ_DEFAULT lock type for
subqueries, is now achieved by setting LEX::lock_option
(or rather its replacement - Yacc_state::m_lock_type) to
TL_READ_DEFAULT in almost all cases.
- Replaced LEX::lock_option with Yacc_state::m_lock_type
in order to emphasize that this value is relevant only
during parsing. Unlike for LEX::lock_option the default
value for Yacc_state::m_lock_type is TL_READ_DEFAULT.
Note that for cases when it is OK to take a "weak" read
lock (e.g. simple SELECT) this lock type will be converted
to TL_READ at open_tables() time. So this change won't
cause negative change in behavior for such statements.
OTOH this change ensures that, for example, for SELECTs
which are used in stored functions TL_READ_NO_INSERT lock
is taken when necessary and as result calls to such stored
functions can be written to the binary log with correct
serialization.
- To be able more easily distinguish high-priority SELECTs
in st_select_lex::print() method we now use new flag
in st_select_lex::options bit-field.
Fix for bug #46947 "Embedded SELECT without FOR UPDATE is
causing a lock", with after-review fixes.
SELECT statements with subqueries referencing InnoDB tables
were acquiring shared locks on rows in these tables when they
were executed in REPEATABLE-READ mode and with statement or
mixed mode binary logging turned on.
This was a regression which were introduced when fixing
bug 39843.
The problem was that for tables belonging to subqueries
parser set TL_READ_DEFAULT as a lock type. In cases when
statement/mixed binary logging at open_tables() time this
type of lock was converted to TL_READ_NO_INSERT lock at
open_tables() time and caused InnoDB engine to acquire
shared locks on reads from these tables. Although in some
cases such behavior was correct (e.g. for subqueries in
DELETE) in case of SELECT it has caused unnecessary locking.
This patch tries to solve this problem by rethinking our
approach to how we handle locking for SELECT and subqueries.
Now we always set TL_READ_DEFAULT lock type for all cases
when we read data. When at open_tables() time this lock
is interpreted as TL_READ_NO_INSERT or TL_READ depending
on whether this statement as a whole or call to function
which uses particular table should be written to the
binary log or not (if yes then statement should be properly
serialized with concurrent statements and stronger lock
should be acquired).
Test coverage is added for both InnoDB and MyISAM.
This patch introduces an "incompatible" change in locking
scheme for subqueries used in SELECT ... FOR UPDATE and
SELECT .. IN SHARE MODE.
In 4.1 the server would use a snapshot InnoDB read for
subqueries in SELECT FOR UPDATE and SELECT .. IN SHARE MODE
statements, regardless of whether the binary log is on or off.
If the user required a different type of read (i.e. locking read),
he/she could request so explicitly by providing FOR UPDATE/IN SHARE MODE
clause for each individual subquery.
On of the patches for 5.0 broke this behaviour (which was not documented
or tested), and started to use locking reads fora all subqueries in SELECT ...
FOR UPDATE/IN SHARE MODE. This patch restored 4.1 behaviour.
Docs/sp-imp-spec.txt:
New sql_mode added.
include/my_base.h:
Flag in frm of create options.
libmysqld/CMakeLists.txt:
New files added.
libmysqld/Makefile.am:
New files added.
mysql-test/r/events_bugs.result:
New sql_mode added.
mysql-test/r/information_schema.result:
New sql_mode added.
mysql-test/r/sp.result:
New sql_mode added.
mysql-test/r/system_mysql_db.result:
New sql_mode added.
mysql-test/suite/funcs_1/r/is_columns_mysql.result:
New sql_mode added.
mysql-test/suite/funcs_1/r/is_columns_mysql_embedded.result:
New sql_mode added.
mysql-test/t/events_bugs.test:
New sql_mode added.
mysql-test/t/sp.test:
New sql_mode added.
scripts/mysql_system_tables.sql:
New sql_mode added.
scripts/mysql_system_tables_fix.sql:
New sql_mode added.
sql/CMakeLists.txt:
New files added.
sql/Makefile.am:
New files added.
sql/event_db_repository.cc:
New sql_mode added.
sql/field.cc:
Create options support added.
sql/field.h:
Create options support added.
sql/ha_partition.cc:
Create options support added.
sql/handler.cc:
Create options support added.
sql/handler.h:
Create options support added.
sql/log_event.h:
New sql_mode added.
sql/mysql_priv.h:
New sql_mode added.
sql/mysqld.cc:
New sql_mode added.
sql/share/errmsg.txt:
New error messages added.
sql/sp.cc:
New sql_mode added.
sql/sp_head.cc:
Create options support added.
sql/sql_class.cc:
Create options support added.
Debug added.
sql/sql_class.h:
Create options support added.
sql/sql_insert.cc:
my_safe_a* moved to mysqld_priv.h
sql/sql_lex.h:
Create options support added.
sql/sql_parse.cc:
Create options support added.
sql/sql_show.cc:
Create options support added.
sql/sql_table.cc:
Create options support added.
sql/sql_view.cc:
New sql_mode added.
sql/sql_yacc.yy:
Create options support added.
sql/structs.h:
Create options support added.
sql/table.cc:
Create options support added.
sql/table.h:
Create options support added.
sql/unireg.cc:
Create options support added.
storage/example/ha_example.cc:
Create options example.
storage/example/ha_example.h:
Create options example.
storage/pbxt/src/discover_xt.cc:
Create options support added.
Conflicts:
Text conflict in mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result
Text conflict in sql/log.cc
Text conflict in sql/set_var.cc
Text conflict in sql/sql_class.cc
Conflicts:
Text conflict in mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result
Text conflict in sql/log.cc
Text conflict in sql/set_var.cc
Text conflict in sql/sql_class.cc
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
BUG#46364 introduced the flag binlog_direct_non_transactional_updates which
would make N-changes to be written to the binary log upon committing the
statement when "ON". On the other hand, when "OFF" the option was supposed
to mimic the behavior in 5.1. However, the implementation was not mimicking
the behavior correctly and the following bugs popped up:
Case #1: N-changes executed within a transaction would go into
the S-cache. When later in the same transaction a
T-change occurs, N-changes following it were written
to the T-cache instead of the S-cache. In some cases,
this raises problems. For example, a
Table_map_log_event being written initially into the
S-cache, together with the initial N-changes, would be
absent from the T-cache. This would log N-changes
orphaned from a Table_map_log_event (thence discarded
at the slave). (MIXED and ROW)
Case #2: When rolling back a transaction, the N-changes that
might be in the T-cache were disregarded and
truncated along with the T-changes. (MIXED and ROW)
Case #3: When a MIXED statement (TN) is ahead of any other
T-changes in the transaction and it fails, it is kept
in the T-cache until the transaction ends. This is
not the case in 5.1 or Betony (5.5.2). In these, the
failed TN statement would be written to the binlog at
the same instant it had failed and not deferred until
transaction end. (SBR)
To fix these problems, we have decided to do what follows:
For Case #1 and #2, we circumvent them:
1. by not letting binlog_direct_non_transactional_updates
affect MIXED and RBR. These modes will keep the behavior
provided by WL#2687. Although this will make Celosia to
behave differently from 5.1, an execution will be always
safe under such modes in the sense that slaves will never
go out sync. In 5.1, using either MIXED or ROW while
mixing N-statements and T-statements was not safe.
For Case #3, we don't actually fix it. We:
1. keep it and make all MIXED statements whether they end
up failing or not or whether they are up front in the
transaction or after some transactional change to always
be stored in the T-cache. This means that it is written
to the binary log on transaction commit/rollback only.
2. We make the warning message even more specific about the
MIXED statement and SBR.
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
Updated the test case to avoid checking inconsistencies between the master and slave
when session.binlog_direct_non_transactional_updates is ON and the format is statement.
In this scenario, they will diverge because a counter (within a triger) is incremented
and associated to the issued statement. However, an n-statement is logged ahead of
the transaction and thus is not executed by the same order in the slave and thus gets
a different value from the counter.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
Updated the test case with the new error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Updated the test case with the new error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
mysql-test/suite/ndb/r/ndb_binlog_format.result:
Updated the test case with the new error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
Updated the test case with the new error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
Updated the test case with the new error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
Updated the test case with the new error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
Updated the test case with the new error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
sql/log.cc:
Checked if either a trx-cache or a non-trx-cache should be used.
If bin_log_direct_non_trans_update is active or the format is either
MIXED or ROW, the cache to be used depends on the flag is_transactional.
When the format is STMT, the non-trx-cache should be used if the statement
is non-transactional and the trx-cache is empty, i.e. if any transactional
statement has not committed yet. Otherwise, the trx-cache should be used.
sql/share/errmsg-utf8.txt:
Added the new unsafe error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
sql/sql_class.cc:
Started printing ER_BINLOG_UNSAFE_MIXED_STATEMENT, when there
is a mixed-statement.
Organized the names of the variables and added comments.
sql/sql_lex.cc:
Added the new unsafe error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
sql/sql_lex.h:
Added the new unsafe error ER_BINLOG_UNSAFE_MIXED_STATEMENT.
BUG#46364 introduced the flag binlog_direct_non_transactional_updates which
would make N-changes to be written to the binary log upon committing the
statement when "ON". On the other hand, when "OFF" the option was supposed
to mimic the behavior in 5.1. However, the implementation was not mimicking
the behavior correctly and the following bugs popped up:
Case #1: N-changes executed within a transaction would go into
the S-cache. When later in the same transaction a
T-change occurs, N-changes following it were written
to the T-cache instead of the S-cache. In some cases,
this raises problems. For example, a
Table_map_log_event being written initially into the
S-cache, together with the initial N-changes, would be
absent from the T-cache. This would log N-changes
orphaned from a Table_map_log_event (thence discarded
at the slave). (MIXED and ROW)
Case #2: When rolling back a transaction, the N-changes that
might be in the T-cache were disregarded and
truncated along with the T-changes. (MIXED and ROW)
Case #3: When a MIXED statement (TN) is ahead of any other
T-changes in the transaction and it fails, it is kept
in the T-cache until the transaction ends. This is
not the case in 5.1 or Betony (5.5.2). In these, the
failed TN statement would be written to the binlog at
the same instant it had failed and not deferred until
transaction end. (SBR)
To fix these problems, we have decided to do what follows:
For Case #1 and #2, we circumvent them:
1. by not letting binlog_direct_non_transactional_updates
affect MIXED and RBR. These modes will keep the behavior
provided by WL#2687. Although this will make Celosia to
behave differently from 5.1, an execution will be always
safe under such modes in the sense that slaves will never
go out sync. In 5.1, using either MIXED or ROW while
mixing N-statements and T-statements was not safe.
For Case #3, we don't actually fix it. We:
1. keep it and make all MIXED statements whether they end
up failing or not or whether they are up front in the
transaction or after some transactional change to always
be stored in the T-cache. This means that it is written
to the binary log on transaction commit/rollback only.
2. We make the warning message even more specific about the
MIXED statement and SBR.
Makefile.am:
add new API files to the check_abi rule,
remove duplicates
client/CMakeLists.txt:
now a client can use dlopen too
client/Makefile.am:
be csh-friendly
include/my_global.h:
add dummy plugs for dlopen and co.
for the code that needs them to work in static builds
mysys/Makefile.am:
be csh-friendly
plugin/auth/dialog.c:
typo fixed
The log event of 'CREATE EVENT' was being binlogged with garbage
at the end of the query if 'CREATE EVENT' is followed by another SQL statement
and they were executed as one command.
for example:
DELIMITER |;
CREATE EVENT e1 ON EVERY DAY DO SELECT 1; SELECT 'a';
DELIMITER ;|
When binlogging 'CREATE EVENT', we always create a new statement with definer
and write it into the log event. The new statement is made from cpp_buf(preprocessed buffer).
which is not a c string(end with '\0'), but it is copied as a c string.
In this patch, cpp_buf is copied with its length.
The log event of 'CREATE EVENT' was being binlogged with garbage
at the end of the query if 'CREATE EVENT' is followed by another SQL statement
and they were executed as one command.
for example:
DELIMITER |;
CREATE EVENT e1 ON EVERY DAY DO SELECT 1; SELECT 'a';
DELIMITER ;|
When binlogging 'CREATE EVENT', we always create a new statement with definer
and write it into the log event. The new statement is made from cpp_buf(preprocessed buffer).
which is not a c string(end with '\0'), but it is copied as a c string.
In this patch, cpp_buf is copied with its length.
Reading from a self-logging engine and updating a transactional engine such as Innodb
generates changes that are written to the binary log in the statement format and may
make slaves diverge. In the mixed mode, such changes should be written to the binary
log in the row format.
Note that the issue does not happen if we mix a self-logging engine and MyIsam
as this case is caught by checking the mixture of non-transactional and transactional
engines.
So, we classify a mixed statement where one reads from NDB and writes into another
engine as unsafe:
if (multi_engine && flags_some_set & HA_HAS_OWN_BINLOGGING)
lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE);
mysql-test/suite/rpl_ndb/r/rpl_ndb_mixed_engines_transactions.result:
Augmented test case to check mixed statements
mysql-test/suite/rpl_ndb/t/rpl_ndb_mixed_engines_transactions.test:
Augmented test case to check mixed statements
sql/share/errmsg-utf8.txt:
Added ER_BINLOG_UNSAFE_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE
sql/sql_class.cc:
Redefined flags' name in order to have two sets of flags: (i) flags that are checked when there
is a write operation; (ii) flags that are checked regardless of the type of the operation.
Classified a mixed statement where one reads from NDB and writes into another engine as unsafe:
if (multi_engine && flags_some_set & HA_HAS_OWN_BINLOGGING)
lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE);
sql/sql_lex.cc:
Added error ER_BINLOG_UNSAFE_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE
sql/sql_lex.h:
Added BINLOG_STMT_UNSAFE_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE
Reading from a self-logging engine and updating a transactional engine such as Innodb
generates changes that are written to the binary log in the statement format and may
make slaves diverge. In the mixed mode, such changes should be written to the binary
log in the row format.
Note that the issue does not happen if we mix a self-logging engine and MyIsam
as this case is caught by checking the mixture of non-transactional and transactional
engines.
So, we classify a mixed statement where one reads from NDB and writes into another
engine as unsafe:
if (multi_engine && flags_some_set & HA_HAS_OWN_BINLOGGING)
lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE);
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_slow_query_log.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Conflict adding files to server-tools. Created directory.
Conflict because server-tools is not versioned, but has versioned children. Versioned directory.
Conflict adding files to server-tools/instance-manager. Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children. Versioned directory.
Contents conflict in server-tools/instance-manager/options.cc
Text conflict in sql/mysqld.cc
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_slow_query_log.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Conflict adding files to server-tools. Created directory.
Conflict because server-tools is not versioned, but has versioned children. Versioned directory.
Conflict adding files to server-tools/instance-manager. Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children. Versioned directory.
Contents conflict in server-tools/instance-manager/options.cc
Text conflict in sql/mysqld.cc
Grouping by a subquery in a query with a distinct aggregate
function lead to a wrong result (wrong and unordered
grouping values).
There are two related problems:
1) The query like this:
SELECT (SELECT t1.a) aa, COUNT(DISTINCT b) c
FROM t1 GROUP BY aa
returned wrong result, because the outer reference "t1.a"
in the subquery was substituted with the Item_ref item.
The Item_ref item obtains data from the result_field object
that refreshes once after the end of each group. This data
is not applicable to filesort since filesort() doesn't care
about groups (and doesn't update result_field objects with
copy_fields() and so on). Also that data is not applicable
to group separation algorithm: end_send_group() checks every
record with test_if_group_changed() that evaluates Item_ref
items, but it refreshes those Item_ref-s only after the end
of group, that is a vicious circle and the grouped column
values in the output are shifted.
Fix: if
a) we grouping by a subquery and
b) that subquery has outer references to FROM list
of the grouping query,
then we substitute these outer references with
Item_direct_ref like references under aggregate
functions: Item_direct_ref obtains data directly
from the current record.
2) The query with a non-trivial grouping expression like:
SELECT (SELECT t1.a) aa, COUNT(DISTINCT b) c
FROM t1 GROUP BY aa+0
also returned wrong result, since JOIN::exec() substitutes
references to top-level aliases in SELECT list with Item_copy
caching items. Item_copy items have same refreshing policy
as Item_ref items, so the whole groping expression with
Item_copy inside returns wrong result in filesort() and
end_send_group().
Fix: include aliased items into GROUP BY item tree instead
of Item_ref references to them.
mysql-test/r/group_by.result:
Test case for bug #45640
mysql-test/t/group_by.test:
Test case for bug #45640
sql/item.cc:
Bug #45640: optimizer bug produces wrong results
Item_field::fix_fields() has been modified to resolve
aliases in GROUP BY item trees into aliased items instead
of Item_ref items.
sql/item.h:
Bug #45640: optimizer bug produces wrong results
- Item::find_item_processor() has been introduced.
- Item_ref::walk() has been modified to apply processors
to itself too (not only to referenced item).
sql/mysql_priv.h:
Bug #45640: optimizer bug produces wrong results
fix_inner_refs() has been modified to accept group_list
parameter.
sql/sql_lex.cc:
Bug #45640: optimizer bug produces wrong results
Initialization of st_select_lex::group_fix_field has
been added.
sql/sql_lex.h:
Bug #45640: optimizer bug produces wrong results
The st_select_lex::group_fix_field field has been introduced
to control alias resolution in Itef_fied::fix_fields.
sql/sql_select.cc:
Bug #45640: optimizer bug produces wrong results
- The fix_inner_refs function has been modified to treat
subquery outer references like outer fields under aggregate
functions, if they are included in GROUP BY item tree.
- The find_order_in_list function has been modified to
fix Item_field alias fields included in the GROUP BY item
trees in a special manner.
Grouping by a subquery in a query with a distinct aggregate
function lead to a wrong result (wrong and unordered
grouping values).
There are two related problems:
1) The query like this:
SELECT (SELECT t1.a) aa, COUNT(DISTINCT b) c
FROM t1 GROUP BY aa
returned wrong result, because the outer reference "t1.a"
in the subquery was substituted with the Item_ref item.
The Item_ref item obtains data from the result_field object
that refreshes once after the end of each group. This data
is not applicable to filesort since filesort() doesn't care
about groups (and doesn't update result_field objects with
copy_fields() and so on). Also that data is not applicable
to group separation algorithm: end_send_group() checks every
record with test_if_group_changed() that evaluates Item_ref
items, but it refreshes those Item_ref-s only after the end
of group, that is a vicious circle and the grouped column
values in the output are shifted.
Fix: if
a) we grouping by a subquery and
b) that subquery has outer references to FROM list
of the grouping query,
then we substitute these outer references with
Item_direct_ref like references under aggregate
functions: Item_direct_ref obtains data directly
from the current record.
2) The query with a non-trivial grouping expression like:
SELECT (SELECT t1.a) aa, COUNT(DISTINCT b) c
FROM t1 GROUP BY aa+0
also returned wrong result, since JOIN::exec() substitutes
references to top-level aliases in SELECT list with Item_copy
caching items. Item_copy items have same refreshing policy
as Item_ref items, so the whole groping expression with
Item_copy inside returns wrong result in filesort() and
end_send_group().
Fix: include aliased items into GROUP BY item tree instead
of Item_ref references to them.
Original revision:
------------------------------------------------------------
revision-id: li-bing.song@sun.com-20100130124925-o6sfex42b6noyc6x
parent: joro@sun.com-20100129145427-0n79l9hnk0q43ajk
committer: <Li-Bing.Song@sun.com>
branch nick: mysql-5.1-bugteam
timestamp: Sat 2010-01-30 20:49:25 +0800
message:
Bug #48321 CURRENT_USER() incorrectly replicated for DROP/RENAME USER;
REVOKE/GRANT; ALTER EVENT.
The following statements support the CURRENT_USER() where a user is needed.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENT
but, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, All above statements are rewritten when they are binlogged.
The CURRENT_USER() is expanded to the real user's name and host.
------------------------------------------------------------
Original revision:
------------------------------------------------------------
revision-id: li-bing.song@sun.com-20100130124925-o6sfex42b6noyc6x
parent: joro@sun.com-20100129145427-0n79l9hnk0q43ajk
committer: <Li-Bing.Song@sun.com>
branch nick: mysql-5.1-bugteam
timestamp: Sat 2010-01-30 20:49:25 +0800
message:
Bug #48321 CURRENT_USER() incorrectly replicated for DROP/RENAME USER;
REVOKE/GRANT; ALTER EVENT.
The following statements support the CURRENT_USER() where a user is needed.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENT
but, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, All above statements are rewritten when they are binlogged.
The CURRENT_USER() is expanded to the real user's name and host.
------------------------------------------------------------
REVOKE/GRANT; ALTER EVENT.
The following statements support the CURRENT_USER() where a user is needed.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENT
but, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, All above statements are rewritten when they are binlogged.
The CURRENT_USER() is expanded to the real user's name and host.
REVOKE/GRANT; ALTER EVENT.
The following statements support the CURRENT_USER() where a user is needed.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENT
but, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, All above statements are rewritten when they are binlogged.
The CURRENT_USER() is expanded to the real user's name and host.
The auto-inc unsafe warning makes sense even though it's just
one auto-inc table could be involved via a trigger or a stored
function.
However its content was not updated by bug@45677 fixes continuing to mention
two tables whereas the fixes refined semantics of replication of auto_increment
in stored routine.
Fixed with updating the error message, renaming the error and an internal unsafe-condition
constants.
A documentation notice
======================
Inserting into an autoincrement column in a stored function or a trigger
is unsafe for replication.
Even with just one autoincrement column, if the routine is invoked more than
once slave is not guaranteed to execute the statement graph same way as
the master.
And since it's impossible to estimate how many times a routine can be invoked at
the query pre-execution phase (see lock_tables), the statement is marked
pessimistically unsafe.
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
results updated to include the expected unsafe warning.
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
regression test for bug#50192 to diplaying the unsafe warning comes out to the user warning stack.
sql/share/errmsg-utf8.txt:
Updating the auto-inc unsafe message to correspond to bug@45677 fixes' new sematics.
sql/share/errmsg.txt:
Updating the auto-inc unsafe message to correspond to bug@45677 fixes' new sematics.
sql/sql_base.cc:
changing a symbolic name to correspond to updated by bug@45677 fixes new sematics.
sql/sql_lex.cc:
changing a symbolic name to correspond to updated by bug@45677 fixes new sematics.
sql/sql_lex.h:
changing a symbolic name to correspond to updated by bug@45677 fixes new sematics
and description comments.
The auto-inc unsafe warning makes sense even though it's just
one auto-inc table could be involved via a trigger or a stored
function.
However its content was not updated by bug@45677 fixes continuing to mention
two tables whereas the fixes refined semantics of replication of auto_increment
in stored routine.
Fixed with updating the error message, renaming the error and an internal unsafe-condition
constants.
A documentation notice
======================
Inserting into an autoincrement column in a stored function or a trigger
is unsafe for replication.
Even with just one autoincrement column, if the routine is invoked more than
once slave is not guaranteed to execute the statement graph same way as
the master.
And since it's impossible to estimate how many times a routine can be invoked at
the query pre-execution phase (see lock_tables), the statement is marked
pessimistically unsafe.
