After BUG#36649, warnings for sub-statements are cleared when a
new sub-statement is started. This is problematic since it suppresses
warnings for unsafe statements in some cases. It is important that we
always give a warning to the client, because the user needs to know
when there is a risk that the slave goes out of sync.
We fixed the problem by generating warning messages for unsafe statements
while returning from a stored procedure, function, trigger or while
executing a top level statement.
We also started checking unsafeness when both performance and log tables are
used. This is necessary after the performance schema which does a distinction
between performance and log tables.
mysql-test/extra/rpl_tests/create_recursive_construct.inc:
Changed the order of the calls in the procedure because the code
that checks if a warning message is printed out expects that the
first statement gives an warning what is not the case for INSERT
INTO ta$CRC_ARG_level VALUES (47);
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
Updated the result file.
mysql-test/suite/binlog/r/binlog_unsafe.result:
There are several changes here:
(1) - Changed the CREATE PROCEDURE $CRC.
(2) - The procedure $CRC was failing and the content of the binlog
was being printed out, after fix (1) the failure disappeared.
(3) - The warning message for unsafeness due to auto-increment collumns was
changed.
(4) - The warning message for unsafeness due to VERSION(), RAND() was changed.
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
Tested filters.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Reenabled the test case binlog_unsafe.
mysql-test/suite/binlog/t/disabled.def:
Reenabled the test case binlog_unsafe.
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
Updated the result file.
mysql-test/suite/rpl/r/rpl_non_direct_stm_mixing_engines.result:
Updated the result file.
mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result:
Updated the result file.
sql/sql_class.cc:
Moved the stmt_accessed_table_flag variable and related information to the
LEX as we need the variable reset after each statement even inside a stored
procedure, what did not happen if the information was in the THD.
Changed the routine in the THD::binlog_query that prints the warning
messages to avoid trying to print them when inside a stored procedure,
function or trigger.
Checked for unsafeness when both performance and log tables where used.
After the introduction of the performance schema, we need to check both.
After BUG#36649, warnings for sub-statements are cleared when a
new sub-statement is started. This is problematic since it suppresses
warnings for unsafe statements in some cases. It is important that we
always give a warning to the client, because the user needs to know
when there is a risk that the slave goes out of sync.
We fixed the problem by generating warning messages for unsafe statements
while returning from a stored procedure, function, trigger or while
executing a top level statement.
We also started checking unsafeness when both performance and log tables are
used. This is necessary after the performance schema which does a distinction
between performance and log tables.
/*![:version:] Query Code */, where [:version:] is a sequence of 5
digits representing the mysql server version(e.g /*!50200 ... */),
is a special comment that the query in it can be executed on those
servers whose versions are larger than the version appearing in the
comment. It leads to a security issue when slave's version is larger
than master's. A malicious user can improve his privileges on slaves.
Because slave SQL thread is running with SUPER privileges, so it can
execute queries that he/she does not have privileges on master.
This bug is fixed with the logic below:
- To replace '!' with ' ' in the magic comments which are not applied on
master. So they become common comments and will not be applied on slave.
- Example:
'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /*!99999 ,(3)*/
will be binlogged as
'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /* 99999 ,(3)*/
mysql-test/suite/rpl/t/rpl_conditional_comments.test:
Test the patch for this bug.
sql/mysql_priv.h:
Rename inBuf as rawBuf and remove the const limitation.
sql/sql_lex.cc:
To replace '!' with ' ' in the magic comments which are not applied on
master.
sql/sql_lex.h:
Remove the const limitation on parameter buff, as it can be modified in the function since
this patch.
Add member function yyUnput for Lex_input_stream. It set a character back the query buff.
sql/sql_parse.cc:
Rename inBuf as rawBuf and remove the const limitation.
sql/sql_partition.cc:
Remove the const limitation on parameter part_buff, as it can be modified in the function since
this patch.
sql/sql_partition.h:
Remove the const limitation on parameter part_buff, as it can be modified in the function since
this patch.
sql/table.h:
Remove the const limitation on variable partition_info, as it can be modified since
this patch.
/*![:version:] Query Code */, where [:version:] is a sequence of 5
digits representing the mysql server version(e.g /*!50200 ... */),
is a special comment that the query in it can be executed on those
servers whose versions are larger than the version appearing in the
comment. It leads to a security issue when slave's version is larger
than master's. A malicious user can improve his privileges on slaves.
Because slave SQL thread is running with SUPER privileges, so it can
execute queries that he/she does not have privileges on master.
This bug is fixed with the logic below:
- To replace '!' with ' ' in the magic comments which are not applied on
master. So they become common comments and will not be applied on slave.
- Example:
'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /*!99999 ,(3)*/
will be binlogged as
'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /* 99999 ,(3)*/
There were two problems that caused wrong results reported with this bug.
1. In some cases stored(persistent) virtual columns were not marked
in the write_set and in the vcol_set bitmaps.
2. If the list of fields in an insert command was empty then the values of
the stored virtual columns were set to default.
To fix the first problem the function st_table::mark_virtual_columns_for_write
was modified. Now the function has a parameter that says whether the virtual
columns are to be marked for insert or for update.
To fix the second problem a special handling of empty insert lists is
added in the function fill_record().
libmysqld/Makefile.am:
The new file added.
mysql-test/r/index_merge_myisam.result:
subquery_cache optimization option added.
mysql-test/r/myisam_mrr.result:
subquery_cache optimization option added.
mysql-test/r/subquery_cache.result:
The subquery cache tests added.
mysql-test/r/subselect3.result:
Subquery cache switched off to avoid changing read statistics.
mysql-test/r/subselect3_jcl6.result:
Subquery cache switched off to avoid changing read statistics.
mysql-test/r/subselect_no_mat.result:
subquery_cache optimization option added.
mysql-test/r/subselect_no_opts.result:
subquery_cache optimization option added.
mysql-test/r/subselect_no_semijoin.result:
subquery_cache optimization option added.
mysql-test/r/subselect_sj.result:
subquery_cache optimization option added.
mysql-test/r/subselect_sj_jcl6.result:
subquery_cache optimization option added.
mysql-test/t/subquery_cache.test:
The subquery cache tests added.
mysql-test/t/subselect3.test:
Subquery cache switched off to avoid changing read statistics.
sql/CMakeLists.txt:
The new file added.
sql/Makefile.am:
The new files added.
sql/item.cc:
Expression cache item (Item_cache_wrapper) added.
Item_ref and Item_field fixed for correct usage of result field and fast resolwing in SP.
sql/item.h:
Expression cache item (Item_cache_wrapper) added.
Item_ref and Item_field fixed for correct usage of result field and fast resolwing in SP.
sql/item_cmpfunc.cc:
Subquery cache added.
sql/item_cmpfunc.h:
Subquery cache added.
sql/item_subselect.cc:
Subquery cache added.
sql/item_subselect.h:
Subquery cache added.
sql/item_sum.cc:
Registration of subquery parameters added.
sql/mysql_priv.h:
subquery_cache optimization option added.
sql/mysqld.cc:
subquery_cache optimization option added.
sql/opt_range.cc:
Fix due to subquery cache.
sql/opt_subselect.cc:
Parameters of the function cahnged.
sql/procedure.h:
.h file guard added.
sql/sql_base.cc:
Registration of subquery parameters added.
sql/sql_class.cc:
Option to allow add indeces to temporary table.
sql/sql_class.h:
Item iterators added.
Option to allow add indeces to temporary table.
sql/sql_expression_cache.cc:
Expression cache for caching subqueries added.
sql/sql_expression_cache.h:
Expression cache for caching subqueries added.
sql/sql_lex.cc:
Registration of subquery parameters added.
sql/sql_lex.h:
Registration of subqueries and subquery parameters added.
sql/sql_select.cc:
Subquery cache added.
sql/sql_select.h:
Subquery cache added.
sql/sql_union.cc:
A new parameter to the function added.
sql/sql_update.cc:
A new parameter to the function added.
sql/table.cc:
Procedures to manage temporarty tables index added.
sql/table.h:
Procedures to manage temporarty tables index added.
storage/maria/ha_maria.cc:
Fix of handler to allow destoy a table in case of error during the table creation.
storage/maria/ha_maria.h:
.h file guard added.
storage/myisam/ha_myisam.cc:
Fix of handler to allow destoy a table in case of error during the table creation.
strict aliasing violations.
One somewhat major source of strict-aliasing violations and
related warnings is the SQL_LIST structure. For example,
consider its member function `link_in_list` which takes
a pointer to pointer of type T (any type) as a pointer to
pointer to unsigned char. Dereferencing this pointer, which
is done to reset the next field, violates strict-aliasing
rules and might cause problems for surrounding code that
uses the next field of the object being added to the list.
The solution is to use templates to parametrize the SQL_LIST
structure in order to deference the pointers with compatible
types. As a side bonus, it becomes possible to remove quite
a few casts related to acessing data members of SQL_LIST.
sql/handler.h:
Use the appropriate template type argument.
sql/item.cc:
Remove now-unnecessary cast.
sql/item_subselect.cc:
Remove now-unnecessary casts.
sql/item_sum.cc:
Use the appropriate template type argument.
Remove now-unnecessary cast.
sql/mysql_priv.h:
Move SQL_LIST structure to sql_list.h
Use the appropriate template type argument.
sql/sp.cc:
Remove now-unnecessary casts.
sql/sql_delete.cc:
Use the appropriate template type argument.
Remove now-unnecessary casts.
sql/sql_derived.cc:
Remove now-unnecessary casts.
sql/sql_lex.cc:
Remove now-unnecessary casts.
sql/sql_lex.h:
SQL_LIST now takes a template type argument which must
match the type of the elements of the list. Use forward
declaration when the type is not available, it is used
in pointers anyway.
sql/sql_list.h:
Rename SQL_LIST to SQL_I_List. The template parameter is
the type of object that is stored in the list.
sql/sql_olap.cc:
Remove now-unnecessary casts.
sql/sql_parse.cc:
Remove now-unnecessary casts.
sql/sql_prepare.cc:
Remove now-unnecessary casts.
sql/sql_select.cc:
Remove now-unnecessary casts.
sql/sql_show.cc:
Remove now-unnecessary casts.
sql/sql_table.cc:
Remove now-unnecessary casts.
sql/sql_trigger.cc:
Remove now-unnecessary casts.
sql/sql_union.cc:
Remove now-unnecessary casts.
sql/sql_update.cc:
Remove now-unnecessary casts.
sql/sql_view.cc:
Remove now-unnecessary casts.
sql/sql_yacc.yy:
Remove now-unnecessary casts.
storage/myisammrg/ha_myisammrg.cc:
Remove now-unnecessary casts.
strict aliasing violations.
One somewhat major source of strict-aliasing violations and
related warnings is the SQL_LIST structure. For example,
consider its member function `link_in_list` which takes
a pointer to pointer of type T (any type) as a pointer to
pointer to unsigned char. Dereferencing this pointer, which
is done to reset the next field, violates strict-aliasing
rules and might cause problems for surrounding code that
uses the next field of the object being added to the list.
The solution is to use templates to parametrize the SQL_LIST
structure in order to deference the pointers with compatible
types. As a side bonus, it becomes possible to remove quite
a few casts related to acessing data members of SQL_LIST.
Bug#46527 COMMIT AND CHAIN RELEASE does not make sense
Bug#53343 completion_type=1, COMMIT/ROLLBACK AND CHAIN don't
preserve the isolation level
Bug#53346 completion_type has strange effect in a stored
procedure/prepared statement
Added test cases to verify the expected behaviour of :
SET SESSION TRANSACTION ISOLATION LEVEL,
SET TRANSACTION ISOLATION LEVEL,
@@completion_type,
COMMIT AND CHAIN,
ROLLBACK AND CHAIN
..and some combinations of the above
Bug#46527 COMMIT AND CHAIN RELEASE does not make sense
Bug#53343 completion_type=1, COMMIT/ROLLBACK AND CHAIN don't
preserve the isolation level
Bug#53346 completion_type has strange effect in a stored
procedure/prepared statement
Added test cases to verify the expected behaviour of :
SET SESSION TRANSACTION ISOLATION LEVEL,
SET TRANSACTION ISOLATION LEVEL,
@@completion_type,
COMMIT AND CHAIN,
ROLLBACK AND CHAIN
..and some combinations of the above
Conflicts:
Text conflict in mysql-test/r/archive.result
Contents conflict in mysql-test/r/innodb_bug38231.result
Text conflict in mysql-test/r/mdl_sync.result
Text conflict in mysql-test/suite/binlog/t/disabled.def
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result
Text conflict in mysql-test/t/archive.test
Contents conflict in mysql-test/t/innodb_bug38231.test
Text conflict in mysql-test/t/mdl_sync.test
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_show.cc
Text conflict in sql/table.cc
Text conflict in sql/table.h
Conflicts:
Text conflict in mysql-test/r/archive.result
Contents conflict in mysql-test/r/innodb_bug38231.result
Text conflict in mysql-test/r/mdl_sync.result
Text conflict in mysql-test/suite/binlog/t/disabled.def
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result
Text conflict in mysql-test/t/archive.test
Contents conflict in mysql-test/t/innodb_bug38231.test
Text conflict in mysql-test/t/mdl_sync.test
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_show.cc
Text conflict in sql/table.cc
Text conflict in sql/table.h
transactional SELECT and ALTER TABLE ... REBUILD PARTITION".
The goal of this patch is to decouple type of metadata
lock acquired for table by open_tables() from type of
table-level lock to be acquired on it.
To achieve this we change approach to how we determine what
type of metadata lock should be acquired on table to be open.
Now instead of inferring it at open_tables() time from flags
and type of table-level lock we rely on that type of metadata
lock is properly set at parsing time and is not changed
further.
sql/ha_ndbcluster.cc:
Now one needs to properly initialize table list element's
MDL_request object before calling mysql_rm_table_part2().
sql/lock.cc:
lock_table_names() no longer initializes table list elements'
MDL_request objects. Now proper initialization of these
requests is a responsibility of the caller.
sql/lock.h:
Removed MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag which became
unnecessary. Thanks to the fact that we don't reset type of
requests for metadata locks between re-executions we now can
figure out that upgradable locks are requested by simply
looking at their type which were set in the parser. As result
this flag became redundant.
sql/mdl.h:
Added version of new operator which simplifies allocation of
MDL_request objects on a MEM_ROOT.
sql/sp_head.cc:
Added comment explaining why it is OK to infer type of
metadata lock to request from type of table-level lock
for prelocking.
Added enum_mdl_type argument to sp_add_to_query_tables()
to simplify its usage in trigger implementation.
sql/sp_head.h:
Added enum_mdl_type argument to sp_add_to_query_tables()
to simplify its usage in trigger implementation.
sql/sql_base.cc:
- open_table_get_mdl_lock():
Preserve type of MDL_request for table list element which
was set in the parser by creating MDL_request objects on
memory root if MYSQL_OPEN_FORCE_SHARED_MDL or
MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag were specified.
Thanks to this and to the fact that we no longer reset
type of requests for metadata locks between re-executions
we no longer need to acquire exclusive metadata lock on
table to be created in a special way. This lock is acquired
by code handling acquiring of upgradable locks.
Also changed signature/calling convention for this function
to simplify its usage.
- Accordingly special lock strategy for table list elements
which was used for such locks became unnecessary and was
removed. Other strategies were renamed.
- Since we no longer have guarantee that MDL_request object
which were not satisfied due to lock conflict belongs to
table list element Open_table_context class and its methods
were extended to remember pointer to MDL_request which has
caused problem at request_backoff_action() time and use it
in recover_from_failed_open(). Similar approach is used
for cases when problem from which we need to recover is
not related to MDL but to the table itself. In this case
we store pointer to the element of table list.
- Changed open_tables()/open_tables_check_upgradable_mdl()/
open_tables_acquire_upgradable_mdl() not to rely on
MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to understand when
upgradable metadata locks should be acquired and not to
infer type of MDL lock from type of table-level lock.
Instead we assume that type of MDL to be acquired was set
in the parser (we can do this as type of MDL_request is
no longer reset between re-executions).
sql/sql_class.h:
Since we no longer have guarantee that MDL_request object
which were not satisfied due to lock conflict belongs to
table list element Open_table_context class and its methods
were extended to remember pointer to MDL_request which has
caused problem at request_backoff_action() time and use it
in recover_from_failed_open(). Similar approach is used
for cases when problem from which we need to recover is
not related to MDL but to the table itself. In this case
we store pointer to the element of table list.
sql/sql_db.cc:
Now one needs to properly initialize table list element's
MDL_request object before calling mysql_rm_table_part2()
or mysql_rename_tables().
sql/sql_lex.cc:
st_select_lex/st_select_lex_node::add_table_to_list() method
now has argument which allows specify type of metadata lock
to be requested for table list element being added.
sql/sql_lex.h:
- st_select_lex/st_select_lex_node::add_table_to_list()
method now has argument which specifies type of metadata
lock to be requested for table list element being added.
This allows to explicitly set type of MDL lock to be
acquired for a DDL statement in parser. It is also more
future-proof than inferring type of MDL request from type
of table-level lock.
- Added Yacc_state::m_mdl_type member which specifies which
type of metadata lock should be requested for tables to be
added to table list by a grammar rule in cases when the same
rule is used in several statements requiring different kinds
of metadata locks.
sql/sql_parse.cc:
- st_select_lex::add_table_to_list() method now has argument
which specifies type of metadata lock to be requested for
table list element being added. This allows to explicitly
set type of MDL lock to be acquired for a DDL statement in
parser. It is also more future-proof than inferring type of
MDL request from type of table-level lock.
- EXCLUSIVE_DOWNGRADABLE_MDL lock strategy has a new name -
OTLS_DOWNGRADE_IF_EXISTS.
- Adjusted LOCK TABLES implementation to the fact that we no
longer infer type of metadata lock to be acquired from table
level lock and that type of MDL request is set at parsing.
And thus MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag became
unnecessary.
sql/sql_prepare.cc:
TABLE_LIST's lock strategy SHARED_MDL was renamed to OTLS_NONE
as now it means that metadata lock should not be changed during
call to open_table() (if it has been already acquired) and is
also used for exclusive metadata lock.
sql/sql_show.cc:
st_select_lex::add_table_to_list() method now has argument
which specifies type of metadata lock to be requested for
table list element being added.
sql/sql_table.cc:
- Adjusted mysql_admin_table()'s code to the fact that
open_tables() no longer determines what kind of metadata
lock should be obtained basing on type of table-level
lock and flags. Instead type of metadata lock for table
to be open should be set before calling open_tables().
- Changed mysql_alter_table() code to the facts:
a) that now it is responsibility of caller to properly
initalize MDL_request in table list elements before calling
lock_table_names()
b) and that MYSQL_OPEN_TAKE_UPGRADABLE_MDL is no longer
necessary since type of metadata lock to be obtained
at open_tables() time is set during parsing.
- Changed code of mysql_recreate_table() to properly set
type of metadata and table-level lock to be obtained
by mysql_alter_table() which it calls.
sql/sql_trigger.cc:
Instead of relying on MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to
force open_tables() to take an upgradable lock we now specify
exact type of lock to be taken when constructing table list
element for table to be open for CREATE/DROP TRIGGER.
sql/sql_view.cc:
We no longer use TABLE_LIST::EXCLUSIVE_MDL strategy to force
open_tables() to take an exclusive metadata lock on view to
be created. Instead we rely on parser setting proper type of
metadata lock to request and open_tables() acquiring it.
This became possible thanks to the fact that we no longer
reset type of MDL_request between statement re-executions.
sql/sql_yacc.yy:
Instead of inferring type of MDL_request for table to be
open from type of table-level lock and flags passed to
open_tables() we now explicitly specify them at parsing.
This became possible thanks to the fact that we no longer
reset type of MDL_request between statement re-executions.
In future this should allow to decouple type of metadata
lock from type of table-level lock.
The only exception to this approach is statements implemented
through mysql_admin_table() which re-uses same table list
element several times with different types of table-level
and metadata locks.
We now also properly initialize MDL_request objects for table
list elements which are later passed to lock_table_names()
function.
sql/table.cc:
Do not reset type of MDL_request between statement
re-executions. This became unnecessesary as we no longer
change type of MDL_request residing in table list element.
In its turn this change allows to set type of MDL_request
only once - at parsing time.
sql/table.h:
Got rid of TABLE_LIST::EXCLUSIVE_MDL lock strategy.
Now we can specify that we need to acquire exclusive lock
on table to be processed by open_tables() through setting
an appropriate type of MDL_request at parsing time (this
became possible thanks to the fact that we no longer reset
types of MDL_request's belonging to table list elements
between statement re-execution).
Strategy SHARED_MDL was renamed to OTLS_NONE as now it
means that metadata lock should not be changed during call
to open_table() (if it has been already acquired) and is
also used for exclusive metadata lock.
Strategy EXCLUSIVE_DOWNGRADABLE_MDL was renamed to
OTLS_DOWNGRADE_IF_EXISTS.
transactional SELECT and ALTER TABLE ... REBUILD PARTITION".
The goal of this patch is to decouple type of metadata
lock acquired for table by open_tables() from type of
table-level lock to be acquired on it.
To achieve this we change approach to how we determine what
type of metadata lock should be acquired on table to be open.
Now instead of inferring it at open_tables() time from flags
and type of table-level lock we rely on that type of metadata
lock is properly set at parsing time and is not changed
further.
Item_hex_string::Item_hex_string
The status of memory allocation in the Lex_input_stream (called
from the Parser_state constructor) was not checked which led to
a parser crash in case of the out-of-memory error.
The solution is to introduce new init() member function in
Parser_state and Lex_input_stream so that status of memory
allocation can be returned to the caller.
mysql-test/r/error_simulation.result:
Added a test case for bug #42064.
mysql-test/t/error_simulation.test:
Added a test case for bug #42064.
mysys/my_alloc.c:
Added error injection code for the regression test.
mysys/my_malloc.c:
Added error injection code for the regression test.
mysys/safemalloc.c:
Added error injection code for the regression test.
sql/event_data_objects.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/mysqld.cc:
Added error injection code for the regression test.
sql/sp.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_lex.cc:
Moved memory allocation from constructor to the separate init()
member function.
Added error injection code for the regression test.
sql/sql_lex.h:
Moved memory allocation from constructor to the separate init()
member function.
sql/sql_parse.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_partition.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_prepare.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_trigger.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_view.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures..
sql/thr_malloc.cc:
Added error injection code for the regression test.
Item_hex_string::Item_hex_string
The status of memory allocation in the Lex_input_stream (called
from the Parser_state constructor) was not checked which led to
a parser crash in case of the out-of-memory error.
The solution is to introduce new init() member function in
Parser_state and Lex_input_stream so that status of memory
allocation can be returned to the caller.
