STATUS OF ROLLBACKED TRANSACTION" and bug #17054007 - "TRANSACTION
IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK".
The problem in the first bug report was that although deadlock involving
metadata locks was reported using the same error code and message as InnoDB
deadlock it didn't rollback transaction like the latter. This caused
confusion to users as in some cases after ER_LOCK_DEADLOCK transaction
could have been restarted immediately and in some cases rollback was
required.
The problem in the second bug report was that although InnoDB deadlock
caused transaction rollback in all storage engines it didn't cause release
of metadata locks. So concurrent DDL on the tables used in transaction was
blocked until implicit or explicit COMMIT or ROLLBACK was issued in the
connection which got InnoDB deadlock.
The former issue has stemmed from the fact that when support for detection
and reporting metadata locks deadlocks was added we erroneously assumed
that InnoDB doesn't rollback transaction on deadlock but only last statement
(while this is what happens on InnoDB lock timeout actually) and so didn't
implement rollback of transactions on MDL deadlocks.
The latter issue was caused by the fact that rollback of transaction due
to deadlock is carried out by setting THD::transaction_rollback_request
flag at the point where deadlock is detected and performing rollback
inside of trans_rollback_stmt() call when this flag is set. And
trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are
released.
This patch solves these two problems in the following way:
- In case when MDL deadlock is detect transaction rollback is requested
by setting THD::transaction_rollback_request flag.
- Code performing rollback of transaction if THD::transaction_rollback_request
is moved out from trans_rollback_stmt(). Now we handle rollback request
on the same level as we call trans_rollback_stmt() and release statement/
transaction MDL locks.
IN TIME RECOVERY FAILURE ON SLAVES
Problem:
DROP TEMP TABLE IF EXISTS commands can cause point
in time recovery (re-applying binlog) failures.
Analyses:
In RBR, 'DROP TEMPORARY TABLE' commands are
always binlogged by adding 'IF EXISTS' clauses.
Also, the slave SQL thread will not check replicate.* filter
rules for "DROP TEMPORARY TABLE IF EXISTS" queries.
If log-slave-updates is enabled on slave, these queries
will be binlogged in the format of "USE `db`;
DROP TEMPORARY TABLE IF EXISTS `t1`;" irrespective
of filtering rules and irrespective of the `db` existence.
When users try to recover slave from it's own binlog,
use `db` command might fail if `db` is not present on slave.
Fix:
At the time of writing the 'DROP TEMPORARY TABLE
IF EXISTS' query into the binlog, 'use `db`' will not be
present and the table name in the query will be a fully
qualified table name.
Eg:
'USE `db`; DROP TEMPORARY TABLE IF EXISTS `t1`;'
will be logged as
'DROP TEMPORARY TABLE IF EXISTS `db`.`t1`;'.
IN TIME RECOVERY FAILURE ON SLAVES
Problem:
DROP TEMP TABLE IF EXISTS commands can cause point
in time recovery (re-applying binlog) failures.
Analyses:
In RBR, 'DROP TEMPORARY TABLE' commands are
always binlogged by adding 'IF EXISTS' clauses.
Also, the slave SQL thread will not check replicate.* filter
rules for "DROP TEMPORARY TABLE IF EXISTS" queries.
If log-slave-updates is enabled on slave, these queries
will be binlogged in the format of "USE `db`;
DROP TEMPORARY TABLE IF EXISTS `t1`;" irrespective
of filtering rules and irrespective of the `db` existence.
When users try to recover slave from it's own binlog,
use `db` command might fail if `db` is not present on slave.
Fix:
At the time of writing the 'DROP TEMPORARY TABLE
IF EXISTS' query into the binlog, 'use `db`' will not be
present and the table name in the query will be a fully
qualified table name.
Eg:
'USE `db`; DROP TEMPORARY TABLE IF EXISTS `t1`;'
will be logged as
'DROP TEMPORARY TABLE IF EXISTS `db`.`t1`;'.
CAN LEAD TO MISSING TABLES
Overview
--------
If the FOREIGN_KEY_CHECKS system variable is set to 0, it is
possible to break a foreign key constraint by changing the type
or character set of the foreign key column, or by dropping the
foreign key index (without carrying out corresponding changes on
another table in the relationship).
If we subsequently set FOREIGN_KEY_CHECKS to 1 and execute ALTER
TABLE involving the COPY algorithm on such a table, the following
happens:
1) If ALTER TABLE does not contain a RENAME clause, the attempt
to install the new version of the table instead of the old one
will fail due to the fact that the inconsistency will be
detected. An attempt to revert the partially executed alter
table operation by restoring the old table definition will
fail as well due to FOREIGN_KEY_CHECKS == 1. As a result, the
table being altered will be lost.
2) If ALTER TABLE contains the RENAME clause, the inconsistency
will not be detected (most probably due to other bugs). But if
an attempt to install the new version of the table fails (for
example, due to a failure when updating triggers associated
with the table), reverting the partially executed alter table
by restoring the old table definition will fail too. So the
table being altered might be lost as well.
Suggested fix
-------------
The suggested fix is to temporarily unset the option bit
representing FOREIGN_KEY_CHECKS when the old table definition is
restored while reverting the partially executed operation.
CAN LEAD TO MISSING TABLES
Overview
--------
If the FOREIGN_KEY_CHECKS system variable is set to 0, it is
possible to break a foreign key constraint by changing the type
or character set of the foreign key column, or by dropping the
foreign key index (without carrying out corresponding changes on
another table in the relationship).
If we subsequently set FOREIGN_KEY_CHECKS to 1 and execute ALTER
TABLE involving the COPY algorithm on such a table, the following
happens:
1) If ALTER TABLE does not contain a RENAME clause, the attempt
to install the new version of the table instead of the old one
will fail due to the fact that the inconsistency will be
detected. An attempt to revert the partially executed alter
table operation by restoring the old table definition will
fail as well due to FOREIGN_KEY_CHECKS == 1. As a result, the
table being altered will be lost.
2) If ALTER TABLE contains the RENAME clause, the inconsistency
will not be detected (most probably due to other bugs). But if
an attempt to install the new version of the table fails (for
example, due to a failure when updating triggers associated
with the table), reverting the partially executed alter table
by restoring the old table definition will fail too. So the
table being altered might be lost as well.
Suggested fix
-------------
The suggested fix is to temporarily unset the option bit
representing FOREIGN_KEY_CHECKS when the old table definition is
restored while reverting the partially executed operation.
bzr merge lp:maria/5.5 -rtag:mariadb-5.5.31
Text conflict in cmake/cpack_rpm.cmake
Text conflict in debian/dist/Debian/control
Text conflict in debian/dist/Ubuntu/control
Text conflict in sql/CMakeLists.txt
Conflict adding file sql/db.opt. Moved existing file to sql/db.opt.moved.
Conflict adding file sql/db.opt.moved. Moved existing file to sql/db.opt.moved.moved.
Text conflict in sql/mysqld.cc
Text conflict in support-files/mysql.spec.sh
8 conflicts encountered.
SHOW ENGINE INNOD
Problem:
The purpose of explain_filename() is to provide useful additional
information regarding the partitions given the filename. This function
was returning an error when it was not able to parse the given filename.
For example, within InnoDB, temporary files are created with #sql-
prefix. But this function was not able to parse it correctly.
Solution:
It is not an error, if explain_filename() could not parse the given
filename. If there is no partition information to explain, then silently
return from the function.
rb#1940 approved by mattiasj
SHOW ENGINE INNOD
Problem:
The purpose of explain_filename() is to provide useful additional
information regarding the partitions given the filename. This function
was returning an error when it was not able to parse the given filename.
For example, within InnoDB, temporary files are created with #sql-
prefix. But this function was not able to parse it correctly.
Solution:
It is not an error, if explain_filename() could not parse the given
filename. If there is no partition information to explain, then silently
return from the function.
rb#1940 approved by mattiasj
sql/sql_table.cc:
Don't call allow_access_to_protected_table() if we haven't protected table against usage.
Table is mainly protected against usage when one disables keys with alter table.
sql/sql_table.cc:
Remove version protection from share when repair has been done.
Without this one can't run SHOW commands on the table if it was locked until it's unlocked.
sql/table.h:
Allow one to remove version protection with allow_access_to_protected_table()
The issue was that there was that SHOW commands could open the table in the store engine, even in cases
where it should not be allowed to do that (ie, the storage engines meta data for that table was under big changes).
The cases where this should not be allowed are:
- ALTER TABLE DISABLE KEYS
- ALTER TABLE ENABLE KEYS
- REPAIR TABLE
- OPTIMIZE TABLE
- DROP TABLE
This patch adds a new mode, protected_against_usage(). If this is used then the SHOW command will wait until the table
is accessable. This is implemented by re-using the already exising 'version' flag for TABLE_SHARE.
It also added functions to be used to change TABLE_SHARE->version instead of changing it directly.
mysql-test/r/myisam-metadata.result:
Added test case
mysql-test/t/myisam-metadata.test:
Added test case
sql/mysqld.cc:
Start from refresh_version 2 as 0 and 1 are reserved.
sql/sql_admin.cc:
Added MYSQL_OPEN_FOR_REPAIR
Updated call to wait_while_table_is_used()
sql/sql_base.cc:
Updated call to wait_while_table_is_used()
- Allow one to specify how the table should be removed (for all commands except show or for all commands).
- Don't allow one to reopen the table if one has called share->protect_against_usage()
sql/sql_base.h:
Added TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE, which is used to mark that no one can reopen this table, except with MYSQL_OPEN_FOR_REPAIR .
- Added MYSQL_OPEN_FOR_REPAIR
- Updated prototype for wait_while_table_is_used()
sql/sql_table.cc:
Updated call to wait_while_table_is_used()
Use MYSQL_OPEN_FOR_REPAIR for open tables that where repaired.
sql/sql_truncate.cc:
Updated call to wait_while_table_is_used()
sql/table.cc:
Use set_refresh_version()
sql/table.h:
Added functions to be used to change TABLE_SHARE->version instead of changing it directly
mysys/errors.c:
revert upstream's fix. use a much simpler one
mysys/my_write.c:
revert upstream's fix. use a simpler one
sql/item_xmlfunc.cc:
useless, but ok
sql/mysqld.cc:
simplify upstream's fix
storage/heap/hp_delete.c:
remove upstream's fix.
we'll use a much less expensive approach.
DOWNGRADED FROM 5.6.11 TO 5.6.10
Problem was new syntax not accepted by previous version.
Fixed by adding version comment of /*!50531 around the
new syntax.
Like this in the .frm file:
'PARTITION BY KEY /*!50611 ALGORITHM = 2 */ () PARTITIONS 3'
and also changing the output from SHOW CREATE TABLE to:
CREATE TABLE t1 (a INT)
/*!50100 PARTITION BY KEY */ /*!50611 ALGORITHM = 1 */ /*!50100 ()
PARTITIONS 3 */
It will always add the ALGORITHM into the .frm for KEY [sub]partitioned
tables, but for SHOW CREATE TABLE it will only add it in case it is the non
default ALGORITHM = 1.
Also notice that for 5.5, it will say /*!50531 instead of /*!50611, which
will make upgrade from 5.5 > 5.5.31 to 5.6 < 5.6.11 fail!
If one downgrades an fixed version to the same major version (5.5 or 5.6) the
bug 14521864 will be visible again, but unless the .frm is updated, it will
work again when upgrading again.
Also fixed so that the .frm does not get updated version
if a single partition check passes.
DOWNGRADED FROM 5.6.11 TO 5.6.10
Problem was new syntax not accepted by previous version.
Fixed by adding version comment of /*!50531 around the
new syntax.
Like this in the .frm file:
'PARTITION BY KEY /*!50611 ALGORITHM = 2 */ () PARTITIONS 3'
and also changing the output from SHOW CREATE TABLE to:
CREATE TABLE t1 (a INT)
/*!50100 PARTITION BY KEY */ /*!50611 ALGORITHM = 1 */ /*!50100 ()
PARTITIONS 3 */
It will always add the ALGORITHM into the .frm for KEY [sub]partitioned
tables, but for SHOW CREATE TABLE it will only add it in case it is the non
default ALGORITHM = 1.
Also notice that for 5.5, it will say /*!50531 instead of /*!50611, which
will make upgrade from 5.5 > 5.5.31 to 5.6 < 5.6.11 fail!
If one downgrades an fixed version to the same major version (5.5 or 5.6) the
bug 14521864 will be visible again, but unless the .frm is updated, it will
work again when upgrading again.
Also fixed so that the .frm does not get updated version
if a single partition check passes.
mysql-test/r/create.result:
Updated test results
mysql-test/t/create.test:
Updated test
sql/sql_base.cc:
Use push_internal_handler/pop_internal_handler to avoid errors & warnings instead of clear_error
Give a warnings instead of an error for CREATE TABLE IF EXISTS
sql/sql_parse.cc:
Check if we failed because of table exists (can only happen from create)
sql/sql_table.cc:
Check if we failed because of table exists (can only happen from create)
If triggers are used for an insert/update/delete statement than the values of
all virtual columns must be computed as any of them may be used by the triggers.
PROBLEM
-------
optimize on partiton will recreate the whole table
instead of just partition.
ANALYSIS
--------
At present innodb doesn't support optimize option ,so we do a rebuild of the
whole table and then call analyze() on the table.Presently for any optimize()
option (on table or partition) we display the following info to the user
"Table does not support optimize, doing recreate + analyze instead".
FIX
---
It was decided for GA versions(5.1 and 5.5) whenever the user tries to
optimize a partition(s) we will will display the following info the user
"Table does not support optimize on partitions.
All partitions will be rebuilt and analyzed."
Earlier partitions were not analyzed.Now all partitions will be analyzed.
If the user wants to optimize the whole table ,we will display the
previous info to the user. i.e
"Table does not support optimize, doing recreate + analyze instead"
For 5.6+ versions we will raise a new bug to support optimize() options
in innodb.
PROBLEM
-------
optimize on partiton will recreate the whole table
instead of just partition.
ANALYSIS
--------
At present innodb doesn't support optimize option ,so we do a rebuild of the
whole table and then call analyze() on the table.Presently for any optimize()
option (on table or partition) we display the following info to the user
"Table does not support optimize, doing recreate + analyze instead".
FIX
---
It was decided for GA versions(5.1 and 5.5) whenever the user tries to
optimize a partition(s) we will will display the following info the user
"Table does not support optimize on partitions.
All partitions will be rebuilt and analyzed."
Earlier partitions were not analyzed.Now all partitions will be analyzed.
If the user wants to optimize the whole table ,we will display the
previous info to the user. i.e
"Table does not support optimize, doing recreate + analyze instead"
For 5.6+ versions we will raise a new bug to support optimize() options
in innodb.
FAILED IN DEACTIVATE_DDL_LOG_ENTRY
deallocate_ddl_log_entry() can be called without having
locked LOCK_gdl. It uses a global buffer for reading and
writing entries in the ddl_log, and since it is not protected
by any mutex, two concurrent threads can overwrite the
content in the global buffer, so it can be different from
what was read.
Thread a reads from entry 1 into global
buffer, thread b reads from entry 2 into global buffer,
thread a writes from global buffer into entry 1
-> entry 1 is not the content of entry 2.
This is especially bad for replace entries, which uses
two phases, and does not deactivate the whole entry
after the first phase, but increases the phase instead.
Fixed by using thread local storage (stack) instead of global
storage (global buffer).
Also added buffer and size arguments to
read/write_ddl_log_file_entry.
Also only read/write first bytes in entries in
deactivate_ddl_log_entry.
Also fixed the scenario where it will try to recover from a server
compiled with a different value of IO_SIZE (very uncommon!)
updated patch with set_ddl_log_entry_from_buf
and removed read_ddl_log_entry.
Manually tested, no test case included.
FAILED IN DEACTIVATE_DDL_LOG_ENTRY
deallocate_ddl_log_entry() can be called without having
locked LOCK_gdl. It uses a global buffer for reading and
writing entries in the ddl_log, and since it is not protected
by any mutex, two concurrent threads can overwrite the
content in the global buffer, so it can be different from
what was read.
Thread a reads from entry 1 into global
buffer, thread b reads from entry 2 into global buffer,
thread a writes from global buffer into entry 1
-> entry 1 is not the content of entry 2.
This is especially bad for replace entries, which uses
two phases, and does not deactivate the whole entry
after the first phase, but increases the phase instead.
Fixed by using thread local storage (stack) instead of global
storage (global buffer).
Also added buffer and size arguments to
read/write_ddl_log_file_entry.
Also only read/write first bytes in entries in
deactivate_ddl_log_entry.
Also fixed the scenario where it will try to recover from a server
compiled with a different value of IO_SIZE (very uncommon!)
updated patch with set_ddl_log_entry_from_buf
and removed read_ddl_log_entry.
Manually tested, no test case included.
