argument of inline_mysql_mutex_init in sql_base.cc.
When initializing LOCK_dd_owns_lock_open mutex pass
correct PSI key instead of NULL value.
mysql-test/suite/perfschema/r/dml_setup_instruments.result:
Updated test results after adding P_S instrumentation
for LOCK_dd_owns_lock_open.
sql/sql_base.cc:
When initializing LOCK_dd_owns_lock_open mutex pass
correct PSI key instead of NULL value.
Fix some bugs where we stored values other than 0 or 1 in my_bool
Fixed some compiler warnings
client/mysql.cc:
Changed interrupted_query from my_bool to int, as we stored 2 in it.
client/mysqladmin.cc:
Changed return variable type to same type as function value type
client/mysqltest.cc:
Changed 'found' to int as we store other values than 0 or 1 into it
Changed type for parameter of set_reconnect() to match usage.
extra/libevent/evbuffer.c:
Added __attribute__((unused))
extra/libevent/event.c:
Added __attribute__((unused))
extra/libevent/signal.c:
Added __attribute__((unused))
sql/event_data_objects.h:
my_bool -> bool
sql/event_db_repository.cc:
my_bool -> bool
sql/event_db_repository.h:
my_bool -> bool
sql/event_parse_data.h:
my_bool -> bool
sql/events.cc:
my_bool -> bool
sql/events.h:
my_bool -> bool
sql/field.cc:
my_bool -> bool
sql/field.h:
my_bool -> bool
sql/hash_filo.h:
my_bool -> bool
sql/item.cc:
my_bool -> bool
sql/item.h:
my_bool -> bool
sql/item_cmpfunc.h:
my_bool -> bool
Changed result_for_null_param from my_bool to int as we stored -1 in it.
sql/item_func.cc:
my_bool -> bool
Modified udf wrapper functions so that the UDF functions would continue to use my_bool. (To keep compatibility with UDF:s)
sql/item_func.h:
my_bool -> bool
sql/item_subselect.h:
my_bool -> bool
sql/item_sum.cc:
Modified udf wrapper functions so that the UDF functions would continue to use my_bool. (To keep compatibility with UDF:s)
sql/parse_file.h:
my_bool -> bool
sql/rpl_mi.h:
my_bool -> bool
sql/sp_rcontext.h:
my_bool -> bool
sql/sql_analyse.h:
my_bool -> bool
sql/sql_base.cc:
Change some assignments so that we don't initialize bool variables with int's.
sql/sql_bitmap.h:
my_bool -> bool
sql/sql_cache.cc:
my_bool -> bool
sql/sql_cache.h:
my_bool -> bool
sql/sql_class.h:
my_bool -> bool
sql/sql_insert.cc:
Change some assignments so that we don't initialize bool variables with int's.
sql/sql_prepare.cc:
my_bool -> bool
sql/table.h:
my_bool -> bool
storage/maria/ma_check.c:
Removed duplicate assignment
strings/decimal.c:
Fixed wrong variable usage.
Don't do complex arithmetic on bool when simple works.
tables".
Attempting to issue an INSERT DELAYED statement for a MERGE
table might have caused a deadlock if it happened as part of
a transaction or under LOCK TABLES, and there was a concurrent
DDL or LOCK TABLES ... WRITE statement which tried to lock one
of its underlying tables.
The problem occurred when a delayed insert handler thread tried
to open a MERGE table and discovered that to do this it had also
to open all underlying tables and hence acquire metadata
locks on them. Since metadata locks on the underlying tables were
not pre-acquired by the connection thread executing INSERT DELAYED,
attempts to do so might lead to waiting. In this case the
connection thread had to wait for the delayed insert thread.
If the thread which was preventing the lock on the underlying table
from being acquired had to wait for the connection thread (due to
this or other metadata locks), a deadlock occurred.
This deadlock was not detected by the MDL deadlock detector since
waiting for the handler thread by the connection thread is not
represented in the wait-for graph.
This patch solves the problem by ensuring that the delayed
insert handler thread never tries to open underlying tables
of a MERGE table. Instead open_tables() is aborted right after
the parent table is opened and a ER_DELAYED_NOT_SUPPORTED
error is emitted (which is passed to the connection thread and
ultimately to the user).
mysql-test/r/merge.result:
Added test for bug #56251 "Deadlock with INSERT DELAYED and
MERGE tables".
mysql-test/t/merge.test:
Added test for bug #56251 "Deadlock with INSERT DELAYED and
MERGE tables".
sql/sql_base.cc:
Changed open_n_lock_single_table() to take prelocking strategy
as an argument instead of always using DML_prelocking_strategy.
sql/sql_base.h:
Changed open_n_lock_single_table() to take prelocking strategy
as an argument instead of always using DML_prelocking_strategy.
Added a version of this function which is compatible with old
signature.
sql/sql_insert.cc:
When opening MERGE table in delayed insert thread stop and emit
ER_DELAYED_NOT_SUPPORTED right after opening main table and
before opening underlying tables. This ensures that we won't
try to acquire metadata lock on underlying tables which might
lead to a deadlock.
This is achieved by using special prelocking strategy which
abort open_tables() process as soon as we discover that we
have opened table with engine which doesn't support delayed
inserts.
tables".
Attempting to issue an INSERT DELAYED statement for a MERGE
table might have caused a deadlock if it happened as part of
a transaction or under LOCK TABLES, and there was a concurrent
DDL or LOCK TABLES ... WRITE statement which tried to lock one
of its underlying tables.
The problem occurred when a delayed insert handler thread tried
to open a MERGE table and discovered that to do this it had also
to open all underlying tables and hence acquire metadata
locks on them. Since metadata locks on the underlying tables were
not pre-acquired by the connection thread executing INSERT DELAYED,
attempts to do so might lead to waiting. In this case the
connection thread had to wait for the delayed insert thread.
If the thread which was preventing the lock on the underlying table
from being acquired had to wait for the connection thread (due to
this or other metadata locks), a deadlock occurred.
This deadlock was not detected by the MDL deadlock detector since
waiting for the handler thread by the connection thread is not
represented in the wait-for graph.
This patch solves the problem by ensuring that the delayed
insert handler thread never tries to open underlying tables
of a MERGE table. Instead open_tables() is aborted right after
the parent table is opened and a ER_DELAYED_NOT_SUPPORTED
error is emitted (which is passed to the connection thread and
ultimately to the user).
table causes assert failure".
Attempting to use FLUSH TABLE table_list WITH READ LOCK
statement for a MERGE table led to an assertion failure if
one of its children was not present in the list of tables
to be flushed. The problem was not visible in non-debug builds.
The assertion failure was caused by the fact that in such
situations FLUSH TABLES table_list WITH READ LOCK implementation
tried to use (e.g. lock) such child tables without acquiring
metadata lock on them. This happened because when opening tables
we assumed metadata locks on all tables were already acquired
earlier during statement execution and a such assumption was
false for MERGE children.
This patch fixes the problem by ensuring at open_tables() time
that we try to acquire metadata locks on all tables to be opened.
For normal tables such requests are satisfied instantly since
locks are already acquired for them. For MERGE children metadata
locks are acquired in normal fashion.
Note that FLUSH TABLES merge_table WITH READ LOCK will lock for
read both the MERGE table and its children but will flush only
the MERGE table. To flush children one has to mention them in table
list explicitly. This is expected behavior and it is consistent with
usage patterns for this statement (e.g. in mysqlhotcopy script).
mysql-test/r/flush.result:
Added test case for bug #55273 "FLUSH TABLE tm WITH READ LOCK
for Merge table causes assert failure".
mysql-test/t/flush.test:
Added test case for bug #55273 "FLUSH TABLE tm WITH READ LOCK
for Merge table causes assert failure".
sql/sql_base.cc:
Changed lock_table_names() to support newly introduced
MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag.
sql/sql_base.h:
Introduced MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag for
open_tables() and lock_table_names() which allows to skip
acquiring of global and schema-scope locks when SNW, SNRW or
X metadata locks are acquired.
sql/sql_reload.cc:
Changed "FLUSH TABLES table_list WITH READ LOCK" code not to
cause assert about missing metadata locks when MERGE table is
flushed without one of its underlying tables.
To achieve this we no longer call open_and_lock_tables() with
MYSQL_OPEN_HAS_MDL_LOCK flag so this function automatically
acquires metadata locks on MERGE children if such lock has
not been already acquired at earlier stage. Instead we call
this function with MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag to
suppress acquiring of global IX lock in order to keep FLUSH
TABLES table_list WITH READ LOCK compatible with FLUSH TABLE
WITH READ LOCK.
Also changed implementation to use lock_table_names() function
for pre-acquiring of metadata locks instead of custom code.
To implement this change moved setting of open_type member for
table list elements to parser.
sql/sql_yacc.yy:
Now we set acceptable type of table for FLUSH TABLES table_list
WITH READ LOCK at parsing time instead of execution time.
table causes assert failure".
Attempting to use FLUSH TABLE table_list WITH READ LOCK
statement for a MERGE table led to an assertion failure if
one of its children was not present in the list of tables
to be flushed. The problem was not visible in non-debug builds.
The assertion failure was caused by the fact that in such
situations FLUSH TABLES table_list WITH READ LOCK implementation
tried to use (e.g. lock) such child tables without acquiring
metadata lock on them. This happened because when opening tables
we assumed metadata locks on all tables were already acquired
earlier during statement execution and a such assumption was
false for MERGE children.
This patch fixes the problem by ensuring at open_tables() time
that we try to acquire metadata locks on all tables to be opened.
For normal tables such requests are satisfied instantly since
locks are already acquired for them. For MERGE children metadata
locks are acquired in normal fashion.
Note that FLUSH TABLES merge_table WITH READ LOCK will lock for
read both the MERGE table and its children but will flush only
the MERGE table. To flush children one has to mention them in table
list explicitly. This is expected behavior and it is consistent with
usage patterns for this statement (e.g. in mysqlhotcopy script).
MDL deadlock detector".
Deadlock could have occurred when workload containing mix
of DML, DDL and FLUSH TABLES statements affecting same
set of tables was executed in heavily concurrent environment.
This deadlock occurred when several connections tried to
perform deadlock detection in metadata locking subsystem.
The first connection started traversing wait-for graph,
encountered sub-graph representing wait for flush, acquired
LOCK_open and dived into sub-graph inspection. When it has
encounterd sub-graph corresponding to wait for metadata lock
and blocked while trying to acquire rd-lock on
MDL_lock::m_rwlock (*) protecting this subgraph, since some
other thread had wr-lock on it. When this wr-lock was released
it could have happened (if there was other pending wr-lock
against this rwlock) that rd-lock from the first connection
was left unsatisfied but at the same time new rd-lock request
from the second connection sneaked in and was satisfied (for
this to be possible second rd- request should come exactly
after wr-lock is released but before pending wr-lock manages
to grab rwlock, which is possible both on Linux and in our
own rwlock implementation). If this second connection
continued traversing wait-for graph and encountered sub-graph
representing wait for flush it tried to acquire LOCK_open
and thus deadlock was created.
This patch tries to workaround this problem but not allowing
deadlock detector to lock LOCK_open mutex if some other thread
doing deadlock detection already owns it and current search
depth is greater than 0. Instead deadlock is reported.
Other possible solutions are either known to have negative
effects on performance or require much more time for proper
implementation and testing.
No test case is provided as this bug is very hard to repeat
in MTR environment but is repeatable with the help of RQG
tests.
sql/mdl.cc:
Moved Deadlock_detection_visitor::m_current_search_depth to
parent class to make it available in
TABLE_SHARE::visit_subgraph().
Added MDL_wait_for_graph_visitor::abort_traversal() method
which allows to abort traversal of a wait-for graph and
report a deadlock.
sql/mdl.h:
Moved Deadlock_detection_visitor::m_current_search_depth to
parent class to make it available in
TABLE_SHARE::visit_subgraph().
Added MDL_wait_for_graph_visitor::abort_traversal() method
which allows to abort traversal of a wait-for graph and
report a deadlock.
sql/sql_base.cc:
Added dd_owns_lock_open counter and mutex protecting it to
track number of connections which do deadlock detection and
own or try to acquire LOCK_open.
sql/sql_base.h:
Added dd_owns_lock_open counter and mutex protecting it to
track number of connections which do deadlock detection and
own or try to acquire LOCK_open.
sql/table.cc:
Workaround bug #56405 but not allowing MDL deadlock detector
to lock LOCK_open mutex if some other thread doing deadlock
detection already owns it and current search depth is greater
than 0. Instead report deadlock.
MDL deadlock detector".
Deadlock could have occurred when workload containing mix
of DML, DDL and FLUSH TABLES statements affecting same
set of tables was executed in heavily concurrent environment.
This deadlock occurred when several connections tried to
perform deadlock detection in metadata locking subsystem.
The first connection started traversing wait-for graph,
encountered sub-graph representing wait for flush, acquired
LOCK_open and dived into sub-graph inspection. When it has
encounterd sub-graph corresponding to wait for metadata lock
and blocked while trying to acquire rd-lock on
MDL_lock::m_rwlock (*) protecting this subgraph, since some
other thread had wr-lock on it. When this wr-lock was released
it could have happened (if there was other pending wr-lock
against this rwlock) that rd-lock from the first connection
was left unsatisfied but at the same time new rd-lock request
from the second connection sneaked in and was satisfied (for
this to be possible second rd- request should come exactly
after wr-lock is released but before pending wr-lock manages
to grab rwlock, which is possible both on Linux and in our
own rwlock implementation). If this second connection
continued traversing wait-for graph and encountered sub-graph
representing wait for flush it tried to acquire LOCK_open
and thus deadlock was created.
This patch tries to workaround this problem but not allowing
deadlock detector to lock LOCK_open mutex if some other thread
doing deadlock detection already owns it and current search
depth is greater than 0. Instead deadlock is reported.
Other possible solutions are either known to have negative
effects on performance or require much more time for proper
implementation and testing.
No test case is provided as this bug is very hard to repeat
in MTR environment but is repeatable with the help of RQG
tests.
Fall back to use ALTER TABLE for engines that doesn't support REPAIR when doing repair for upgrade.
Nicer output from mysql_upgrade and mysql_check
Updated all arrays that used NAME_LEN to use SAFE_NAME_LEN to ensure that we don't break things accidently as names can now have a #mysql50# prefix.
client/mysql_upgrade.c:
If we are using verbose, also run mysqlcheck in verbose mode.
client/mysqlcheck.c:
Add more information if running in verbose mode
Print 'Needs upgrade' instead of complex error if table needs to be upgraded
Don't write connect information if verbose is not 2 or above
mysql-test/r/drop.result:
Updated test and results as we now support full table names
mysql-test/r/grant.result:
Now you get a correct error message if using #mysql with paths
mysql-test/r/show_check.result:
Update results as table names can temporarly be bigger than NAME_LEN (during upgrade)
mysql-test/r/upgrade.result:
Test upgrade for long table names.
mysql-test/suite/funcs_1/r/is_tables_is.result:
Updated old test result (had note been updated in a while)
mysql-test/t/drop.test:
Updated test and results as we now support full table names
mysql-test/t/grant.test:
Now you get a correct error message if using #mysql with paths
mysql-test/t/upgrade.test:
Test upgrade for long table names.
sql/ha_partition.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/item.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/log_event.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/mysql_priv.h:
Added SAFE_NAME_LEN
sql/rpl_filter.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sp.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sp_head.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_acl.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_base.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_connect.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_parse.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_prepare.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_select.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_show.cc:
NAME_LEN -> SAFE_NAME_LEN
Enlarge table names for SHOW TABLES to also include optional #mysql50#
sql/sql_table.cc:
Fall back to use ALTER TABLE for engines that doesn't support REPAIR when doing repair for upgrade.
sql/sql_trigger.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_udf.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_view.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/table.cc:
Fixed check_table_name() to not count #mysql50# as part of name
If #mysql50# is part of the name, don't allow path characters in name.
to allow temp table operations) -- prerequisite patch #3.
Rename open_temporary_table() to open_table_uncached().
open_temporary_table() will be introduced in following patches
to open temporary tables for a statement.
to allow temp table operations) -- prerequisite patch #3.
Rename open_temporary_table() to open_table_uncached().
open_temporary_table() will be introduced in following patches
to open temporary tables for a statement.
to allow temp table operations) -- prerequisite patch #2.
Introduce a new form of find_temporary_table() function:
find_temporary_table() by a table key. It will be used
in further patches.
Replace find_temporary_table(table_list->db, table_list->name)
by more appropiate find_temporary_table(table_list) across
the codebase.
to allow temp table operations) -- prerequisite patch #2.
Introduce a new form of find_temporary_table() function:
find_temporary_table() by a table key. It will be used
in further patches.
Replace find_temporary_table(table_list->db, table_list->name)
by more appropiate find_temporary_table(table_list) across
the codebase.
- Changed to still use bcmp() in certain cases becasue
- Faster for short unaligneed strings than memcmp()
- Bettern when using valgrind
- Changed to use my_sprintf() instead of sprintf() to get higher portability for old systems
- Changed code to use MariaDB version of select->skip_record()
- Removed -%::SCCS/s.% from Makefile.am:s to remove automake warnings
to allow temp table operations) -- prerequisite patch #1.
Move a piece of code that initialiazes TABLE instance
after it was successfully opened into a separate function.
This function will be reused in the following patches.
to allow temp table operations) -- prerequisite patch #1.
Move a piece of code that initialiazes TABLE instance
after it was successfully opened into a separate function.
This function will be reused in the following patches.
on CREATE TABLE .. SELECT I_S.PART
This assert was triggered if an InnoDB table was created using
CREATE TABLE ... AS SELECT where the query used an I_S table, and
a view existed in the database. It would also be triggered for
any statement changing an InnoDB table (e.g. INSERT, UPDATE, DELETE)
which had a subquery referencing an I_S table.
The assert was triggered if open_normal_and_derived_tables() failed
and a statement transaction had been started. This will usually not
happen as tables are opened before a statement transaction is started.
However, e.g. CREATE TABLE ... AS SELECT starts a transaction in order
to insert tuples into the new table. And if the subquery references
an I_S table, all current tables and views can be opened in order to
fill the I_S table on the fly. If a view is discovered, open will fail
as it is instructed to open tables only (OPEN_TABLE_ONLY). This would
cause the assert to be triggered.
The assert was added in the patch for Bug#52044 and was therefore
not in any released versions of the server.
This patch fixes the problem by adjusting the assert to take into
consideration the possibility of tables being opened as part of
an I_S query. This is similar to what is already done for
close_tables_for_reopen().
Test case added to information_schema_inno.test.
on CREATE TABLE .. SELECT I_S.PART
This assert was triggered if an InnoDB table was created using
CREATE TABLE ... AS SELECT where the query used an I_S table, and
a view existed in the database. It would also be triggered for
any statement changing an InnoDB table (e.g. INSERT, UPDATE, DELETE)
which had a subquery referencing an I_S table.
The assert was triggered if open_normal_and_derived_tables() failed
and a statement transaction had been started. This will usually not
happen as tables are opened before a statement transaction is started.
However, e.g. CREATE TABLE ... AS SELECT starts a transaction in order
to insert tuples into the new table. And if the subquery references
an I_S table, all current tables and views can be opened in order to
fill the I_S table on the fly. If a view is discovered, open will fail
as it is instructed to open tables only (OPEN_TABLE_ONLY). This would
cause the assert to be triggered.
The assert was added in the patch for Bug#52044 and was therefore
not in any released versions of the server.
This patch fixes the problem by adjusting the assert to take into
consideration the possibility of tables being opened as part of
an I_S query. This is similar to what is already done for
close_tables_for_reopen().
Test case added to information_schema_inno.test.
temp table
This patch introduces two key changes in the replication's behavior.
Firstly, it reverts part of BUG#51894 which puts any update to temporary tables
into the trx-cache. Now, updates to temporary tables are handled according to
the type of their engines as a regular table.
Secondly, an unsafe mixed statement, (i.e. a statement that access transactional
table as well non-transactional or temporary table, and writes to any of them),
are written into the trx-cache in order to minimize errors in the execution when
the statement logging format is in use.
Such changes has a direct impact on which statements are classified as unsafe
statements and thus part of BUG#53259 is reverted.
temp table
This patch introduces two key changes in the replication's behavior.
Firstly, it reverts part of BUG#51894 which puts any update to temporary tables
into the trx-cache. Now, updates to temporary tables are handled according to
the type of their engines as a regular table.
Secondly, an unsafe mixed statement, (i.e. a statement that access transactional
table as well non-transactional or temporary table, and writes to any of them),
are written into the trx-cache in order to minimize errors in the execution when
the statement logging format is in use.
Such changes has a direct impact on which statements are classified as unsafe
statements and thus part of BUG#53259 is reverted.
check_if_table_exists()
This assert was triggered when the server tried to load plugins
while running in embedded server mode. In embedded server mode,
check_if_table_exists() was used to check if mysql.plugin existed
so that ER_NO_SUCH_TABLE could be silently ignored.
The problem was that this check was done without acquiring a metadata
lock on mysql.plugin first. This triggered the assert.
This patch fixes the problem by removing the call to
check_if_table_exists() from plugin_load(). Instead an error handler
which traps ER_NO_SUCH_TABLE is installed before trying to open
mysql.plugin when running in embedded server mode.
No test coverage added since this assert was triggered by
existing tests running in embedded server mode.
sql/sql_base.cc:
Renamed Prelock_error_handler to No_such_table_error_handler
and moved the declaration to sql_base.h to make it usable
in plugin_load().
sql/sql_base.h:
Renamed Prelock_error_handler to No_such_table_error_handler
and moved the declaration to sql_base.h to make it usable
in plugin_load().
sql/sql_plugin.cc:
Removed call to check_if_table_exists() used to check for mysql.plugin
in plugin_load() for embedded server. Instead install error handler
which traps ER_NO_SUCH_TABLE during open_and_lock_tables().
check_if_table_exists()
This assert was triggered when the server tried to load plugins
while running in embedded server mode. In embedded server mode,
check_if_table_exists() was used to check if mysql.plugin existed
so that ER_NO_SUCH_TABLE could be silently ignored.
The problem was that this check was done without acquiring a metadata
lock on mysql.plugin first. This triggered the assert.
This patch fixes the problem by removing the call to
check_if_table_exists() from plugin_load(). Instead an error handler
which traps ER_NO_SUCH_TABLE is installed before trying to open
mysql.plugin when running in embedded server mode.
No test coverage added since this assert was triggered by
existing tests running in embedded server mode.
'CREATE TABLE IF NOT EXISTS ... SELECT' behaviour
BUG#47132, BUG#47442, BUG49494, BUG#23992 and BUG#48814 will disappear
automatically after the this patch.
BUG#55617 is fixed by this patch too.
This is the 5.5 part.
It implements:
- 'CREATE TABLE IF NOT EXISTS ... SELECT' statement will not insert
anything and binlog anything if the table already exists.
It only generate a warning that table already exists.
- A couple of test cases for the behavior changing.
'CREATE TABLE IF NOT EXISTS ... SELECT' behaviour
BUG#47132, BUG#47442, BUG49494, BUG#23992 and BUG#48814 will disappear
automatically after the this patch.
BUG#55617 is fixed by this patch too.
This is the 5.5 part.
It implements:
- 'CREATE TABLE IF NOT EXISTS ... SELECT' statement will not insert
anything and binlog anything if the table already exists.
It only generate a warning that table already exists.
- A couple of test cases for the behavior changing.
corruption on ADD PARTITION and LOCK TABLE
Bug#53770: Server crash at handler.cc:2076 on
LOAD DATA after timed out COALESCE PARTITION
5.5 fix for:
Bug#51042: REORGANIZE PARTITION can leave table in an
inconsistent state in case of crash
Needs to be back-ported to 5.1
5.5 fix for:
Bug#50418: DROP PARTITION does not interact with
transactions
Main problem was non-persistent operations done
before meta-data lock was taken (53770+53676).
And 53676 needed to keep the table/partitions opened and locked
while copying the data to the new partitions.
Also added thorough tests to spot some additional bugs
in the ddl_log code, which could result in bad state
between the .frm and partitions.
Collapsed patch, includes all fixes required from the reviewers.
mysql-test/r/partition_innodb.result:
updated result with new test
mysql-test/suite/parts/inc/partition_crash.inc:
crash test include file
mysql-test/suite/parts/inc/partition_crash_add.inc:
test all states in fast_alter_partition_table
ADD PARTITION branch
mysql-test/suite/parts/inc/partition_crash_change.inc:
test all states in fast_alter_partition_table
CHANGE PARTITION branch
mysql-test/suite/parts/inc/partition_crash_drop.inc:
test all states in fast_alter_partition_table
DROP PARTITION branch
mysql-test/suite/parts/inc/partition_fail.inc:
recovery test including injecting errors
mysql-test/suite/parts/inc/partition_fail_add.inc:
test all states in fast_alter_partition_table
ADD PARTITION branch
mysql-test/suite/parts/inc/partition_fail_change.inc:
test all states in fast_alter_partition_table
CHANGE PARTITION branch
mysql-test/suite/parts/inc/partition_fail_drop.inc:
test all states in fast_alter_partition_table
DROP PARTITION branch
mysql-test/suite/parts/inc/partition_mgm_crash.inc:
include file that runs all crash and failure injection tests.
mysql-test/suite/parts/r/partition_debug_innodb.result:
new test result file
mysql-test/suite/parts/r/partition_debug_myisam.result:
new test result file
mysql-test/suite/parts/r/partition_special_innodb.result:
updated result
mysql-test/suite/parts/r/partition_special_myisam.result:
updated result
mysql-test/suite/parts/t/partition_debug_innodb-master.opt:
opt file for using with crashing tests of partitioned innodb
mysql-test/suite/parts/t/partition_debug_innodb.test:
partitioned innodb test that require debug builds
mysql-test/suite/parts/t/partition_debug_myisam-master.opt:
opt file for using with crashing tests of partitioned myisam
mysql-test/suite/parts/t/partition_debug_myisam.test:
partitioned myisam test that require debug builds
mysql-test/suite/parts/t/partition_special_innodb-master.opt:
added innodb-file-per-table to easier verify partition status on disk
mysql-test/suite/parts/t/partition_special_innodb.test:
added test case
mysql-test/suite/parts/t/partition_special_myisam.test:
added test case
mysql-test/t/partition_innodb.test:
added test case
sql/sql_base.cc:
Moved alter_close_tables to sql_partition.cc
sql/sql_base.h:
removed some non existing and duplicated functions.
sql/sql_partition.cc:
fast_alter_partition_table:
Spletted abort_and_upgrad_lock_and_close_table
to its parts (wait_while_table_is_used and
alter_close_tables) and always have
wait_while_table_is_used before any persistent
operations (including logs, which will be executed
on failure) and alter_close_tables after
create/read/write operations and before
drop operations.
moved alter_close_tables here from sql_base.cc
Added error injections for better test coverage.
write_log_final_change_partition:
fixed a log_entry linking bug (delete_frm was not
linked to change/drop partition)
and drop partition must be executed before
change partition (change partition can rename a
partition to an old name, like REORG p1 INTO (p1,p2).
write_log_add_change_partition:
need to use drop_frm first, and relinking that entry
and reusing its execute entry.
sql/sql_table.cc:
added initialization of next_active_log_entry.
sql/table.h:
removed a duplicate declaration.
corruption on ADD PARTITION and LOCK TABLE
Bug#53770: Server crash at handler.cc:2076 on
LOAD DATA after timed out COALESCE PARTITION
5.5 fix for:
Bug#51042: REORGANIZE PARTITION can leave table in an
inconsistent state in case of crash
Needs to be back-ported to 5.1
5.5 fix for:
Bug#50418: DROP PARTITION does not interact with
transactions
Main problem was non-persistent operations done
before meta-data lock was taken (53770+53676).
And 53676 needed to keep the table/partitions opened and locked
while copying the data to the new partitions.
Also added thorough tests to spot some additional bugs
in the ddl_log code, which could result in bad state
between the .frm and partitions.
Collapsed patch, includes all fixes required from the reviewers.
Implement a few simple asserts in my_rwlock_t locks.
include/my_pthread.h:
Declare two simple assert functions.
include/mysql/psi/mysql_thread.h:
Add wrappers for new assert functions.
mysys/thr_rwlock.c:
Add asserts.
sql/sql_base.cc:
Silence a compiler warning for the case when
SAFE_MUTEX is not ON.
Remove acquisition of LOCK_open around file system operations,
since such operations are now protected by metadata locks.
Rework table discovery algorithm to not require LOCK_open.
No new tests added since all MDL locking operations are covered
in lock.test and mdl_sync.test, and as long as these tests
pass despite the increased concurrency, consistency must be
unaffected.
mysql-test/t/disabled.def:
Disable NDB tests due to Bug#55799.
sql/datadict.cc:
No longer necessary to protect ha_create_table() with
LOCK_open. Serial execution is now ensured by metadata
locks.
sql/ha_ndbcluster.cc:
Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.cc:
Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.h:
Update function signature.
sql/handler.cc:
Implement ha_check_if_table_exists().
@todo: some engines provide ha_table_exists_in_engine()
handlerton call, for those we perhaps shouldn't
call ha_discover(), to be more efficient.
Since currently it's only NDB, postpone till
integration with NDB.
sql/handler.h:
Declare ha_check_if_table_exists() function.
sql/mdl.cc:
Remove an obsolete comment.
sql/sql_base.cc:
Update to a new signature of close_cached_tables():
from now on we always call it without LOCK_open.
Update comments.
Remove get_table_share_with_create(), we should
not attempt to create a table under LOCK_open.
Introduce get_table_share_with_discover() instead,
which would request a back off action if the table
exists in engine.
Remove acquisition of LOCK_open for
data dictionary operations, such as check_if_table_exists().
Do not use get_table_share_with_create/discover for views,
where it's not needed.
Make tdc_remove_table() optionally acquire LOCK_open
to simplify usage of this function.
Use the right mutex in the partitioning code when
manipulating with thd->open_tables.
sql/sql_base.h:
Update signatures of changes functions.
sql/sql_insert.cc:
Do not wrap quick_rm_table() with LOCK_open acquisition,
this is unnecessary.
sql/sql_parse.cc:
Update to the new calling convention of tdc_remove_table().
Update to the new signature of close_cached_tables().
Update comments.
sql/sql_rename.cc:
Update to the new calling convention of tdc_remove_table().
Remove acquisition of LOCK_open around filesystem
operations.
sql/sql_show.cc:
Remove get_trigger_table_impl().
Do not acquire LOCK_open for a dirty read of the trigger
file.
sql/sql_table.cc:
Do not acquire LOCK_open for filesystem operations.
sql/sql_trigger.cc:
Do not require LOCK_open for trigger file I/O.
sql/sql_truncate.cc:
Update to the new signature of tdc_remove_table().
sql/sql_view.cc:
Do not require LOCK_open for view I/O.
Use tdc_remove_table() to expel view share.
Update comments.
sql/sys_vars.cc:
Update to the new signature of close_cached_tables().
Remove acquisition of LOCK_open around file system operations,
since such operations are now protected by metadata locks.
Rework table discovery algorithm to not require LOCK_open.
No new tests added since all MDL locking operations are covered
in lock.test and mdl_sync.test, and as long as these tests
pass despite the increased concurrency, consistency must be
unaffected.
FLUSH TABLES <list> WITH READ LOCK are incompatible" to
be pushed as separate patch.
Replaced thread state name "Waiting for table", which was
used by threads waiting for a metadata lock or table flush,
with a set of names which better reflect types of resources
being waited for.
Also replaced "Table lock" thread state name, which was used
by threads waiting on thr_lock.c table level lock, with more
elaborate "Waiting for table level lock", to make it
more consistent with other thread state names.
Updated test cases and their results according to these
changes.
Fixed sys_vars.query_cache_wlock_invalidate_func test to not
to wait for timeout of wait_condition.inc script.
mysql-test/r/query_cache.result:
Added test coverage for query_cache_wlock_invalidate
behavior for implicitly locked tables.
mysql-test/suite/sys_vars/r/query_cache_wlock_invalidate_func.result:
Fixed sys_vars.query_cache_wlock_invalidate_func test to not
to wait for timeout of wait_condition.inc script. Reverted
changes to test which introduced timeout and replaced waiting
condition with a more appropriate one.
Test coverage for query_cache_wlock_invalidate behavior for
implicitly locked tables was added to query_cache.test.
mysql-test/suite/sys_vars/t/query_cache_wlock_invalidate_func.test:
Fixed sys_vars.query_cache_wlock_invalidate_func test to not
to wait for timeout of wait_condition.inc script. Reverted
changes to test which introduced timeout and replaced waiting
condition with a more appropriate one.
Test coverage for query_cache_wlock_invalidate behavior for
implicitly locked tables was added to query_cache.test.
mysql-test/t/query_cache.test:
Added test coverage for query_cache_wlock_invalidate
behavior for implicitly locked tables.
mysys/thr_lock.c:
Replaced "Table lock" thread state name, which was used by
threads waiting on thr_lock.c table level lock, with more
elaborate "Waiting for table level lock", to make it
consistent with thread state names which are used while
waiting for metadata locks and table flush.
sql/mdl.cc:
Replaced thread state name "Waiting for table", which was
used by threads waiting for a metadata lock or table flush,
with a set of names which better reflect types of resources
being waited for.
To implement this:
- Adjusted MDL_wait::timed_wait() to take thread state name
as parameter.
- Introduced method of MDL_key class which allows to get
thread state name to be used while waiting for resource
corresponding to the key and changed code to use it.
Added array translating namespaces to thread state names
as part of this change.
sql/mdl.h:
To implement this:
- Adjusted MDL_wait::timed_wait() to take thread state name
as parameter.
- Introduced method of MDL_key class which allows to get
thread state name to be used while waiting for resource
corresponding to the key and changed code to use it.
Added array translating namespaces to thread state names
as part of this change.
sql/sql_base.cc:
Replaced thread state name "Waiting for table", which was
used by threads waiting for table flush, with a more elaborate
"Waiting for table flush".
