database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-08-08 11:22:35 +03:00
Code Activity

MDEV-25506 (3 of 3): Do not delete .ibd files before commit

Browse Source 
This is a complete rewrite of DROP TABLE, also as part of other DDL,
such as ALTER TABLE, CREATE TABLE...SELECT, TRUNCATE TABLE.

The background DROP TABLE queue hack is removed.
If a transaction needs to drop and create a table by the same name
(like TRUNCATE TABLE does), it must first rename the table to an
internal #sql-ib name. No committed version of the data dictionary
will include any #sql-ib tables, because whenever a transaction
renames a table to a #sql-ib name, it will also drop that table.
Either the rename will be rolled back, or the drop will be committed.

Data files will be unlinked after the transaction has been committed
and a FILE_RENAME record has been durably written. The file will
actually be deleted when the detached file handle returned by
fil_delete_tablespace() will be closed, after the latches have been
released. It is possible that a purge of the delete of the SYS_INDEXES
record for the clustered index will execute fil_delete_tablespace()
concurrently with the DDL transaction. In that case, the thread that
arrives later will wait for the other thread to finish.

HTON_TRUNCATE_REQUIRES_EXCLUSIVE_USE: A new handler flag.
ha_innobase::truncate() now requires that all other references to
the table be released in advance. This was implemented by Monty.

ha_innobase::delete_table(): If CREATE TABLE..SELECT is detected,
we will "hijack" the current transaction, drop the table in
the current transaction and commit the current transaction.
This essentially fixes MDEV-21602. There is a FIXME comment about
making the check less failure-prone.

ha_innobase::truncate(), ha_innobase::delete_table():
Implement a fast path for temporary tables. We will no longer allow
temporary tables to use the adaptive hash index.

dict_table_t::mdl_name: The original table name for the purpose of
acquiring MDL in purge, to prevent a race condition between a
DDL transaction that is dropping a table, and purge processing
undo log records of DML that had executed before the DDL operation.
For #sql-backup- tables during ALTER TABLE...ALGORITHM=COPY, the
dict_table_t::mdl_name will differ from dict_table_t::name.

dict_table_t::parse_name(): Use mdl_name instead of name.

dict_table_rename_in_cache(): Update mdl_name.

For the internal FTS_ tables of FULLTEXT INDEX, purge would
acquire MDL on the FTS_ table name, but not on the main table,
and therefore it would be able to run concurrently with a
DDL transaction that is dropping the table. Previously, the
DROP TABLE queue hack prevented a race between purge and DDL.
For now, we introduce purge_sys.stop_FTS() to prevent purge from
opening any table, while a DDL transaction that may drop FTS_
tables is in progress. The function fts_lock_table(), which will
be invoked before the dictionary is locked, will wait for
purge to release any table handles.

trx_t::drop_table_statistics(): Drop statistics for the table.
This replaces dict_stats_drop_index(). We will drop or rename
persistent statistics atomically as part of DDL transactions.
On lock conflict for dropping statistics, we will fail instantly
with DB_LOCK_WAIT_TIMEOUT, because we will be holding the
exclusive data dictionary latch.

trx_t::commit_cleanup(): Separated from trx_t::commit_in_memory().
Relax an assertion around fts_commit() and allow DB_LOCK_WAIT_TIMEOUT
in addition to DB_DUPLICATE_KEY. The call to fts_commit() is
entirely misplaced here and may obviously break the consistency
of transactions that affect FULLTEXT INDEX. It needs to be fixed
separately.

dict_table_t::n_foreign_key_checks_running: Remove (MDEV-21175).
The counter was a work-around for missing meta-data locking (MDL)
on the SQL layer, and not really needed in MariaDB.

ER_TABLE_IN_FK_CHECK: Replaced with ER_UNUSED_28.

HA_ERR_TABLE_IN_FK_CHECK: Remove.

row_ins_check_foreign_constraints(): Do not acquire
dict_sys.latch either. The SQL-layer MDL will protect us.

This was reviewed by Thirunarayanan Balathandayuthapani
and tested by Matthias Leich.
This commit is contained in:
Marko Mäkelä
2021-06-09 17:02:55 +03:00
parent 3f78fbc582
commit 1bd681c8b3
98 changed files with 2180 additions and 3804 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
mysql-test/suite/innodb/r/innodb-index-online.result
View File

@@ -8,7 +8,6 @@ SET GLOBAL innodb_monitor_enable = module_ddl;
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -27,7 +26,6 @@ ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -81,7 +79,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR scanned';
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 1
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
@@ -99,7 +96,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR created';
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
@@ -113,7 +109,6 @@ ALTER TABLE t1 ADD UNIQUE INDEX(c2);
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -163,7 +158,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR c2d_created';
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 1
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
@@ -175,7 +169,6 @@ ERROR 70100: Query execution was interrupted
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -243,7 +236,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR c2e_created';
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 1
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
@@ -283,7 +275,6 @@ ROLLBACK;
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 1
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
@@ -316,7 +307,6 @@ ERROR HY000: Creating index 'c2e' required more than 'innodb_online_alter_log_ma
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 1
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -328,7 +318,6 @@ name pos
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 1
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -338,7 +327,6 @@ ALTER TABLE t1 COMMENT 'testing if c2e will be dropped';
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -363,7 +351,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR c2f_created';
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 1
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
@@ -387,7 +374,6 @@ ROLLBACK;
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 1
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
@@ -400,7 +386,6 @@ ALTER TABLE t1 CHANGE c2 c22f INT;
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -464,7 +449,6 @@ name pos
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 1
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -473,7 +457,6 @@ connection default;
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 1
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0
@@ -493,7 +476,6 @@ ALTER TABLE t1 DROP INDEX c2d, DROP INDEX c2f;
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
ddl_background_drop_tables 0
ddl_online_create_index 0
ddl_pending_alter_table 0
ddl_sort_file_alter_table 0