database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-08-08 11:22:35 +03:00
Code Activity

MDEV-15250 UPSERT during ALTER TABLE results in 'Duplicate entry' error for alter

Browse Source 
- InnoDB DDL results in `Duplicate entry' if concurrent DML throws
duplicate key error. The following scenario explains the problem

connection con1:
  ALTER TABLE t1 FORCE;

connection con2:
  INSERT INTO t1(pk, uk) VALUES (2, 2), (3, 2);

In connection con2, InnoDB throws the 'DUPLICATE KEY' error because
of unique index. Alter operation will throw the error when applying
the concurrent DML log.

- Inserting the duplicate key for unique index logs the insert
operation for online ALTER TABLE. When insertion fails,
transaction does rollback and it leads to logging of
delete operation for online ALTER TABLE.
While applying the insert log entries, alter operation
encounters 'DUPLICATE KEY' error.

- To avoid the above fake duplicate scenario, InnoDB should
not write any log for online ALTER TABLE before DML transaction
commit.

- User thread which does DML can apply the online log if
InnoDB ran out of online log and index is marked as completed.
Set online log error if apply phase encountered any error.
It can also clear all other indexes log, marks the newly
added indexes as corrupted.

- Removed the old online code which was a part of DML operations

commit_inplace_alter_table() : Does apply the online log
for the last batch of secondary index log and does frees
the log for the completed index.

trx_t::apply_online_log: Set to true while writing the undo
log if the modified table has active DDL

trx_t::apply_log(): Apply the DML changes to online DDL tables

dict_table_t::is_active_ddl(): Returns true if the table
has an active DDL

dict_index_t::online_log_make_dummy(): Assign dummy value
for clustered index online log to indicate the secondary
indexes are being rebuild.

dict_index_t::online_log_is_dummy(): Check whether the online
log has dummy value

ha_innobase_inplace_ctx::log_failure(): Handle the apply log
failure for online DDL transaction

row_log_mark_other_online_index_abort(): Clear out all other
online index log after encountering the error during
row_log_apply()

row_log_get_error(): Get the error happened during row_log_apply()

row_log_online_op(): Does apply the online log if index is
completed and ran out of memory. Returns false if apply log fails

UndorecApplier: Introduced a class to maintain the undo log
record, latched undo buffer page, parse the undo log record,
maintain the undo record type, info bits and update vector

UndorecApplier::get_old_rec(): Get the correct version of the
clustered index record that was modified by the current undo
log record

UndorecApplier::clear_undo_rec(): Clear the undo log related
information after applying the undo log record

UndorecApplier::log_update(): Handle the update, delete undo
log and apply it on online indexes

UndorecApplier::log_insert(): Handle the insert undo log
and apply it on online indexes

UndorecApplier::is_same(): Check whether the given roll pointer
is generated by the current undo log record information

trx_t::rollback_low(): Set apply_online_log for the transaction
after partially rollbacked transaction has any active DDL

prepare_inplace_alter_table_dict(): After allocating the online
log, InnoDB does create fulltext common tables. Fulltext index
doesn't allow the index to be online. So removed the dead
code of online log removal

Thanks to Marko Mäkelä for providing the initial prototype and
Matthias Leich for testing the issue patiently.
This commit is contained in:
Thirunarayanan Balathandayuthapani
2022-04-25 13:36:56 +05:30
parent 4ed30b2ac5
commit 4b80c11f52
57 changed files with 1648 additions and 1471 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
mysql-test/suite/innodb/r/innodb-table-online.result
View File

@@ -97,11 +97,11 @@ ddl_online_create_index 1
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
ddl_log_file_alter_table 0
BEGIN;
INSERT INTO t1 VALUES(4,7,2);
SET DEBUG_SYNC = 'now SIGNAL insert_done';
connection con1;
ERROR 23000: Duplicate entry '4' for key 'PRIMARY'
DELETE FROM t1 WHERE c1=4 and c2=7;
connection default;
ROLLBACK;
connection con1;
@@ -213,30 +213,22 @@ ddl_online_create_index 1
ddl_pending_alter_table 1
ddl_sort_file_alter_table 0
ddl_log_file_alter_table 1
BEGIN;
DELETE FROM t1;
ROLLBACK;
UPDATE t1 SET c2 = c2 + 1;
BEGIN;
UPDATE t1 SET c2 = c2 + 2;
UPDATE t1 SET c2 = c2 + 1;
DELETE FROM t1;
ROLLBACK;
BEGIN;
DELETE FROM t1;
ROLLBACK;
UPDATE t1 SET c2 = c2 + 2;
UPDATE t1 SET c2 = c2 + 1;
BEGIN;
UPDATE t1 SET c2 = c2 + 2;
UPDATE t1 SET c2 = c2 + 1;
DELETE FROM t1;
ROLLBACK;
BEGIN;
DELETE FROM t1;
ROLLBACK;
UPDATE t1 SET c2 = c2 + 2;
UPDATE t1 SET c2 = c2 + 1;
BEGIN;
UPDATE t1 SET c2 = c2 + 2;
UPDATE t1 SET c2 = c2 + 1;
DELETE FROM t1;
ROLLBACK;
UPDATE t1 SET c2 = c2 + 2;
UPDATE t1 SET c2 = c2 + 1;
UPDATE t1 SET c2 = c2 + 2;
UPDATE t1 SET c2 = c2 + 1;
UPDATE t1 SET c2 = c2 + 2;
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
@@ -286,7 +278,7 @@ SET DEBUG_SYNC = 'row_log_table_apply1_before SIGNAL rebuilt3 WAIT_FOR dml3_done
ALTER TABLE t1 ADD PRIMARY KEY(c22f), CHANGE c2 c22f INT;
ERROR 42000: Multiple primary key defined
ALTER TABLE t1 DROP PRIMARY KEY, ADD PRIMARY KEY(c22f), CHANGE c2 c22f INT;
ERROR 23000: Duplicate entry '5' for key 'PRIMARY'
ERROR 23000: Duplicate entry '26' for key 'PRIMARY'
ALTER TABLE t1 DROP PRIMARY KEY, ADD PRIMARY KEY(c22f,c1,c4(5)),
CHANGE c2 c22f INT, CHANGE c3 c3 CHAR(255) NULL, CHANGE c1 c1 INT AFTER c22f,
ADD COLUMN c4 VARCHAR(6) DEFAULT 'Online', LOCK=NONE;
@@ -302,11 +294,8 @@ ddl_log_file_alter_table 1
BEGIN;
INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 240;
DELETE FROM t1 WHERE c1 > 320;
ROLLBACK;
BEGIN;
UPDATE t1 SET c2 = c2 + 1;
DELETE FROM t1;
ROLLBACK;
COMMIT;
SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
name count
ddl_background_drop_indexes 0
@@ -374,12 +363,11 @@ SET DEBUG_SYNC = 'now WAIT_FOR c3p5_created0';
BEGIN;
INSERT INTO t1 VALUES(347,33101,'Pikku kakkosen posti','YLETV2');
INSERT INTO t1 VALUES(33101,347,NULL,'');
COMMIT;
SET DEBUG_SYNC = 'now SIGNAL ins_done0';
connection con1;
ERROR 01000: Data truncated for column 'c3' at row 323
connection default;
ROLLBACK;
connection con1;
ERROR 22004: Invalid use of NULL value
DELETE FROM t1 WHERE c1= 347 and c22f = 33101;
ALTER TABLE t1 MODIFY c3 CHAR(255) NOT NULL;
SET DEBUG_SYNC = 'row_log_table_apply1_before SIGNAL c3p5_created WAIT_FOR ins_done';
ALTER TABLE t1 DROP PRIMARY KEY, DROP COLUMN c22f,
@@ -405,20 +393,20 @@ ddl_log_file_alter_table 2
connection default;
SELECT COUNT(*) FROM t1;
COUNT(*)
321
322
ALTER TABLE t1 ROW_FORMAT=REDUNDANT;
SELECT * FROM t1 LIMIT 10;
c22f c1 c3 c4
5 1 1foo Online
5 6 6foofoofoofoofoofoo Online
5 11 11foofoofoofoofoofoofoofoofoofoofoo Online
5 16 16foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
5 21 21foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
5 26 26foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
5 31 31foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
5 36 36foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
5 41 41foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
5 46 46foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
27 1 1foo Online
27 6 6foofoofoofoofoofoo Online
27 11 11foofoofoofoofoofoofoofoofoofoofoo Online
27 16 16foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
27 21 21foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
27 26 26foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
27 31 31foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
27 36 36foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
27 41 41foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
27 46 46foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo Online
connection con1;
ALTER TABLE t1 DISCARD TABLESPACE;
connection default;