database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Code Activity

MDEV-37467: InnoDB Instant ALTER TABLE is not crash safe

Browse Source 
The hidden metadata record of instant ALTER TABLE was not written
crash-safely, and recovery could fail to roll it back. These are
independent problems.

First, the metadata record may include externally stored BLOB
metadata. The existing BLOB storage path in
btr_store_big_rec_extern_fields() writes the clustered index record
first, with zero BLOB pointers, and only fills in the BLOB pointers
afterwards. If the server is killed after the mini-transaction that
wrote the (incomplete) metadata record was durably committed, but
before the BLOB pointers were written, the table could become
inaccessible on recovery.

Make metadata BLOB storage crash-safe by writing the BLOB pages and
computing their pointers before the metadata record itself is
inserted or updated, so that the record is always written with
complete BLOB pointers. If the server is killed before the metadata
record is written, the already-written BLOB pages are merely
orphaned, which is safe.

Second, trx_undo_report_row_operation() writes the undo log record in
a mini-transaction of its own, which is committed before the
mini-transaction that writes the metadata record. Because
innobase_instant_try() had already updated SYS_COLUMNS and SYS_TABLES
in earlier mini-transactions, a kill in between left a durable undo
log record for the table while the metadata record was unchanged. On
recovery, trx_resurrect_table_locks() would then load the table
definition before the incomplete transaction was rolled back. The
data dictionary described the table as it would be after the
operation, while the metadata record still described it as it was
before, and btr_cur_instant_init() failed on that disagreement.

Write the undo log record of the metadata record in the same
mini-transaction that inserts or updates the record, so that the two
cannot be separated by a crash: until that mini-transaction is
committed, neither of them is durable.

An undo log record is never split between pages. If the DEFAULT
values of the columns being added are large enough that the undo log
record for updating the metadata record would not fit on one page,
innobase_instant_try() would fail. Determine this before the operation
starts, so that it can be performed by another algorithm instead.

Third, the table definition that recovery loads need not correspond to
the metadata record. dict_load_table_one() reads the committed version
of the SYS_TABLES record, and escalates to READ UNCOMMITTED only when
it finds a SYS_COLUMNS record that was written by a transaction that is
still active. The number of SYS_COLUMNS records that dict_load_columns()
reads is derived from SYS_TABLES.N_COLS, which was read from the
committed version. The record of a column that the operation appended is
located after that many records, so it is never read and the operation
goes unnoticed. Only an instant ALTER TABLE that merely appends columns
can escape this way: ADD COLUMN ... FIRST, DROP COLUMN and column
reordering rewrite the SYS_COLUMNS records of already existing columns.

Detect this on the SYS_TABLES record itself, which is located by table
name and therefore does not depend on N_COLS. Every instant ALTER TABLE
that changes the columns updates that record, because
innobase_instant_try() invokes innodb_update_cols().

Fourth, the rollback writes a metadata record that comprises fewer
fields than the table definition describes, because
btr_cur_trim_alter_metadata() shortens it to the number of fields that
it comprised before the operation. That number determines the size of
the null flag bitmap, and hence the position of the array of field
lengths. rec_init_offsets_comp_ordinary() derives it from the record,
while the two functions that write the record derived it from the table
definition and asserted that the two agree.

- btr_store_big_rec_metadata():
  New function to store the off-page columns of a metadata record
  ahead of time. Each BLOB page is allocated and linked in its own
  mini-transaction, and the resulting BLOB pointers are written
  directly into the (heap-resident) index entry. On failure, it frees
  any pages it already allocated and resets the pointers to zero.

- btr_free_big_rec_metadata():
  New helper to free the BLOB pages written by
  btr_store_big_rec_metadata() and reset the entry's BLOB pointers
  to zero, used both on failure inside that function and by its
  callers when the metadata record ends up not being written.

- row_ins_clust_index_entry_low():
  For a metadata entry that needs external storage, convert it to a
  big record and call btr_store_big_rec_metadata() (with
  log_free_check() allowed, since no latches are held yet) before
  inserting the record. On failure, free the metadata BLOBs and
  convert the entry back.

- btr_cur_pessimistic_update():
  When updating a metadata record that requires external storage,
  call btr_store_big_rec_metadata() (without log_free_check(),
  since index and page latches are held) before modifying the record,
  and free the temporary big_rec vector via btr_free_big_rec_metadata()
  or dtuple_big_rec_free() on the various failure/success paths.

- btr_cur_optimistic_insert():
  Remove the special-cased jump to convert_big_rec for metadata
  entries, since their BLOBs are now always stored ahead of time by
  the caller; assert that a metadata entry never needs external
  storage at this point.

- innobase_instant_try():
  Since btr_cur_pessimistic_update() now stores metadata BLOBs
  before updating the record, big_rec is always NULL here; assert
  this instead of calling btr_store_big_rec_extern_fields().

- trx_undo_report_row_operation():
  New parameter caller_mtr. If it is specified, the undo log record
  is written in that mini-transaction, which is never committed or
  restarted here. An undo log page is added within the same
  mini-transaction if the record does not fit on the current one. A
  temporary table never uses the caller's mini-transaction, because
  that would require changing its logging mode. All other callers
  pass NULL and are unaffected.
  Encapsulate the parameters that describe the row change
  (clust_entry, update, cmpl_info, rec, offsets) in the new type
  trx_undo_row_op, which of the fields are set depends on the
  operation, which the type documents.

- btr_cur_ins_lock_and_undo(), btr_cur_upd_lock_and_undo():
  For an instant ALTER TABLE metadata record, pass the
  mini-transaction that is going to insert or modify the record.

- trx_undo_max_rec_size():
  New function to determine the maximum size of an undo log record,
  that is, the space available on an empty undo log page.

- ha_innobase::check_if_supported_inplace_alter():
  Refuse ALGORITHM=INSTANT if the metadata record already exists and
  the undo log record for updating it would exceed
  trx_undo_max_rec_size(). trx_undo_page_report_modify() stores the
  DEFAULT value of each column that is being added in that record in
  full, inline. No such limit applies when the metadata record is
  being inserted, because trx_undo_page_report_insert() writes
  TRX_UNDO_INSERT_METADATA and no field data.

- dict_load_table_one():
  If the SYS_TABLES record was written by a transaction that is still
  active, load the table definition as READ UNCOMMITTED. A
  delete-marked record is excluded, because SYS_TABLES.NAME is the
  clustered index key: RENAME TABLE delete-marks the record of the old
  name, and the definition that corresponds to that name is the one
  that precedes the rename.

- dict_sys_tables_rec_read():
  Report whether the current version of the record was written by a
  transaction that has not been committed. The function already
  determines this in order to decide whether to read an older
  version of the record, and used to discard the answer. Store the
  fields that are read in the new type dict_sys_tables_rec, instead
  of in separate output parameters. A caller that does not want a
  delete-marked record to be reported as not found used to indicate
  that by specifying trx_id as nullptr; that is now the parameter
  skip_deleted.

- dict_load_table_low():
  New parameter uncommitted_rec, which is passed on to
  dict_sys_tables_rec_read().

- trx_undo_add_page():
  Correct the documented restriction on the mini-transaction of the
  caller.

- rec_get_converted_size_comp_prefix_low(),
  rec_convert_dtuple_to_rec_comp():
  For a record that includes a metadata BLOB, determine the number of
  nullable fields from the tuple, by way of
  dict_index_t::get_n_nullable(), and not from
  dict_index_t::n_nullable. This is what
  rec_init_offsets_comp_ordinary() does, and it is equivalent for a
  tuple that comprises all fields of the index. Relax the assertions
  that required the tuple to comprise all of them.

- Added test in innodb.instant_alter and innodb.instant_alter_crash
  to test normal working of INSTANT ALTER, crash safety and full table.
This commit is contained in:
Mohammad Tafzeel Shams
2026-08-03 12:15:17 +05:30
parent f6869f9068
commit 1b71b64d19
15 changed files with 1283 additions and 170 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
mysql-test/suite/innodb/r/instant_alter.result
View File

@@ -2939,4 +2939,60 @@ index(id, msg)
FLUSH TABLES;
ALTER TABLE mdev28822_100427_innodb ADD i1 INTEGER, ALGORITHM=INSTANT;
DROP TABLE mdev28822_100427_innodb;
#
# MDEV-37467 InnoDB Instant ALTER TABLE is not crash safe
#
CREATE TABLE t1(
id INT PRIMARY KEY,
c1 INT,
c2 VARCHAR(100),
c3 TEXT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1, 100, 'test1', 'data1');
INSERT INTO t1 VALUES(2, 200, 'test2', 'data2');
INSERT INTO t1 VALUES(3, 300, 'test3', 'data3');
ALTER TABLE t1 DROP COLUMN c2, ALGORITHM=INSTANT;
INSERT INTO t1 VALUES(4, 400, 'data4');
ALTER TABLE t1 DROP COLUMN c3, ALGORITHM=INSTANT;
INSERT INTO t1 VALUES(5, 500);
SELECT * FROM t1;
id c1
1 100
2 200
3 300
4 400
5 500
DROP TABLE t1;
CREATE TABLE t2(id INT PRIMARY KEY, c1 INT) ENGINE=InnoDB;
INSERT INTO t2 VALUES(1, 100);
INSERT INTO t2 VALUES(2, 200);
ALTER TABLE t2 ADD COLUMN c2 VARCHAR(100) DEFAULT 'default_value', ALGORITHM=INSTANT;
INSERT INTO t2 VALUES(3, 300, 'custom');
SELECT * FROM t2;
id c1 c2
1 100 default_value
2 200 default_value
3 300 custom
ALTER TABLE t2 DROP COLUMN c2, ALGORITHM=INSTANT;
INSERT INTO t2 VALUES(4, 400);
SELECT * FROM t2;
id c1
1 100
2 200
3 300
4 400
DROP TABLE t2;
CREATE TABLE t3(
id INT PRIMARY KEY,
c1 INT,
c2 VARCHAR(100)) ENGINE=InnoDB;
INSERT INTO t3 VALUES(1, 100, 'test1');
INSERT INTO t3 VALUES(2, 200, 'test2');
ALTER TABLE t3 DROP COLUMN c2, ADD COLUMN c3 BLOB DEFAULT (REPEAT(0x41, 50000)), ALGORITHM=INSTANT;
INSERT INTO t3 VALUES(3, 300, REPEAT('y', 1024));
SELECT id, c1, LENGTH(c3) AS blob_length, MD5(c3) AS blob_md5 FROM t3;
id c1 blob_length blob_md5
1 100 50000 b1a7de895f33fde6fba7a3029ef7db68
2 200 50000 b1a7de895f33fde6fba7a3029ef7db68
3 300 1024 deb88981eeb769584b258b701d09b3d7
DROP TABLE t3;
SET GLOBAL innodb_stats_persistent = @save_stats_persistent;

192
mysql-test/suite/innodb/r/instant_alter_crash.result
View File

@@ -201,4 +201,196 @@ CHECK TABLE t2;
Table Op Msg_type Msg_text
test.t2 check status OK
DROP TABLE t1,t2;
#
# MDEV-37467 InnoDB Instant ALTER TABLE is not crash safe
#
CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1,2,3),(4,5,6);
connect con1,localhost,root,,,;
SET DEBUG="+d,btr_store_metadata_crash";
SET DEBUG_SYNC="btr_store_metadata_crash SIGNAL con_start WAIT_FOR con_finish";
ALTER TABLE t1 DROP COLUMN c, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR con_start";
# restart
disconnect con1;
CHECK TABLE t1;
Table Op Msg_type Msg_text
test.t1 check status OK
SELECT * FROM t1;
a b c
1 2 3
4 5 6
ALTER TABLE t1 DROP COLUMN c, ALGORITHM=INSTANT;
SELECT * FROM t1;
a b
1 2
4 5
ALTER TABLE t1 ADD COLUMN d INT NOT NULL DEFAULT 7, ALGORITHM=INSTANT;
SELECT * FROM t1;
a b d
1 2 7
4 5 7
connect con1,localhost,root,,,;
SET DEBUG="+d,btr_store_metadata_crash";
SET DEBUG_SYNC="btr_store_metadata_crash SIGNAL con_continue WAIT_FOR ever";
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
connection default;
set DEBUG_SYNC="now WAIT_FOR con_continue";
# restart
disconnect con1;
CHECK TABLE t1;
Table Op Msg_type Msg_text
test.t1 check status OK
SELECT * FROM t1;
a b d
1 2 7
4 5 7
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
SELECT * FROM t1;
a d
1 7
4 7
CHECK TABLE t1;
Table Op Msg_type Msg_text
test.t1 check status OK
DROP TABLE t1;
CREATE TABLE t1(
id INT PRIMARY KEY,
c1 INT,
c2 VARCHAR(100),
c3 TEXT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1, 100, 'test1', 'data1');
INSERT INTO t1 VALUES(2, 200, 'test2', 'data2');
CREATE TABLE t2 LIKE t1;
INSERT INTO t2 VALUES(1, 100, 'test1', 'data1');
INSERT INTO t2 VALUES(2, 200, 'test2', 'data2');
SET STATEMENT DEBUG_DBUG='d,btr_page_alloc_fail' FOR
ALTER TABLE t1 DROP COLUMN c2, ALGORITHM=INSTANT;
ERROR HY000: The table 't1' is full
INSERT INTO t1 VALUES(3, 300, 'test3', 'data3');
SELECT * FROM t1;
id c1 c2 c3
1 100 test1 data1
2 200 test2 data2
3 300 test3 data3
ALTER TABLE t1 ADD COLUMN c4 VARCHAR(100) DEFAULT 'default_value', ALGORITHM=INSTANT;
INSERT INTO t1 VALUES(4, 400, 'test4', 'data4', 'custom');
SELECT * FROM t1;
id c1 c2 c3 c4
1 100 test1 data1 default_value
2 200 test2 data2 default_value
3 300 test3 data3 default_value
4 400 test4 data4 custom
SET STATEMENT DEBUG_DBUG='d,btr_page_alloc_fail' FOR
ALTER TABLE t1 DROP COLUMN c4, ALGORITHM=INSTANT;
ERROR HY000: The table 't1' is full
INSERT INTO t1 VALUES(5, 500, 'test5', 'data5', 'custom');
SELECT * FROM t1;
id c1 c2 c3 c4
1 100 test1 data1 default_value
2 200 test2 data2 default_value
3 300 test3 data3 default_value
4 400 test4 data4 custom
5 500 test5 data5 custom
SET STATEMENT DEBUG_DBUG='d,btr_page_alloc_fail_4' FOR
ALTER TABLE t2 DROP COLUMN c2, ADD COLUMN c4 BLOB DEFAULT (REPEAT(0x41, 50000)), ALGORITHM=INSTANT;
ERROR HY000: The table 't2' is full
INSERT INTO t2 VALUES(3, 300, 'test3', 'data3');
SELECT * FROM t2;
id c1 c2 c3
1 100 test1 data1
2 200 test2 data2
3 300 test3 data3
DROP TABLE t1, t2;
CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1,2,3),(4,5,6);
ALTER TABLE t1 DROP COLUMN c, ALGORITHM=INSTANT;
ALTER TABLE t1 ADD COLUMN d INT NOT NULL DEFAULT 7, ALGORITHM=INSTANT;
connect con1,localhost,root,,,;
SET DEBUG="+d,btr_metadata_undo_logged";
SET DEBUG_SYNC="btr_metadata_undo_logged SIGNAL con1_go WAIT_FOR ever";
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR con1_go";
# restart
disconnect con1;
CHECK TABLE t1;
Table Op Msg_type Msg_text
test.t1 check status OK
SELECT * FROM t1;
a b d
1 2 7
4 5 7
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
SELECT * FROM t1;
a d
1 7
4 7
CREATE TABLE t2(a INT PRIMARY KEY, b INT, c INT) ENGINE=InnoDB;
INSERT INTO t2 VALUES(1,2,3),(4,5,6);
ALTER TABLE t2 ADD COLUMN d INT NOT NULL DEFAULT 7, ALGORITHM=INSTANT;
connect con2,localhost,root,,,;
SET DEBUG="+d,btr_metadata_undo_logged";
SET DEBUG_SYNC="btr_metadata_undo_logged SIGNAL con2_go WAIT_FOR ever";
ALTER TABLE t2 DROP COLUMN b, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR con2_go";
# restart
disconnect con2;
CHECK TABLE t2;
Table Op Msg_type Msg_text
test.t2 check status OK
SELECT * FROM t2;
a b c d
1 2 3 7
4 5 6 7
ALTER TABLE t2 DROP COLUMN b, ALGORITHM=INSTANT;
SELECT * FROM t2;
a c d
1 3 7
4 6 7
DROP TABLE t1,t2;
CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT, d INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1,2,3,4);
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
CREATE TABLE t2(a INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t2 VALUES(1);
BEGIN;
DELETE FROM t2;
connect ddl, localhost, root;
SET DEBUG_SYNC="innodb_alter_inplace_before_commit SIGNAL ddl WAIT_FOR ever";
ALTER TABLE t1 ADD COLUMN e INT, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR ddl";
SET GLOBAL innodb_flush_log_at_trx_commit=1;
COMMIT;
# restart
disconnect ddl;
ALTER TABLE t1 ADD COLUMN e INT, ALGORITHM=INSTANT;
SELECT * FROM t1;
a c d e
1 3 4 NULL
DROP TABLE t1,t2;
CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT, d INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1,2,3,4);
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
CREATE TABLE t2(a INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t2 VALUES(1);
BEGIN;
DELETE FROM t2;
connect ddl, localhost, root;
SET DEBUG_SYNC="innodb_alter_inplace_before_commit SIGNAL ddl WAIT_FOR ever";
ALTER TABLE t1 ADD COLUMN e INT, DROP COLUMN c, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR ddl";
SET GLOBAL innodb_flush_log_at_trx_commit=1;
COMMIT;
# restart
disconnect ddl;
ALTER TABLE t1 ADD COLUMN e INT, DROP COLUMN c, ALGORITHM=INSTANT;
SELECT * FROM t1;
a d e
1 4 NULL
DROP TABLE t1,t2;
db.opt

48
mysql-test/suite/innodb/t/instant_alter.test
View File

@@ -973,4 +973,52 @@ remove_file $datadir/test/mdev28822_100427_innodb.frm;
copy_file std_data/mysql_upgrade/mdev28822_100427_innodb.frm $datadir/test/mdev28822_100427_innodb.frm;
ALTER TABLE mdev28822_100427_innodb ADD i1 INTEGER, ALGORITHM=INSTANT;
DROP TABLE mdev28822_100427_innodb;
--echo #
--echo # MDEV-37467 InnoDB Instant ALTER TABLE is not crash safe
--echo #
# Test 1: Simple ALTER DROP
CREATE TABLE t1(
id INT PRIMARY KEY,
c1 INT,
c2 VARCHAR(100),
c3 TEXT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1, 100, 'test1', 'data1');
INSERT INTO t1 VALUES(2, 200, 'test2', 'data2');
INSERT INTO t1 VALUES(3, 300, 'test3', 'data3');
ALTER TABLE t1 DROP COLUMN c2, ALGORITHM=INSTANT;
INSERT INTO t1 VALUES(4, 400, 'data4');
ALTER TABLE t1 DROP COLUMN c3, ALGORITHM=INSTANT;
INSERT INTO t1 VALUES(5, 500);
SELECT * FROM t1;
DROP TABLE t1;
# Test 2: ALTER ADD followed by ALTER DROP
CREATE TABLE t2(id INT PRIMARY KEY, c1 INT) ENGINE=InnoDB;
INSERT INTO t2 VALUES(1, 100);
INSERT INTO t2 VALUES(2, 200);
ALTER TABLE t2 ADD COLUMN c2 VARCHAR(100) DEFAULT 'default_value', ALGORITHM=INSTANT;
INSERT INTO t2 VALUES(3, 300, 'custom');
SELECT * FROM t2;
ALTER TABLE t2 DROP COLUMN c2, ALGORITHM=INSTANT;
INSERT INTO t2 VALUES(4, 400);
SELECT * FROM t2;
DROP TABLE t2;
# Test 3: ALTER DROP ADD BLOB with 50kb default value
CREATE TABLE t3(
id INT PRIMARY KEY,
c1 INT,
c2 VARCHAR(100)) ENGINE=InnoDB;
INSERT INTO t3 VALUES(1, 100, 'test1');
INSERT INTO t3 VALUES(2, 200, 'test2');
ALTER TABLE t3 DROP COLUMN c2, ADD COLUMN c3 BLOB DEFAULT (REPEAT(0x41, 50000)), ALGORITHM=INSTANT;
INSERT INTO t3 VALUES(3, 300, REPEAT('y', 1024));
SELECT id, c1, LENGTH(c3) AS blob_length, MD5(c3) AS blob_md5 FROM t3;
DROP TABLE t3;
SET GLOBAL innodb_stats_persistent = @save_stats_persistent;

213
mysql-test/suite/innodb/t/instant_alter_crash.test
View File

@@ -229,4 +229,217 @@ disconnect ddl;
CHECK TABLE t2;
DROP TABLE t1,t2;
--echo #
--echo # MDEV-37467 InnoDB Instant ALTER TABLE is not crash safe
--echo #
CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1,2,3),(4,5,6);
connect(con1,localhost,root,,,);
# Crash after writing the metadata BLOB, before inserting the
# metadata record.
SET DEBUG="+d,btr_store_metadata_crash";
SET DEBUG_SYNC="btr_store_metadata_crash SIGNAL con_start WAIT_FOR con_finish";
send ALTER TABLE t1 DROP COLUMN c, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR con_start";
let $shutdown_timeout=0;
--source include/restart_mysqld.inc
disconnect con1;
CHECK TABLE t1;
SELECT * FROM t1;
ALTER TABLE t1 DROP COLUMN c, ALGORITHM=INSTANT;
SELECT * FROM t1;
ALTER TABLE t1 ADD COLUMN d INT NOT NULL DEFAULT 7, ALGORITHM=INSTANT;
SELECT * FROM t1;
# Crash after writing the metadata BLOB, before updating the
# existing metadata record.
connect(con1,localhost,root,,,);
SET DEBUG="+d,btr_store_metadata_crash";
SET DEBUG_SYNC="btr_store_metadata_crash SIGNAL con_continue WAIT_FOR ever";
send ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
connection default;
set DEBUG_SYNC="now WAIT_FOR con_continue";
let $shutdown_timeout=0;
--source include/restart_mysqld.inc
disconnect con1;
CHECK TABLE t1;
SELECT * FROM t1;
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
SELECT * FROM t1;
CHECK TABLE t1;
DROP TABLE t1;
# Test: Table full during Simple ALTER DROP
CREATE TABLE t1(
id INT PRIMARY KEY,
c1 INT,
c2 VARCHAR(100),
c3 TEXT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1, 100, 'test1', 'data1');
INSERT INTO t1 VALUES(2, 200, 'test2', 'data2');
CREATE TABLE t2 LIKE t1;
INSERT INTO t2 VALUES(1, 100, 'test1', 'data1');
INSERT INTO t2 VALUES(2, 200, 'test2', 'data2');
--error ER_RECORD_FILE_FULL
SET STATEMENT DEBUG_DBUG='d,btr_page_alloc_fail' FOR
ALTER TABLE t1 DROP COLUMN c2, ALGORITHM=INSTANT;
INSERT INTO t1 VALUES(3, 300, 'test3', 'data3');
SELECT * FROM t1;
# Test: Table full during ALTER DROP done after ALTER ADD
ALTER TABLE t1 ADD COLUMN c4 VARCHAR(100) DEFAULT 'default_value', ALGORITHM=INSTANT;
INSERT INTO t1 VALUES(4, 400, 'test4', 'data4', 'custom');
SELECT * FROM t1;
--error ER_RECORD_FILE_FULL
SET STATEMENT DEBUG_DBUG='d,btr_page_alloc_fail' FOR
ALTER TABLE t1 DROP COLUMN c4, ALGORITHM=INSTANT;
INSERT INTO t1 VALUES(5, 500, 'test5', 'data5', 'custom');
SELECT * FROM t1;
# Test: Table full during ALTER DROP ADD BLOB with 50kb default value while allocating 4th page
--error ER_RECORD_FILE_FULL
SET STATEMENT DEBUG_DBUG='d,btr_page_alloc_fail_4' FOR
ALTER TABLE t2 DROP COLUMN c2, ADD COLUMN c4 BLOB DEFAULT (REPEAT(0x41, 50000)), ALGORITHM=INSTANT;
INSERT INTO t2 VALUES(3, 300, 'test3', 'data3');
SELECT * FROM t2;
DROP TABLE t1, t2;
# Test: Kill right after the undo log record for the metadata record was
# written, before the mini-transaction that writes both of them was
# committed.
# The metadata record is an "alter metadata" (BLOB-carrying) record.
CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1,2,3),(4,5,6);
ALTER TABLE t1 DROP COLUMN c, ALGORITHM=INSTANT;
ALTER TABLE t1 ADD COLUMN d INT NOT NULL DEFAULT 7, ALGORITHM=INSTANT;
connect(con1,localhost,root,,,);
SET DEBUG="+d,btr_metadata_undo_logged";
SET DEBUG_SYNC="btr_metadata_undo_logged SIGNAL con1_go WAIT_FOR ever";
send ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR con1_go";
let $shutdown_timeout=0;
--source include/restart_mysqld.inc
disconnect con1;
# Wait for the rollback so that the results do not depend on its timing.
let $wait_condition=
SELECT COUNT(*)=0 FROM information_schema.innodb_trx;
--source include/wait_condition.inc
CHECK TABLE t1;
SELECT * FROM t1;
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
SELECT * FROM t1;
# The metadata record is a plain "instant ADD COLUMN" record.
CREATE TABLE t2(a INT PRIMARY KEY, b INT, c INT) ENGINE=InnoDB;
INSERT INTO t2 VALUES(1,2,3),(4,5,6);
ALTER TABLE t2 ADD COLUMN d INT NOT NULL DEFAULT 7, ALGORITHM=INSTANT;
connect(con2,localhost,root,,,);
SET DEBUG="+d,btr_metadata_undo_logged";
SET DEBUG_SYNC="btr_metadata_undo_logged SIGNAL con2_go WAIT_FOR ever";
send ALTER TABLE t2 DROP COLUMN b, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR con2_go";
let $shutdown_timeout=0;
--source include/restart_mysqld.inc
disconnect con2;
let $wait_condition=
SELECT COUNT(*)=0 FROM information_schema.innodb_trx;
--source include/wait_condition.inc
CHECK TABLE t2;
SELECT * FROM t2;
ALTER TABLE t2 DROP COLUMN b, ALGORITHM=INSTANT;
SELECT * FROM t2;
DROP TABLE t1,t2;
# Test: Kill an instant ALTER TABLE whose changes to SYS_COLUMNS and
# SYS_TABLES are durable, but which has not been committed. Appending a
# column does not rewrite the SYS_COLUMNS records of the already
# existing columns, so dict_load_columns() will not notice the
# uncommitted operation. The recovery must detect it on the SYS_TABLES
# record and load the uncommitted table definition.
CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT, d INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1,2,3,4);
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
CREATE TABLE t2(a INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t2 VALUES(1);
BEGIN;
DELETE FROM t2;
connect ddl, localhost, root;
SET DEBUG_SYNC="innodb_alter_inplace_before_commit SIGNAL ddl WAIT_FOR ever";
send ALTER TABLE t1 ADD COLUMN e INT, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR ddl";
SET GLOBAL innodb_flush_log_at_trx_commit=1;
COMMIT;
let $shutdown_timeout=0;
--source include/restart_mysqld.inc
disconnect ddl;
# Wait for the rollback so that the results do not depend on its timing.
let $wait_condition=
SELECT COUNT(*)=0 FROM information_schema.innodb_trx;
--source include/wait_condition.inc
ALTER TABLE t1 ADD COLUMN e INT, ALGORITHM=INSTANT;
SELECT * FROM t1;
DROP TABLE t1,t2;
# The same, but the operation also drops a column, so that the rollback
# will have to shorten the metadata record to the number of fields that
# it comprised before the operation.
CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT, d INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1,2,3,4);
ALTER TABLE t1 DROP COLUMN b, ALGORITHM=INSTANT;
CREATE TABLE t2(a INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t2 VALUES(1);
BEGIN;
DELETE FROM t2;
connect ddl, localhost, root;
SET DEBUG_SYNC="innodb_alter_inplace_before_commit SIGNAL ddl WAIT_FOR ever";
send ALTER TABLE t1 ADD COLUMN e INT, DROP COLUMN c, ALGORITHM=INSTANT;
connection default;
SET DEBUG_SYNC="now WAIT_FOR ddl";
SET GLOBAL innodb_flush_log_at_trx_commit=1;
COMMIT;
let $shutdown_timeout=0;
--source include/restart_mysqld.inc
disconnect ddl;
# Wait for the rollback so that the results do not depend on its timing.
let $wait_condition=
SELECT COUNT(*)=0 FROM information_schema.innodb_trx;
--source include/wait_condition.inc
ALTER TABLE t1 ADD COLUMN e INT, DROP COLUMN c, ALGORITHM=INSTANT;
SELECT * FROM t1;
DROP TABLE t1,t2;
--list_files $MYSQLD_DATADIR/test

354
storage/innobase/btr/btr0cur.cc
View File

@@ -2349,9 +2349,13 @@ btr_cur_ins_lock_and_undo(
roll_ptr_t roll_ptr = dummy_roll_ptr;
if (!(flags & BTR_NO_UNDO_LOG_FLAG)) {
/* For the instant ALTER TABLE metadata record, write the
undo log record in the same mini-transaction that is
going to insert the record. */
if (dberr_t err = trx_undo_report_row_operation(
thr, index, entry, NULL, 0, NULL, NULL,
&roll_ptr)) {
thr, index,
{entry, NULL, 0, NULL, NULL}, &roll_ptr,
entry->is_metadata() ? mtr : nullptr)) {
return err;
}
@@ -2468,19 +2472,13 @@ btr_cur_optimistic_insert(
#endif /* HAVE_valgrind */
leaf = page_is_leaf(page);
if (UNIV_UNLIKELY(entry->is_alter_metadata())) {
ut_ad(leaf);
goto convert_big_rec;
}
/* Calculate the record size when entry is converted to a record */
rec_size = rec_get_converted_size(index, entry, n_ext);
if (page_zip_rec_needs_ext(rec_size, page_is_comp(page),
dtuple_get_n_fields(entry),
block->zip_size())) {
convert_big_rec:
ut_ad(!entry->is_alter_metadata());
/* The record is so big that we have to store some fields
externally on separate database pages */
big_rec_vec = dtuple_convert_big_rec(index, 0, entry, &n_ext);
@@ -2923,13 +2921,18 @@ btr_cur_upd_lock_and_undo(
}
}
/* Append the info about the update in the undo log */
/* Append the info about the update in the undo log.
For the instant ALTER TABLE metadata record, write the undo log
record in the same mini-transaction that is going to modify the
record. */
return((flags & BTR_NO_UNDO_LOG_FLAG)
? DB_SUCCESS
: trx_undo_report_row_operation(
thr, index, NULL, update,
cmpl_info, rec, offsets, roll_ptr));
thr, index,
{NULL, update, cmpl_info, rec, offsets}, roll_ptr,
rec_is_metadata(rec, *index) ? mtr : nullptr));
}
/** Write DB_TRX_ID,DB_ROLL_PTR to a clustered index entry.
@@ -4013,7 +4016,9 @@ btr_cur_pessimistic_update(
committed before latching any further pages */
{
big_rec_t* big_rec_vec = NULL;
big_rec_t* metadata_big = NULL;
big_rec_t* dummy_big_rec;
dtuple_t* new_entry = NULL;
dict_index_t* index;
buf_block_t* block;
page_zip_des_t* page_zip;
@@ -4076,6 +4081,15 @@ btr_cur_pessimistic_update(
ibuf_update_free_bits_zip(block, mtr);
}
if (metadata_big != NULL) {
/* The metadata record was not modified. Free the
off-page columns that were already written by
btr_store_big_rec_metadata(). */
btr_free_big_rec_metadata(index, new_entry,
metadata_big, mtr);
dtuple_big_rec_free(metadata_big);
}
if (big_rec_vec != NULL) {
dtuple_big_rec_free(big_rec_vec);
}
@@ -4086,8 +4100,6 @@ btr_cur_pessimistic_update(
rec = btr_cur_get_rec(cursor);
ut_ad(rec_offs_validate(rec, index, *offsets));
dtuple_t* new_entry;
const bool is_metadata = rec_is_metadata(rec, *index);
if (UNIV_UNLIKELY(is_metadata)) {
@@ -4179,7 +4191,31 @@ btr_cur_pessimistic_update(
}
}
/* Do lock checking and undo logging */
if (UNIV_UNLIKELY(is_metadata) && big_rec_vec) {
ut_ad(flags & BTR_KEEP_POS_FLAG);
/* Store the off-page columns and write complete BLOB
pointers into new_entry before the metadata record is
modified, so that no metadata record with incomplete
(zero) BLOB pointers can ever be written. If the server
is killed before the record is modified below, the BLOB
pages will merely be orphaned. Unlike in
btr_store_big_rec_extern_fields(), it is not safe to
invoke log_free_check() here, because we are holding
index and page latches. */
err = btr_store_big_rec_metadata(index, new_entry,
big_rec_vec, false);
if (err != DB_SUCCESS) {
/* The BLOB pages were already freed and the
BLOB pointers in new_entry were reset by
btr_store_big_rec_metadata(). */
goto err_exit;
}
metadata_big = big_rec_vec;
big_rec_vec = NULL;
}
/* Do lock checking and undo logging. */
err = btr_cur_upd_lock_and_undo(flags, cursor, *offsets,
update, cmpl_info,
thr, mtr, &roll_ptr);
@@ -4187,6 +4223,16 @@ btr_cur_pessimistic_update(
goto err_exit;
}
if (UNIV_UNLIKELY(is_metadata)) {
/* The undo log record was written into this
mini-transaction, which has not been committed yet.
Allow tests to kill the server here, after making
everything that was written before durable. */
DBUG_EXECUTE_IF("btr_metadata_undo_logged",
log_buffer_flush_to_disk(););
DEBUG_SYNC_C("btr_metadata_undo_logged");
}
if (optim_err == DB_OVERFLOW) {
/* First reserve enough free space for the file segments
of the index tree, so that the update will not fail because
@@ -4246,6 +4292,13 @@ btr_cur_pessimistic_update(
if (!page_cur_move_to_prev(page_cursor)) {
err = DB_CORRUPTION;
if (metadata_big != NULL) {
/* The metadata record was deleted and will not
be re-inserted. Free the off-page columns that
were written by btr_store_big_rec_metadata(). */
btr_free_big_rec_metadata(index, new_entry,
metadata_big, mtr);
}
goto return_after_reservations;
}
@@ -4377,6 +4430,13 @@ btr_cur_pessimistic_update(
/* This should happen when InnoDB tries to extend the
tablespace */
ut_ad(err == DB_OUT_OF_FILE_SPACE);
if (metadata_big != NULL) {
/* The metadata record was not re-inserted. Free
the off-page columns that were already written. */
btr_free_big_rec_metadata(index, new_entry,
metadata_big, mtr);
dtuple_big_rec_free(metadata_big);
}
return err;
}
ut_a(rec);
@@ -4445,6 +4505,13 @@ return_after_reservations:
btr_cur_get_page(cursor), index));
#endif /* UNIV_ZIP_DEBUG */
if (UNIV_UNLIKELY(metadata_big != NULL)) {
/* Free the vector only. Either the BLOB pages are
owned by the metadata record that was written above,
or btr_free_big_rec_metadata() already freed them. */
dtuple_big_rec_free(metadata_big);
}
index->table->space->release_free_extents(n_reserved);
*big_rec = big_rec_vec;
return(err);
@@ -4526,9 +4593,12 @@ btr_cur_del_mark_set_clust_rec(
return(DB_SUCCESS);
}
/* A metadata pseudo-record is never delete-marked by DML. */
ut_ad(!rec_is_metadata(rec, *index));
err = trx_undo_report_row_operation(thr, index,
entry, NULL, 0, rec, offsets,
&roll_ptr);
{entry, NULL, 0, rec, offsets},
&roll_ptr, nullptr);
if (err != DB_SUCCESS) {
return(err);
@@ -6477,6 +6547,256 @@ func_exit:
return(error);
}
/** Free the BLOB pages that were allocated by
btr_store_big_rec_metadata(), after the insert or update of the
metadata record itself failed, and reset the BLOB pointers in the
index entry to zero.
@param index clustered index
@param entry metadata index entry
@param big_rec externally stored fields created by
dtuple_convert_big_rec() from entry
@param caller_mtr the mini-transaction of the caller, or NULL to free
each BLOB page in a separate mini-transaction. It must
be specified if the caller holds a mini-transaction
that may have acquired the latch of the tablespace,
because mtr_t::x_lock_space() would acquire it again. */
void btr_free_big_rec_metadata(dict_index_t *index, dtuple_t *entry,
const big_rec_t *big_rec, mtr_t *caller_mtr)
{
ut_ad(entry->is_metadata());
ut_ad(index->is_primary());
/* When the caller's mini-transaction is being used, it must never be
committed or restarted here. */
mtr_t own_mtr;
mtr_t &mtr= caller_mtr ? *caller_mtr : own_mtr;
for (ulint i= 0; i < big_rec->n_fields; i++)
{
dfield_t *field= dtuple_get_nth_field(entry, big_rec->fields[i].field_no);
ut_ad(dfield_is_ext(field));
ut_ad(dfield_get_len(field) >= BTR_EXTERN_FIELD_REF_SIZE);
byte *field_ref= static_cast<byte*>(dfield_get_data(field)) +
dfield_get_len(field) - BTR_EXTERN_FIELD_REF_SIZE;
if (!memcmp(field_ref, field_ref_zero, BTR_EXTERN_FIELD_REF_SIZE))
{
/* The BLOB for this field was not stored yet. */
continue;
}
ut_ad(mach_read_from_4(field_ref + BTR_EXTERN_SPACE_ID) ==
index->table->space_id);
uint32_t page_no= mach_read_from_4(field_ref + BTR_EXTERN_PAGE_NO);
while (page_no != FIL_NULL)
{
if (!caller_mtr)
{
mtr.start();
index->set_modified(mtr);
}
dberr_t err;
buf_block_t *block=
buf_page_get_gen(page_id_t(index->table->space_id, page_no), 0,
RW_X_LATCH, nullptr, BUF_GET, &mtr, &err);
if (UNIV_UNLIKELY(!block))
{
if (!caller_mtr)
mtr.commit();
break;
}
ut_ad(fil_page_get_type(block->page.frame) == FIL_PAGE_TYPE_BLOB);
page_no= mach_read_from_4(block->page.frame + FIL_PAGE_DATA +
BTR_BLOB_HDR_NEXT_PAGE_NO);
err= btr_page_free(index, block, &mtr, true, false);
if (!caller_mtr)
mtr.commit();
if (UNIV_UNLIKELY(err != DB_SUCCESS))
break;
}
/* Reset the BLOB pointer to zero. */
memset(field_ref, 0, BTR_EXTERN_FIELD_REF_SIZE);
}
}
/** Store the off-page columns of a metadata record before the record
itself is inserted into or updated in the clustered index, and write
complete BLOB pointers into the index entry.
btr_store_big_rec_extern_fields() writes the off-page columns after
the clustered index record has already been written with zero BLOB
pointers. That order is not crash-safe for the metadata record of
instant ALTER TABLE: if the server is killed after the mini-transaction
that wrote the record was durably committed, but before all BLOB
pointers were written, the incomplete metadata record could make the
table inaccessible on recovery.
By storing the off-page columns first, the metadata record will be
written with complete BLOB pointers in one atomic mini-transaction.
If the server is killed before the metadata record itself is written,
the already written BLOB pages will merely be orphaned.
@param index clustered index
@param entry metadata index entry, whose externally stored fields
were shortened by dtuple_convert_big_rec() and end in
zero BLOB pointers that will be filled in
@param big_rec externally stored fields created by
dtuple_convert_big_rec() from entry
@param free_check whether it is safe to invoke log_free_check()
(the caller must not be holding any index or page latches)
@return DB_SUCCESS or error code
On failure, any BLOB pages that were already allocated will be freed,
and the BLOB pointers in entry will be reset to zero. */
dberr_t btr_store_big_rec_metadata(dict_index_t *index, dtuple_t *entry,
const big_rec_t *big_rec, bool free_check)
{
mtr_t mtr;
dberr_t error= DB_SUCCESS;
ut_ad(entry->is_metadata());
ut_ad(index->is_primary());
ut_ad(index->is_instant());
ut_ad(!index->table->is_temporary());
/* Instant ALTER TABLE is never supported on ROW_FORMAT=COMPRESSED
tables. */
ut_ad(!index->table->space->zip_size());
const uint32_t space_id= index->table->space->id;
/* Space available in an uncompressed page to carry BLOB data */
const ulint payload_size= index->table->space->physical_size() -
(FIL_PAGE_DATA + BTR_BLOB_HDR_SIZE + FIL_PAGE_DATA_END);
for (ulint i= 0; i < big_rec->n_fields; i++)
{
const big_rec_field_t &f= big_rec->fields[i];
dfield_t *field= dtuple_get_nth_field(entry, f.field_no);
ut_ad(dfield_is_ext(field));
ut_ad(dfield_get_len(field) >= BTR_EXTERN_FIELD_REF_SIZE);
byte *field_ref= static_cast<byte*>(dfield_get_data(field)) +
dfield_get_len(field) - BTR_EXTERN_FIELD_REF_SIZE;
/* The BLOB pointer must still be zero; it will be written below. */
ut_ad(!memcmp(field_ref, field_ref_zero, BTR_EXTERN_FIELD_REF_SIZE));
ulint extern_len= f.len;
MEM_CHECK_DEFINED(f.data, extern_len);
ut_a(extern_len > 0);
uint32_t prev_page_no= FIL_NULL;
for (;;)
{
if (free_check)
log_free_check();
mtr.start();
index->set_modified(mtr);
buf_block_t *block=
btr_page_alloc(index,
(prev_page_no == FIL_NULL
? index->page : prev_page_no) + 1,
FSP_NO_DIR, 0, &mtr, &mtr, &error);
DBUG_EXECUTE_IF("btr_page_alloc_fail",
block= nullptr;
error= DB_OUT_OF_FILE_SPACE;);
DBUG_EXECUTE_IF("btr_page_alloc_fail_4",
{
static unsigned calls;
if (++calls == 4)
{
block= nullptr;
error= DB_OUT_OF_FILE_SPACE;
}
});
if (UNIV_UNLIKELY(!block))
{
mtr.commit();
goto func_exit;
}
const uint32_t page_no= block->page.id().page_no();
if (prev_page_no == FIL_NULL)
{
/* Write the BLOB pointer into the entry. The entry resides in a
memory heap, not in a buffer pool page; this write must not be
redo logged. */
mach_write_to_4(field_ref + BTR_EXTERN_SPACE_ID, space_id);
mach_write_to_4(field_ref + BTR_EXTERN_PAGE_NO, page_no);
mach_write_to_4(field_ref + BTR_EXTERN_OFFSET, FIL_PAGE_DATA);
}
else if (buf_block_t *prev_block=
buf_page_get_gen(page_id_t(space_id, prev_page_no), 0,
RW_X_LATCH, nullptr, BUF_GET, &mtr, &error))
mtr.write<4>(*prev_block, BTR_BLOB_HDR_NEXT_PAGE_NO + FIL_PAGE_DATA +
prev_block->page.frame, page_no);
else
{
mtr.commit();
goto func_exit;
}
ut_ad(!page_has_siblings(block->page.frame));
ut_ad(!fil_page_get_type(block->page.frame));
mtr.write<1>(*block, FIL_PAGE_TYPE + 1 + block->page.frame,
FIL_PAGE_TYPE_BLOB);
const ulint store_len=
extern_len > payload_size ? payload_size : extern_len;
mtr.memcpy<mtr_t::MAYBE_NOP>(*block, FIL_PAGE_DATA +
BTR_BLOB_HDR_SIZE + block->page.frame,
static_cast<const byte*>(f.data) +
f.len - extern_len, store_len);
mtr.write<4>(*block, BTR_BLOB_HDR_PART_LEN + FIL_PAGE_DATA +
block->page.frame, store_len);
compile_time_assert(FIL_NULL == 0xffffffff);
mtr.memset(block, BTR_BLOB_HDR_NEXT_PAGE_NO + FIL_PAGE_DATA, 4, 0xff);
extern_len-= store_len;
prev_page_no= page_no;
mtr.commit();
if (extern_len == 0)
break;
}
mach_write_to_4(field_ref + BTR_EXTERN_LEN, 0);
mach_write_to_4(field_ref + BTR_EXTERN_LEN + 4, f.len);
DBUG_EXECUTE_IF("btr_store_big_rec_extern",
error= DB_OUT_OF_FILE_SPACE;
goto func_exit;);
}
/* Allow tests to kill the server here, after making the BLOB pages
durable while no metadata record refers to them yet. */
DBUG_EXECUTE_IF("btr_store_metadata_crash", log_buffer_flush_to_disk(););
DEBUG_SYNC_C("btr_store_metadata_crash");
return DB_SUCCESS;
func_exit:
ut_ad(error != DB_SUCCESS);
/* Free any BLOB pages that were already allocated, and reset the BLOB
pointers in the entry to zero. Our own mini-transaction was committed
above, and the caller cannot be holding the latch of the tablespace
yet, because the BLOB pages are stored before the metadata record is
written. */
btr_free_big_rec_metadata(index, entry, big_rec, nullptr);
return error;
}
/** Check the FIL_PAGE_TYPE on an uncompressed BLOB page.
@param block uncompressed BLOB page
@param op operation

176
storage/innobase/dict/dict0load.cc
View File

@@ -634,17 +634,34 @@ uint32_t dict_sys_tables_type_to_tf(uint32_t type, bool not_redundant)
/** Outcome of dict_sys_tables_rec_read() */
enum table_read_status { READ_OK= 0, READ_ERROR, READ_NOT_FOUND };
/** Read and return 5 integer fields from a SYS_TABLES record.
/** The fields that dict_sys_tables_rec_read() reads from a SYS_TABLES
record. */
struct dict_sys_tables_rec
{
/** SYS_TABLES.ID */
table_id_t table_id;
/** SYS_TABLES.SPACE */
uint32_t space_id;
/** SYS_TABLES.N_COLS, still encoded */
uint32_t n_cols;
/** table flags, derived from SYS_TABLES.TYPE */
uint32_t flags;
/** table flags2, derived from SYS_TABLES.MIX_LEN */
uint32_t flags2;
/** DB_TRX_ID of the committed SYS_TABLES record */
trx_id_t trx_id;
/** whether the current version of the record was written by a
transaction that has not been committed */
bool uncommitted;
};
/** Read the fields of a SYS_TABLES record.
@param[in] rec A record of SYS_TABLES
@param[in] uncommitted true=use READ UNCOMMITTED, false=READ COMMITTED
@param[in] mtr mini-transaction
@param[out] table_id Pointer to the table_id for this table
@param[out] space_id Pointer to the space_id for this table
@param[out] n_cols Pointer to number of columns for this table.
@param[out] flags Pointer to table flags
@param[out] flags2 Pointer to table flags2
@param[out] trx_id DB_TRX_ID of the committed SYS_TABLES record,
or nullptr to perform READ UNCOMMITTED
@param[out] table_rec the fields that were read
@param[in] skip_deleted whether a delete-marked record means that
the table does not exist
@return whether the record was read correctly */
MY_ATTRIBUTE((warn_unused_result))
static
@@ -653,22 +670,21 @@ dict_sys_tables_rec_read(
const rec_t* rec,
bool uncommitted,
mtr_t* mtr,
table_id_t* table_id,
uint32_t* space_id,
uint32_t* n_cols,
uint32_t* flags,
uint32_t* flags2,
trx_id_t* trx_id)
dict_sys_tables_rec& table_rec,
bool skip_deleted = true)
{
const byte* field;
ulint len;
mem_heap_t* heap = nullptr;
table_rec.uncommitted = false;
field = rec_get_nth_field_old(
rec, DICT_FLD__SYS_TABLES__DB_TRX_ID, &len);
ut_ad(len == 6 || len == UNIV_SQL_NULL);
trx_id_t id = len == 6 ? trx_read_trx_id(field) : 0;
if (id && !uncommitted && trx_sys.is_registered_nonzero(id)) {
table_rec.uncommitted = true;
const auto savepoint = mtr->get_savepoint();
heap = mem_heap_create(1024);
dict_index_t* index = UT_LIST_GET_FIRST(
@@ -696,7 +712,7 @@ dict_sys_tables_rec_read(
if (rec_get_deleted_flag(rec, 0)) {
ut_ad(id);
if (trx_id) {
if (skip_deleted) {
if (heap) {
mem_heap_free(heap);
}
@@ -704,19 +720,17 @@ dict_sys_tables_rec_read(
}
}
if (trx_id) {
*trx_id = id;
}
table_rec.trx_id = id;
field = rec_get_nth_field_old(
rec, DICT_FLD__SYS_TABLES__ID, &len);
ut_ad(len == 8);
*table_id = static_cast<table_id_t>(mach_read_from_8(field));
table_rec.table_id = static_cast<table_id_t>(mach_read_from_8(field));
field = rec_get_nth_field_old(
rec, DICT_FLD__SYS_TABLES__SPACE, &len);
ut_ad(len == 4);
*space_id = mach_read_from_4(field);
table_rec.space_id = mach_read_from_4(field);
/* Read the 4 byte flags from the TYPE field */
field = rec_get_nth_field_old(
@@ -806,9 +820,10 @@ dict_sys_tables_rec_read(
field = rec_get_nth_field_old(
rec, DICT_FLD__SYS_TABLES__N_COLS, &len);
ut_a(len == 4);
*n_cols = mach_read_from_4(field);
table_rec.n_cols = mach_read_from_4(field);
const bool not_redundant = 0 != (*n_cols & DICT_N_COLS_COMPACT);
const bool not_redundant = 0 != (table_rec.n_cols
& DICT_N_COLS_COMPACT);
if (!dict_sys_tables_type_valid(type, not_redundant)) {
sql_print_error("InnoDB: Table %.*s in InnoDB"
@@ -816,7 +831,7 @@ dict_sys_tables_rec_read(
" SYS_TABLES.TYPE=" UINT32PF
" SYS_TABLES.N_COLS=" UINT32PF,
int(rec_get_field_start_offs(rec, 1)), rec,
type, *n_cols);
type, table_rec.n_cols);
err_exit:
if (UNIV_LIKELY_NULL(heap)) {
mem_heap_free(heap);
@@ -824,23 +839,24 @@ err_exit:
return READ_ERROR;
}
*flags = dict_sys_tables_type_to_tf(type, not_redundant);
table_rec.flags = dict_sys_tables_type_to_tf(type, not_redundant);
/* For tables created before MySQL 4.1, there may be
garbage in SYS_TABLES.MIX_LEN where flags2 are found. Such tables
would always be in ROW_FORMAT=REDUNDANT which do not have the
high bit set in n_cols, and flags would be zero.
MySQL 4.1 was the first version to support innodb_file_per_table,
that is, *space_id != 0. */
if (not_redundant || *space_id != 0 || *n_cols & DICT_N_COLS_COMPACT
that is, table_rec.space_id != 0. */
if (not_redundant || table_rec.space_id != 0
|| table_rec.n_cols & DICT_N_COLS_COMPACT
|| fil_system.sys_space->full_crc32()) {
/* Get flags2 from SYS_TABLES.MIX_LEN */
field = rec_get_nth_field_old(
rec, DICT_FLD__SYS_TABLES__MIX_LEN, &len);
*flags2 = mach_read_from_4(field);
table_rec.flags2 = mach_read_from_4(field);
if (!dict_tf2_is_valid(*flags, *flags2)) {
if (!dict_tf2_is_valid(table_rec.flags, table_rec.flags2)) {
sql_print_error("InnoDB: Table %.*s in InnoDB"
" data dictionary"
" contains invalid flags."
@@ -848,17 +864,17 @@ err_exit:
" SYS_TABLES.MIX_LEN=" UINT32PF,
int(rec_get_field_start_offs(rec, 1)),
rec,
type, *flags2);
type, table_rec.flags2);
goto err_exit;
}
/* DICT_TF2_FTS will be set when indexes are being loaded */
*flags2 &= ~DICT_TF2_FTS;
table_rec.flags2 &= ~DICT_TF2_FTS;
/* Now that we have used this bit, unset it. */
*n_cols &= ~DICT_N_COLS_COMPACT;
table_rec.n_cols &= ~DICT_N_COLS_COMPACT;
} else {
*flags2 = 0;
table_rec.flags2 = 0;
}
if (UNIV_LIKELY_NULL(heap)) {
@@ -913,11 +929,7 @@ void dict_check_tablespaces_and_store_max_id(const std::set<uint32_t> *spaces)
dict_sys.sys_tables);
rec; rec = dict_getnext_system_low(&pcur, &mtr)) {
ulint len;
table_id_t table_id;
uint32_t space_id;
uint32_t n_cols;
uint32_t flags;
uint32_t flags2;
dict_sys_tables_rec table_rec;
/* If a table record is not useable, ignore it and continue
on to the next record. Error messages were logged. */
@@ -932,11 +944,9 @@ void dict_check_tablespaces_and_store_max_id(const std::set<uint32_t> *spaces)
DBUG_PRINT("dict_check_sys_tables",
("name: %*.s", static_cast<int>(len), field));
if (dict_sys_tables_rec_read(rec, false,
&mtr, &table_id, &space_id,
&n_cols, &flags, &flags2, nullptr)
!= READ_OK
|| space_id == TRX_SYS_SPACE) {
if (dict_sys_tables_rec_read(rec, false, &mtr, table_rec,
false) != READ_OK
|| table_rec.space_id == TRX_SYS_SPACE) {
continue;
}
@@ -948,16 +958,17 @@ void dict_check_tablespaces_and_store_max_id(const std::set<uint32_t> *spaces)
newly created or rebuilt tables or partitions, but
will otherwise ignore the flag. */
if (fil_space_for_table_exists_in_mem(space_id, flags)) {
if (fil_space_for_table_exists_in_mem(table_rec.space_id,
table_rec.flags)) {
continue;
}
if (spaces && spaces->find(uint32_t(space_id))
if (spaces && spaces->find(uint32_t(table_rec.space_id))
== spaces->end()) {
continue;
}
if (flags2 & DICT_TF2_DISCARDED) {
if (table_rec.flags2 & DICT_TF2_DISCARDED) {
sql_print_information("InnoDB: Ignoring tablespace"
" for %.*s because "
"the DISCARD flag is set",
@@ -973,7 +984,8 @@ void dict_check_tablespaces_and_store_max_id(const std::set<uint32_t> *spaces)
const bool not_dropped{!rec_get_deleted_flag(rec, 0)};
/* Check that the .ibd file exists. */
if (fil_ibd_open(space_id, dict_tf_to_fsp_flags(flags),
if (fil_ibd_open(table_rec.space_id,
dict_tf_to_fsp_flags(table_rec.flags),
not_dropped
? fil_space_t::VALIDATE_NOTHING
: fil_space_t::MAYBE_MISSING,
@@ -994,7 +1006,7 @@ void dict_check_tablespaces_and_store_max_id(const std::set<uint32_t> *spaces)
static_cast<int>(len), field);
}
max_space_id = ut_max(max_space_id, space_id);
max_space_id = ut_max(max_space_id, table_rec.space_id);
ut_free(filepath);
}
@@ -2192,12 +2204,11 @@ Do not load any columns or indexes.
@retval nullptr on success, or if the record is not visible, in
which case *table will be nullptr */
const char *dict_load_table_low(mtr_t *mtr, bool uncommitted,
const rec_t *rec, dict_table_t **table)
const rec_t *rec, dict_table_t **table,
bool *uncommitted_rec)
{
table_id_t table_id;
uint32_t space_id, t_num, flags, flags2;
dict_sys_tables_rec table_rec;
ulint n_cols, n_v_col;
trx_id_t trx_id;
if (const char* error_text = dict_sys_tables_rec_check(rec)) {
*table = NULL;
@@ -2205,23 +2216,26 @@ const char *dict_load_table_low(mtr_t *mtr, bool uncommitted,
}
if (auto r = dict_sys_tables_rec_read(rec, uncommitted, mtr,
&table_id, &space_id,
&t_num, &flags, &flags2,
&trx_id)) {
table_rec)) {
*table = NULL;
return r == READ_ERROR ? dict_load_table_flags : nullptr;
}
dict_table_decode_n_col(t_num, &n_cols, &n_v_col);
if (uncommitted_rec) {
*uncommitted_rec = table_rec.uncommitted;
}
dict_table_decode_n_col(table_rec.n_cols, &n_cols, &n_v_col);
*table = dict_table_t::create(
span<const char>(reinterpret_cast<const char*>(rec),
rec_get_field_start_offs(rec, 1)),
nullptr, n_cols + n_v_col, n_v_col, flags, flags2);
(*table)->space_id = space_id;
(*table)->id = table_id;
(*table)->file_unreadable = !!(flags2 & DICT_TF2_DISCARDED);
(*table)->def_trx_id = trx_id;
nullptr, n_cols + n_v_col, n_v_col,
table_rec.flags, table_rec.flags2);
(*table)->space_id = table_rec.space_id;
(*table)->id = table_rec.table_id;
(*table)->file_unreadable = !!(table_rec.flags2 & DICT_TF2_DISCARDED);
(*table)->def_trx_id = table_rec.trx_id;
return(NULL);
}
@@ -2388,8 +2402,10 @@ err_exit:
}
dict_table_t* table;
bool uncommitted_rec = false;
if (const char* err_msg =
dict_load_table_low(&mtr, uncommitted, rec, &table)) {
dict_load_table_low(&mtr, uncommitted, rec, &table,
&uncommitted_rec)) {
if (err_msg != dict_load_table_flags) {
ib::error() << err_msg;
}
@@ -2405,6 +2421,44 @@ err_exit:
rec + rec_get_field_start_offs(
rec, DICT_FLD__SYS_TABLES__ID));
/* This is the first attempt (not READ UNCOMMITTED), and the table
identifier was not changed by an uncommitted transaction.
If the current version of this record was written by
a transaction that has not been committed,
dict_sys_tables_rec_read() read an older version of it.
When that transaction is an instant ALTER TABLE, the
clustered index metadata record already describes the
table definition that the transaction is creating, and
we must load that definition instead of the preceding
one.
This cannot be detected while reading SYS_COLUMNS. The
number of SYS_COLUMNS records that dict_load_columns()
reads is derived from SYS_TABLES.N_COLS, which we would
be reading from the older version. A column that the
operation appended is located after that many records,
and its SYS_COLUMNS record would never be read. Here we
are looking at a record that is located by table name,
so no such limit applies.
A delete-marked record is excluded. SYS_TABLES.NAME is
the clustered index key, so RENAME TABLE delete-marks the
record of the old name and inserts one for the new name.
The definition that corresponds to the old name is the
one that precedes the rename, which is what
dict_sys_tables_rec_read() already returned. Every
operation that modifies a table definition in place,
including instant ALTER TABLE, updates a non-key column
and leaves the record unmarked. */
if (use_uncommitted == 1 && uncommitted_rec
&& !rec_get_deleted_flag(rec, 0)) {
uncommitted = true;
mtr.commit();
dict_mem_table_free(table);
goto reload;
}
mtr.commit();
mem_heap_t* heap = mem_heap_create(32000);

91
storage/innobase/handler/handler0alter.cc
View File

@@ -49,6 +49,7 @@ Smart ALTER TABLE
#include "row0upd.h"
#include "trx0trx.h"
#include "trx0purge.h"
#include "trx0rec.h"
#include "handler0alter.h"
#include "srv0mon.h"
#include "srv0srv.h"
@@ -2629,6 +2630,10 @@ innodb_instant_alter_column_allowed_reason:
need_rebuild = need_rebuild
|| innobase_need_rebuild(ha_alter_info, table);
/* Size of the undo log record that innobase_instant_try() will
write if it updates the metadata record. */
ulint instant_undo_size = 50 + FIELD_REF_SIZE;
for (Create_field& cf : ha_alter_info->alter_info->create_list) {
DBUG_ASSERT(cf.field
|| (ha_alter_info->handler_flags
@@ -2675,6 +2680,33 @@ innodb_instant_alter_column_allowed_reason:
ER_ALTER_OPERATION_NOT_SUPPORTED_REASON_NOT_NULL);
} else if (!is_non_const_value(*af)
&& set_default_value(*af)) {
if ((*af)->stored_in_db()) {
instant_undo_size += 10;
if (!(*af)->is_real_null()) {
switch ((*af)->type()) {
case MYSQL_TYPE_VARCHAR:
instant_undo_size +=
reinterpret_cast<const
Field_varstring*>(*af)
->get_length();
break;
case MYSQL_TYPE_GEOMETRY:
case MYSQL_TYPE_TINY_BLOB:
case MYSQL_TYPE_MEDIUM_BLOB:
case MYSQL_TYPE_BLOB:
case MYSQL_TYPE_LONG_BLOB:
instant_undo_size +=
reinterpret_cast<const
Field_blob*>(*af)
->get_length();
break;
default:
instant_undo_size +=
(*af)->pack_length();
}
}
}
if (fulltext_indexes > 1
&& !my_strcasecmp(system_charset_info,
(*af)->field_name.str,
@@ -2696,9 +2728,41 @@ next_column:
af++;
}
const bool supports_instant = instant_alter_column_possible(
*m_prebuilt->table, ha_alter_info, table, altered_table,
is_innodb_strict_mode());
/* An undo log record is never split between pages. If the
metadata record already exists, it will be updated rather than
inserted, and trx_undo_page_report_modify() would have to write
an undo log record of the above size, plus the PRIMARY KEY of
the metadata record. */
const dict_index_t* pk = dict_table_get_first_index(
m_prebuilt->table);
ulint pk_size = ulint{pk->trx_id_offset};
if (!pk_size) {
/* Some PRIMARY KEY column is of variable length or is
a column prefix. The variable-length columns will be
written with zero length. */
for (uint i = pk->n_uniq; i--; ) {
pk_size += ulint{pk->fields[i].fixed_len};
}
}
pk_size += ulint{pk->n_uniq} * 5;
const bool instant_undo_too_big = pk->is_instant()
&& instant_undo_size + pk_size
> trx_undo_max_rec_size();
if (instant_undo_too_big) {
ha_alter_info->unsupported_reason =
"DEFAULT values are too large for the undo log"
" record that updates the instant ALTER TABLE"
" metadata";
}
const bool supports_instant = !instant_undo_too_big
&& instant_alter_column_possible(
*m_prebuilt->table, ha_alter_info, table,
altered_table, is_innodb_strict_mode());
if (add_drop_v_cols) {
ulonglong flags = ha_alter_info->handler_flags;
@@ -6310,22 +6374,11 @@ func_exit:
&offsets, &offsets_heap, ctx->heap,
&big_rec, update, UPD_NODE_NO_ORD_CHANGE,
thr, trx->id, &mtr);
if (err == DB_SUCCESS) {
offsets = rec_get_offsets(
btr_pcur_get_rec(&pcur), index, offsets,
index->n_core_fields, ULINT_UNDEFINED,
&offsets_heap);
}
if (big_rec) {
if (err == DB_SUCCESS) {
err = btr_store_big_rec_extern_fields(
&pcur, offsets, big_rec, &mtr,
BTR_STORE_UPDATE);
}
dtuple_big_rec_free(big_rec);
}
/* Any off-page columns of the metadata record were
stored by btr_cur_pessimistic_update() before the
record itself was updated, so that the record never
contains incomplete BLOB pointers. */
ut_ad(!big_rec);
if (offsets_heap) {
mem_heap_free(offsets_heap);
}

44
storage/innobase/include/btr0cur.h
View File

@@ -537,6 +537,50 @@ btr_store_big_rec_extern_fields(
enum blob_op op) /*! in: operation code */
MY_ATTRIBUTE((nonnull, warn_unused_result));
/** Store the off-page columns of a metadata record before the record
itself is inserted into or updated in the clustered index, and write
complete BLOB pointers into the index entry.
Unlike btr_store_big_rec_extern_fields(), this is invoked before the
metadata record exists in the index. The record will be written with
complete BLOB pointers in one atomic mini-transaction, so that no
metadata record with zero (incomplete) BLOB pointers can ever be
observed, even if the server is killed in the middle of the operation.
If the server is killed before the metadata record itself is written,
the already written BLOB pages will merely be orphaned.
@param index clustered index
@param entry metadata index entry, whose externally stored fields
were shortened by dtuple_convert_big_rec() and end in
zero BLOB pointers that will be filled in
@param big_rec externally stored fields created by
dtuple_convert_big_rec() from entry
@param free_check whether it is safe to invoke log_free_check()
(the caller must not be holding any index or page latches)
@return DB_SUCCESS or error code
On failure, any BLOB pages that were already allocated will be freed,
and the BLOB pointers in entry will be reset to zero. */
dberr_t btr_store_big_rec_metadata(dict_index_t *index, dtuple_t *entry,
const big_rec_t *big_rec, bool free_check)
MY_ATTRIBUTE((nonnull, warn_unused_result));
/** Free the BLOB pages that were allocated by
btr_store_big_rec_metadata(), after the insert or update of the
metadata record itself failed, and reset the BLOB pointers in the
index entry to zero.
@param index clustered index
@param entry metadata index entry
@param big_rec externally stored fields created by
dtuple_convert_big_rec() from entry
@param caller_mtr the mini-transaction of the caller, or NULL to free
each BLOB page in a separate mini-transaction. It must
be specified if the caller holds a mini-transaction
that may have acquired the latch of the tablespace,
because mtr_t::x_lock_space() would acquire it again. */
void btr_free_big_rec_metadata(dict_index_t *index, dtuple_t *entry,
const big_rec_t *big_rec, mtr_t *caller_mtr)
MY_ATTRIBUTE((nonnull(1,2,3)));
/*******************************************************************//**
Frees the space in an externally stored field to the file space
management if the field in data is owned the externally stored field,

8
storage/innobase/include/dict0load.h
View File

@@ -122,11 +122,15 @@ Do not load any columns or indexes.
@param[in] uncommitted whether to use READ UNCOMMITTED isolation level
@param[in] rec SYS_TABLES record
@param[out,own] table table, or nullptr
@param[out] uncommitted_rec whether the current version of the SYS_TABLES
record was written by a transaction that has
not been committed
@return error message
@retval nullptr on success */
const char *dict_load_table_low(mtr_t *mtr, bool uncommitted,
const rec_t *rec, dict_table_t **table)
MY_ATTRIBUTE((nonnull, warn_unused_result));
const rec_t *rec, dict_table_t **table,
bool *uncommitted_rec= nullptr)
MY_ATTRIBUTE((nonnull(1,3,4), warn_unused_result));
/********************************************************************//**
This function parses a SYS_INDEXES record and populate a dict_index_t

58
storage/innobase/include/trx0rec.h
View File

@@ -122,6 +122,28 @@ trx_undo_update_rec_get_update(
@return DB_SUCCESS or error code */
dberr_t trx_undo_report_rename(trx_t* trx, const dict_table_t* table)
MY_ATTRIBUTE((nonnull, warn_unused_result));
/** The change to a clustered index record that is being reported to the
undo log. Which fields are set depends on the operation: clust_entry for
an insert, rec and offsets for an update or a delete marking, and update
and cmpl_info only for an update. */
struct trx_undo_row_op
{
/** in the case of an insert, the index entry to insert into the
clustered index; in updates, may contain a clustered index record
tuple that also contains virtual columns of the table; otherwise
NULL */
const dtuple_t* clust_entry;
/** in the case of an update, the update vector, otherwise NULL */
const upd_t* update;
/** compiler info on secondary index updates */
ulint cmpl_info;
/** in the case of an update or delete marking, the record in the
clustered index; NULL if insert */
const rec_t* rec;
/** rec_get_offsets(rec) */
const rec_offs* offsets;
};
/***********************************************************************//**
Writes information to an undo log about an insert, update, or a delete marking
of a clustered index record. This information is used in a rollback of the
@@ -131,27 +153,25 @@ transaction.
dberr_t
trx_undo_report_row_operation(
/*==========================*/
que_thr_t* thr, /*!< in: query thread */
dict_index_t* index, /*!< in: clustered index */
const dtuple_t* clust_entry, /*!< in: in the case of an insert,
index entry to insert into the
clustered index; in updates,
may contain a clustered index
record tuple that also contains
virtual columns of the table;
otherwise, NULL */
const upd_t* update, /*!< in: in the case of an update,
the update vector, otherwise NULL */
ulint cmpl_info, /*!< in: compiler info on secondary
index updates */
const rec_t* rec, /*!< in: case of an update or delete
marking, the record in the clustered
index; NULL if insert */
const rec_offs* offsets, /*!< in: rec_get_offsets(rec) */
roll_ptr_t* roll_ptr) /*!< out: DB_ROLL_PTR to the
undo log record */
que_thr_t* thr, /*!< in: query thread */
dict_index_t* index, /*!< in: clustered index */
const trx_undo_row_op& op, /*!< in: the row change */
roll_ptr_t* roll_ptr,
/*!< out: DB_ROLL_PTR to the undo log record */
mtr_t* caller_mtr)
/*!< in/out: the mini-transaction that is
going to modify op.rec, so that the undo
log record and the modification of op.rec
cannot be separated by a crash; NULL to
use a separate mini-transaction */
MY_ATTRIBUTE((nonnull(1,2), warn_unused_result));
/** Determine how large an undo log record may be. An undo log record
is never split between pages, so a record that does not fit on an empty
undo log page cannot be written at all.
@return the maximum size of an undo log record, in bytes */
ulint trx_undo_max_rec_size();
/** status bit used for trx_undo_prev_version_build() */
/** TRX_UNDO_PREV_IN_PURGE tells trx_undo_prev_version_build() that it

4
storage/innobase/include/trx0undo.h
View File

@@ -134,7 +134,9 @@ void trx_undo_page_init(const buf_block_t &block);
/** Allocate an undo log page.
@param[in,out] undo undo log
@param[in,out] mtr mini-transaction that does not hold any page latch
@param[in,out] mtr mini-transaction. Until it is committed, the
calling thread must not allocate or free pages of
the tablespace of undo in another mini-transaction.
@param[out] err error code
@return X-latched block if success
@retval nullptr on failure */

21
storage/innobase/rem/rem0rec.cc
View File

@@ -1141,8 +1141,16 @@ rec_get_converted_size_comp_prefix_low(
ut_ad(index->table->instant);
ut_ad(!redundant_temp && index->is_instant());
ut_ad(status == REC_STATUS_INSTANT);
ut_ad(n_fields == ulint(index->n_fields) + 1);
extra_size += UT_BITS_IN_BYTES(index->n_nullable)
/* The number of fields must be taken from the tuple, and
not from the table definition. The rollback of instant
ALTER TABLE shortens the metadata record in
btr_cur_trim_alter_metadata(), while the table definition
still describes the operation that is being rolled back.
rec_init_offsets_comp_ordinary() reads the number of
fields from the record in the same way. */
ut_ad(n_fields <= ulint(index->n_fields) + 1);
extra_size += UT_BITS_IN_BYTES(
index->get_n_nullable(n_fields - 1))
+ rec_get_n_add_field_len(n_fields - 1
- n_core_fields);
} else if (status == REC_STATUS_INSTANT
@@ -1173,7 +1181,7 @@ rec_get_converted_size_comp_prefix_low(
if (mblob && i == index->first_user_field()) {
data_size += FIELD_REF_SIZE;
if (++dfield == end) {
ut_ad(i == index->n_fields);
ut_ad(i + 1 == n_fields);
break;
}
}
@@ -1497,14 +1505,15 @@ rec_convert_dtuple_to_rec_comp(
ut_ad(index->table->instant);
ut_ad(!redundant_temp && index->is_instant());
ut_ad(status == REC_STATUS_INSTANT);
ut_ad(n_fields == ulint(index->n_fields) + 1);
ut_ad(n_fields <= ulint(index->n_fields) + 1);
rec_set_n_add_field(nulls, n_fields - 1 - n_core_fields);
rec_set_bit_field_2(rec, PAGE_HEAP_NO_USER_LOW,
REC_NEW_HEAP_NO, REC_HEAP_NO_MASK,
REC_HEAP_NO_SHIFT);
rec_set_status(rec, REC_STATUS_INSTANT);
n_node_ptr_field = ULINT_UNDEFINED;
lens = nulls - UT_BITS_IN_BYTES(index->n_nullable);
lens = nulls - UT_BITS_IN_BYTES(
index->get_n_nullable(n_fields - 1));
goto start;
}
switch (status) {
@@ -1574,7 +1583,7 @@ start:
memcpy(end, dfield_get_data(field), len);
end += len;
if (++field == fend) {
ut_ad(i == index->n_fields);
ut_ad(i + 1 == n_fields);
break;
}
len = dfield_get_len(field);

58
storage/innobase/row/row0ins.cc
View File

@@ -31,6 +31,7 @@ Created 4/20/1996 Heikki Tuuri
#include "trx0undo.h"
#include "btr0btr.h"
#include "btr0cur.h"
#include "page0zip.h"
#include "mach0data.h"
#include "ibuf0ibuf.h"
#include "que0que.h"
@@ -2662,6 +2663,7 @@ row_ins_clust_index_entry_low(
btr_pcur_t pcur;
dberr_t err = DB_SUCCESS;
big_rec_t* big_rec = NULL;
big_rec_t* metadata_big = NULL;
mtr_t mtr;
uint64_t auto_inc = 0;
mem_heap_t* offsets_heap = NULL;
@@ -2679,6 +2681,44 @@ row_ins_clust_index_entry_low(
ut_ad(!n_uniq || n_uniq == dict_index_get_n_unique(index));
ut_ad(!trx->in_rollback);
if (UNIV_UNLIKELY(entry->info_bits != 0)
&& (entry->is_alter_metadata()
? !dfield_is_ext(dtuple_get_nth_field(
entry,
index->first_user_field()))
: page_zip_rec_needs_ext(
rec_get_converted_size(index, entry, n_ext),
index->table->not_redundant(),
dtuple_get_n_fields(entry), 0))) {
ut_ad(entry->is_metadata());
ut_ad(index->is_instant());
ut_ad(mode == BTR_MODIFY_TREE);
ut_ad(flags == BTR_NO_LOCKING_FLAG);
metadata_big = dtuple_convert_big_rec(index, 0, entry, &n_ext);
if (UNIV_UNLIKELY(!metadata_big)) {
DBUG_RETURN(DB_TOO_BIG_RECORD);
}
/* Store the off-page columns and write complete BLOB
pointers into entry before the metadata record is
inserted, so that no metadata record with incomplete
(zero) BLOB pointers can ever be written. If the server
is killed before the metadata record is inserted below,
the BLOB pages will merely be orphaned. No latches are
being held yet; it is safe to invoke log_free_check(). */
err = btr_store_big_rec_metadata(index, entry,
metadata_big, true);
if (err != DB_SUCCESS) {
/* The BLOB pages were already freed and the
BLOB pointers in entry were reset by
btr_store_big_rec_metadata(). */
dtuple_convert_back_big_rec(index, entry,
metadata_big);
DBUG_RETURN(err);
}
}
mtr.start();
if (index->table->is_temporary()) {
@@ -2826,9 +2866,9 @@ avoid_bulk:
/* Write TRX_UNDO_EMPTY undo log and
start buffering the insert operation */
err = trx_undo_report_row_operation(
thr, index, entry,
nullptr, 0, nullptr, nullptr,
nullptr);
thr, index,
{entry, nullptr, 0, nullptr, nullptr},
nullptr, nullptr);
if (err != DB_SUCCESS) {
goto avoid_bulk;
@@ -3006,6 +3046,18 @@ do_insert:
}
func_exit:
if (UNIV_UNLIKELY(metadata_big != NULL)) {
if (err != DB_SUCCESS) {
/* The mini-transaction was committed on every
path that leads here, so no latch of the
tablespace can be held. */
btr_free_big_rec_metadata(index, entry,
metadata_big, nullptr);
}
dtuple_convert_back_big_rec(index, entry, metadata_big);
}
if (offsets_heap != NULL) {
mem_heap_free(offsets_heap);
}

126
storage/innobase/trx/trx0rec.cc
View File

@@ -70,6 +70,15 @@ static ulint trx_undo_left(const buf_block_t *undo_block, const byte *ptr)
return left < 0 ? 0 : static_cast<ulint>(left);
}
ulint trx_undo_max_rec_size()
{
/* This must agree with trx_undo_left() on a freshly initialized
undo log page, where TRX_UNDO_PAGE_FREE is
TRX_UNDO_PAGE_HDR + TRX_UNDO_PAGE_HDR_SIZE. */
return srv_page_size - (TRX_UNDO_PAGE_HDR + TRX_UNDO_PAGE_HDR_SIZE) -
(10 + FIL_PAGE_DATA_END);
}
/**********************************************************************//**
Set the next and previous pointers in the undo page for the undo record
that was written to ptr. Update the first free value by the number of bytes
@@ -1840,25 +1849,17 @@ transaction.
dberr_t
trx_undo_report_row_operation(
/*==========================*/
que_thr_t* thr, /*!< in: query thread */
dict_index_t* index, /*!< in: clustered index */
const dtuple_t* clust_entry, /*!< in: in the case of an insert,
index entry to insert into the
clustered index; in updates,
may contain a clustered index
record tuple that also contains
virtual columns of the table;
otherwise, NULL */
const upd_t* update, /*!< in: in the case of an update,
the update vector, otherwise NULL */
ulint cmpl_info, /*!< in: compiler info on secondary
index updates */
const rec_t* rec, /*!< in: case of an update or delete
marking, the record in the clustered
index; NULL if insert */
const rec_offs* offsets, /*!< in: rec_get_offsets(rec) */
roll_ptr_t* roll_ptr) /*!< out: DB_ROLL_PTR to the
undo log record */
que_thr_t* thr, /*!< in: query thread */
dict_index_t* index, /*!< in: clustered index */
const trx_undo_row_op& op, /*!< in: the row change */
roll_ptr_t* roll_ptr,
/*!< out: DB_ROLL_PTR to the undo log record */
mtr_t* caller_mtr)
/*!< in/out: the mini-transaction that is
going to modify op.rec, so that the undo
log record and the modification of op.rec
cannot be separated by a crash; NULL to
use a separate mini-transaction */
{
trx_t* trx;
#ifdef UNIV_DEBUG
@@ -1866,8 +1867,8 @@ trx_undo_report_row_operation(
#endif /* UNIV_DEBUG */
ut_a(dict_index_is_clust(index));
ut_ad(!update || rec);
ut_ad(!rec || rec_offs_validate(rec, index, offsets));
ut_ad(!op.update || op.rec);
ut_ad(!op.rec || rec_offs_validate(op.rec, index, op.offsets));
ut_ad(!srv_read_only_mode);
trx = thr_get_trx(thr);
@@ -1885,7 +1886,7 @@ trx_undo_report_row_operation(
trx->apply_online_log= true;
}
bool bulk = !rec;
bool bulk = !op.rec;
if (!bulk) {
/* An UPDATE or DELETE must not be covered by an
@@ -1911,22 +1912,34 @@ trx_undo_report_row_operation(
thd_sql_command(trx->mysql_thd) != SQLCOM_LOAD);
if (dberr_t err = m.first->second.bulk_insert_buffered(
*clust_entry, *index, trx)) {
*op.clust_entry, *index, trx)) {
return err;
}
} else {
bulk = false;
}
mtr_t mtr;
/* When the undo log record must be written in the same
mini-transaction as the modification of op.rec, we must never
commit or restart that mini-transaction here. */
mtr_t own_mtr;
mtr_t& mtr = caller_mtr ? *caller_mtr : own_mtr;
dberr_t err;
mtr.start();
if (!caller_mtr) {
mtr.start();
}
trx_undo_t** pundo;
trx_rseg_t* rseg;
const bool is_temp = index->table->is_temporary();
buf_block_t* undo_block;
if (is_temp) {
/* A temporary table never has an instant ALTER TABLE
metadata record, so the caller's mini-transaction is
never used for one. That is essential here: changing the
log mode would also disable redo logging for the page
modifications that the caller has already buffered. */
ut_ad(!caller_mtr);
mtr.set_log_mode(MTR_LOG_NO_REDO);
rseg = trx->get_temp_rseg();
pundo = &trx->rsegs.m_noredo.undo;
@@ -1945,7 +1958,7 @@ trx_undo_report_row_operation(
undo_block = trx_undo_assign_low<false>(trx, rseg, pundo,
&mtr, &err);
if (clear_ignore) {
ut_ad(!rec);
ut_ad(!op.rec);
m = trx_undo_rewrite_ignore(&mtr, undo_block,
index->table, trx, *pundo);
}
@@ -1955,20 +1968,23 @@ trx_undo_report_row_operation(
ut_ad((err == DB_SUCCESS) == (undo_block != NULL));
if (UNIV_UNLIKELY(undo_block == NULL)) {
err_exit:
mtr.commit();
if (!caller_mtr) {
mtr.commit();
}
return err;
}
ut_ad(undo != NULL);
do {
uint16_t offset = !rec
uint16_t offset = !op.rec
? trx_undo_page_report_insert(
undo_block, trx, index, clust_entry, &mtr,
undo_block, trx, index, op.clust_entry, &mtr,
bulk)
: trx_undo_page_report_modify(
undo_block, trx, index, rec, offsets, update,
cmpl_info, clust_entry, &mtr);
undo_block, trx, index, op.rec,
op.offsets, op.update, op.cmpl_info,
op.clust_entry, &mtr);
if (UNIV_UNLIKELY(offset == 0)) {
const uint16_t first_free = mach_read_from_2(
@@ -1991,7 +2007,26 @@ err_exit:
tree latch, which is the rseg
mutex. We must commit the mini-transaction
first, because it may be holding lower-level
latches, such as SYNC_FSP_PAGE. */
latches, such as SYNC_FSP_PAGE.
We cannot commit the caller's
mini-transaction. In that case, keep the
page allocated to the undo log; it will be
used by the next undo log record, or freed
when the transaction ends. The unused tail
of the page was zero-filled above, and
that must be redo logged, just like in the
non-empty case below. */
if (caller_mtr) {
mtr.memset(*undo_block, first_free,
srv_page_size - first_free
- FIL_PAGE_DATA_END, 0);
if (m.second) {
trx->mod_tables.erase(m.first);
}
err = DB_UNDO_RECORD_TOO_BIG;
goto err_exit;
}
mtr.commit();
mtr.start();
@@ -2030,11 +2065,15 @@ err_exit:
- FIL_PAGE_DATA_END, 0);
}
mtr.commit();
if (!caller_mtr) {
mtr.commit();
}
} else {
/* Success */
undo->top_page_no = undo_block->page.id().page_no();
mtr.commit();
if (!caller_mtr) {
mtr.commit();
}
undo->old_offset = offset;
undo->top_offset = offset;
@@ -2048,16 +2087,18 @@ err_exit:
if (!time.is_versioned()
&& index->table->versioned_by_id()
&& (!rec /* INSERT */
|| (update
&& update->affects_versioned()))) {
&& (!op.rec /* INSERT */
|| (op.update
&& op.update
->affects_versioned()))) {
time.set_versioned(undo->top_undo_no);
}
}
if (!bulk) {
*roll_ptr = trx_undo_build_roll_ptr(
!rec, trx_sys.rseg_id(rseg, !is_temp),
!op.rec,
trx_sys.rseg_id(rseg, !is_temp),
undo->top_page_no, offset);
}
@@ -2069,10 +2110,13 @@ err_exit:
/* We have to extend the undo log by one page */
ut_ad(++loop_count < 2);
mtr.start();
if (is_temp) {
mtr.set_log_mode(MTR_LOG_NO_REDO);
if (!caller_mtr) {
mtr.start();
if (is_temp) {
mtr.set_log_mode(MTR_LOG_NO_REDO);
}
}
undo_block = trx_undo_add_page(undo, &mtr, &err);

4
storage/innobase/trx/trx0undo.cc
View File

@@ -640,7 +640,9 @@ trx_undo_read_xid(const trx_ulogf_t* log_hdr, XID* xid)
/** Allocate an undo log page.
@param[in,out] undo undo log
@param[in,out] mtr mini-transaction that does not hold any page latch
@param[in,out] mtr mini-transaction. Until it is committed, the
calling thread must not allocate or free pages of
the tablespace of undo in another mini-transaction.
@param[out] err error code
@return X-latched block if success
@retval nullptr on failure */