mirror of
https://github.com/MariaDB/server.git
synced 2025-11-10 23:02:54 +03:00
ab0190101b
Until now, the attribute EXTENDED of CHECK TABLE was ignored by InnoDB, and InnoDB only counted the records in each index according to the current read view. Unless the attribute QUICK was specified, the function btr_validate_index() would be invoked to validate the B-tree structure (the sibling and child links between index pages). The EXTENDED check will not only count all index records according to the current read view, but also ensure that any delete-marked records in the clustered index are waiting for the purge of history, and that all secondary index records point to a version of the clustered index record that is waiting for the purge of history. In other words, no index may contain orphan records. Normal MVCC reads and the non-EXTENDED version of CHECK TABLE would ignore these orphans. Unpurged records merely result in warnings (at most one per index), not errors, and no indexes will be flagged as corrupted due to such garbage. It will remain possible to SELECT data from such indexes or tables (which will skip such records) or to rebuild the table to reclaim some space. We introduce purge_sys.end_view that will be (almost) a copy of purge_sys.view at the end of a batch of purging committed transaction history. It is not an exact copy, because if the size of a purge batch is limited by innodb_purge_batch_size, some records that purge_sys.view would allow to be purged will be left over for subsequent batches. The purge_sys.view is relevant in the purge of committed transaction history, to determine if records are safe to remove. The new purge_sys.end_view is relevant in MVCC operations and in CHECK TABLE ... EXTENDED. It tells which undo log records are safe to access (have not been discarded at the end of a purge batch). purge_sys.clone_oldest_view<true>(): In trx_lists_init_at_db_start(), clone the oldest read view similar to purge_sys_t::clone_end_view() so that CHECK TABLE ... EXTENDED will not report bogus failures between InnoDB restart and the completed purge of committed transaction history. purge_sys_t::is_purgeable(): Replaces purge_sys_t::changes_visible() in the case that purge_sys.latch will not be held by the caller. Among other things, this guards access to BLOBs. It is not safe to dereference any BLOBs of a delete-marked purgeable record, because they may have already been freed. purge_sys_t::view_guard::view(): Return a reference to purge_sys.view that will be protected by purge_sys.latch, held by purge_sys_t::view_guard. purge_sys_t::end_view_guard::view(): Return a reference to purge_sys.end_view while it is protected by purge_sys.end_latch. Whenever a thread needs to retrieve an older version of a clustered index record, it will hold a page latch on the clustered index page and potentially also on a secondary index page that points to the clustered index page. If these pages contain purgeable records that would be accessed by a currently running purge batch, the progress of the purge batch would be blocked by the page latches. Hence, it is safe to make a copy of purge_sys.end_view while holding an index page latch, and consult the copy of the view to determine whether a record should already have been purged. btr_validate_index(): Remove a redundant check. row_check_index_match(): Check if a secondary index record and a version of a clustered index record match each other. row_check_index(): Replaces row_scan_index_for_mysql(). Count the records in each index directly, duplicating the relevant logic from row_search_mvcc(). Initialize check_table_extended_view for CHECK ... EXTENDED while holding an index leaf page latch. If we encounter an orphan record, the copy of purge_sys.end_view that we make is safe for visibility checks, and trx_undo_get_undo_rec() will check for the safety to access each undo log record. Should that check fail, we should return DB_MISSING_HISTORY to report a corrupted index. The EXTENDED check tries to match each secondary index record with every available clustered index record version, by duplicating the logic of row_vers_build_for_consistent_read() and invoking trx_undo_prev_version_build() directly. Before invoking row_check_index_match() on delete-marked clustered index record versions, we will consult purge_sys.is_purgeable() in order to avoid accessing freed BLOBs. We will always check that the DB_TRX_ID or PAGE_MAX_TRX_ID does not exceed the global maximum. Orphan secondary index records will be flagged only if everything up to PAGE_MAX_TRX_ID has been purged. We warn also about clustered index records whose nonzero DB_TRX_ID should have been reset in purge or rollback. trx_set_rw_mode(): Move an assertion from ReadView::set_creator_trx_id(). trx_undo_prev_version_build(): Remove two debug-only parameters, and return an error code instead of a Boolean. trx_undo_get_undo_rec(): Return a pointer to the undo log record, or nullptr if one cannot be retrieved. Instead of consulting the purge_sys.view, consult the purge_sys.end_view to determine which records can be accessed. trx_undo_get_rec_if_purgeable(): A variant of trx_undo_get_undo_rec() that will consult purge_sys.view instead of purge_sys.end_view. TRX_UNDO_CHECK_PURGEABILITY: A new parameter to trx_undo_prev_version_build(), passed by row_vers_old_has_index_entry() so that purge_sys.view instead of purge_sys.end_view will be consulted to determine whether a secondary index record may be safely purged. row_upd_changes_disowned_external(): Remove. This should be more expensive than briefly latching purge_sys in trx_undo_prev_version_build() (which may make use of transactional memory). row_sel_reset_old_vers_heap(): New function, split from row_sel_build_prev_vers_for_mysql(). row_sel_build_prev_vers_for_mysql(): Reorder some parameters to simplify the call to row_sel_reset_old_vers_heap(). row_search_for_mysql(): Replaced with direct calls to row_search_mvcc(). sel_node_get_nth_plan(): Define inline in row0sel.h open_step(): Define at the call site, in simplified form. sel_node_reset_cursor(): Merged with the only caller open_step(). --- ReadViewBase::check_trx_id_sanity(): Remove. Let us handle "future" DB_TRX_ID in a more meaningful way: row_sel_clust_sees(): Return DB_SUCCESS if the record is visible, DB_SUCCESS_LOCKED_REC if it is invisible, and DB_CORRUPTION if the DB_TRX_ID is in the future. row_undo_mod_must_purge(), row_undo_mod_clust(): Silently ignore corrupted DB_TRX_ID. We are in ROLLBACK, and we should have noticed that corruption when we were about to modify the record in the first place (leading us to refuse the operation). row_vers_build_for_consistent_read(): Return DB_CORRUPTION if DB_TRX_ID is in the future. Tested by: Matthias Leich Reviewed by: Vladislav Lesin
177 lines
4.2 KiB
Plaintext
177 lines
4.2 KiB
Plaintext
--source include/have_innodb.inc
|
|
--source include/count_sessions.inc
|
|
|
|
# Ensure that the history list length will actually be decremented by purge.
|
|
SET @saved_frequency = @@GLOBAL.innodb_purge_rseg_truncate_frequency;
|
|
SET GLOBAL innodb_purge_rseg_truncate_frequency = 1;
|
|
|
|
--echo #
|
|
--echo # Bug#21869656 UNDO LOG DOES NOT CONTAIN ENOUGH INFORMATION
|
|
--echo # ON INDEXED VIRTUAL COLUMNS
|
|
--echo #
|
|
|
|
CREATE TABLE t1 (a INT, b INT,
|
|
a1 INT GENERATED ALWAYS AS (a) VIRTUAL, INDEX(a1)
|
|
) ENGINE=InnoDB;
|
|
|
|
INSERT INTO t1 (a,b) VALUES(1,1);
|
|
|
|
connect (con1,localhost,root,,);
|
|
# disable purge
|
|
CREATE TABLE t0 (a INT) ENGINE=InnoDB;
|
|
BEGIN; SELECT * FROM t0;
|
|
|
|
connection default;
|
|
# write the problematic update_undo log record
|
|
UPDATE t1 SET a=0;
|
|
|
|
ALTER TABLE t1 DROP COLUMN a1, ALGORITHM=INPLACE;
|
|
ALTER TABLE t1 ADD COLUMN b1 INT GENERATED ALWAYS AS (b) VIRTUAL, ADD
|
|
INDEX(b1),
|
|
ALGORITHM=INPLACE;
|
|
|
|
connection con1;
|
|
# enable purge
|
|
COMMIT;
|
|
UPDATE t1 SET a=1;
|
|
|
|
connection default;
|
|
--source ../../innodb/include/wait_all_purged.inc
|
|
|
|
CHECK TABLE t1 EXTENDED;
|
|
SELECT b1 FROM t1;
|
|
|
|
|
|
# Create multi-virtual column, more ADD/DROP to test it
|
|
ALTER TABLE t1
|
|
ADD COLUMN a1 INT GENERATED ALWAYS AS (a) VIRTUAL,
|
|
ADD COLUMN a2 INT GENERATED ALWAYS AS (a + b) VIRTUAL,
|
|
ADD COLUMN a3 INT GENERATED ALWAYS AS (a - b) VIRTUAL,
|
|
ADD COLUMN a4 INT GENERATED ALWAYS AS (a - b) VIRTUAL,
|
|
ADD INDEX(a1), ADD INDEX(a2), ADD INDEX(a3), ALGORITHM=INPLACE;
|
|
|
|
CREATE TABLE t2 (
|
|
a BLOB,
|
|
b BLOB,
|
|
c BLOB GENERATED ALWAYS AS (CONCAT(a,b)) VIRTUAL,
|
|
h VARCHAR(10) DEFAULT NULL
|
|
) ENGINE=InnoDB;
|
|
|
|
INSERT INTO t2 VALUES (REPEAT('g', 16000), REPEAT('x', 16000), DEFAULT, 'kk');
|
|
|
|
INSERT INTO t2 VALUES (REPEAT('a', 16000), REPEAT('b', 16000), DEFAULT, 'mm');
|
|
|
|
CREATE INDEX idx ON t2(c(100));
|
|
|
|
INSERT INTO t1 (a, b) VALUES(1,1);
|
|
|
|
connection con1;
|
|
# disable purge
|
|
BEGIN; SELECT * FROM t0;
|
|
|
|
connection default;
|
|
--enable_info
|
|
|
|
# write the problematic update_undo log record
|
|
UPDATE t1 SET a=0;
|
|
UPDATE t1 SET b=0;
|
|
|
|
ALTER TABLE t1 DROP COLUMN a3, ALGORITHM=INPLACE;
|
|
|
|
UPDATE t1 SET a=2;
|
|
ALTER TABLE t1 DROP COLUMN a2, ALGORITHM=INPLACE;
|
|
UPDATE t1 SET b=3;
|
|
|
|
ALTER TABLE t1 ADD COLUMN b2 INT GENERATED ALWAYS AS (b) VIRTUAL,
|
|
ADD INDEX(b2), ALGORITHM=INPLACE;
|
|
UPDATE t1 SET b=9;
|
|
|
|
ALTER TABLE t1 ADD COLUMN b3 INT GENERATED ALWAYS AS (a) VIRTUAL,
|
|
ADD INDEX(b3), ALGORITHM=INPLACE;
|
|
|
|
UPDATE t1 SET b=10;
|
|
|
|
ALTER TABLE t2 DROP COLUMN c;
|
|
|
|
UPDATE t2 SET a = REPEAT('s', 6000) WHERE a like 'aaa%';
|
|
|
|
ALTER TABLE t2 ADD COLUMN x1 BLOB GENERATED ALWAYS AS (CONCAT(a,b)) VIRTUAL,
|
|
ADD COLUMN x2 BLOB GENERATED ALWAYS AS (CONCAT(a,b)) VIRTUAL,
|
|
ADD INDEX(x1(100), x2(120)), ADD INDEX (x1(20));
|
|
|
|
UPDATE t1 SET a=5;
|
|
|
|
UPDATE t2 SET a = REPEAT('m', 16000) WHERE a like 'sss%';
|
|
|
|
ALTER TABLE t1 DROP COLUMN b2, ALGORITHM=INPLACE;
|
|
|
|
UPDATE t1 SET a=6;
|
|
|
|
ALTER TABLE t2 DROP COLUMN x1, DROP COLUMN x2, ALGORITHM=INPLACE;
|
|
|
|
UPDATE t2 SET a = REPEAT('x', 1000) WHERE a like 'mmm%';
|
|
|
|
ALTER TABLE t1 DROP INDEX b3;
|
|
UPDATE t1 SET a=100;
|
|
--disable_info
|
|
|
|
connection con1;
|
|
# enable purge
|
|
COMMIT;
|
|
disconnect con1;
|
|
|
|
connection default;
|
|
--source ../../innodb/include/wait_all_purged.inc
|
|
|
|
CHECK TABLE t1 EXTENDED;
|
|
SELECT b1 FROM t1;
|
|
|
|
SELECT * FROM t1;
|
|
CHECK TABLE t2 EXTENDED;
|
|
DROP TABLE t2, t1, t0;
|
|
|
|
CREATE TABLE t1 (a VARCHAR(30), b INT, a2 VARCHAR(30) GENERATED ALWAYS AS (a) VIRTUAL);
|
|
|
|
--error ER_DUP_FIELDNAME
|
|
CREATE INDEX idx ON t1(a2(10), b, a2(20));
|
|
|
|
DROP TABLE t1;
|
|
|
|
--echo #
|
|
--echo # MDEV-17540 Server crashes in row_purge after TRUNCATE TABLE
|
|
--echo #
|
|
|
|
# Note: this test case is nondeterministic and should depend on
|
|
# MDEV-12288 to trigger the needed purge activity.
|
|
# The test does not seem to repeat the bug on MariaDB 10.2.
|
|
|
|
CREATE TABLE t1 (a BIT(14)) ENGINE=InnoDB;
|
|
INSERT INTO t1 VALUES
|
|
(b'01110110101011'),(b'01100111111000'),(b'00001011110100'),
|
|
(b'01110110111010'),(b'10001010101011'),(b'01100111001111');
|
|
|
|
CREATE TABLE t2 (
|
|
pk INT DEFAULT 1,
|
|
b YEAR,
|
|
c BIT(14),
|
|
d YEAR AS (b),
|
|
e BIT(14) AS (c),
|
|
UNIQUE(pk),
|
|
KEY(e)
|
|
) ENGINE=InnoDB;
|
|
|
|
# Run a few times in order to improve the chances of triggering the bug.
|
|
--disable_query_log
|
|
let $n=10;
|
|
while ($n) {
|
|
REPLACE INTO t2 (c) SELECT a FROM t1;
|
|
TRUNCATE TABLE t2;
|
|
dec $n;
|
|
}
|
|
--enable_query_log
|
|
|
|
DROP TABLE t1, t2;
|
|
|
|
--source include/wait_until_count_sessions.inc
|
|
SET GLOBAL innodb_purge_rseg_truncate_frequency = @saved_frequency;
|