MDEV-30671 InnoDB undo log truncation fails to wait for purge of history

It is not safe to invoke trx_purge_free_segment() or execute
innodb_undo_log_truncate=ON before all undo log records in
the rollback segment has been processed.

A prominent failure that would occur due to premature freeing of
undo log pages is that trx_undo_get_undo_rec() would crash when
trying to copy an undo log record to fetch the previous version
of a record.

If trx_undo_get_undo_rec() was not invoked in the unlucky time frame,
then the symptom would be that some committed transaction history is
never removed. This would be detected by CHECK TABLE...EXTENDED that
was impleented in commit ab0190101b.
Such a garbage collection leak should be possible even when using
innodb_undo_log_truncate=OFF, just involving trx_purge_free_segment().

trx_rseg_t::needs_purge: Change the type from Boolean to a transaction
identifier, noting the most recent non-purged transaction, or 0 if
everything has been purged. On transaction start, we initialize this
to 1 more than the transaction start ID. On recovery, the field may be
adjusted to the transaction end ID (TRX_UNDO_TRX_NO) if it is larger.

The field TRX_UNDO_NEEDS_PURGE becomes write-only; only some debug
assertions that would validate the value. The field reflects the old
inaccurate Boolean field trx_rseg_t::needs_purge.

trx_undo_mem_create_at_db_start(), trx_undo_lists_init(),
trx_rseg_mem_restore(): Remove the parameter max_trx_id.
Instead, store the maximum in trx_rseg_t::needs_purge,
where trx_rseg_array_init() will find it.

trx_purge_free_segment(): Contiguously hold a lock on
trx_rseg_t to prevent any concurrent allocation of undo log.

trx_purge_truncate_rseg_history(): Only invoke trx_purge_free_segment()
if the rollback segment is empty and there are no pending transactions
associated with it.

trx_purge_truncate_history(): Only proceed with innodb_undo_log_truncate=ON
if trx_rseg_t::needs_purge indicates that all history has been purged.

Tested by: Matthias Leich

mysql-test/suite/innodb/t/innodb_bug84958.test

@@ -13,12 +13,10 @@ DELIMITER ~~;
 CREATE PROCEDURE insert_n(start int, end int)
 BEGIN
     DECLARE i INT DEFAULT start;
-    START TRANSACTION;
     WHILE i <= end do
         INSERT INTO t1 VALUES (1, 2, 3) ON DUPLICATE KEY UPDATE c = i;
         SET i = i + 1;
     END WHILE;
-    COMMIT;
 END~~
 
 CREATE FUNCTION num_pages_get()
@@ -36,7 +34,8 @@ DELIMITER ;~~
 --echo # Create a table with one record in it and start an RR transaction
 --echo #
 CREATE TABLE t1 (a INT, b INT, c INT, PRIMARY KEY(a,b), KEY (b,c))
-ENGINE=InnoDB;
+ENGINE=InnoDB STATS_PERSISTENT=0;
+--source include/wait_all_purged.inc
 BEGIN;
 SELECT * FROM t1;
 
@@ -45,18 +44,22 @@ SELECT * FROM t1;
 --echo #
 connect (con2, localhost, root,,);
 connection con2;
+BEGIN;
 INSERT INTO t1 VALUES (1, 2, 3) ON DUPLICATE KEY UPDATE c = NULL;
 --send CALL insert_n(1, 50);
 
 connect (con3, localhost, root,,);
 connection con3;
+BEGIN;
 --send CALL insert_n(51, 100);
 
 connection con2;
 reap;
+COMMIT;
 connection con3;
 reap;
 INSERT INTO t1 VALUES (1, 2, 1) ON DUPLICATE KEY UPDATE c = NULL;
+COMMIT;
 
 connection default;
 
@@ -64,7 +67,7 @@ connection default;
 --echo # Connect to default and record how many pages were accessed
 --echo # when selecting the record using the secondary key.
 --echo #
---let $wait_all_purged=4
+--let $wait_all_purged=2
 --source include/wait_all_purged.inc
 SET @num_pages_1 = num_pages_get();
 SELECT * FROM t1 force index (b);