database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2026-10-23 13:29:09 +03:00
Code Activity

Reorderbuffer: handle catalog-change-bearing aborted subxacts better

Browse Source 
When constructing a historic snapshot for logical decoding,
reorderbuffer needs to keep track of movement of tuples in catalog
tables.  For the most part this works correctly, except that when a
subtransaction that wrote a catalog tuple aborts, we failed to "forget"
about that change.  There's no known reports of anything actually
breaking except assertion failures, but it is likely possible to
construct cases where historical snapshots produce bogus visibility
answers, potentially causing incorrect decoding output.

To fix, have the decode of a subtransaction abort go over the TUPLECID
list that has been accumulated on the ReorderBufferTXN for its topmost
parent transaction, and remove all the entries that were marked as
originating from the aborted subtransaction or its children.  There are
some little details in which various devils lie, which the added code
comments explain (we hope).

Add a couple of isolation test cases that trip the assertions with the
original coding.  They're not ideal test design, because they pass in
non-assert builds without the fix.  But we'd rather have the fix be
verified in that limited scenario than nowhere at all, and they are
cheap enough.

This bug is sibling to the one fixed in commit 8c67d29fd5 and, like
that one, it goes all the way back to the introduction of
reorderbuffers, so backpatch to all live branches.

While reviewing this, I (Álvaro) was bothered by the snapmgr.c
tuplecid_data file-scoped variable, as it's easily confused with a
function parameter used widely in reorderbuffer.c.  In branches 19 and
master, rename that to avoid confusion.

Reported-by: Arseny Sher <a.sher@postgrespro.ru>
Reported-by: Mark Dilger <mark.dilger@enterprisedb.com>
Reported-by: Alexander Kozhemyakin <a.kozhemyakin@postgrespro.ru>
Author: Bingshuai Li <lucian1412@outlook.com>
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Author: Zhijie Hou <houzhijie22@gmail.com>
Backpatch-through: 14
Bug: #19555
Discussion: https://postgr.es/m/87r2c8xocq.fsf@ars-thinkpad
Discussion: https://postgr.es/m/CAHgHdKu5e3XY5e90Tuaxq_R4WrKxSV734Q%2BLwo5y39Omp2A-Gg@mail.gmail.com
Discussion: https://postgr.es/m/19555-3c700f40a13045cd@postgresql.org
This commit is contained in:
Álvaro Herrera
2026-10-14 19:42:04 +02:00
parent 4118012791
commit beff3cd009
11 changed files with 389 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
contrib/test_decoding/Makefile
View File

@@ -15,7 +15,8 @@ SLOT_CREATION_TST = $(if $(filter win32,$(PORTNAME)),,slot_creation_error)
ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
twophase_snapshot $(SLOT_CREATION_TST) catalog_change_snapshot \
skip_snapshot_restore invalidation_distribution parallel_session_origin
skip_snapshot_restore invalidation_distribution parallel_session_origin \
tuplecid_restart tuplecid_nested
REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf
ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf

32
contrib/test_decoding/expected/tuplecid_nested.out Normal file
View File

@@ -0,0 +1,32 @@
Parsed test spec with 3 sessions
starting permutation: s1_init s0_begin s0_topwrite s0_outer s0_inner s0_catinsert1 s0_release_inner s0_rollback_outer s2_vacuum s0_catinsert2 s0_commit s1_get_changes
step s1_init: SELECT 'init' FROM pg_create_logical_replication_slot('tuplecid_nested_slot', 'test_decoding');
?column?
--------
init
(1 row)
step s0_begin: BEGIN;
step s0_topwrite: INSERT INTO tbl1 VALUES (0);
step s0_outer: SAVEPOINT outer_sp;
step s0_inner: SAVEPOINT inner_sp;
step s0_catinsert1: INSERT INTO user_cat VALUES (5, 'e');
step s0_release_inner: RELEASE SAVEPOINT inner_sp;
step s0_rollback_outer: ROLLBACK TO SAVEPOINT outer_sp;
step s2_vacuum: VACUUM user_cat;
step s0_catinsert2: INSERT INTO user_cat VALUES (6, 'f');
step s0_commit: COMMIT;
step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('tuplecid_nested_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0') WHERE data NOT LIKE '%user_cat%';
data
------------------------------------------
BEGIN
table public.tbl1: INSERT: val1[integer]:0
COMMIT
(3 rows)
?column?
--------
stop
(1 row)

68
contrib/test_decoding/expected/tuplecid_restart.out Normal file
View File

@@ -0,0 +1,68 @@
Parsed test spec with 3 sessions
starting permutation: s0_init s0_begin s0_savepoint s0_insert s1_checkpoint s1_get_changes s0_catinsert1 s0_rollback s2_vacuum s0_catinsert2 s1_checkpoint s1_get_changes s0_commit s1_get_changes
step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
?column?
--------
init
(1 row)
step s0_begin: BEGIN;
step s0_savepoint: SAVEPOINT sp1;
step s0_insert: INSERT INTO tbl1 VALUES (1);
step s1_checkpoint: CHECKPOINT;
step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0') WHERE data NOT LIKE '%user_cat%';
data
----
(0 rows)
step s0_catinsert1: INSERT INTO user_cat VALUES (5, 'e');
step s0_rollback: ROLLBACK TO SAVEPOINT sp1;
step s2_vacuum: VACUUM user_cat;
step s0_catinsert2: INSERT INTO user_cat VALUES (6, 'f');
step s1_checkpoint: CHECKPOINT;
step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0') WHERE data NOT LIKE '%user_cat%';
data
----
(0 rows)
step s0_commit: COMMIT;
step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0') WHERE data NOT LIKE '%user_cat%';
data
------
BEGIN
COMMIT
(2 rows)
?column?
--------
stop
(1 row)
starting permutation: s0_begin s0_savepoint s0_insert s1_init s0_catinsert1 s0_rollback s2_vacuum s0_catinsert2 s0_commit s1_get_changes
step s0_begin: BEGIN;
step s0_savepoint: SAVEPOINT sp1;
step s0_insert: INSERT INTO tbl1 VALUES (1);
step s1_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); <waiting ...>
step s0_catinsert1: INSERT INTO user_cat VALUES (5, 'e');
step s0_rollback: ROLLBACK TO SAVEPOINT sp1;
step s2_vacuum: VACUUM user_cat;
step s0_catinsert2: INSERT INTO user_cat VALUES (6, 'f');
step s0_commit: COMMIT;
step s1_init: <... completed>
?column?
--------
init
(1 row)
step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0') WHERE data NOT LIKE '%user_cat%';
data
----
(0 rows)
?column?
--------
stop
(1 row)

2
contrib/test_decoding/meson.build
View File

@@ -30,6 +30,8 @@ test_decoding_isolation = [
'skip_snapshot_restore',
'invalidation_distribution',
'parallel_session_origin',
'tuplecid_nested',
'tuplecid_restart',
]
# slot_creation_error terminates another backend, whose FATAL message can be

77
contrib/test_decoding/specs/tuplecid_nested.spec Normal file
View File

@@ -0,0 +1,77 @@
# Test for BUG #19555.
#
# Test decoding of a partially rolled back transaction where the aborted part
# is nested: the aborting outer subtransaction never produced any WAL of its
# own, so decoding cannot know its toplevel transaction, while an inner
# subtransaction it rolls back did produce WAL and its tuplecid records still
# have to be removed
#
# The tuplecid records written by the aborted inner subtransaction are queued
# on the toplevel transaction and must be removed when the outer
# subtransaction's abort is decoded; otherwise they collide with the records
# of the line pointer that gets reused by the toplevel transaction's later
# catalog insert.
#
# - s0_topwrite makes the toplevel transaction produce its first WAL record
# before any subtransaction exists.
# - s0_outer establishes the outer savepoint, which never gets a WAL record
# of its own.
# - s0_catinsert1 generates an xl_heap_new_cid record in the inner
# subtransaction, which is then released.
# - s0_rollback_outer rolls the outer savepoint back. Its abort record is
# written once the backend is already in TRANS_ABORT, so it carries no
# toplevel xid in its header and the outer subtransaction's association
# with the toplevel can never be established. The released inner
# subtransaction is listed in the same abort record, though, and its
# tuplecid records still must be removed from the toplevel's list.
# - s2_vacuum reclaims the aborted row's line pointer (dead inserts are
# removed regardless of the xid horizon, so this is deterministic even
# while the toplevel transaction is still open); s0_catinsert2 then reuses
# the same tid with a different cmin.
# - s1_get_changes then decodes the toplevel commit and builds the tuplecid
# hash.
setup
{
DROP TABLE IF EXISTS tbl1;
DROP TABLE IF EXISTS user_cat;
CREATE TABLE tbl1 (val1 integer);
-- Four rows of ~1.5kB fill the first heap page to within a few hundred
-- bytes, so that vacuuming away the aborted fifth row leaves exactly one
-- line pointer for the next insert to reuse.
CREATE TABLE user_cat (c1 int, filler char(1500)) WITH (user_catalog_table = true);
INSERT INTO user_cat VALUES (1, 'a'), (2, 'b'), (3, 'c'), (4, 'd');
}
teardown
{
DROP TABLE tbl1;
DROP TABLE user_cat;
SELECT 'stop' FROM pg_drop_replication_slot('tuplecid_nested_slot');
}
session "s0"
setup { SET synchronous_commit=on; }
step "s0_begin" { BEGIN; }
step "s0_topwrite" { INSERT INTO tbl1 VALUES (0); }
step "s0_outer" { SAVEPOINT outer_sp; }
step "s0_inner" { SAVEPOINT inner_sp; }
step "s0_catinsert1" { INSERT INTO user_cat VALUES (5, 'e'); }
step "s0_release_inner" { RELEASE SAVEPOINT inner_sp; }
step "s0_rollback_outer" { ROLLBACK TO SAVEPOINT outer_sp; }
step "s0_catinsert2" { INSERT INTO user_cat VALUES (6, 'f'); }
step "s0_commit" { COMMIT; }
session "s1"
setup { SET synchronous_commit=on; }
step "s1_init" { SELECT 'init' FROM pg_create_logical_replication_slot('tuplecid_nested_slot', 'test_decoding'); }
# The user_cat rows carry ~1.5kB filler values which would flood the expected
# output; filter them out (decoding still processes them server-side).
step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('tuplecid_nested_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0') WHERE data NOT LIKE '%user_cat%'; }
session "s2"
setup { SET synchronous_commit=on; }
step "s2_vacuum" { VACUUM user_cat; }
# The slot is created before the transaction begins, so the final
# s1_get_changes decodes the whole transaction through its commit.
permutation "s1_init" "s0_begin" "s0_topwrite" "s0_outer" "s0_inner" "s0_catinsert1" "s0_release_inner" "s0_rollback_outer" "s2_vacuum" "s0_catinsert2" "s0_commit" "s1_get_changes"

74
contrib/test_decoding/specs/tuplecid_restart.spec Normal file
View File

@@ -0,0 +1,74 @@
# Test for BUG #19555.
#
# Test decoding of a transaction in which a catalog-modifying subtransaction
# was rolled back, where the transaction spans a logical decoding restart
# point.
#
# The tuplecid records written by the aborted subtransaction are queued on the
# toplevel transaction and must be removed when the subtransaction's abort is
# decoded; otherwise they collide with the records of the line pointer that
# gets reused by the toplevel transaction's later catalog insert.
#
# - s0_insert runs in a subtransaction and produces its first WAL record.
# - s0_catinsert1 generates an xl_heap_new_cid record in that subtransaction.
# - s0_rollback aborts the subtransaction.
# - s2_vacuum reclaims the aborted row's line pointer (dead inserts are
# removed regardless of the xid horizon, so this is deterministic even
# while the toplevel transaction is still open); s0_catinsert2 then reuses
# the same tid with a different cmin.
# - The transaction stays open across both checkpoints, so the final
# s1_get_changes replays the transaction's records from the first
# checkpoint and decodes its commit for output, building the tuplecid
# hash.
setup
{
DROP TABLE IF EXISTS tbl1;
DROP TABLE IF EXISTS user_cat;
CREATE TABLE tbl1 (val1 integer);
-- Four rows of ~1.5kB fill the first heap page to within a few hundred
-- bytes, so that vacuuming away the aborted fifth row leaves exactly one
-- line pointer for the next insert to reuse.
CREATE TABLE user_cat (c1 int, filler char(1500)) WITH (user_catalog_table = true);
INSERT INTO user_cat VALUES (1, 'a'), (2, 'b'), (3, 'c'), (4, 'd');
}
teardown
{
DROP TABLE tbl1;
DROP TABLE user_cat;
SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
}
session "s0"
setup { SET synchronous_commit=on; }
step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
step "s0_begin" { BEGIN; }
step "s0_savepoint" { SAVEPOINT sp1; }
step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
step "s0_catinsert1" { INSERT INTO user_cat VALUES (5, 'e'); }
step "s0_rollback" { ROLLBACK TO SAVEPOINT sp1; }
step "s0_catinsert2" { INSERT INTO user_cat VALUES (6, 'f'); }
step "s0_commit" { COMMIT; }
session "s1"
setup { SET synchronous_commit=on; }
step "s1_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
step "s1_checkpoint" { CHECKPOINT; }
# The user_cat rows carry ~1.5kB filler values which would flood the expected
# output; filter them out (decoding still processes them server-side).
step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0') WHERE data NOT LIKE '%user_cat%'; }
session "s2"
setup { SET synchronous_commit=on; }
step "s2_vacuum" { VACUUM user_cat; }
# The transaction commits only after the second s1_get_changes, so the last
# s1_get_changes both replays the transaction from the first checkpoint and
# decodes its commit for output.
permutation "s0_init" "s0_begin" "s0_savepoint" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_catinsert1" "s0_rollback" "s2_vacuum" "s0_catinsert2" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes"
# Variant with the slot created while the transaction is already open: slot
# creation waits for a consistent point past the open transaction, so the
# transaction is never decoded by this slot at all (no output). This locks in
# the behavior that decoding cannot start in the middle of a transaction.
permutation "s0_begin" "s0_savepoint" "s0_insert" "s1_init" "s0_catinsert1" "s0_rollback" "s2_vacuum" "s0_catinsert2" "s0_commit" "s1_get_changes"