database/postgres
database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-05-01 01:04:50 +03:00
Code

Fix the bugs in selecting the transaction for streaming.

Browse Source 
There were two problems:
a. We were always selecting the next available txn instead of selecting it
when it is larger than the previous transaction.
b. We were selecting the transactions which haven't made any changes to
the database (base snapshot is not set). Later it was hitting an Assert
because we don't decode such transactions and the changes in txn remain as
it is. It is better not to choose such transactions for streaming in the
first place.

Reported-by: Haiying Tang
Author: Dilip Kumar
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/OS0PR01MB61133B94E63177040F7ECDA1FB429@OS0PR01MB6113.jpnprd01.prod.outlook.com
This commit is contained in:
Amit Kapila 2021-04-30 10:49:52 +05:30
parent 3c80e96dff
commit ee4ba01dbb
1 changed files with 23 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
src/backend/replication/logical/reorderbuffer.c
View File

@ -3362,19 +3362,22 @@ ReorderBufferLargestTXN(ReorderBuffer *rb)
* This can be seen as an optimized version of ReorderBufferLargestTXN, which * This can be seen as an optimized version of ReorderBufferLargestTXN, which
* should give us the same transaction (because we don't update memory account * should give us the same transaction (because we don't update memory account
* for subtransaction with streaming, so it's always 0). But we can simply * for subtransaction with streaming, so it's always 0). But we can simply
* iterate over the limited number of toplevel transactions. * iterate over the limited number of toplevel transactions that have a base
* snapshot. There is no use of selecting a transaction that doesn't have base
* snapshot because we don't decode such transactions.
* *
* Note that, we skip transactions that contains incomplete changes. There * Note that, we skip transactions that contains incomplete changes. There
* is a scope of optimization here such that we can select the largest transaction * is a scope of optimization here such that we can select the largest
* which has complete changes. But that will make the code and design quite complex * transaction which has incomplete changes. But that will make the code and
* and that might not be worth the benefit. If we plan to stream the transactions * design quite complex and that might not be worth the benefit. If we plan to
* that contains incomplete changes then we need to find a way to partially * stream the transactions that contains incomplete changes then we need to
* stream/truncate the transaction changes in-memory and build a mechanism to * find a way to partially stream/truncate the transaction changes in-memory
* partially truncate the spilled files. Additionally, whenever we partially * and build a mechanism to partially truncate the spilled files.
* stream the transaction we need to maintain the last streamed lsn and next time * Additionally, whenever we partially stream the transaction we need to
* we need to restore from that segment and the offset in WAL. As we stream the * maintain the last streamed lsn and next time we need to restore from that
* changes from the top transaction and restore them subtransaction wise, we need * segment and the offset in WAL. As we stream the changes from the top
* to even remember the subxact from where we streamed the last change. * transaction and restore them subtransaction wise, we need to even remember
* the subxact from where we streamed the last change.
*/ */
static ReorderBufferTXN * static ReorderBufferTXN *
ReorderBufferLargestTopTXN(ReorderBuffer *rb) ReorderBufferLargestTopTXN(ReorderBuffer *rb)
@ -3383,14 +3386,19 @@ ReorderBufferLargestTopTXN(ReorderBuffer *rb)
Size largest_size = 0; Size largest_size = 0;
ReorderBufferTXN *largest = NULL; ReorderBufferTXN *largest = NULL;
/* Find the largest top-level transaction. */ /* Find the largest top-level transaction having a base snapshot. */
dlist_foreach(iter, &rb->toplevel_by_lsn) dlist_foreach(iter, &rb->txns_by_base_snapshot_lsn)
{ {
ReorderBufferTXN *txn; ReorderBufferTXN *txn;
txn = dlist_container(ReorderBufferTXN, node, iter.cur); txn = dlist_container(ReorderBufferTXN, base_snapshot_node, iter.cur);
if ((largest != NULL || txn->total_size > largest_size) && /* must not be a subtxn */
Assert(!rbtxn_is_known_subxact(txn));
/* base_snapshot must be set */
Assert(txn->base_snapshot != NULL);
if ((largest == NULL || txn->total_size > largest_size) &&
(txn->total_size > 0) && !(rbtxn_has_incomplete_tuple(txn))) (txn->total_size > 0) && !(rbtxn_has_incomplete_tuple(txn)))
{ {
largest = txn; largest = txn;