database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-11-09 06:21:09 +03:00
Code Activity

Add support for prepared transactions to built-in logical replication.

Browse Source 
To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:

* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.

* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.

* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.

We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.

The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.

We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.

Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
This commit is contained in:
Amit Kapila
2021-07-14 07:33:50 +05:30
parent 6c9c283166
commit a8fd13cab0
43 changed files with 2382 additions and 191 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
src/backend/replication/logical/snapbuild.c
View File

@@ -165,15 +165,15 @@ struct SnapBuild
XLogRecPtr start_decoding_at;
/*
* LSN at which we found a consistent point at the time of slot creation.
* This is also the point where we have exported a snapshot for the
* initial copy.
* LSN at which two-phase decoding was enabled or LSN at which we found a
* consistent point at the time of slot creation.
*
* The prepared transactions that are not covered by initial snapshot
* needs to be sent later along with commit prepared and they must be
* before this point.
* The prepared transactions, that were skipped because previously
* two-phase was not enabled or are not covered by initial snapshot, need
* to be sent later along with commit prepared and they must be before
* this point.
*/
XLogRecPtr initial_consistent_point;
XLogRecPtr two_phase_at;
/*
* Don't start decoding WAL until the "xl_running_xacts" information
@@ -281,7 +281,7 @@ AllocateSnapshotBuilder(ReorderBuffer *reorder,
TransactionId xmin_horizon,
XLogRecPtr start_lsn,
bool need_full_snapshot,
XLogRecPtr initial_consistent_point)
XLogRecPtr two_phase_at)
{
MemoryContext context;
MemoryContext oldcontext;
@@ -309,7 +309,7 @@ AllocateSnapshotBuilder(ReorderBuffer *reorder,
builder->initial_xmin_horizon = xmin_horizon;
builder->start_decoding_at = start_lsn;
builder->building_full_snapshot = need_full_snapshot;
builder->initial_consistent_point = initial_consistent_point;
builder->two_phase_at = two_phase_at;
MemoryContextSwitchTo(oldcontext);
@@ -370,12 +370,21 @@ SnapBuildCurrentState(SnapBuild *builder)
}
/*
* Return the LSN at which the snapshot was exported
* Return the LSN at which the two-phase decoding was first enabled.
*/
XLogRecPtr
SnapBuildInitialConsistentPoint(SnapBuild *builder)
SnapBuildGetTwoPhaseAt(SnapBuild *builder)
{
return builder->initial_consistent_point;
return builder->two_phase_at;
}
/*
* Set the LSN at which two-phase decoding is enabled.
*/
void
SnapBuildSetTwoPhaseAt(SnapBuild *builder, XLogRecPtr ptr)
{
builder->two_phase_at = ptr;
}
/*