database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-10-13 18:28:01 +03:00
Code Activity

Add support for prepared transactions to built-in logical replication.

Browse Source 
To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:

* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.

* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.

* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.

We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.

The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.

We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.

Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
This commit is contained in:
Amit Kapila
2021-07-14 07:33:50 +05:30
parent 6c9c283166
commit a8fd13cab0
43 changed files with 2382 additions and 191 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
src/backend/replication/logical/logical.c
View File

@@ -207,7 +207,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder = ReorderBufferAllocate();
ctx->snapshot_builder =
AllocateSnapshotBuilder(ctx->reorder, xmin_horizon, start_lsn,
need_full_snapshot, slot->data.initial_consistent_point);
need_full_snapshot, slot->data.two_phase_at);
ctx->reorder->private_data = ctx;
@@ -432,10 +432,12 @@ CreateInitDecodingContext(const char *plugin,
MemoryContextSwitchTo(old_context);
/*
* We allow decoding of prepared transactions iff the two_phase option is
* enabled at the time of slot creation.
* We allow decoding of prepared transactions when the two_phase is
* enabled at the time of slot creation, or when the two_phase option is
* given at the streaming start, provided the plugin supports all the
* callbacks for two-phase.
*/
ctx->twophase &= MyReplicationSlot->data.two_phase;
ctx->twophase &= slot->data.two_phase;
ctx->reorder->output_rewrites = ctx->options.receive_rewrites;
@@ -538,10 +540,22 @@ CreateDecodingContext(XLogRecPtr start_lsn,
MemoryContextSwitchTo(old_context);
/*
* We allow decoding of prepared transactions iff the two_phase option is
* enabled at the time of slot creation.
* We allow decoding of prepared transactions when the two_phase is
* enabled at the time of slot creation, or when the two_phase option is
* given at the streaming start, provided the plugin supports all the
* callbacks for two-phase.
*/
ctx->twophase &= MyReplicationSlot->data.two_phase;
ctx->twophase &= (slot->data.two_phase || ctx->twophase_opt_given);
/* Mark slot to allow two_phase decoding if not already marked */
if (ctx->twophase && !slot->data.two_phase)
{
slot->data.two_phase = true;
slot->data.two_phase_at = start_lsn;
ReplicationSlotMarkDirty();
ReplicationSlotSave();
SnapBuildSetTwoPhaseAt(ctx->snapshot_builder, start_lsn);
}
ctx->reorder->output_rewrites = ctx->options.receive_rewrites;
@@ -602,7 +616,8 @@ DecodingContextFindStartpoint(LogicalDecodingContext *ctx)
SpinLockAcquire(&slot->mutex);
slot->data.confirmed_flush = ctx->reader->EndRecPtr;
slot->data.initial_consistent_point = ctx->reader->EndRecPtr;
if (slot->data.two_phase)
slot->data.two_phase_at = ctx->reader->EndRecPtr;
SpinLockRelease(&slot->mutex);
}