database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-11-09 06:21:09 +03:00
Code Activity

Add support for prepared transactions to built-in logical replication.

Browse Source 
To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:

* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.

* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.

* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.

We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.

The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.

We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.

Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
This commit is contained in:
Amit Kapila
2021-07-14 07:33:50 +05:30
parent 6c9c283166
commit a8fd13cab0
43 changed files with 2382 additions and 191 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
src/backend/access/transam/twophase.c
View File

@@ -2458,3 +2458,71 @@ PrepareRedoRemove(TransactionId xid, bool giveWarning)
RemoveTwoPhaseFile(xid, giveWarning);
RemoveGXact(gxact);
}
/*
* LookupGXact
* Check if the prepared transaction with the given GID, lsn and timestamp
* exists.
*
* Note that we always compare with the LSN where prepare ends because that is
* what is stored as origin_lsn in the 2PC file.
*
* This function is primarily used to check if the prepared transaction
* received from the upstream (remote node) already exists. Checking only GID
* is not sufficient because a different prepared xact with the same GID can
* exist on the same node. So, we are ensuring to match origin_lsn and
* origin_timestamp of prepared xact to avoid the possibility of a match of
* prepared xact from two different nodes.
*/
bool
LookupGXact(const char *gid, XLogRecPtr prepare_end_lsn,
TimestampTz origin_prepare_timestamp)
{
int i;
bool found = false;
LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
{
GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
/* Ignore not-yet-valid GIDs. */
if (gxact->valid && strcmp(gxact->gid, gid) == 0)
{
char *buf;
TwoPhaseFileHeader *hdr;
/*
* We are not expecting collisions of GXACTs (same gid) between
* publisher and subscribers, so we perform all I/O while holding
* TwoPhaseStateLock for simplicity.
*
* To move the I/O out of the lock, we need to ensure that no
* other backend commits the prepared xact in the meantime. We can
* do this optimization if we encounter many collisions in GID
* between publisher and subscriber.
*/
if (gxact->ondisk)
buf = ReadTwoPhaseFile(gxact->xid, false);
else
{
Assert(gxact->prepare_start_lsn);
XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, NULL);
}
hdr = (TwoPhaseFileHeader *) buf;
if (hdr->origin_lsn == prepare_end_lsn &&
hdr->origin_timestamp == origin_prepare_timestamp)
{
found = true;
pfree(buf);
break;
}
pfree(buf);
}
}
LWLockRelease(TwoPhaseStateLock);
return found;
}