database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-30 11:03:19 +03:00
Code Activity

Add support for streaming to built-in logical replication.

Browse Source 
To add support for streaming of in-progress transactions into the
built-in logical replication, we need to do three things:

* Extend the logical replication protocol, so identify in-progress
transactions, and allow adding additional bits of information (e.g.
XID of subtransactions).

* Modify the output plugin (pgoutput) to implement the new stream
API callbacks, by leveraging the extended replication protocol.

* Modify the replication apply worker, to properly handle streamed
in-progress transaction by spilling the data to disk and then
replaying them on commit.

We however must explicitly disable streaming replication during
replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover we don't have a replication connection open so we
don't have where to send the data anyway.

Author: Tomas Vondra, Dilip Kumar and Amit Kapila
Reviewed-by: Amit Kapila, Kuntal Ghosh and Ajin Cherian
Tested-by: Neha Sharma, Mahendra Singh Thalor and Ajin Cherian
Discussion: https://postgr.es/m/688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com
This commit is contained in:
Amit Kapila
2020-09-03 07:54:07 +05:30
parent 66f1630680
commit 464824323e
23 changed files with 1766 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
doc/src/sgml/monitoring.sgml
View File

@ -1509,6 +1509,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry><literal>WALWrite</literal></entry>
<entry>Waiting for a write to a WAL file.</entry>
</row>
<row>
<entry><literal>LogicalChangesRead</literal></entry>
<entry>Waiting for a read from a logical changes file.</entry>
</row>
<row>
<entry><literal>LogicalChangesWrite</literal></entry>
<entry>Waiting for a write to a logical changes file.</entry>
</row>
<row>
<entry><literal>LogicalSubxactRead</literal></entry>
<entry>Waiting for a read from a logical subxact file.</entry>
</row>
<row>
<entry><literal>LogicalSubxactWrite</literal></entry>
<entry>Waiting for a write to a logical subxact file.</entry>
</row>
</tbody>
</tgroup>
</table>

5
doc/src/sgml/ref/alter_subscription.sgml
View File

@ -165,8 +165,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
<xref linkend="sql-createsubscription"/>. See there for more
information. The parameters that can be altered
are <literal>slot_name</literal>,
<literal>synchronous_commit</literal>, and
<literal>binary</literal>.
<literal>synchronous_commit</literal>,
<literal>binary</literal>, and
<literal>streaming</literal>.
</para>
</listitem>
</varlistentry>

11
doc/src/sgml/ref/create_subscription.sgml
View File

@ -228,6 +228,17 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>streaming</literal> (<type>boolean</type>)</term>
<listitem>
<para>
Specifies whether streaming of in-progress transactions should
be enabled for this subscription. By default, all transactions
are fully decoded on the publisher, and only then sent to the
subscriber as a whole.
</para>
</listitem>
</varlistentry>
</variablelist></para>
</listitem>
</varlistentry>