database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-10-16 17:07:43 +03:00
Code Activity

Add max_retention_duration option to subscriptions.

Browse Source 
This commit introduces a new subscription parameter,
max_retention_duration, aimed at mitigating excessive accumulation of dead
tuples when retain_dead_tuples is enabled and the apply worker lags behind
the publisher.

When the time spent advancing a non-removable transaction ID exceeds the
max_retention_duration threshold, the apply worker will stop retaining
conflict detection information. In such cases, the conflict slot's xmin
will be set to InvalidTransactionId, provided that all apply workers
associated with the subscription (with retain_dead_tuples enabled) confirm
the retention duration has been exceeded.

To ensure retention status persists across server restarts, a new column
subretentionactive has been added to the pg_subscription catalog. This
prevents unnecessary reactivation of retention logic after a restart.

The conflict detection slot will not be automatically re-initialized
unless a new subscription is created with retain_dead_tuples = true, or
the user manually re-enables retain_dead_tuples.

A future patch will introduce support for automatic slot re-initialization
once at least one apply worker confirms that the retention duration is
within the configured max_retention_duration.

Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/OS0PR01MB5716BE80DAEB0EE2A6A5D1F5949D2@OS0PR01MB5716.jpnprd01.prod.outlook.com
This commit is contained in:
Amit Kapila
2025-09-02 03:20:18 +00:00
parent 36aed19fd9
commit a850be2fe6
20 changed files with 777 additions and 216 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
doc/src/sgml/catalogs.sgml
View File

@@ -8094,6 +8094,31 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>submaxretention</structfield> <type>int4</type>
</para>
<para>
The maximum duration (in milliseconds) for which information (e.g., dead
tuples, commit timestamps, and origins) useful for conflict detection can
be retained.
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subretentionactive</structfield> <type>bool</type>
</para>
<para>
The retention status of information (e.g., dead tuples, commit
timestamps, and origins) useful for conflict detection. True if
<link linkend="sql-createsubscription-params-with-retain-dead-tuples"><literal>retain_dead_tuples</literal></link>
is enabled, and the retention duration has not exceeded
<link linkend="sql-createsubscription-params-with-max-retention-duration"><literal>max_retention_duration</literal></link>,
when defined.
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subconninfo</structfield> <type>text</type>

5
doc/src/sgml/ref/alter_subscription.sgml
View File

@@ -236,8 +236,9 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
<link linkend="sql-createsubscription-params-with-run-as-owner"><literal>run_as_owner</literal></link>,
<link linkend="sql-createsubscription-params-with-origin"><literal>origin</literal></link>,
<link linkend="sql-createsubscription-params-with-failover"><literal>failover</literal></link>,
<link linkend="sql-createsubscription-params-with-two-phase"><literal>two_phase</literal></link>, and
<link linkend="sql-createsubscription-params-with-retain-dead-tuples"><literal>retain_dead_tuples</literal></link>.
<link linkend="sql-createsubscription-params-with-two-phase"><literal>two_phase</literal></link>,
<link linkend="sql-createsubscription-params-with-retain-dead-tuples"><literal>retain_dead_tuples</literal></link>, and
<link linkend="sql-createsubscription-params-with-max-retention-duration"><literal>max_retention_duration</literal></link>.
Only a superuser can set <literal>password_required = false</literal>.
</para>

43
doc/src/sgml/ref/create_subscription.sgml
View File

@@ -448,7 +448,7 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
If set to <literal>true</literal>, the detection of
<xref linkend="conflict-update-deleted"/> is enabled, and a physical
replication slot named <quote><literal>pg_conflict_detection</literal></quote>
created on the subscriber to prevent the information for detecting
is created on the subscriber to prevent the information for detecting
conflicts from being removed.
</para>
@@ -521,6 +521,47 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
<varlistentry id="sql-createsubscription-params-with-max-retention-duration">
<term><literal>max_retention_duration</literal> (<type>integer</type>)</term>
<listitem>
<para>
Maximum duration in milliseconds for which this subscription's apply worker
is allowed to retain the information useful for conflict detection when
<literal>retain_dead_tuples</literal> is enabled. The default value
is <literal>0</literal>, indicating that the information is retained
until it is no longer needed for detection purposes.
</para>
<para>
The information useful for conflict detection is no longer retained if
all apply workers associated with the subscriptions, where
<literal>retain_dead_tuples</literal> is enabled, confirm that the
retention duration has exceeded the
<literal>max_retention_duration</literal> set within the corresponding
subscription. The retention will not be automatically resumed unless a
new subscription is created with <literal>retain_dead_tuples =
true</literal>, or the user manually re-enables
<literal>retain_dead_tuples</literal>.
</para>
<para>
Note that overall retention will not stop if other subscriptions that
have a value greater than 0 for this parameter have not exceeded it,
or if they set this option to 0.
</para>
<para>
This option is effective only when
<literal>retain_conflict_info</literal> is enabled and the apply
worker associated with the subscription is active.
</para>
<warning>
<para>
Note that setting a non-zero value for this option could lead to
information for conflict detection being removed prematurely,
potentially resulting in incorrect conflict detection.
</para>
</warning>
</listitem>
</varlistentry>
</variablelist></para>
</listitem>