database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-30 11:03:19 +03:00
Code Activity

Revert "Logical decoding of sequences"

Browse Source 
This reverts a sequence of commits, implementing features related to
logical decoding and replication of sequences:

 - 0da92dc530
 - 80901b3291
 - b779d7d8fd
 - d5ed9da41d
 - a180c2b34d
 - 75b1521dae
 - 2d2232933b
 - 002c9dd97a
 - 05843b1aa4

The implementation has issues, mostly due to combining transactional and
non-transactional behavior of sequences. It's not clear how this could
be fixed, but it'll require reworking significant part of the patch.

Discussion: https://postgr.es/m/95345a19-d508-63d1-860a-f5c2f41e8d40@enterprisedb.com
This commit is contained in:
Tomas Vondra
2022-04-07 18:13:13 +02:00
parent d7ab2a9a3c
commit 2c7ea57e56
73 changed files with 605 additions and 4762 deletions
Download Patch File Download Diff File Expand all files Collapse all files

81
doc/src/sgml/catalogs.sgml
View File

@ -6354,16 +6354,6 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
Reference to schema
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>pntype</structfield> <type>char</type>
Determines which object type is included from this schema.
</para>
<para>
Reference to schema
</para></entry>
</row>
</tbody>
</tgroup>
</table>
@ -9699,11 +9689,6 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
<entry>prepared transactions</entry>
</row>
<row>
<entry><link linkend="view-pg-publication-sequences"><structname>pg_publication_sequences</structname></link></entry>
<entry>publications and their associated sequences</entry>
</row>
<row>
<entry><link linkend="view-pg-publication-tables"><structname>pg_publication_tables</structname></link></entry>
<entry>publications and their associated tables</entry>
@ -11641,72 +11626,6 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</sect1>
<sect1 id="view-pg-publication-sequences">
<title><structname>pg_publication_sequences</structname></title>
<indexterm zone="view-pg-publication-sequences">
<primary>pg_publication_sequences</primary>
</indexterm>
<para>
The view <structname>pg_publication_sequences</structname> provides
information about the mapping between publications and the sequences they
contain. Unlike the underlying catalog
<link linkend="catalog-pg-publication-rel"><structname>pg_publication_rel</structname></link>,
this view expands
publications defined as <literal>FOR ALL SEQUENCES</literal>, so for such
publications there will be a row for each eligible sequence.
</para>
<table>
<title><structname>pg_publication_sequences</structname> Columns</title>
<tgroup cols="1">
<thead>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
Column Type
</para>
<para>
Description
</para></entry>
</row>
</thead>
<tbody>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>pubname</structfield> <type>name</type>
(references <link linkend="catalog-pg-publication"><structname>pg_publication</structname></link>.<structfield>pubname</structfield>)
</para>
<para>
Name of publication
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>schemaname</structfield> <type>name</type>
(references <link linkend="catalog-pg-namespace"><structname>pg_namespace</structname></link>.<structfield>nspname</structfield>)
</para>
<para>
Name of schema containing sequence
</para></entry>
</row>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>sequencename</structfield> <type>name</type>
(references <link linkend="catalog-pg-class"><structname>pg_class</structname></link>.<structfield>relname</structfield>)
</para>
<para>
Name of sequence
</para></entry>
</row>
</tbody>
</tgroup>
</table>
</sect1>
<sect1 id="view-pg-publication-tables">
<title><structname>pg_publication_tables</structname></title>

65
doc/src/sgml/logicaldecoding.sgml
View File

@ -458,7 +458,6 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
LogicalDecodeSequenceCB sequence_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
LogicalDecodeFilterPrepareCB filter_prepare_cb;
@ -473,7 +472,6 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
LogicalDecodeStreamSequenceCB stream_sequence_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
@ -483,11 +481,9 @@ typedef void (*LogicalOutputPluginInit) (struct OutputPluginCallbacks *cb);
and <function>commit_cb</function> callbacks are required,
while <function>startup_cb</function>,
<function>filter_by_origin_cb</function>, <function>truncate_cb</function>,
<function>sequence_cb</function>, and <function>shutdown_cb</function> are
optional. If <function>truncate_cb</function> is not set but a
and <function>shutdown_cb</function> are optional.
If <function>truncate_cb</function> is not set but a
<command>TRUNCATE</command> is to be decoded, the action will be ignored.
Similarly, if <function>sequence_cb</function> is not set and a sequence
change is to be decoded, the action will be ignored.
</para>
<para>
@ -496,8 +492,7 @@ typedef void (*LogicalOutputPluginInit) (struct OutputPluginCallbacks *cb);
<function>stream_stop_cb</function>, <function>stream_abort_cb</function>,
<function>stream_commit_cb</function>, <function>stream_change_cb</function>,
and <function>stream_prepare_cb</function>
are required, while <function>stream_message_cb</function>,
<function>stream_sequence_cb</function>, and
are required, while <function>stream_message_cb</function> and
<function>stream_truncate_cb</function> are optional.
</para>
@ -813,35 +808,6 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
</para>
</sect3>
<sect3 id="logicaldecoding-output-plugin-sequence">
<title>Sequence Callback</title>
<para>
The optional <function>sequence_cb</function> callback is called for
actions that update a sequence value.
<programlisting>
typedef void (*LogicalDecodeSequenceCB) (struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
XLogRecPtr sequence_lsn,
Relation rel,
bool transactional,
int64 last_value,
int64 log_cnt,
bool is_called);
</programlisting>
The <parameter>txn</parameter> parameter contains meta information about
the transaction the sequence change is part of. Note however that for
non-transactional increments, the transaction may be either NULL or not
NULL, depending on if the transaction already has an XID assigned.
The <parameter>sequence_lsn</parameter> has the WAL location of the
sequence update. <parameter>transactional</parameter> says if the
sequence has to be replayed as part of the transaction or directly.
The <parameter>last_value</parameter>, <parameter>log_cnt</parameter> and
<parameter>is_called</parameter> parameters describe the sequence change.
</para>
</sect3>
<sect3 id="logicaldecoding-output-plugin-filter-prepare">
<title>Prepare Filter Callback</title>
@ -1051,26 +1017,6 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
</para>
</sect3>
<sect3 id="logicaldecoding-output-plugin-stream-sequence">
<title>Stream Sequence Callback</title>
<para>
The optional <function>stream_sequence_cb</function> callback is called
for actions that change a sequence in a block of streamed changes
(demarcated by <function>stream_start_cb</function> and
<function>stream_stop_cb</function> calls).
<programlisting>
typedef void (*LogicalDecodeStreamSequenceCB) (struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
XLogRecPtr sequence_lsn,
Relation rel,
bool transactional,
int64 last_value,
int64 log_cnt,
bool is_called);
</programlisting>
</para>
</sect3>
<sect3 id="logicaldecoding-output-plugin-stream-truncate">
<title>Stream Truncate Callback</title>
<para>
@ -1251,9 +1197,8 @@ OutputPluginWrite(ctx, true);
in-progress transactions. There are multiple required streaming callbacks
(<function>stream_start_cb</function>, <function>stream_stop_cb</function>,
<function>stream_abort_cb</function>, <function>stream_commit_cb</function>
and <function>stream_change_cb</function>) and multiple optional callbacks
(<function>stream_message_cb</function>, <function>stream_sequence_cb</function>,
and <function>stream_truncate_cb</function>).
and <function>stream_change_cb</function>) and two optional callbacks
(<function>stream_message_cb</function> and <function>stream_truncate_cb</function>).
Also, if streaming of two-phase commands is to be supported, then additional
callbacks must be provided. (See <xref linkend="logicaldecoding-two-phase-commits"/>
for details).

119
doc/src/sgml/protocol.sgml
View File

@ -7072,125 +7072,6 @@ Relation
</listitem>
</varlistentry>
<varlistentry id="protocol-logicalrep-message-formats-Sequence">
<term>
Sequence
</term>
<listitem>
<para>
<variablelist>
<varlistentry>
<term>
Byte1('X')
</term>
<listitem>
<para>
Identifies the message as a sequence message.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
Int32 (TransactionId)
</term>
<listitem>
<para>
Xid of the transaction (only present for streamed transactions).
This field is available since protocol version 2.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
Int8(0)
</term>
<listitem>
<para>
Flags; currently unused.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
Int64 (XLogRecPtr)
</term>
<listitem>
<para>
The LSN of the sequence increment.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
String
</term>
<listitem>
<para>
Namespace (empty string for <literal>pg_catalog</literal>).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
String
</term>
<listitem>
<para>
Relation name.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
Int8
</term>
<listitem>
<para>
1 if the sequence update is transactional, 0 otherwise.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
Int64
</term>
<listitem>
<para>
<structfield>last_value</structfield> value of the sequence.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
Int64
</term>
<listitem>
<para>
<structfield>log_cnt</structfield> value of the sequence.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
Int8
</term>
<listitem>
<para>
<structfield>is_called</structfield> value of the sequence.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
</listitem>
</varlistentry>
<varlistentry id="protocol-logicalrep-message-formats-Type">
<term>
Type

25
doc/src/sgml/ref/alter_publication.sgml
View File

@ -31,9 +31,7 @@ ALTER PUBLICATION <replaceable class="parameter">name</replaceable> RENAME TO <r
<phrase>where <replaceable class="parameter">publication_object</replaceable> is one of:</phrase>
TABLE [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ] [ ( <replaceable class="parameter">column_name</replaceable> [, ... ] ) ] [ WHERE ( <replaceable class="parameter">expression</replaceable> ) ] [, ... ]
SEQUENCE <replaceable class="parameter">sequence_name</replaceable> [, ... ]
ALL TABLES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA } [, ... ]
ALL SEQUENCES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA } [, ... ]
</synopsis>
</refsynopsisdiv>
@ -46,13 +44,13 @@ ALTER PUBLICATION <replaceable class="parameter">name</replaceable> RENAME TO <r
</para>
<para>
The first three variants change which objects (tables, sequences or schemas)
are part of the publication. The <literal>SET</literal> clause will replace
the list of objects in the publication with the specified list; the existing
objects that were present in the publication will be removed.
The <literal>ADD</literal> and <literal>DROP</literal> clauses will add and
remove one or more objects from the publication. Note that adding objects
to a publication that is already subscribed to will require an
The first three variants change which tables/schemas are part of the
publication. The <literal>SET</literal> clause will replace the list of
tables/schemas in the publication with the specified list; the existing
tables/schemas that were present in the publication will be removed. The
<literal>ADD</literal> and <literal>DROP</literal> clauses will add and
remove one or more tables/schemas from the publication. Note that adding
tables/schemas to a publication that is already subscribed to will require an
<literal>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</literal> action on the
subscribing side in order to become effective. Note also that the combination
of <literal>DROP</literal> with a <literal>WHERE</literal> clause is not
@ -132,15 +130,6 @@ ALTER PUBLICATION <replaceable class="parameter">name</replaceable> RENAME TO <r
</listitem>
</varlistentry>
<varlistentry>
<term><replaceable class="parameter">sequence_name</replaceable></term>
<listitem>
<para>
Name of an existing sequence.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><replaceable class="parameter">schema_name</replaceable></term>
<listitem>

8
doc/src/sgml/ref/alter_subscription.sgml
View File

@ -150,8 +150,8 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
<listitem>
<para>
Fetch missing table information from publisher. This will start
replication of tables and sequences that were added to the subscribed-to
publications since <command>CREATE SUBSCRIPTION</command> or
replication of tables that were added to the subscribed-to publications
since <command>CREATE SUBSCRIPTION</command> or
the last invocation of <command>REFRESH PUBLICATION</command>.
</para>
@ -169,8 +169,8 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
The default is <literal>true</literal>.
</para>
<para>
Previously subscribed tables and sequences are not copied, even if a
table's row filter <literal>WHERE</literal> clause has since been modified.
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
</listitem>
</varlistentry>

51
doc/src/sgml/ref/create_publication.sgml
View File

@ -22,21 +22,14 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
[ FOR ALL <replaceable class="parameter">object_type</replaceable> [, ...]
[ FOR ALL TABLES
| FOR <replaceable class="parameter">publication_object</replaceable> [, ... ] ]
[ WITH ( <replaceable class="parameter">publication_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) ]
<phrase>where <replaceable class="parameter">object type</replaceable> is one of:</phrase>
TABLES
SEQUENCES
<phrase>where <replaceable class="parameter">publication_object</replaceable> is one of:</phrase>
TABLE [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ] [ ( <replaceable class="parameter">column_name</replaceable> [, ... ] ) ] [ WHERE ( <replaceable class="parameter">expression</replaceable> ) ] [, ... ]
SEQUENCE <replaceable class="parameter">sequence_name</replaceable> [ * ] [, ... ]
ALL TABLES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA } [, ... ]
ALL SEQUENCES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA } [, ... ]
</synopsis>
</refsynopsisdiv>
@ -121,43 +114,27 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>FOR SEQUENCE</literal></term>
<listitem>
<para>
Specifies a list of sequences to add to the publication.
</para>
<para>
Specifying a sequence that is part of a schema specified by <literal>FOR
ALL SEQUENCES IN SCHEMA</literal> is not supported.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>FOR ALL TABLES</literal></term>
<term><literal>FOR ALL SEQUENCES</literal></term>
<listitem>
<para>
Marks the publication as one that replicates changes for all tables/sequences in
the database, including tables/sequences created in the future.
Marks the publication as one that replicates changes for all tables in
the database, including tables created in the future.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>FOR ALL TABLES IN SCHEMA</literal></term>
<term><literal>FOR ALL SEQUENCES IN SCHEMA</literal></term>
<listitem>
<para>
Marks the publication as one that replicates changes for all sequences/tables in
the specified list of schemas, including sequences/tables created in the future.
Marks the publication as one that replicates changes for all tables in
the specified list of schemas, including tables created in the future.
</para>
<para>
Specifying a schema along with a sequence/table which belongs to the specified
schema using <literal>FOR SEQUENCE</literal>/<literal>FOR TABLE</literal> is not supported.
Specifying a schema along with a table which belongs to the specified
schema using <literal>FOR TABLE</literal> is not supported.
</para>
<para>
@ -232,9 +209,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
<title>Notes</title>
<para>
If <literal>FOR TABLE</literal>, <literal>FOR SEQUENCE</literal>, etc. is
not specified, then the publication starts out with an empty set of tables
and sequences. That is useful if objects are to be added later.
If <literal>FOR TABLE</literal>, <literal>FOR ALL TABLES</literal> or
<literal>FOR ALL TABLES IN SCHEMA</literal> are not specified, then the
publication starts out with an empty set of tables. That is useful if
tables or schemas are to be added later.
</para>
<para>
@ -249,9 +227,10 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
To add a table or a sequence to a publication, the invoking user must
have ownership rights on the object. The <command>FOR ALL ...</command>
clauses require the invoking user to be a superuser.
To add a table to a publication, the invoking user must have ownership
rights on the table. The <command>FOR ALL TABLES</command> and
<command>FOR ALL TABLES IN SCHEMA</command> clauses require the invoking
user to be a superuser.
</para>
<para>