database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-10-31 10:30:33 +03:00
Code Activity

Restore archive_command documentation

Browse Source 
Commit 5ef1eefd76, which added
archive_library, purged most mentions of archive_command from the
documentation.  This is inappropriate, since archive_command is still
a feature in use and users will want to see information about it.

This restores all the removed mentions and rephrases things so that
archive_command and archive_library are presented as alternatives of
each other.

Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://www.postgresql.org/message-id/9366d634-a917-85a9-4991-b2a4859edaf9@enterprisedb.com
This commit is contained in:
Peter Eisentraut
2022-09-17 11:34:20 +02:00
parent 18ac08f0b4
commit ba50834551
5 changed files with 90 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

96
doc/src/sgml/backup.sgml
View File

@@ -593,7 +593,7 @@ tar -cf backup.tar /usr/local/pgsql/data
provide the database administrator with flexibility,
<productname>PostgreSQL</productname> tries not to make any assumptions about how
the archiving will be done. Instead, <productname>PostgreSQL</productname> lets
the administrator specify an archive library to be executed to copy a
the administrator specify a shell command or an archive library to be executed to copy a
completed segment file to wherever it needs to go. This could be as simple
as a shell command that uses <literal>cp</literal>, or it could invoke a
complex C function &mdash; it's all up to you.
@@ -603,13 +603,15 @@ tar -cf backup.tar /usr/local/pgsql/data
To enable WAL archiving, set the <xref linkend="guc-wal-level"/>
configuration parameter to <literal>replica</literal> or higher,
<xref linkend="guc-archive-mode"/> to <literal>on</literal>,
and specify the library to use in the <xref
specify the shell command to use in the <xref
linkend="guc-archive-command"/> configuration parameter
or specify the library to use in the <xref
linkend="guc-archive-library"/> configuration parameter. In practice
these settings will always be placed in the
<filename>postgresql.conf</filename> file.
One simple way to archive is to set <varname>archive_library</varname> to
an empty string and to specify a shell command in
<xref linkend="guc-archive-command"/>.
</para>
<para>
In <varname>archive_command</varname>,
<literal>%p</literal> is replaced by the path name of the file to
archive, while <literal>%f</literal> is replaced by only the file name.
@@ -633,6 +635,24 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
A similar command will be generated for each new file to be archived.
</para>
<para>
The archive command will be executed under the ownership of the same
user that the <productname>PostgreSQL</productname> server is running as. Since
the series of WAL files being archived contains effectively everything
in your database, you will want to be sure that the archived data is
protected from prying eyes; for example, archive into a directory that
does not have group or world read access.
</para>
<para>
It is important that the archive command return zero exit status if and
only if it succeeds. Upon getting a zero result,
<productname>PostgreSQL</productname> will assume that the file has been
successfully archived, and will remove or recycle it. However, a nonzero
status tells <productname>PostgreSQL</productname> that the file was not archived;
it will try again periodically until it succeeds.
</para>
<para>
Another way to archive is to use a custom archive module as the
<varname>archive_library</varname>. Since such modules are written in
@@ -644,41 +664,17 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
</para>
<para>
The archive library will be executed under the ownership of the same
user that the <productname>PostgreSQL</productname> server is running as. Since
the series of WAL files being archived contains effectively everything
in your database, you will want to be sure that the archived data is
protected from prying eyes; for example, archive into a directory that
does not have group or world read access.
When the archive command is terminated by a signal (other than
<systemitem>SIGTERM</systemitem> that is used as part of a server
shutdown) or an error by the shell with an exit status greater than
125 (such as command not found), or if the archive function emits an
<literal>ERROR</literal> or <literal>FATAL</literal>, the archiver process
aborts and gets restarted by the postmaster. In such cases, the failure is
not reported in <xref linkend="pg-stat-archiver-view"/>.
</para>
<para>
It is important that the archive function return <literal>true</literal> if
and only if it succeeds. If <literal>true</literal> is returned,
<productname>PostgreSQL</productname> will assume that the file has been
successfully archived, and will remove or recycle it. However, a return
value of <literal>false</literal> tells
<productname>PostgreSQL</productname> that the file was not archived; it
will try again periodically until it succeeds. If you are archiving via a
shell command, the appropriate return values can be achieved by returning
<literal>0</literal> if the command succeeds and a nonzero value if it
fails.
</para>
<para>
If the archive function emits an <literal>ERROR</literal> or
<literal>FATAL</literal>, the archiver process aborts and gets restarted by
the postmaster. If you are archiving via shell command,
<literal>FATAL</literal> is emitted if the command is terminated by a signal
(other than <systemitem>SIGTERM</systemitem>
that is used as part of a server shutdown)
or an error by the shell with an exit status greater than 125 (such as
command not found). In such cases, the failure is not reported in
<xref linkend="pg-stat-archiver-view"/>.
</para>
<para>
The archive library should generally be designed to refuse to overwrite
Archive commands and libraries should generally be designed to refuse to overwrite
any pre-existing archive file. This is an important safety feature to
preserve the integrity of your archive in case of administrator error
(such as sending the output of two different servers to the same archive
@@ -691,13 +687,13 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
re-archive a WAL file that was previously archived. For example, if the
system crashes before the server makes a durable record of archival
success, the server will attempt to archive the file again after
restarting (provided archiving is still enabled). When an archive library
encounters a pre-existing file, it should return <literal>true</literal>
restarting (provided archiving is still enabled). When an archive command or library
encounters a pre-existing file, it should return a zero status or <literal>true</literal>, respectively,
if the WAL file has identical contents to the pre-existing archive and the
pre-existing archive is fully persisted to storage. If a pre-existing
file contains different contents than the WAL file being archived, the
archive library <emphasis>must</emphasis> return
<literal>false</literal>.
archive command or library <emphasis>must</emphasis> return a nonzero status or
<literal>false</literal>, respectively.
</para>
<para>
@@ -713,7 +709,7 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
<para>
While designing your archiving setup, consider what will happen if
the archive library fails repeatedly because some aspect requires
the archive command or library fails repeatedly because some aspect requires
operator intervention or the archive runs out of space. For example, this
could occur if you write to tape without an autochanger; when the tape
fills, nothing further can be archived until the tape is swapped.
@@ -728,7 +724,7 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
</para>
<para>
The speed of the archive library is unimportant as long as it can keep up
The speed of the archive command or library is unimportant as long as it can keep up
with the average rate at which your server generates WAL data. Normal
operation continues even if the archiving process falls a little behind.
If archiving falls significantly behind, this will increase the amount of
@@ -740,11 +736,11 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
</para>
<para>
In writing your archive library, you should assume that the file names to
In writing your archive command or library, you should assume that the file names to
be archived can be up to 64 characters long and can contain any
combination of ASCII letters, digits, and dots. It is not necessary to
preserve the original relative path but it is necessary to preserve the file
name.
preserve the original relative path (<literal>%p</literal>) but it is necessary to
preserve the file name (<literal>%f</literal>).
</para>
<para>
@@ -761,7 +757,7 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
</para>
<para>
The archive function is only invoked on completed WAL segments. Hence,
The archive command or function is only invoked on completed WAL segments. Hence,
if your server generates only little WAL traffic (or has slack periods
where it does so), there could be a long delay between the completion
of a transaction and its safe recording in archive storage. To put
@@ -790,7 +786,7 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 &amp;&amp; cp pg_wal/0
turned on during execution of one of these statements, WAL would not
contain enough information for archive recovery. (Crash recovery is
unaffected.) For this reason, <varname>wal_level</varname> can only be changed at
server start. However, <varname>archive_library</varname> can be changed with a
server start. However, <varname>archive_command</varname> and <varname>archive_library</varname> can be changed with a
configuration file reload. If you are archiving via shell and wish to
temporarily stop archiving,
one way to do it is to set <varname>archive_command</varname> to the empty
@@ -960,12 +956,12 @@ SELECT * FROM pg_backup_stop(wait_for_archive => true);
On a standby, <varname>archive_mode</varname> must be <literal>always</literal> in order
for <function>pg_backup_stop</function> to wait.
Archiving of these files happens automatically since you have
already configured <varname>archive_library</varname> or
already configured <varname>archive_command</varname> or <varname>archive_library</varname> or
<varname>archive_command</varname>.
In most cases this happens quickly, but you are advised to monitor your
archive system to ensure there are no delays.
If the archive process has fallen behind because of failures of the
archive library or archive command, it will keep retrying
archive command or library, it will keep retrying
until the archive succeeds and the backup is complete.
If you wish to place a time limit on the execution of
<function>pg_backup_stop</function>, set an appropriate

61
doc/src/sgml/config.sgml
View File

@@ -3503,7 +3503,7 @@ include_dir 'conf.d'
Maximum size to let the WAL grow during automatic
checkpoints. This is a soft limit; WAL size can exceed
<varname>max_wal_size</varname> under special circumstances, such as
heavy load, a failing <varname>archive_library</varname>, or a high
heavy load, a failing <varname>archive_command</varname> or <varname>archive_library</varname>, or a high
<varname>wal_keep_size</varname> setting.
If this value is specified without units, it is taken as megabytes.
The default is 1 GB.
@@ -3552,6 +3552,7 @@ include_dir 'conf.d'
<para>
When <varname>archive_mode</varname> is enabled, completed WAL segments
are sent to archive storage by setting
<xref linkend="guc-archive-command"/> or
<xref linkend="guc-archive-library"/>. In addition to <literal>off</literal>,
to disable, there are two modes: <literal>on</literal>, and
<literal>always</literal>. During normal operation, there is no
@@ -3562,6 +3563,12 @@ include_dir 'conf.d'
<xref linkend="continuous-archiving-in-standby"/> for details.
</para>
<para>
<varname>archive_mode</varname> is a separate setting from
<varname>archive_command</varname> and
<varname>archive_library</varname> so that
<varname>archive_command</varname> and
<varname>archive_library</varname> can be changed without leaving
archiving mode.
This parameter can only be set at server start.
<varname>archive_mode</varname> cannot be enabled when
<varname>wal_level</varname> is set to <literal>minimal</literal>.
@@ -3569,28 +3576,6 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
<varlistentry id="guc-archive-library" xreflabel="archive_library">
<term><varname>archive_library</varname> (<type>string</type>)
<indexterm>
<primary><varname>archive_library</varname> configuration parameter</primary>
</indexterm>
</term>
<listitem>
<para>
The library to use for archiving completed WAL file segments. If set to
an empty string (the default), archiving via shell is enabled, and
<xref linkend="guc-archive-command"/> is used. Otherwise, the specified
shared library is used for archiving. For more information, see
<xref linkend="backup-archiving-wal"/> and
<xref linkend="archive-modules"/>.
</para>
<para>
This parameter can only be set in the
<filename>postgresql.conf</filename> file or on the server command line.
</para>
</listitem>
</varlistentry>
<varlistentry id="guc-archive-command" xreflabel="archive_command">
<term><varname>archive_command</varname> (<type>string</type>)
<indexterm>
@@ -3614,10 +3599,10 @@ include_dir 'conf.d'
This parameter can only be set in the <filename>postgresql.conf</filename>
file or on the server command line. It is ignored unless
<varname>archive_mode</varname> was enabled at server start and
<varname>archive_library</varname> specifies to archive via shell command.
<varname>archive_library</varname> is set to an empty string.
If <varname>archive_command</varname> is an empty string (the default) while
<varname>archive_mode</varname> is enabled and <varname>archive_library</varname>
specifies archiving via shell, WAL archiving is temporarily
<varname>archive_mode</varname> is enabled (and <varname>archive_library</varname>
is set to an empty string), WAL archiving is temporarily
disabled, but the server continues to accumulate WAL segment files in
the expectation that a command will soon be provided. Setting
<varname>archive_command</varname> to a command that does nothing but
@@ -3629,6 +3614,28 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
<varlistentry id="guc-archive-library" xreflabel="archive_library">
<term><varname>archive_library</varname> (<type>string</type>)
<indexterm>
<primary><varname>archive_library</varname> configuration parameter</primary>
</indexterm>
</term>
<listitem>
<para>
The library to use for archiving completed WAL file segments. If set to
an empty string (the default), archiving via shell is enabled, and
<xref linkend="guc-archive-command"/> is used. Otherwise, the specified
shared library is used for archiving. For more information, see
<xref linkend="backup-archiving-wal"/> and
<xref linkend="archive-modules"/>.
</para>
<para>
This parameter can only be set in the
<filename>postgresql.conf</filename> file or on the server command line.
</para>
</listitem>
</varlistentry>
<varlistentry id="guc-archive-timeout" xreflabel="archive_timeout">
<term><varname>archive_timeout</varname> (<type>integer</type>)
<indexterm>
@@ -3637,7 +3644,7 @@ include_dir 'conf.d'
</term>
<listitem>
<para>
The <xref linkend="guc-archive-library"/> is only invoked for
The <xref linkend="guc-archive-command"/> or <xref linkend="guc-archive-library"/> is only invoked for
completed WAL segments. Hence, if your server generates little WAL
traffic (or has slack periods where it does so), there could be a
long delay between the completion of a transaction and its safe

6
doc/src/sgml/high-availability.sgml
View File

@@ -935,7 +935,7 @@ primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
In lieu of using replication slots, it is possible to prevent the removal
of old WAL segments using <xref linkend="guc-wal-keep-size"/>, or by
storing the segments in an archive using
<xref linkend="guc-archive-library"/>.
<xref linkend="guc-archive-command"/> or <xref linkend="guc-archive-library"/>.
However, these methods often result in retaining more WAL segments than
required, whereas replication slots retain only the number of segments
known to be needed. On the other hand, replication slots can retain so
@@ -1386,10 +1386,10 @@ synchronous_standby_names = 'ANY 2 (s1, s2, s3)'
to <literal>always</literal>, and the standby will call the archive
command for every WAL segment it receives, whether it's by restoring
from the archive or by streaming replication. The shared archive can
be handled similarly, but the <varname>archive_library</varname> must
be handled similarly, but the <varname>archive_command</varname> or <varname>archive_library</varname> must
test if the file being archived exists already, and if the existing file
has identical contents. This requires more care in the
<varname>archive_library</varname>, as it must
<varname>archive_command</varname> or <varname>archive_library</varname>, as it must
be careful to not overwrite an existing file with different contents,
but return success if the exactly same file is archived twice. And
all that must be done free of race conditions, if two servers attempt

7
doc/src/sgml/ref/pg_receivewal.sgml
View File

@@ -40,7 +40,8 @@ PostgreSQL documentation
<para>
<application>pg_receivewal</application> streams the write-ahead
log in real time as it's being generated on the server, and does not wait
for segments to complete like <xref linkend="guc-archive-library"/> does.
for segments to complete like <xref linkend="guc-archive-command"/> and
<xref linkend="guc-archive-library"/> do.
For this reason, it is not necessary to set
<xref linkend="guc-archive-timeout"/> when using
<application>pg_receivewal</application>.
@@ -488,11 +489,13 @@ PostgreSQL documentation
<para>
When using <application>pg_receivewal</application> instead of
<xref linkend="guc-archive-command"/> or
<xref linkend="guc-archive-library"/> as the main WAL backup method, it is
strongly recommended to use replication slots. Otherwise, the server is
free to recycle or remove write-ahead log files before they are backed up,
because it does not have any information, either
from <xref linkend="guc-archive-library"/> or the replication slots, about
from <xref linkend="guc-archive-command"/> or
<xref linkend="guc-archive-library"/> or the replication slots, about
how far the WAL stream has been archived. Note, however, that a
replication slot will fill up the server's disk space if the receiver does
not keep up with fetching the WAL data.

3
doc/src/sgml/wal.sgml
View File

@@ -637,7 +637,8 @@
WAL files plus one additional WAL file are
kept at all times. Also, if WAL archiving is used, old segments cannot be
removed or recycled until they are archived. If WAL archiving cannot keep up
with the pace that WAL is generated, or if <varname>archive_library</varname>
with the pace that WAL is generated, or if <varname>archive_command</varname>
or <varname>archive_library</varname>
fails repeatedly, old WAL files will accumulate in <filename>pg_wal</filename>
until the situation is resolved. A slow or failed standby server that
uses a replication slot will have the same effect (see