postgres/doc/src/sgml/ref/pg_receivewal.sgml

<!--
doc/src/sgml/ref/pg_receivewal.sgml
PostgreSQL documentation
-->

<refentry id="app-pgreceivewal">
 <indexterm zone="app-pgreceivewal">
  <primary>pg_receivewal</primary>
 </indexterm>

 <refmeta>
  <refentrytitle><application>pg_receivewal</application></refentrytitle>
  <manvolnum>1</manvolnum>
  <refmiscinfo>Application</refmiscinfo>
 </refmeta>

 <refnamediv>
  <refname>pg_receivewal</refname>
  <refpurpose>stream write-ahead logs from a <productname>PostgreSQL</productname> server</refpurpose>
 </refnamediv>

 <refsynopsisdiv>
  <cmdsynopsis>
   <command>pg_receivewal</command>
   <arg rep="repeat"><replaceable>option</replaceable></arg>
  </cmdsynopsis>
 </refsynopsisdiv>

 <refsect1>
  <title>Description</title>
  <para>
   <application>pg_receivewal</application> is used to stream the write-ahead log
   from a running <productname>PostgreSQL</productname> cluster. The write-ahead
   log is streamed using the streaming replication protocol, and is written
   to a local directory of files. This directory can be used as the archive
   location for doing a restore using point-in-time recovery (see
   <xref linkend="continuous-archiving"/>).
  </para>

  <para>
   <application>pg_receivewal</application> streams the write-ahead
   log in real time as it's being generated on the server, and does not wait
   for segments to complete like <xref linkend="guc-archive-command"/> and
   <xref linkend="guc-archive-library"/> do.
   For this reason, it is not necessary to set
   <xref linkend="guc-archive-timeout"/> when using
    <application>pg_receivewal</application>.
  </para>

  <para>
   Unlike the WAL receiver of a PostgreSQL standby server, <application>pg_receivewal</application>
   by default flushes WAL data only when a WAL file is closed.
   The option <option>--synchronous</option> must be specified to flush WAL data
   in real time. Since <application>pg_receivewal</application> does not
   apply WAL, you should not allow it to become a synchronous standby when
   <xref linkend="guc-synchronous-commit"/> equals
   <literal>remote_apply</literal>. If it does, it will appear to be a
   standby that never catches up, and will cause transaction commits to
   block. To avoid this, you should either configure an appropriate value
   for <xref linkend="guc-synchronous-standby-names"/>, or specify
   <varname>application_name</varname> for
   <application>pg_receivewal</application> that does not match it, or
   change the value of <varname>synchronous_commit</varname> to
   something other than <literal>remote_apply</literal>.
  </para>

  <para>
   The write-ahead log is streamed over a regular
   <productname>PostgreSQL</productname> connection and uses the replication
   protocol. The connection must be made with a user having
   <literal>REPLICATION</literal> permissions (see
   <xref linkend="role-attributes"/>) or a superuser, and
   <filename>pg_hba.conf</filename> must permit the replication connection.
   The server must also be configured with
   <xref linkend="guc-max-wal-senders"/> set high enough to leave at least
   one session available for the stream.
  </para>

  <para>
   The starting point of the write-ahead log streaming is calculated when
   <application>pg_receivewal</application> starts:
   <orderedlist>
    <listitem>
     <para>
      First, scan the directory where the WAL segment files are written and
      find the newest completed segment file, using as the starting point the
      beginning of the next WAL segment file.
     </para>
    </listitem>

    <listitem>
     <para>
      If a starting point cannot be calculated with the previous method,
      and if a replication slot is used, an extra
      <command>READ_REPLICATION_SLOT</command> command is issued to retrieve
      the slot's <literal>restart_lsn</literal> to use as the starting point.
      This option is only available when streaming write-ahead logs from
      <productname>PostgreSQL</productname> 15 and up.
     </para>
    </listitem>

    <listitem>
     <para>
      If a starting point cannot be calculated with the previous method,
      the latest WAL flush location is used as reported by the server from
      an <literal>IDENTIFY_SYSTEM</literal> command.
     </para>
    </listitem>
   </orderedlist>
  </para>

  <para>
   If the connection is lost, or if it cannot be initially established,
   with a non-fatal error, <application>pg_receivewal</application> will
   retry the connection indefinitely, and reestablish streaming as soon
   as possible. To avoid this behavior, use the <literal>-n</literal>
   parameter.
  </para>

  <para>
   In the absence of fatal errors, <application>pg_receivewal</application>
   will run until terminated by the <systemitem>SIGINT</systemitem>
   (<keycombo action="simul"><keycap>Control</keycap><keycap>C</keycap></keycombo>)
   or <systemitem>SIGTERM</systemitem> signal.
  </para>
 </refsect1>

 <refsect1>
  <title>Options</title>

    <variablelist>
     <varlistentry>
      <term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
      <term><option>--directory=<replaceable class="parameter">directory</replaceable></option></term>
      <listitem>
       <para>
        Directory to write the output to.
       </para>
       <para>
        This parameter is required.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-E <replaceable>lsn</replaceable></option></term>
      <term><option>--endpos=<replaceable>lsn</replaceable></option></term>
      <listitem>
       <para>
        Automatically stop replication and exit with normal exit status 0 when
        receiving reaches the specified LSN.
       </para>

       <para>
        If there is a record with LSN exactly equal to <replaceable>lsn</replaceable>,
        the record will be processed.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>--if-not-exists</option></term>
      <listitem>
       <para>
        Do not error out when <option>--create-slot</option> is specified
        and a slot with the specified name already exists.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-n</option></term>
      <term><option>--no-loop</option></term>
      <listitem>
       <para>
        Don't loop on connection errors. Instead, exit right away with
        an error.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>--no-sync</option></term>
      <listitem>
       <para>
        This option causes <command>pg_receivewal</command> to not force WAL
        data to be flushed to disk.  This is faster, but means that a
        subsequent operating system crash can leave the WAL segments corrupt.
        Generally, this option is useful for testing but should not be used
        when doing WAL archiving on a production deployment.
       </para>

       <para>
        This option is incompatible with <literal>--synchronous</literal>.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-s <replaceable class="parameter">interval</replaceable></option></term>
      <term><option>--status-interval=<replaceable class="parameter">interval</replaceable></option></term>
      <listitem>
       <para>
        Specifies the number of seconds between status packets sent back to the
        server. This allows for easier monitoring of the progress from server.
        A value of zero disables the periodic status updates completely,
        although an update will still be sent when requested by the server, to
        avoid timeout disconnect. The default value is 10 seconds.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-S <replaceable>slotname</replaceable></option></term>
      <term><option>--slot=<replaceable class="parameter">slotname</replaceable></option></term>
      <listitem>
        <para>
         Require <application>pg_receivewal</application> to use an existing
         replication slot (see <xref linkend="streaming-replication-slots"/>).
         When this option is used, <application>pg_receivewal</application> will report
         a flush position to the server, indicating when each segment has been
         synchronized to disk so that the server can remove that segment if it
         is not otherwise needed.
        </para>

        <para>
         When the replication client
         of <application>pg_receivewal</application> is configured on the
         server as a synchronous standby, then using a replication slot will
         report the flush position to the server, but only when a WAL file is
         closed.  Therefore, that configuration will cause transactions on the
         primary to wait for a long time and effectively not work
         satisfactorily.  The option <literal>--synchronous</literal> (see
         below) must be specified in addition to make this work correctly.
        </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>--synchronous</option></term>
      <listitem>
       <para>
        Flush the WAL data to disk immediately after it has been received. Also
        send a status packet back to the server immediately after flushing,
        regardless of <literal>--status-interval</literal>.
       </para>

       <para>
        This option should be specified if the replication client
        of <application>pg_receivewal</application> is configured on the
        server as a synchronous standby, to ensure that timely feedback is
        sent to the server.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-v</option></term>
      <term><option>--verbose</option></term>
      <listitem>
       <para>
        Enables verbose mode.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-Z <replaceable class="parameter">level</replaceable></option></term>
      <term><option>-Z <replaceable class="parameter">method</replaceable>[:<replaceable>detail</replaceable>]</option></term>
      <term><option>--compress=<replaceable class="parameter">level</replaceable></option></term>
      <term><option>--compress=<replaceable class="parameter">method</replaceable>[:<replaceable>detail</replaceable>]</option></term>
      <listitem>
       <para>
        Enables compression of write-ahead logs.
       </para>
       <para>
        The compression method can be set to <literal>gzip</literal>,
        <literal>lz4</literal> (if <productname>PostgreSQL</productname>
        was compiled with <option>--with-lz4</option>) or
        <literal>none</literal> for no compression.
        A compression detail string can optionally be specified.  If the
        detail string is an integer, it specifies the compression level.
        Otherwise, it should be a comma-separated list of items, each of the
        form <literal>keyword</literal> or <literal>keyword=value</literal>.
        Currently, the only supported keyword is <literal>level</literal>.
       </para>
       <para>
        If no compression level is specified, the default compression level
        will be used. If only a level is specified without mentioning an
        algorithm, <literal>gzip</literal> compression will be used if the
        level is greater than 0, and no compression will be used if the level
        is 0.
       </para>
       <para>
        The suffix <filename>.gz</filename> will automatically be added to
        all filenames when using <literal>gzip</literal>, and the suffix
        <filename>.lz4</filename> is added when using <literal>lz4</literal>.
       </para>
      </listitem>
     </varlistentry>
    </variablelist>

   <para>
    The following command-line options control the database connection parameters.

    <variablelist>
     <varlistentry>
      <term><option>-d <replaceable class="parameter">connstr</replaceable></option></term>
      <term><option>--dbname=<replaceable class="parameter">connstr</replaceable></option></term>
      <listitem>
       <para>
        Specifies parameters used to connect to the server, as a <link
        linkend="libpq-connstring">connection string</link>;  these
        will override any conflicting command line options.
       </para>
       <para>
        The option is called <literal>--dbname</literal> for consistency with other
        client applications, but because <application>pg_receivewal</application>
        doesn't connect to any particular database in the cluster, any database
        name in the connection string will be ignored by
        <productname>PostgreSQL</productname>. Middleware, or proxies, used in
        connecting to <productname>PostgreSQL</productname> might however
        utilize the value.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
      <term><option>--host=<replaceable class="parameter">host</replaceable></option></term>
      <listitem>
       <para>
        Specifies the host name of the machine on which the server is
        running.  If the value begins with a slash, it is used as the
        directory for the Unix domain socket. The default is taken
        from the <envar>PGHOST</envar> environment variable, if set,
        else a Unix domain socket connection is attempted.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-p <replaceable class="parameter">port</replaceable></option></term>
      <term><option>--port=<replaceable class="parameter">port</replaceable></option></term>
      <listitem>
       <para>
        Specifies the TCP port or local Unix domain socket file
        extension on which the server is listening for connections.
        Defaults to the <envar>PGPORT</envar> environment variable, if
        set, or a compiled-in default.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-U <replaceable>username</replaceable></option></term>
      <term><option>--username=<replaceable class="parameter">username</replaceable></option></term>
      <listitem>
       <para>
        User name to connect as.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-w</option></term>
      <term><option>--no-password</option></term>
      <listitem>
       <para>
        Never issue a password prompt.  If the server requires
        password authentication and a password is not available by
        other means such as a <filename>.pgpass</filename> file, the
        connection attempt will fail.  This option can be useful in
        batch jobs and scripts where no user is present to enter a
        password.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>-W</option></term>
      <term><option>--password</option></term>
      <listitem>
       <para>
        Force <application>pg_receivewal</application> to prompt for a
        password before connecting to a database.
       </para>

       <para>
        This option is never essential, since
        <application>pg_receivewal</application> will automatically prompt
        for a password if the server demands password authentication.
        However, <application>pg_receivewal</application> will waste a
        connection attempt finding out that the server wants a password.
        In some cases it is worth typing <option>-W</option> to avoid the extra
        connection attempt.
       </para>
      </listitem>
     </varlistentry>
    </variablelist>
   </para>

   <para>
    <application>pg_receivewal</application> can perform one of the two
    following actions in order to control physical replication slots:

    <variablelist>
     <varlistentry>
      <term><option>--create-slot</option></term>
      <listitem>
       <para>
        Create a new physical replication slot with the name specified in
        <option>--slot</option>, then exit.
       </para>
      </listitem>
     </varlistentry>

     <varlistentry>
      <term><option>--drop-slot</option></term>
      <listitem>
       <para>
        Drop the replication slot with the name specified in
        <option>--slot</option>, then exit.
       </para>
      </listitem>
     </varlistentry>
    </variablelist>
   </para>

   <para>
    Other options are also available:

    <variablelist>
     <varlistentry>
       <term><option>-V</option></term>
       <term><option>--version</option></term>
       <listitem>
       <para>
       Print the <application>pg_receivewal</application> version and exit.
       </para>
       </listitem>
     </varlistentry>

     <varlistentry>
       <term><option>-?</option></term>
       <term><option>--help</option></term>
       <listitem>
       <para>
       Show help about <application>pg_receivewal</application> command line
       arguments, and exit.
       </para>
       </listitem>
     </varlistentry>

    </variablelist>
   </para>

 </refsect1>

 <refsect1>
  <title>Exit Status</title>

  <para>
   <application>pg_receivewal</application> will exit with status 0 when
   terminated by the <systemitem>SIGINT</systemitem> or
   <systemitem>SIGTERM</systemitem> signal.  (That is the
   normal way to end it.  Hence it is not an error.)  For fatal errors or
   other signals, the exit status will be nonzero.
  </para>

 </refsect1>

 <refsect1>
  <title>Environment</title>

  <para>
   This utility, like most other <productname>PostgreSQL</productname> utilities,
   uses the environment variables supported by <application>libpq</application>
   (see <xref linkend="libpq-envars"/>).
  </para>

  <para>
   The environment variable <envar>PG_COLOR</envar> specifies whether to use
   color in diagnostic messages. Possible values are
   <literal>always</literal>, <literal>auto</literal> and
   <literal>never</literal>.
  </para>
 </refsect1>

 <refsect1>
  <title>Notes</title>

  <para>
   When using <application>pg_receivewal</application> instead of
   <xref linkend="guc-archive-command"/> or
   <xref linkend="guc-archive-library"/> as the main WAL backup method, it is
   strongly recommended to use replication slots.  Otherwise, the server is
   free to recycle or remove write-ahead log files before they are backed up,
   because it does not have any information, either
   from <xref linkend="guc-archive-command"/> or
   <xref linkend="guc-archive-library"/> or the replication slots, about
   how far the WAL stream has been archived.  Note, however, that a
   replication slot will fill up the server's disk space if the receiver does
   not keep up with fetching the WAL data.
  </para>

  <para>
   <application>pg_receivewal</application> will preserve group permissions on
   the received WAL files if group permissions are enabled on the source
   cluster.
  </para>

 </refsect1>

 <refsect1>
  <title>Examples</title>

  <para>
   To stream the write-ahead log from the server at
   <literal>mydbserver</literal> and store it in the local directory
   <filename>/usr/local/pgsql/archive</filename>:
<screen>
<prompt>$</prompt> <userinput>pg_receivewal -h mydbserver -D /usr/local/pgsql/archive</userinput>
</screen></para>
 </refsect1>

 <refsect1>
  <title>See Also</title>

  <simplelist type="inline">
   <member><xref linkend="app-pgbasebackup"/></member>
  </simplelist>
 </refsect1>

</refentry>