database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-27 12:41:57 +03:00
Code Activity

Allow parallel zstd compression when taking a base backup.

Browse Source 
libzstd allows transparent parallel compression just by setting
an option when creating the compression context, so permit that
for both client and server-side backup compression. To use this,
use something like pg_basebackup --compress WHERE-zstd:workers=N
where WHERE is "client" or "server" and N is an integer.

When compression is performed on the server side, this will spawn
threads inside the PostgreSQL backend. While there is almost no
PostgreSQL server code which is thread-safe, the threads here are used
internally by libzstd and touch only data structures controlled by
libzstd.

Patch by me, based in part on earlier work by Dipesh Pandit
and Jeevan Ladhe. Reviewed by Justin Pryzby.

Discussion: http://postgr.es/m/CA+Tgmobj6u-nWF-j=FemygUhobhryLxf9h-wJN7W-2rSsseHNA@mail.gmail.com
This commit is contained in:
Robert Haas
2022-03-30 09:35:14 -04:00
parent c6863b8582
commit 51c0d186d9
9 changed files with 147 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
doc/src/sgml/protocol.sgml
View File

@ -2739,17 +2739,23 @@ The commands accepted in replication mode are:
option. If the value is an integer, it specifies the compression
level. Otherwise, it should be a comma-separated list of items,
each of the form <literal>keyword</literal> or
<literal>keyword=value</literal>. Currently, the only supported
keyword is <literal>level</literal>, which sets the compression
level.
<literal>keyword=value</literal>. Currently, the supported keywords
are <literal>level</literal> and <literal>workers</literal>.
</para>
<para>
The <literal>level</literal> keyword sets the compression level.
For <literal>gzip</literal> the compression level should be an
integer between 1 and 9, for <literal>lz4</literal> an integer
between 1 and 12, and for <literal>zstd</literal> an integer
between 1 and 22.
</para>
<para>
The <literal>workers</literal> keyword sets the number of threads
that should be used for parallel compression. Parallel compression
is supported only for <literal>zstd</literal>.
</para>
</listitem>
</varlistentry>

4
doc/src/sgml/ref/pg_basebackup.sgml
View File

@ -424,8 +424,8 @@ PostgreSQL documentation
integer, it specifies the compression level. Otherwise, it should be
a comma-separated list of items, each of the form
<literal>keyword</literal> or <literal>keyword=value</literal>.
Currently, the only supported keyword is <literal>level</literal>,
which sets the compression level.
Currently, the supported keywords are <literal>level</literal>
and <literal>workers</literal>.
</para>
<para>
If no compression level is specified, the default compression level