1
0
mirror of https://github.com/postgres/postgres.git synced 2025-07-30 11:03:19 +03:00

Add support for LZ4 with compression of full-page writes in WAL

The logic is implemented so as there can be a choice in the compression
used when building a WAL record, and an extra per-record bit is used to
track down if a block is compressed with PGLZ, LZ4 or nothing.

wal_compression, the existing parameter, is changed to an enum with
support for the following backward-compatible values:
- "off", the default, to not use compression.
- "pglz" or "on", to compress FPWs with PGLZ.
- "lz4", the new mode, to compress FPWs with LZ4.

Benchmarking has showed that LZ4 outclasses easily PGLZ.  ZSTD would be
also an interesting choice, but going just with LZ4 for now makes the
patch minimalistic as toast compression is already able to use LZ4, so
there is no need to worry about any build-related needs for this
implementation.

Author: Andrey Borodin, Justin Pryzby
Reviewed-by: Dilip Kumar, Michael Paquier
Discussion: https://postgr.es/m/3037310D-ECB7-4BF1-AF20-01C10BB33A33@yandex-team.ru
This commit is contained in:
Michael Paquier
2021-06-29 11:17:55 +09:00
parent cc2c7d65fc
commit 4035cd5d4e
13 changed files with 191 additions and 52 deletions

View File

@ -114,8 +114,8 @@ typedef struct XLogRecordBlockHeader
* present is (BLCKSZ - <length of "hole" bytes>).
*
* Additionally, when wal_compression is enabled, we will try to compress full
* page images using the PGLZ compression algorithm, after removing the "hole".
* This can reduce the WAL volume, but at some extra cost of CPU spent
* page images using one of the supported algorithms, after removing the
* "hole". This can reduce the WAL volume, but at some extra cost of CPU spent
* on the compression during WAL logging. In this case, since the "hole"
* length cannot be calculated by subtracting the number of page image bytes
* from BLCKSZ, basically it needs to be stored as an extra information.
@ -134,7 +134,7 @@ typedef struct XLogRecordBlockImageHeader
uint8 bimg_info; /* flag bits, see below */
/*
* If BKPIMAGE_HAS_HOLE and BKPIMAGE_IS_COMPRESSED, an
* If BKPIMAGE_HAS_HOLE and BKPIMAGE_COMPRESSED(), an
* XLogRecordBlockCompressHeader struct follows.
*/
} XLogRecordBlockImageHeader;
@ -144,9 +144,13 @@ typedef struct XLogRecordBlockImageHeader
/* Information stored in bimg_info */
#define BKPIMAGE_HAS_HOLE 0x01 /* page image has "hole" */
#define BKPIMAGE_IS_COMPRESSED 0x02 /* page image is compressed */
#define BKPIMAGE_APPLY 0x04 /* page image should be restored during
* replay */
#define BKPIMAGE_APPLY 0x02 /* page image should be restored
* during replay */
/* compression methods supported */
#define BKPIMAGE_COMPRESS_PGLZ 0x04
#define BKPIMAGE_COMPRESS_LZ4 0x08
#define BKPIMAGE_COMPRESSED(info) \
((info & (BKPIMAGE_COMPRESS_PGLZ | BKPIMAGE_COMPRESS_LZ4)) != 0)
/*
* Extra header information used when page image has "hole" and