database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-28 23:42:10 +03:00
Code Activity

Switch to CRC-32C in WAL and other places.

Browse Source 
The old algorithm was found to not be the usual CRC-32 algorithm, used by
Ethernet et al. We were using a non-reflected lookup table with code meant
for a reflected lookup table. That's a strange combination that AFAICS does
not correspond to any bit-wise CRC calculation, which makes it difficult to
reason about its properties. Although it has worked well in practice, seems
safer to use a well-known algorithm.

Since we're changing the algorithm anyway, we might as well choose a
different polynomial. The Castagnoli polynomial has better error-correcting
properties than the traditional CRC-32 polynomial, even if we had
implemented it correctly. Another reason for picking that is that some new
CPUs have hardware support for calculating CRC-32C, but not CRC-32, let
alone our strange variant of it. This patch doesn't add any support for such
hardware, but a future patch could now do that.

The old algorithm is kept around for tsquery and pg_trgm, which use the
values in indexes that need to remain compatible so that pg_upgrade works.
While we're at it, share the old lookup table for CRC-32 calculation
between hstore, ltree and core. They all use the same table, so might as
well.
This commit is contained in:
Heikki Linnakangas
2014-11-04 11:35:15 +02:00
parent 404bc51cde
commit 5028f22f6e
20 changed files with 299 additions and 340 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
src/backend/replication/logical/snapbuild.c
View File

@ -1517,9 +1517,9 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
ondisk->magic = SNAPBUILD_MAGIC;
ondisk->version = SNAPBUILD_VERSION;
ondisk->length = needed_length;
INIT_CRC32(ondisk->checksum);
COMP_CRC32(ondisk->checksum,
((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
INIT_CRC32C(ondisk->checksum);
COMP_CRC32C(ondisk->checksum,
((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
ondisk_c += sizeof(SnapBuildOnDisk);
@ -1531,20 +1531,20 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
ondisk->builder.running.xip = NULL;
ondisk->builder.committed.xip = NULL;
COMP_CRC32(ondisk->checksum,
&ondisk->builder,
sizeof(SnapBuild));
COMP_CRC32C(ondisk->checksum,
&ondisk->builder,
sizeof(SnapBuild));
/* copy running xacts */
sz = sizeof(TransactionId) * builder->running.xcnt_space;
memcpy(ondisk_c, builder->running.xip, sz);
COMP_CRC32(ondisk->checksum, ondisk_c, sz);
COMP_CRC32C(ondisk->checksum, ondisk_c, sz);
ondisk_c += sz;
/* copy committed xacts */
sz = sizeof(TransactionId) * builder->committed.xcnt;
memcpy(ondisk_c, builder->committed.xip, sz);
COMP_CRC32(ondisk->checksum, ondisk_c, sz);
COMP_CRC32C(ondisk->checksum, ondisk_c, sz);
ondisk_c += sz;
/* we have valid data now, open tempfile and write it there */
@ -1672,8 +1672,8 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
(errmsg("snapbuild state file \"%s\" has unsupported version %u instead of %u",
path, ondisk.version, SNAPBUILD_VERSION)));
INIT_CRC32(checksum);
COMP_CRC32(checksum,
INIT_CRC32C(checksum);
COMP_CRC32C(checksum,
((char *) &ondisk) + SnapBuildOnDiskNotChecksummedSize,
SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
@ -1687,7 +1687,7 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
errmsg("could not read file \"%s\", read %d of %d: %m",
path, readBytes, (int) sizeof(SnapBuild))));
}
COMP_CRC32(checksum, &ondisk.builder, sizeof(SnapBuild));
COMP_CRC32C(checksum, &ondisk.builder, sizeof(SnapBuild));
/* restore running xacts information */
sz = sizeof(TransactionId) * ondisk.builder.running.xcnt_space;
@ -1701,7 +1701,7 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
errmsg("could not read file \"%s\", read %d of %d: %m",
path, readBytes, (int) sz)));
}
COMP_CRC32(checksum, ondisk.builder.running.xip, sz);
COMP_CRC32C(checksum, ondisk.builder.running.xip, sz);
/* restore committed xacts information */
sz = sizeof(TransactionId) * ondisk.builder.committed.xcnt;
@ -1715,12 +1715,12 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
errmsg("could not read file \"%s\", read %d of %d: %m",
path, readBytes, (int) sz)));
}
COMP_CRC32(checksum, ondisk.builder.committed.xip, sz);
COMP_CRC32C(checksum, ondisk.builder.committed.xip, sz);
CloseTransientFile(fd);
/* verify checksum of what we've read */
if (!EQ_CRC32(checksum, ondisk.checksum))
if (!EQ_CRC32C(checksum, ondisk.checksum))
ereport(ERROR,
(errcode_for_file_access(),
errmsg("snapbuild state file %s: checksum mismatch, is %u, should be %u",