database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-11-09 06:21:09 +03:00
Code Activity

Switch to CRC-32C in WAL and other places.

Browse Source 
The old algorithm was found to not be the usual CRC-32 algorithm, used by
Ethernet et al. We were using a non-reflected lookup table with code meant
for a reflected lookup table. That's a strange combination that AFAICS does
not correspond to any bit-wise CRC calculation, which makes it difficult to
reason about its properties. Although it has worked well in practice, seems
safer to use a well-known algorithm.

Since we're changing the algorithm anyway, we might as well choose a
different polynomial. The Castagnoli polynomial has better error-correcting
properties than the traditional CRC-32 polynomial, even if we had
implemented it correctly. Another reason for picking that is that some new
CPUs have hardware support for calculating CRC-32C, but not CRC-32, let
alone our strange variant of it. This patch doesn't add any support for such
hardware, but a future patch could now do that.

The old algorithm is kept around for tsquery and pg_trgm, which use the
values in indexes that need to remain compatible so that pg_upgrade works.
While we're at it, share the old lookup table for CRC-32 calculation
between hstore, ltree and core. They all use the same table, so might as
well.
This commit is contained in:
Heikki Linnakangas
2014-11-04 11:35:15 +02:00
parent 404bc51cde
commit 5028f22f6e
20 changed files with 299 additions and 340 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
src/backend/access/transam/twophase.c
View File

@@ -847,9 +847,9 @@ TwoPhaseGetDummyProc(TransactionId xid)
* 6. TwoPhaseRecordOnDisk
* 7. ...
* 8. TwoPhaseRecordOnDisk (end sentinel, rmid == TWOPHASE_RM_END_ID)
* 9. CRC32
* 9. checksum (CRC-32C)
*
* Each segment except the final CRC32 is MAXALIGN'd.
* Each segment except the final checksum is MAXALIGN'd.
*/
/*
@@ -1056,11 +1056,11 @@ EndPrepare(GlobalTransaction gxact)
path)));
/* Write data to file, and calculate CRC as we pass over it */
INIT_CRC32(statefile_crc);
INIT_CRC32C(statefile_crc);
for (record = records.head; record != NULL; record = record->next)
{
COMP_CRC32(statefile_crc, record->data, record->len);
COMP_CRC32C(statefile_crc, record->data, record->len);
if ((write(fd, record->data, record->len)) != record->len)
{
CloseTransientFile(fd);
@@ -1070,7 +1070,7 @@ EndPrepare(GlobalTransaction gxact)
}
}
FIN_CRC32(statefile_crc);
FIN_CRC32C(statefile_crc);
/*
* Write a deliberately bogus CRC to the state file; this is just paranoia
@@ -1289,13 +1289,13 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
return NULL;
}
INIT_CRC32(calc_crc);
COMP_CRC32(calc_crc, buf, crc_offset);
FIN_CRC32(calc_crc);
INIT_CRC32C(calc_crc);
COMP_CRC32C(calc_crc, buf, crc_offset);
FIN_CRC32C(calc_crc);
file_crc = *((pg_crc32 *) (buf + crc_offset));
if (!EQ_CRC32(calc_crc, file_crc))
if (!EQ_CRC32C(calc_crc, file_crc))
{
pfree(buf);
return NULL;
@@ -1540,9 +1540,9 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
int fd;
/* Recompute CRC */
INIT_CRC32(statefile_crc);
COMP_CRC32(statefile_crc, content, len);
FIN_CRC32(statefile_crc);
INIT_CRC32C(statefile_crc);
COMP_CRC32C(statefile_crc, content, len);
FIN_CRC32C(statefile_crc);
TwoPhaseFilePath(path, xid);