database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-11-28 11:44:57 +03:00
Code Activity

Avoid creating archive status ".ready" files too early

Browse Source 
WAL records may span multiple segments, but XLogWrite() does not
wait for the entire record to be written out to disk before
creating archive status files.  Instead, as soon as the last WAL page of
the segment is written, the archive status file is created, and the
archiver may process it.  If PostgreSQL crashes before it is able to
write and flush the rest of the record (in the next WAL segment), the
wrong version of the first segment file lingers in the archive, which
causes operations such as point-in-time restores to fail.

To fix this, keep track of records that span across segments and ensure
that segments are only marked ready-for-archival once such records have
been completely written to disk.

This has always been wrong, so backpatch all the way back.

Author: Nathan Bossart <bossartn@amazon.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Ryo Matsumura <matsumura.ryo@fujitsu.com>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/CBDDFA01-6E40-46BB-9F98-9340F4379505@amazon.com
This commit is contained in:
Alvaro Herrera
2021-08-23 15:50:35 -04:00
parent f7bda63a48
commit 515e3d84a0
8 changed files with 234 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
src/backend/access/transam/xlogarchive.c
View File

@@ -433,7 +433,7 @@ KeepFileRestoredFromArchive(const char *path, const char *xlogfname)
if (XLogArchiveMode != ARCHIVE_MODE_ALWAYS)
XLogArchiveForceDone(xlogfname);
else
XLogArchiveNotify(xlogfname);
XLogArchiveNotify(xlogfname, true);
/*
* If the existing file was replaced, since walsenders might have it open,
@@ -462,9 +462,12 @@ KeepFileRestoredFromArchive(const char *path, const char *xlogfname)
* by the archiver, e.g. we write 0000000100000001000000C6.ready
* and the archiver then knows to archive XLOGDIR/0000000100000001000000C6,
* then when complete, rename it to 0000000100000001000000C6.done
*
* Optionally, nudge the archiver process so that it'll notice the file we
* create.
*/
void
XLogArchiveNotify(const char *xlog)
XLogArchiveNotify(const char *xlog, bool nudge)
{
char archiveStatusPath[MAXPGPATH];
FILE *fd;
@@ -489,8 +492,8 @@ XLogArchiveNotify(const char *xlog)
return;
}
/* Notify archiver that it's got something to do */
if (IsUnderPostmaster)
/* If caller requested, let archiver know it's got work to do */
if (nudge)
PgArchWakeup();
}
@@ -498,12 +501,12 @@ XLogArchiveNotify(const char *xlog)
* Convenience routine to notify using segment number representation of filename
*/
void
XLogArchiveNotifySeg(XLogSegNo segno)
XLogArchiveNotifySeg(XLogSegNo segno, bool nudge)
{
char xlog[MAXFNAMELEN];
XLogFileName(xlog, ThisTimeLineID, segno, wal_segment_size);
XLogArchiveNotify(xlog);
XLogArchiveNotify(xlog, nudge);
}
/*
@@ -608,7 +611,7 @@ XLogArchiveCheckDone(const char *xlog)
return true;
/* Retry creation of the .ready file */
XLogArchiveNotify(xlog);
XLogArchiveNotify(xlog, true);
return false;
}