database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-02 09:02:37 +03:00
Code Activity

Move temporary file cleanup to before_shmem_exit().

Browse Source 
As reported by a few OSX buildfarm animals there exist at least one path where
temporary files exist during AtProcExit_Files() processing. As temporary file
cleanup causes pgstat reporting, the assertions added in ee3f8d3d3a caused
failures.

This is not an OSX specific issue, we were just lucky that timing on OSX
reliably triggered the problem.  The known way to cause this is a FATAL error
during perform_base_backup() with a MANIFEST used - adding an elog(FATAL)
after InitializeBackupManifest() reliably reproduces the problem in isolation.

The problem is that the temporary file created in InitializeBackupManifest()
is not cleaned up via resource owner cleanup as WalSndResourceCleanup()
currently is only used for non-FATAL errors. That then allows to reach
AtProcExit_Files() with existing temporary files, causing the assertion
failure.

To fix this problem, move temporary file cleanup to a before_shmem_exit() hook
and add assertions ensuring that no temporary files are created before / after
temporary file management has been initialized / shut down. The cleanest way
to do so seems to be to split fd.c initialization into two, one for plain file
access and one for temporary file access.

Right now there's no need to perform further fd.c cleanup during process exit,
so I just renamed AtProcExit_Files() to BeforeShmemExit_Files(). Alternatively
we could perform another pass through the files to check that no temporary
files exist, but the added assertions seem to provide enough protection
against that.

It might turn out that the assertions added in ee3f8d3d3a will cause too much
noise - in that case we'll have to downgrade them to a WARNING, at least
temporarily.

This commit is not necessarily the best approach to address this issue, but it
should resolve the buildfarm failures. We can revise later.

Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210807190131.2bm24acbebl4wl6i@alap3.anarazel.de
This commit is contained in:
Andres Freund
2021-08-07 19:14:20 -07:00
parent 256909c6c1
commit 675c945394
3 changed files with 65 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
src/backend/utils/init/postinit.c
View File

@ -517,6 +517,12 @@ BaseInit(void)
*/
DebugFileOpen();
/*
* Initialize file access. Done early so other subsystems can access
* files.
*/
InitFileAccess();
/*
* Initialize statistics reporting. This needs to happen early to ensure
* that pgstat's shutdown callback runs after the shutdown callbacks of
@ -525,11 +531,16 @@ BaseInit(void)
*/
pgstat_initialize();
/* Do local initialization of file, storage and buffer managers */
InitFileAccess();
/* Do local initialization of storage and buffer managers */
InitSync();
smgrinit();
InitBufferPoolAccess();
/*
* Initialize temporary file access after pgstat, so that the temorary
* file shutdown hook can report temporary file statistics.
*/
InitTemporaryFileAccess();
}