mirror of
https://github.com/postgres/postgres.git
synced 2025-08-24 09:27:52 +03:00
bd7c348d83
The fact that multixact truncations are not WAL logged has caused a fair share of problems. Amongst others it requires to do computations during recovery while the database is not in a consistent state, delaying truncations till checkpoints, and handling members being truncated, but offset not. We tried to put bandaids on lots of these issues over the last years, but it seems time to change course. Thus this patch introduces WAL logging for multixact truncations. This allows: 1) to perform the truncation directly during VACUUM, instead of delaying it to the checkpoint. 2) to avoid looking at the offsets SLRU for truncation during recovery, we can just use the master's values. 3) simplify a fair amount of logic to keep in memory limits straight, this has gotten much easier During the course of fixing this a bunch of additional bugs had to be fixed: 1) Data was not purged from memory the member's SLRU before deleting segments. This happened to be hard or impossible to hit due to the interlock between checkpoints and truncation. 2) find_multixact_start() relied on SimpleLruDoesPhysicalPageExist - but that doesn't work for offsets that haven't yet been flushed to disk. Add code to flush the SLRUs to fix. Not pretty, but it feels slightly safer to only make decisions based on actual on-disk state. 3) find_multixact_start() could be called concurrently with a truncation and thus fail. Via SetOffsetVacuumLimit() that could lead to a round of emergency vacuuming. The problem remains in pg_get_multixact_members(), but that's quite harmless. For now this is going to only get applied to 9.5+, leaving the issues in the older branches in place. It is quite possible that we need to backpatch at a later point though. For the case this gets backpatched we need to handle that an updated standby may be replaying WAL from a not-yet upgraded primary. We have to recognize that situation and use "old style" truncation (i.e. looking at the SLRUs) during WAL replay. In contrast to before, this now happens in the startup process, when replaying a checkpoint record, instead of the checkpointer. Doing truncation in the restartpoint is incorrect, they can happen much later than the original checkpoint, thereby leading to wraparound. To avoid "multixact_redo: unknown op code 48" errors standbys would have to be upgraded before primaries. A later patch will bump the WAL page magic, and remove the legacy truncation codepaths. Legacy truncation support is just included to make a possible future backpatch easier. Discussion: 20150621192409.GA4797@alap3.anarazel.de Reviewed-By: Robert Haas, Alvaro Herrera, Thomas Munro Backpatch: 9.5 for now
pgindent
========
This can format all PostgreSQL *.c and *.h files, but excludes *.y, and
*.l files.
1) Install pg_bsd_indent (see below for details).
2) Install entab (src/tools/entab/).
3) Change directory to the top of the build tree.
4) Remove all derived files (pgindent has trouble with one of the flex macros):
make maintainer-clean
Or:
git clean -fdx
5) Download the typedef file from the buildfarm:
wget -O src/tools/pgindent/typedefs.list http://buildfarm.postgresql.org/cgi-bin/typedefs.pl
(see http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list for a full list of typedefs,
also http://adpgtech.blogspot.com/2015/05/running-pgindent-on-non-core-code-or.html)
6) Run pgindent:
src/tools/pgindent/pgindent
7) Remove any files that generate errors and restore their original
versions.
8) Indent the Perl code:
(
find . -name \*.pl -o -name \*.pm
find . -type f -exec file {} \; |
egrep -i ':.*perl[0-9]*\>' |
cut -d: -f1
) |
sort -u |
xargs perltidy --profile=src/tools/pgindent/perltidyrc
9) Do a full test build:
> run configure
# stop is only necessary if it's going to install in a location with an
# already running server
pg_ctl stop
run configure
make -C src install
make -C contrib install
run initdb
pg_ctl start
make installcheck-world
10) Remove Perl backup files after testing (*.bak)
---------------------------------------------------------------------------
BSD indent
----------
We have standardized on NetBSD's indent, and renamed it pg_bsd_indent.
We have fixed a few bugs which requre the NetBSD source to be patched
with indent.bsd.patch patch. A fully patched version is available at
ftp://ftp.postgresql.org/pub/dev.
GNU indent, version 2.2.6, has several problems, and is not recommended.
These bugs become pretty major when you are doing >500k lines of code.
If you don't believe me, take a directory and make a copy. Run pgindent
on the copy using GNU indent, and do a diff -r. You will see what I
mean. GNU indent does some things better, but mangles too. For details,
see:
http://archives.postgresql.org/pgsql-hackers/2003-10/msg00374.php
http://archives.postgresql.org/pgsql-hackers/2011-04/msg01436.php
---------------------------------------------------------------------------
Notes about excluded files
--------------------------
src/include/storage/s_lock.h and src/include/port/atomics/ are excluded
because they contain assembly code that pgindent tends to mess up.
src/include/snowball/libstemmer/ and src/backend/snowball/libstemmer/
are excluded because those files are imported from an external project,
not maintained locally, and are machine-generated anyway.
src/interfaces/ecpg/test/expected/ is excluded to avoid breaking the ecpg
regression tests. Several *.h files are included in regression output so
should not be changed.
---------------------------------------------------------------------------
Obsolete typedef list creation instructions
-------------------------------------------
To use pgindent:
1) Build the source tree with _debug_ symbols and all possible configure options
2) Install to /usr/local/pgsql
3) Install all contrib modules
4) Save a list of typedefs by running:
src/tools/find_typedef /usr/local/pgsql/bin /usr/local/pgsql/lib > /tmp/pgtypedefs