The build tool in src/build becomes build/build.py with its library in build/lib, and the C part of the documentation tool in doc/src becomes doc/doc.py with its library in doc/lib. They migrate together because doc/src was built on src/build, linking its config and help parsing along with its extensions to the core types. doc.pl runs doc.py rather than building and running a C binary, and meson runs build.py for the code it generates as it builds.
The three libraries form the hierarchy the C include path did: build/lib is the base, doc/lib sits on it, and test/lib sits on both. They are implicit namespace packages so a package spans them, and a module lives in exactly one library and holds the union of what the tools need from it, since Python resolves a module name to a single file and cannot extend one the way the C included a core module and added to it. define.yaml declares which libraries each tool may import and the test runner puts only those on the path, so a test of the build library cannot reach the doc or test library any more than the build tool can.
The declarations move with the code. config.yaml, error.yaml, and postgres.yaml are now at the root of build, and help.xml becomes doc/xml/reference.xml since it is documentation that the help happens to be generated from, so the generator is told where it is rather than knowing. dist.sh moves to build as well. The documentation tool's own option declarations are gone because they existed only to generate the option parser for the C binary and doc.py parses its command line directly.
Three modules in src/build/common were never build tool code. exec, regExp, and string extend the core types for the test harness and the unit tests, which were their only users, so they move to test/src/harness. The json and xml extensions had no users outside the two tools and are removed with them, and the tests for both are trimmed to the core functions that remain.
The help unit test built its fixture by calling the generator's C functions directly, which was the last thing outside the tools that depended on them. It now runs build.py help-data to write the packed help and loads that, so it still runs against the current declarations without needing the generated C, which may not have been written when the test runs.
The manual page wrapping is a port of the wrapper the binary uses for its own help, so the manual page and pgbackrest help break lines the same way, including where a word longer than a line is left unbroken.
libyaml is no longer needed to build pgBackRest since the only thing that read yaml was the C generator. It is dropped from the development container and from every CI platform, and meson no longer looks for it. Code generation needs nothing compiled first either, so a test run no longer builds a generator before it can select the tests.
The common source list is merged into the pgbackrest source list since it existed only so the generator could be built from the same files, and the entries that were out of alphabetical order are put in order while the list is being rewritten anyway.
Every file the generator writes is byte identical to what the C wrote, for the project declarations and for the documentation's own. The documentation is checked the same way: the HTML rendered on this branch is byte identical to the HTML rendered on integration, every file of it, which exercises the whole pipeline on the real content rather than only the four files the tool writes.
Every module in both libraries is declared in define.yaml and has full line and branch coverage. The coverage list is keyed by file rather than by module name, since the same name in two languages is two files and one of them would have silently lost its requirement while both existed.
TestError becomes ToolError, since it is the error type for three tools now rather than for the harness alone.
test.pl becomes test.py and ci.pl becomes ci.py, so an invocation only changes from .pl to .py. Running the tests needs no command since that is what a developer does, and the unit and coverage commands that a run drives are commands of test.py because a test runs inside a container where only the repository copy is available.
The Perl library in test/lib is gone. The modules the documentation depends on move to doc/lib, with ExecuteTest becoming Execute, and are trimmed to what the documentation calls, which removes the storage manifest, info, and owner along with everything else the tests were the only user of. release.pl no longer needs the vm definitions since all it did with them was compare two vm names.
Nothing in a test container or on a CI runner runs on Perl now so none of it is installed in either place, which was a substantial part of a base image. The packaging tools go with it since they are written in Perl and nothing builds a package, and ca-certificates is installed explicitly because it was arriving as a dependency of a package that is no longer there. Every base image is rebuilt as a result.
The generated Dockerfile is otherwise unchanged. The one difference is that the PostgreSQL repository for RHEL is installed for the architecture being built rather than for the architecture of the host, which only differs when cross building.
The list of tests to run is unchanged. It was compared against the Perl across the vm, module, test, run, and PostgreSQL version options before the Perl was removed.
The vm definitions, the user lookup, the test list, the container build, the job, the test run itself, and the code format are covered by tests, which the Perl equivalents never were. The container build is checked by comparing the generated script in full, since the hash of that script is the cache key.
Make the code format a command rather than two options of a test run, and fold in the black check that CI has been running as a step of its own. It gets the files to format from git rather than from the repository copy, so it no longer depends on a test run having made that copy and is told nothing about where either path is.
Wire up --log-level-test, which the driver has been collecting and discarding. Remove --psql-bin, --gen-check, --no-coverage-report, --build-max, --log-level-test-file, --no-optimize, --no-debug, and --debug-test-trace, none of which have done anything for some time.
Allow --vm=all only for --vm-build. Building every container at once works but running the tests on every vm never has, since a test needs the binary and the build for the vm it runs on and there is one of each.
The C harness in test/src/command/test is replaced by test/test.py with its library in test/lib, and test.pl runs that per test rather than building and running a C binary. Python is already required by meson so the only new dependency is PyYAML, and the harness no longer needs to be compiled before any test can run.
The harness is now tested the same way the rest of the code is. Test modules live in test/src/module/test and are declared in define.yaml with lang: python, so they run through the existing test.pl --module and --test interface. A python test is a plain function rather than a run inside testBegin(), since these tests are fast enough that selecting a single run is not useful, and --test-name selects one function when debugging.
Coverage from both languages merges into a single report, so a run that covered C and python produces one result. Full line and branch coverage is required and the uncovered and uncoverable annotations work as they do for C. Python reports the lines and arcs it knows about rather than everything a module contains, and which arcs a run reports depends on which ones it took, so python coverage merges by line number and by where an arc goes rather than by position.
A python test may import only the library modules it declares, which is what the C link step enforced by compiling in only what was declared. An import that is already in sys.modules never reaches an import hook, so the test module is loaded in its own interpreter with the hook installed ahead of the library.
Fix coverage for a shimmed .c.inc module, which has been missing from the report rather than reported as uncovered. The build writes the shimmed copy to the include path so it resolves ahead of the repository copy, which puts it outside src, and the filter only matched the path such a module is normally compiled from. In practice this is command/backup/process.inc, the only shimmed .c.inc.
Install PyYAML wherever the harness runs and a coverage module wherever the python tests run. rh8 installs python3.12 since PyYAML is not packaged for the platform python there, u24 and rh10 package a coverage new enough to report branch detail per line, and rh9 installs one with pip since the packaged version predates it. The jobs that run outside a container install PyYAML as each platform allows, with pip on macOS since homebrew has no formula for it.
Check the format of the harness and its tests with black, which previously covered only the smoke test.
The RHEL documentation builds have been slow and intermittently failing. The mirrors are not down: three consecutive full metadata refreshes from the same host measured 73, 124, and 157 seconds with a full mirror list and no dead mirrors, so the variance is mirror throughput rather than availability. Every documentation container pays this cost because Rocky sets metadata_expire to six hours and the cached base image is normally older than that by the time it is used.
Tune dnf to fail over from a slow mirror. By default a mirror is only abandoned when it delivers less than 1000 bytes per second for 30 seconds, so a mirror that is crawling rather than failing is retried instead of dropped. Raise the rate floor to 50k, shorten the timeout, sort mirrors by latency, and fetch in parallel.
Disable repositories the documentation does not use so their metadata is not refreshed in every container. The epel-release package adds an openh264 repository and the PGDG repository enables one repository per PostgreSQL version, of which only the documented version and the upgrade version are needed. This takes the enabled repository count from twelve to eight. The enable list is driven from the existing version variables so it follows a version bump.
After both changes the same three-run measurement was 81, 53, and 51 seconds, though the mirror variance being addressed also makes that comparison noisy.
The container revision is left alone since the cache key hashes the generated Dockerfile and these edits invalidate it on their own.
Add a u24 vm and split the PostgreSQL versions between it and u22 so the integration tests run as two jobs instead of one. u22 tests 9.6 and 11 and no longer installs anything above 14, while u24 installs the full range and tests 12, 13, and 19. The unit test jobs (coverage/backtrace and valgrind), the coverage summary generated for a release, and the Debian/Ubuntu documentation build all move to u24, so the PostgreSQL 19 beta repository is now enabled there rather than on u22. The manual container revision for the beta is dropped since the u24 base image is new and has no cached tag to invalidate.
Build the contributing guide development environment and the user guide host from ubuntu:24.04 rather than ubuntu:22.04.
Remove the guard that skipped PostgreSQL 18 unless the container was being built for a particular vm and architecture. It dates from when 18 was in beta and not built for ppc64le and s390x, and those architectures no longer reach it since they install a single version via the postgresql metapackage.
Drop python3-distutils, which no longer exists on Ubuntu 24.04. Nothing here uses distutils and on the older distributions it is pulled in as a meson dependency anyway.
Install the AWS CLI from the vendor installer on the Debian/Ubuntu documentation host since awscli is no longer packaged for Ubuntu 24.04. This matches what the RHEL host already does with awscli2 and means ca_bundle must now be set for both, since the vendored CLI ships its own certificate bundle instead of using the system store.
Leave the arch workflow and the code format check on u22 and ubuntu-22.04. Ubuntu 22.04 is still supported and uncrustify formats differently on 24.04, which is a separate change.
Banner comments become docstrings with a separator before each top level definition and each method, matching the rest of Python and making a description reachable by an editor rather than only by reading the file.
Two of the banners introduced a group of helpers rather than the function that followed, so those titles are dropped and each helper describes itself. A few functions had no description at all and were given one.
YAML mappings are unordered, but shim key order determined the order the modules were included in the harness, e.g. the fd harness includes common/io/fd before common/io/fdWrite. A sequence makes that order explicit instead of relying on parser behavior.
A shim entry is now either a bare module name or a map with a name and a function list, matching how harness entries are already written, and the null placeholder the map form needed is gone.
The harness integration flag is also applied to its shims after the harness has been parsed, so it no longer depends on integration appearing before shim.
The restore file module is only used internally by the restore protocol handler so move it into a file.c.inc that is included directly into protocol.c, matching the approach used for the other restore modules. This makes restoreFile() static and avoids maintaining an extra header and build entry.
Move the RestoreResult enum to protocol.h since it is still needed by process.c.inc and the restore tests but file.h no longer exists to provide it.
backupFile() is only called by the backup protocol handler so include file.c into protocol.c directly, matching how the rest of the backup module is included into backup.c. This makes backupFile() static and removes the file.h header and build entry. The BackupFile/BackupFileResult structs are only used by backupFile() and the protocol handler so they become file-private as well.
The BackupCopyResult enum is still needed by the backup process to interpret protocol job results so move it to protocol.h.
The backup module has grown large so split it into .c.inc files that are included directly into backup.c, matching the approach used for the restore module. These functions are only used internally by backup so this avoids maintaining extra header and build files and keeps them static.
Since backupProcess() is shimmed for testing and now lives in an included module, the shim uses the inc option to locate and shim it there.
The shim mechanism renames a function in the shim module's C file so the harness can supply an alternate implementation for testing, which required the shimmed function to be defined in the shim module itself. Now that large modules are being split into included .c.inc files a function to be shimmed may be defined outside the shim module.
Add an inc option to a shim function that names the included module where the function is defined. That module is shimmed in addition to the shim module and written to the include path so the shim module finds the shimmed copy (via the unit include directory) before the repo copy.
The Ceph RGW host was slow and unreliable to start, frequently returning connection failures and requiring long startup waits during documentation builds and integration tests.
SeaweedFS starts in a few seconds, ships a much smaller multi-arch image, and supports S3 object versioning, which the repository target time feature requires. The image is pinned to 4.40 to include recent versioned-listing fixes.
A single weed server command runs the master, volume, filer, and S3 gateway with TLS on 443, host-style addressing via s3.domainName, and the demo credentials in a mounted identity config, so no bootstrap script is needed. The entrypoint is overridden to run weed as root because the default drops to an unprivileged user that cannot bind 443.
After a stanza delete on the versioned repository, SeaweedFS keeps listing the now-empty stanza prefix, so info reports missing stanza data rather than missing stanza path; the user guide output is updated to match.
The documentation build builds a base host image per distribution before running the commands the docs contain, installing PostgreSQL, the Azure and AWS CLIs, and other packages from scratch on every run. On a fresh CI runner nothing is cached, so this rebuild made the doc builds the slowest jobs in CI.
Cache each base image in a new ghcr.io/pgbackrest/doc registry package, keyed by a hash of the generated Dockerfile combined with a manual revision on the host-define. A run pulls the image when the key matches and only builds and pushes it on a miss, the same approach the test containers already use; forks pull from the pgbackrest cache and push to their own.
The revision is set date-style (e.g. 20260721A) like the test container revisions in test/container.yaml and forces a rebuild when the Dockerfile is unchanged but the upstream packages have changed, e.g. a new PostgreSQL minor.
Factor the cache logic into pgBackRestDoc::Common::DocContainer to keep doc.pl tidy, grant the doc CI job packages:write and log it in to GHCR so a build can be pushed, and prune the doc package on the same schedule as the other caches.
During restore, files bundled together in the repository are read back one file or range at a time, and each new read from an object store pays the latency of establishing a request. The restore copy loop worked around this by manually coalescing adjacent bundled files into a single read, but it could neither hide request latency nor skip over unneeded data within a bundle.
Add StorageReadMulti, which is given the full list of files and ranges up front and reads them back in order. Requests are submitted ahead of time (prefetch) so the latency of the next request overlaps reading the current one, and ranges separated by a small gap (e.g. a file that already exists on the host) are merged into a single request by skipping over the gap ("read over") rather than closing and reopening at a new offset, which is faster and cheaper than a new request for most storage classes.
To support prefetch, the S3, Azure, and GCS read drivers gain an async open. The HTTP request is submitted when the read is opened and the response is deferred until the first read via a new optional openAsync driver function, so multiple requests can be in flight at once. storageReadNew() gains an async flag; the async open is completed lazily on first read and is never combined with ignore missing.
The default StorageReadMulti driver queues reads directly on the storage. When the repository is remote a dedicated remote driver is used instead so the entire request list is forwarded to the remote in a single open and prefetch and read over run there, closer to the storage. The remote read and remote read-multi share stream state and chunk handling.
Two options control the behavior: repo-storage-prefetch (default 3, range 0-1024) sets how many requests to queue ahead, and repo-storage-read-over (default 64KiB, range 0B-1MiB) sets the largest gap to read over rather than issue a new request. The number of open requests per read is one greater than prefetch since the initial read cannot be prefetched.
Rewrite the restore copy loop to add each file that will be copied to a StorageReadMulti and read through it, removing the manual offset and limit tracking across bundled files.
Only same-page section links were verified, so a page/section link with a bad section or a missing leading slash would silently render a broken anchor.
Require a link section to begin with a slash and, after all pages have been rendered, verify that each page/section link points at a section that exists on the target page. Links to a page that is not part of the build (e.g. a partial build) are skipped.
Also fix the news link to the 2.59.0 release notes, which was missing the leading slash and pointed at a broken anchor.
The distribution README (DIST.md) duplicated the Debian build dependency list from the user guide, so the two could drift. Move it into a shared variable used by both, drop python3-distutils, and link DIST.md to the user guide build section.
The variable is split with a backslash continuation so each line stays within the command line length limit. The doc executor now keeps an existing continuation rather than adding its own.
doc/DIST.md, the README shipped in the distribution tarball, was maintained by hand. Source it from doc/xml/distribution.xml and render it with the doc tool as is done for the other generated markdown, and gitignore the output like NEWS.md.
release.pl --dist renders DIST.md so meson dist can copy it into the tarball as README.md.
List items were separated by a blank line, which reads poorly when the markdown is viewed as plain text.
Regenerate README.md, CODING.md, and CONTRIBUTING.md.
The paragraph following a code block was rendered directly against the closing fence, which reads poorly when the markdown is viewed as plain text.
Regenerate CODING.md and CONTRIBUTING.md.
This command occasionally failed even though the preceding bucket creation had succeeded, presumably because the new bucket was not yet available. Retry for up to 30 seconds to give it time to appear.
Also lower the bucket creation retry from 60 to 30 seconds since it normally completes well within that.
Meson 0.55 and later name the unit test private build directory test-unit.p rather than test-unit@exe, so the harness no longer needs to check for both when clearing old coverage data.
All supported platforms ship a newer meson (RHEL 8, the oldest, has 0.58.2), so raise the minimum to 0.55.3 and always use test-unit.p.
Debian 11 (bullseye) reaches end of life before the next release, so replace it with Debian 12 (bookworm) in the distribution build matrix.
Debian 12 ships a newer meson that configures the Python installation without setuptools, so remove the debian:11-only setuptools workaround.
MEM_CONTEXT_PRIOR_END() and MEM_CONTEXT_OBJ_END() are both defined as MEM_CONTEXT_END(), so closing a MEM_CONTEXT_PRIOR_BEGIN() or MEM_CONTEXT_OBJ_BEGIN() block with the base MEM_CONTEXT_END() expands identically and cannot be detected by the compiler. Eight sites had drifted to the base closer: four in info.c, one in io/session.c, and one each in the s3, azure, and gcs storage drivers.
The headers document the matching END as the intended pairing, so use it. Preprocessed output was verified byte-identical before and after, so the change is a no-op.
These were found while prototyping a lint check for block-macro pairing.
The test and test-ccache packages grow without bound: test gains a new tag every time a base image Dockerfile changes, and test-ccache orphans the previous digest every time a per-architecture tag is overwritten. Both are pure caches that are rebuilt on demand, so add a weekly workflow to prune the stale entries, with a dry-run input for manual runs.
The test package keys base images by a hash of the Dockerfile, so a change abandons the old tag while leaving it tagged and out of reach of an untagged-only sweep. Age these out by deleting every tag older than two months; anything still in use is simply rebuilt and re-pushed on the next run. The test-ccache package overwrites its per-architecture tags on every save, leaving the previous digest untagged and referenced by nothing, so delete all untagged versions there.
The owner defaults to the account the workflow runs as, so a fork prunes its own caches rather than the canonical pgbackrest ones, mirroring where each environment pushes.
NOTE TO PACKAGERS: A new distribution tarball is available which simplifies the build process by providing pregenerated HTML documentation, man page, and code. The tarball is attached to each release as an asset and is named pgbackrest-{version}.tar.gz. See the README.md in the tarball for details. Please use the new distribution tarball to avoid the additional build tooling required to generate code in future releases.
NOTE TO PACKAGERS: There is a new optional dependency on libsystemd.
IMPORTANT NOTE: Starting with this release, only the restore command may be run as root by default. Use allow-root to run other commands as root (though this is not recommended).
Bug Fixes:
* Fix expire removing a backup in progress on another host. (Reviewed by Douglas J Hunley. Reported by Dmitrii.)
* Fix archive-push-queue-max not enforced when a WAL file errors. (Reviewed by Stefan Fercot. Reported by vut12.)
* Fix async archive-get failure during recovery across a timeline switch. (Reviewed by Douglas J Hunley. Reported by Jimmy Yih.)
* Fix potential buffer overrun in error module. (Fixed by Christophe Pettus. Reviewed by David Steele.)
* Fix use-after-free of parallel protocol client. (Reviewed by Douglas J Hunley. Reported by Georgy Shelkovy.)
* Fix heap overflow when a pg option is set without pg1. (Reviewed by Douglas J Hunley. Reported by Naveen.)
Features:
* PostgreSQL 19beta2 support. (Reviewed by Stefan Fercot.)
* Add archive-expire-before option to clean up WAL archive. (Contributed by Stefan Fercot. Reviewed by David Steele.)
* Add batch delete for Azure storage. (Reviewed by Douglas J Hunley, Crispy.)
* Add support for S3 Outposts. (Contributed by Shiva Kumar Ambigi. Reviewed by David Steele, Roberto Mello.)
* Add S3 process authentication. (Reviewed by Andrew Charlton. Suggested by Andrew Charlton.)
Improvements:
* Reconnect SFTP storage after the server drops an idle connection. (Reviewed by Stefan Fercot. Suggested by mustafa0x, tisserat.)
* Add user/group caching for faster manifest build. (Contributed by Gunnar Lindholm. Reviewed by David Steele.)
* Add per-repo backup progress to info command output. (Contributed by Will Morland. Reviewed by David Steele, Stefan Fercot.)
* Add backup.info checks to verify command. (Contributed by Denis Garsh. Reviewed by David Steele, Douglas J Hunley.)
* Allow the S3 STS endpoint to be configured. (Contributed by Simon Gratton. Reviewed by David Steele.)
* Add archive-push-batch-size to limit WAL pushed per asynchronous run. (Reviewed by Stefan Fercot.)
* Exit async archive-push on first error. (Reviewed by Stefan Fercot. Suggested by Lardière Sébastien.)
* Harden HTTP chunked response parsing. (Contributed by Shubham. Reviewed by David Steele.)
* Error when running as root unless allow-root is enabled. (Reviewed by Douglas J Hunley.)
* Fewer binary searches when building manifest. (Contributed by Gunnar Lindholm. Reviewed by David Steele.)
* Improve seek performance during block incremental delta restore. (Reviewed by David Christensen, Douglas J Hunley.)
* Bundle backup files in order rather than scanning for a better fit. (Reviewed by Stefan Fercot.)
* Report the underlying error when a query fails to complete. (Reviewed by Andrew Pogrebnoi. Suggested by Marco Fontana.)
* Report the actual error when an S3 credential request fails. (Reviewed by Douglas J Hunley. Suggested by Mitchell Grice.)
* Report archive-push spool path errors to the PostgreSQL log. (Reviewed by Douglas J Hunley. Suggested by Don Seiler.)
* Hint that pgBackRest may be out of date when a version is unsupported.
* Add systemd notify integration. (Contributed by Andrew Jackson. Reviewed by David Steele.)
* Suppress unused parameter errors in meson compiler probes. (Contributed by Jörg Plate. Reviewed by David Steele.)
* C11 is now the minimum C standard. (Reviewed by Mohammad Ali Nazir Kosar.)
Documentation Improvements:
* Make stanza option internal for the repo-* commands. (Reviewed by Stefan Fercot.)
* Document that info reads encryption settings from global section. (Reviewed by Stefan Fercot. Suggested by Ron Johnson.)
* Document that expire-auto uses the backup command configuration. (Reviewed by Stefan Fercot. Suggested by Lardière Sébastien.)
* Document adjusting the logrotate su directive for a dedicated user. (Reviewed by Douglas J Hunley. Suggested by lkanbus.)
* Document that end-of-line comments are not supported in config files. (Reviewed by Douglas J Hunley. Suggested by Alex Richman.)
Test Suite Improvements:
* Fix Alpine group conflicts in CI containers. (Contributed by Artur Zakirov. Reviewed by David Steele.)
A markdown link that specified both a page and a section failed to render when the target page was itself rendered as markdown, because markdown cannot reliably anchor into another generated file.
Resolve these links to the website URL with the section anchor, matching how links to pages that are not rendered as markdown already behave, so a markdown document can link to a section of another.
The site now serves over HTTPS at the apex domain, so switch from http://www.pgbackrest.org to https://pgbackrest.org.
Change the backrest-url-base variable that generates the documentation links and the URL hardcoded in the man page SEE ALSO section, then regenerate README.md and CONTRIBUTING.md and update the doc build test to match.
NEWS.md is the release announcement copied to a postgresql.org news item. It was maintained by hand for each release, but that content now lives in news.xml and is rendered to NEWS.md by the documentation build.
Remove the committed file and add it to .gitignore so it is treated as generated output and regenerated when an announcement is needed.
The distribution tarball ships the generated source and the rendered documentation so it builds without the code generation or documentation tooling that a git checkout requires. Add a workflow that exercises this contract: build the tarball, then on a range of distributions build and smoke test it in a fresh container that has only the required packages, which is both a test and a record of what a packager needs on each.
The tarball is built once, since it is distribution-independent by design, and uploaded as an artifact; a matrix job then consumes it per distribution and drives the built binary through a full backup, restore, and verify cycle against that distribution's native PostgreSQL. The containers are run with docker directly rather than as job containers because the runner's glibc Node.js cannot execute the checkout and artifact actions inside a musl container, which keeps one path for glibc and musl alike.
Debian, Ubuntu, Rocky, Fedora, and Alpine are covered, and the tarball is built with -Dwerror so a compiler warning on any of them fails the build. A separate check confirms the tarball ships exactly the expected generated and rendered files and that none of them is empty, catching a broken code generation or documentation render before it is uploaded.
Also remove the smoke job from the misc workflow, since building and smoke testing the tarball now covers the check a packager would run, and move its format check for smoke.py, the only Python file, into the code-format job with the other formatting checks.
The bsd test group cannot run the integration tests because there are no containers available, but the smoke test drives a backup/restore cycle against each installed PostgreSQL and so provides basic coverage on these platforms. Teach the smoke test the Homebrew and FreeBSD bin paths so it can find the installations there.
Discover the PostgreSQL versions to test at install time rather than naming them so the tests do not break when a version becomes unavailable. Read the supported versions from the generated header, intersect them with what the platform offers, and test the oldest and newest of the result.
On MacOS only keg-only formulae that are neither deprecated nor disabled are considered since the binaries in a formula that is not keg-only load libpq from the linked prefix and fail unless the formula is linked.
On FreeBSD the ports all install into the same prefix and conflict, so only one version can be installed at a time and the oldest goes on the older OS while the newest goes on the newer OS. Install the versioned client and server packages instead of postgresql-libpqxx, which depends on the default PostgreSQL version.
Now that a persistent ccache has made compiling under emulation affordable, broaden the tests run there to better check that checksums are correct on big-endian. Add the command restore, archive-get, archive-push, and verify tests alongside backup, and the common type-pack, crypto, and compress tests, which exercise the serialization, checksum, and compression code that a byte-order bug would break.
Also add a smoke test that drives a real PostgreSQL through a full backup, restore, and verify cycle, checking checksums end-to-end rather than against synthetic harness data. This needs a PostgreSQL server, so install a single version in the base image for these architectures; the full version matrix is skipped to keep the emulated build fast.
Consolidate the per-vm and per-vm-index ccache directories into a single cache shared by the host and every container, passed with CCACHE_DIR instead of mounted at ~/.ccache. A file compiled in one place is then a hit in the others, e.g. test-pgbackrest is built both on the host and in the build container, and since ccache is safe for concurrent access there is no need to split the cache. It stays in the test path so --clean removes it.
Store and restore that cache in GHCR via a new composite action. Unlike actions/cache a registry is not branch-scoped, so a short-lived branch starts warm from the last run on the fork rather than cold, which matters most for the emulated architecture jobs where compiling is slow. The action pulls the fork's own cache first and falls back to the canonical pgbackrest cache for a fork that has not run yet; the push is best effort so pull requests from forks that cannot write just use what they built.
The smoke test now reads the ELF header of the pgbackrest binary being tested and prints its machine architecture and endianness, e.g. "s390x (64-bit big-endian)", right after the user it runs as. This makes it easy to confirm which architecture is actually being exercised, which matters when a binary built for a foreign architecture is run under emulation.
test.yml had grown large enough to be awkward to navigate, but splitting it into independent workflows would have produced a separate run in the Actions UI for each file and required the push/pull_request trigger to be duplicated in every one.
Instead test.yml becomes a dispatcher that calls reusable workflows, so it remains a readable index of what runs while still producing a single run. Only test.yml has a trigger and the called workflows are all on: workflow_call. Permissions are granted in the dispatcher since a called workflow can only maintain or reduce the permissions of the token it is given, never elevate them.
The jobs are grouped by platform into linux, arch, and bsd, with doc for the documentation builds and misc for the checks that do not fit any of them.
The documentation builds are worth separating because they have little in common with the tests they were mixed in with. They do not use the test containers, so they need no GHCR login or packages: write permission to maintain the container cache, and they generate no coverage, so there is no report to upload on failure. The build-max parameter was already ignored for doc since ci.pl only passes parameters through on the test path, so it is not carried over.
Job names gain the calling job as a prefix, e.g. linux / test, which is the cost of getting a single run.
Also move symbol.out into a data directory since it is not a workflow and GitHub only scans the top level of .github/workflows.
These instructions are long out of date and it does not seem worth updating them since packagers should now be using the pre-generated documentation in the distribution tarball introduced in 8fcfef5d.
The user guide build section demonstrates building from the distribution tarball but the download command still points at the GitHub source archive url for the release tag. That tarball is just the repository contents, so it has neither the generated files nor the dist marker, and building it requires the code generation tooling that the documented dependencies do not include. It also unpacks to pgbackrest-release-<version> rather than the pgbackrest-<version> path that build-br-path has used since the distribution tarball was introduced.
Point github-url-release at the distribution tarball attached to the release instead. The release tag contains a slash, e.g. release/2.59.0, which must be percent-encoded as %2F in the asset download path.
The url renders on a single line of 110 characters, which is why the download command carries a cmd-line-len override.
This was missed in 8fcfef5d.
The documentation build errors when a rendered command has a line longer than cmd-line-len, which is configured per document and defaults to 80. The check keeps commands readable in the rendered output, but a command containing a single token longer than the limit cannot satisfy it no matter where the line is broken, e.g. a url.
Add a cmd-line-len parameter to execute that overrides the document setting for a single command so the limit continues to apply to every other command.
curl is already used elsewhere in the documentation and is preinstalled on the container images, while wget is not installed on any of them by default. Standardize on curl.
Update the systemctl replacement and release tarball downloads in the user guide to use curl -fsSL. This is equivalent to the wget flags it replaces, except that -f makes curl fail on an HTTP error rather than piping the error page into tar.
Remove wget from the RHEL documentation container instead of replacing it with curl, since curl is already provided by curl-minimal. Installing the curl package there would fail because it conflicts with curl-minimal.
Remove wget from the test containers entirely. Nothing has invoked it since the lcov, nodejs, pip, minio, and s3 server downloads that needed it were removed. The PGDG repository keys it once fetched are now retrieved by rpm/dnf directly on RHEL, and on Debian the key is shipped with postgresql-common.
Move to the last release of the 1.5 branch to pick up a year of fixes.
The 1.7 branch is where upstream development continues, but it lowered the default log level from ERROR to NOTE, which surfaces a warning about postgresql-{[pg-version]}.service using /bin/kill for ExecReload. The warning is not new, it was simply below the log level before, and 1.7 offers no way to suppress it since --no-warn is accepted but ignored and -c is applied after the log level is set.
The runner image comes with a large amount of preinstalled software that cannot be controlled, which makes it a poor stand-in for the environment a packager builds in. Run the job in a minimal ubuntu:24.04 container instead so the smoke test runs with nothing but the build environment that the job explicitly installs.
Install the build environment before checkout since the container does not include git, which checkout requires in order to clone rather than download the code via the GitHub API. Also add build-essential for the compiler and ca-certificates/curl/gnupg/lsb-release for the PGDG script, all of which were previously provided by the runner image.
Since the container runs as root, the smoke test now drops to the postgres user, which exercises a path that was not covered when the job ran as the runner user.
The GitHub runner image adds taps such as aws/tap but never trusts them, so Homebrew 6.0 tap trust emits a warning on every brew command that auto-updates. Untap everything except the official Homebrew taps since no formula used in test comes from a third-party tap.
Uninstall the packages provided by a tap before untapping it. The runner image installs packer from hashicorp/tap and bicep from azure/bicep, and untapping a tap that still has packages installed warns just as the trust check does.
The run parameter of cross-platform-actions/action is deprecated in favor of starting the VM in one step and running commands in subsequent steps with the cpa.sh custom shell.
Bump the u22 revision on x86_64 and aarch64 so the base images are rebuilt and pick up the new beta from apt.postgresql.org. Key the revision by architecture rather than by VM so the ppc64le and s390x images, which do not install PostgreSQL 19, are not rebuilt under emulation for no reason.
Since there are no code changes beta1 continues to work.
Both tests only report the major version of PostgreSQL, which is not enough to identify the build. The integration test takes the major version from the test matrix and the smoke test parses it out of pg_ctl during discovery, so neither shows the minor version. This matters most during the beta period, where a rebuild of a container or a package update can move the installed version from one beta to the next without anything in the test output changing.
Log the output of pg_ctl --version in both tests, e.g. "pg_ctl (PostgreSQL) 19beta2". The integration test logs it after the hosts are built, so the version is reported even when cluster creation fails. The smoke test logs it under the header for each version it tests.
Use pg_ctl rather than postgres since the tests already require pg_ctl to run a cluster, so no error handling is needed when it cannot be found. The postmaster symlink is not an option because it was removed in PostgreSQL 16.
A base image is built per VM and architecture but a revision could only be keyed by VM, so bumping one rebuilt every architecture. This is expensive for u22, which is built for x86_64, aarch64, ppc64le, and s390x, since the last two are built under emulation in CI and are very slow. A new PostgreSQL beta packaged for only some architectures does not change the others, so those rebuilds just reproduce identical images under a new tag.
Allow a revision to be keyed by VM and architecture, e.g. u22-x86_64. A container uses the revision for its VM and architecture if it has one, else the revision for its VM, else the required "all" revision. Keying by VM alone is unchanged and still rebuilds every architecture.
Validate the revision keys. Only the presence of "all" was checked before, so a key with a misspelled VM or architecture, or with no value at all, silently fell back to a less specific revision and the expected rebuild never happened. Add VM_ARCH_PPC64LE, VM_ARCH_S390X, and a list of valid architectures to support the check, since these architectures were previously named only in the CI workflow.
The test suite is too complex and time-consuming for packagers to run, so provide a basic smoke test that exercises major functionality to show that a build works. test/smoke.py searches the paths used by the test harness for each PostgreSQL installation and runs the built pgbackrest through a full cycle for every supported version found: initdb, stanza-create, check, full backup, incremental backup, restore, and verification that the restored data matches. Each version runs in a temporary directory with its own repository and a private unix socket, so a running cluster is not disturbed.
Run the test with meson test --suite smoke. It uses only the Python standard library since meson already requires Python. The test fails rather than skips when there is no PostgreSQL to test against, since a test that silently does nothing does not verify the build. PostgreSQL will not run as root, so running as root drops to the postgres user and fails when that user does not exist.
The supported versions are read from the generated src/postgres/version.auto.h so that versions pgBackRest does not support are skipped rather than failing. The header is used rather than src/build/postgres/postgres.yaml because the latter is not shipped in the distribution.
The test is defined in test/meson.build rather than the root meson.build because unit test builds comment out all subdirs, which excludes the smoke test from them. .gitattributes ships test/smoke.py and test/meson.build in the distribution tarball, and DIST.md and the user guide document how to run it.
Add a CI job that builds on Ubuntu 24.04 and runs the smoke test against every supported version, installed from PGDG with a component added for versions that are not released yet. The version list comes from postgres.yaml so it does not need to be maintained in the workflow. The job also checks that smoke.py is formatted with black.
The message read "expected mode for '0644' for 'src/file.c' but found '0755'", where the extra "for" made the expected mode read as a file name. Drop it so the message reads "expected mode '0644' for 'src/file.c' but found '0755'".
On failure the coverage report was written to the job log with cat, which meant copying the HTML out of the log and into a file before it could be viewed. Upload it as an artifact instead so it can be downloaded directly from a failed run.
The upload ignores a missing report (most matrix jobs run with no coverage) and skips the zip archive since the report is a single HTML file.
meson dist produces the tarball via src/build/dist.sh, which regenerates help.auto.c.inc and interface.auto.c.inc, copies the man page and HTML into doc, and writes a dist marker to the root. When the marker is present the build uses the shipped files and skips build-code, libyaml, and the doc and test tools, so the tarball compiles and installs without the code generation or documentation tooling.
.gitattributes export-ignore rules limit the tarball to what is needed to build: src (except the src/build code generation sources), meson.build, meson_options.txt, and LICENSE, plus the generated files, docs, marker, and a README copied from doc/DIST.md.
RELEASE.md documents building the tarball, attaching it to a draft release, verifying it, and publishing. The user guides demonstrate building from the tarball.
