4 Commits

Author	SHA1	Message	Date
David Steele	cc9ff041f4	Represent the values that vary by version as interface data. ... A value that varies by version was read inside a function, so two versions got a rendering of it that differed only by that value. pgInterfaceControlVersion() returned PG_CONTROL_VERSION, pgInterfaceWalIs() compared XLOG_PAGE_MAGIC, and pgInterfaceControlIs() compared both PG_CONTROL_VERSION and CATALOG_VERSION_NO, so none of them could ever be shared no matter how little of the version they actually depended on. Include them in the interface struct instead. A version block captures each as a constant named for it, since a value only has a value inside the block and the struct is built after every block has undefined what it defined. A define cannot do this because it is expanded where it is used rather than where it is made, by which time the value it was given is gone. The constant also names the member it is assigned to, so adding a value is a change to the header and to nothing else. Each value is captured by a macro of its own, so the analysis that decides which versions can share a function decides the same for a value. One that does not vary between two versions is captured once for both and named for the range it covers, e.g. pgInterfaceControlVersion13_16, which makes what a version shares readable where it is assigned rather than something to work out from the vendored header. The control version is the same from 13 through 16 and the crc offset for 15 and 16. The functions that only compared a value go with them. Matching pg_control against an interface is now a comparison of the fields that are at the same offset in every version, which pgControlCommon and pgWalCommon already read to report a version that is not supported, and the WAL magic is compared the same way. The comparison is rendered by the generator in the form that fits the versions rendered, so the range that an unreleased version needs is only rendered while there is one and there is never a comparison that no supported version can fail. Whether a version has been released is set from the declaration, which is where it is stated, rather than derived in the header. The harness shims the interface so it writes with the types that interface declares rather than declaring them again, and takes the defaults a test does not ask for from the interface the code matches on rather than from a second copy of them. This brings the binary interface to 22 rendered functions from 62 and the harness to 12 from 32.	2026-08-12 10:16:57 +07:00
David Steele	6e7a9d19d2	Render a PostgreSQL interface function once per set of types it uses. ... Every supported version got a rendering of every interface function, which is 77 functions for the seven macros the binary interface declares and 33 for the three the harness declares. Many of them compile to identical code, since most of the vendored types do not change in a given release. The functions are trivial enough today that the duplication costs little, but more ambitious ones would multiply by the number of versions. Read the vendored header as a chain of version branches per type and define, resolve each of them for each version, and take the transitive closure of what an interface macro reaches. Two versions share a rendering when every entity in that closure resolves to the same branch for both, since that is what makes the code the compiler sees the same. Following the closure rather than the names a macro uses directly is what makes the comparison sound: ControlFileData is unchanged from 13 to 14, but CheckPoint is embedded in it and does change, so the layout differs. The oldest version of a group renders the function and the struct points the rest at it, so adding a version later never renames what is already there. A rendering more than one version shares is named for the range it covers, e.g. pgInterfaceWal14_19, and the block it is rendered in says which versions share it. This brings the binary interface to 62 functions and the harness to 32. A macro that does not take a version is now read as a helper the function macros are written with rather than as a function of its own. The dependency scan follows a helper to the types it names, so declaring one is now possible, where before it would have been rendered as a function.	2026-08-11 22:19:35 +07:00
David Steele	2b51a7f733	Generate the PostgreSQL interfaces the test harness uses. ... The harness had a file per version that did nothing but expand a macro for it, a block of declarations for the functions those files defined, and the struct that dispatched to them, all maintained by hand, so adding a PostgreSQL version meant adding a file and three more entries to keep in step. The interfaces the binary uses are generated from the same versions, so the harness generates its own the same way with 'build.py postgres-harness'. What differs between the two interfaces is a declaration: the header the macros are read from, how the generated code includes it, the prefix of the functions the macros define, and where the interface is rendered. The versions, the vendored types that are renamed for each of them, and the defines that are undefined after each one are the same, so one generator renders both. The harness interface is one translation unit with the vendored header included again for each version, the way the binary's is, rather than a translation unit per version. The functions are static, so the declarations and the pragma that silenced the missing prototypes go away with the files, and they are named for the version the way the binary's are, e.g. 19 rather than 190. The harness macros are renamed to say which function each one defines, since the generator names a function after the macro that defines it, and the macro that expanded all of them for a version is gone because the generated file expands each one itself. The header they are declared in loses its include guard and everything that does not vary by version, since it is now included once per version and the module that expands the macros includes the rest. The system id constants are generated into the harness header rather than into a file of their own, so the header still holds everything the harness declares. The defines are replaced where they are, which is what the version in src/version.h is already generated by. Every version gets the offset form that only two of them had, since a block that reads the same for every version is easier to follow than one that does not. The interface is generated rather than committed, the same as the interface the binary uses, so a test run generates it and the repository copy duplicates it for a unit build to compile.	2026-08-11 13:36:36 +07:00
David Steele	59bd2cd96c	Migrate the build and documentation code from C to Python. ... The build tool in src/build becomes build/build.py with its library in build/lib, and the C part of the documentation tool in doc/src becomes doc/doc.py with its library in doc/lib. They migrate together because doc/src was built on src/build, linking its config and help parsing along with its extensions to the core types. doc.pl runs doc.py rather than building and running a C binary, and meson runs build.py for the code it generates as it builds. The three libraries form the hierarchy the C include path did: build/lib is the base, doc/lib sits on it, and test/lib sits on both. They are implicit namespace packages so a package spans them, and a module lives in exactly one library and holds the union of what the tools need from it, since Python resolves a module name to a single file and cannot extend one the way the C included a core module and added to it. define.yaml declares which libraries each tool may import and the test runner puts only those on the path, so a test of the build library cannot reach the doc or test library any more than the build tool can. The declarations move with the code. config.yaml, error.yaml, and postgres.yaml are now at the root of build, and help.xml becomes doc/xml/reference.xml since it is documentation that the help happens to be generated from, so the generator is told where it is rather than knowing. dist.sh moves to build as well. The documentation tool's own option declarations are gone because they existed only to generate the option parser for the C binary and doc.py parses its command line directly. Three modules in src/build/common were never build tool code. exec, regExp, and string extend the core types for the test harness and the unit tests, which were their only users, so they move to test/src/harness. The json and xml extensions had no users outside the two tools and are removed with them, and the tests for both are trimmed to the core functions that remain. The help unit test built its fixture by calling the generator's C functions directly, which was the last thing outside the tools that depended on them. It now runs build.py help-data to write the packed help and loads that, so it still runs against the current declarations without needing the generated C, which may not have been written when the test runs. The manual page wrapping is a port of the wrapper the binary uses for its own help, so the manual page and pgbackrest help break lines the same way, including where a word longer than a line is left unbroken. libyaml is no longer needed to build pgBackRest since the only thing that read yaml was the C generator. It is dropped from the development container and from every CI platform, and meson no longer looks for it. Code generation needs nothing compiled first either, so a test run no longer builds a generator before it can select the tests. The common source list is merged into the pgbackrest source list since it existed only so the generator could be built from the same files, and the entries that were out of alphabetical order are put in order while the list is being rewritten anyway. Every file the generator writes is byte identical to what the C wrote, for the project declarations and for the documentation's own. The documentation is checked the same way: the HTML rendered on this branch is byte identical to the HTML rendered on integration, every file of it, which exercises the whole pipeline on the real content rather than only the four files the tool writes. Every module in both libraries is declared in define.yaml and has full line and branch coverage. The coverage list is keyed by file rather than by module name, since the same name in two languages is two files and one of them would have silently lost its requirement while both existed. TestError becomes ToolError, since it is the error type for three tools now rather than for the harness alone.	2026-07-30 19:37:43 +07:00