The help describes the values an option allows with an allow list. An option that allows the same values as another option can inherit that list instead of describing the values again.
This only worked for a list documented by an option on its own. A list documented under a command could not be inherited, so an option that only the commands document has its values written out once per command. The output option describes the same two values under manifest, repo-ls, info, verify, and version.
Record the list each command documents as well, and let inherit name the command to take it from. The command has to be named because more than one command can document the same option and the lists need not match. The type option allows full, diff, and incr for backup but nine values for restore.
A list is still checked against the values the inheriting option or command allows, so it can only be inherited where both allow the same values.
In the test declaration set is now taken by backup and restore so that backup can describe the values and restore can inherit them.
main.c included the generated help data and passed it to cmdHelp() as a buffer, so no unit build ever compiled it and the help test had to run the generator and load the packed help from a file to have anything to render. Including it in help.c puts the data in the module that reads it and lets the test render exactly what the binary carries.
The generated file is added to the files the repository copy needs, since git does not list it, and the test command generates it along with the PostgreSQL interfaces. Those are the two files that are generated rather than committed, and a unit build needs both from the copy because it does not run the code generation the build does. The file is regenerated at the start of every run, so the test still runs against the current declarations without generating them itself.
cmdHelp() and helpRender() no longer take the data as a parameter and help.h no longer needs the buffer type. The help-data generator command is removed along with its only caller.
The values an option allows are declared in config.yaml and were written out again by hand in reference.xml as an ordinary list, so nothing held the two together. A value added, removed, or renamed in the declaration left the documentation describing something else and only a reader would find it.
An allow list is now marked as what it is and its values are checked against the declaration. Where each value needs an explanation allow-item gives it and caption says what the list is a list of, which renders the sentence that was written above each list by hand. Where the values speak for themselves the tag is left empty and the values are rendered from the declaration into the sentence around it, so they are not written out at all. Where an option allows what another option allows the list is inherited, the way the declaration inherits the values themselves, which is what log-level-file, log-level-stderr, and pg-host-type do.
Documenting the list is required rather than optional, since a value a user cannot read about is a value they cannot use. That turned up four options that never listed their values. buffer-size and repo-sftp-host-key-hash-type render the sentence they already had from the declaration, and repo-retention-archive-type and repo-retention-full-type described a value per paragraph, which is now the list it always was.
The markup is resolved where the help is parsed, so the help in the binary and the documentation are both rendered from it and no renderer knows it exists. Everything else in the documentation renders exactly as it did, except for four introductions reworded to fit the caption.
Most of these are prose in a python docstring or comment wrapped a few characters wide, which black does not reflow, and macro continuation backslashes left stranded past the margin. The rest are long calls broken after the first parenthesis, section separator comments with too many dashes, and long expectation strings split along what they hold. A few long shell commands in the workflows are wrapped with continuations.
hrnTestResultDiff() had a continuation backslash left over from when it was a macro, which spliced the closing brace onto the line above it. It is removed.
The linter that enforces the limit follows, so this commit is only the reformatting and can be ignored by blame.
The build tool in src/build becomes build/build.py with its library in build/lib, and the C part of the documentation tool in doc/src becomes doc/doc.py with its library in doc/lib. They migrate together because doc/src was built on src/build, linking its config and help parsing along with its extensions to the core types. doc.pl runs doc.py rather than building and running a C binary, and meson runs build.py for the code it generates as it builds.
The three libraries form the hierarchy the C include path did: build/lib is the base, doc/lib sits on it, and test/lib sits on both. They are implicit namespace packages so a package spans them, and a module lives in exactly one library and holds the union of what the tools need from it, since Python resolves a module name to a single file and cannot extend one the way the C included a core module and added to it. define.yaml declares which libraries each tool may import and the test runner puts only those on the path, so a test of the build library cannot reach the doc or test library any more than the build tool can.
The declarations move with the code. config.yaml, error.yaml, and postgres.yaml are now at the root of build, and help.xml becomes doc/xml/reference.xml since it is documentation that the help happens to be generated from, so the generator is told where it is rather than knowing. dist.sh moves to build as well. The documentation tool's own option declarations are gone because they existed only to generate the option parser for the C binary and doc.py parses its command line directly.
Three modules in src/build/common were never build tool code. exec, regExp, and string extend the core types for the test harness and the unit tests, which were their only users, so they move to test/src/harness. The json and xml extensions had no users outside the two tools and are removed with them, and the tests for both are trimmed to the core functions that remain.
The help unit test built its fixture by calling the generator's C functions directly, which was the last thing outside the tools that depended on them. It now runs build.py help-data to write the packed help and loads that, so it still runs against the current declarations without needing the generated C, which may not have been written when the test runs.
The manual page wrapping is a port of the wrapper the binary uses for its own help, so the manual page and pgbackrest help break lines the same way, including where a word longer than a line is left unbroken.
libyaml is no longer needed to build pgBackRest since the only thing that read yaml was the C generator. It is dropped from the development container and from every CI platform, and meson no longer looks for it. Code generation needs nothing compiled first either, so a test run no longer builds a generator before it can select the tests.
The common source list is merged into the pgbackrest source list since it existed only so the generator could be built from the same files, and the entries that were out of alphabetical order are put in order while the list is being rewritten anyway.
Every file the generator writes is byte identical to what the C wrote, for the project declarations and for the documentation's own. The documentation is checked the same way: the HTML rendered on this branch is byte identical to the HTML rendered on integration, every file of it, which exercises the whole pipeline on the real content rather than only the four files the tool writes.
Every module in both libraries is declared in define.yaml and has full line and branch coverage. The coverage list is keyed by file rather than by module name, since the same name in two languages is two files and one of them would have silently lost its requirement while both existed.
TestError becomes ToolError, since it is the error type for three tools now rather than for the harness alone.
