You've already forked pgbackrest
mirror of
https://github.com/pgbackrest/pgbackrest.git
synced 2026-10-23 21:28:56 +03:00
59bd2cd96c
The build tool in src/build becomes build/build.py with its library in build/lib, and the C part of the documentation tool in doc/src becomes doc/doc.py with its library in doc/lib. They migrate together because doc/src was built on src/build, linking its config and help parsing along with its extensions to the core types. doc.pl runs doc.py rather than building and running a C binary, and meson runs build.py for the code it generates as it builds. The three libraries form the hierarchy the C include path did: build/lib is the base, doc/lib sits on it, and test/lib sits on both. They are implicit namespace packages so a package spans them, and a module lives in exactly one library and holds the union of what the tools need from it, since Python resolves a module name to a single file and cannot extend one the way the C included a core module and added to it. define.yaml declares which libraries each tool may import and the test runner puts only those on the path, so a test of the build library cannot reach the doc or test library any more than the build tool can. The declarations move with the code. config.yaml, error.yaml, and postgres.yaml are now at the root of build, and help.xml becomes doc/xml/reference.xml since it is documentation that the help happens to be generated from, so the generator is told where it is rather than knowing. dist.sh moves to build as well. The documentation tool's own option declarations are gone because they existed only to generate the option parser for the C binary and doc.py parses its command line directly. Three modules in src/build/common were never build tool code. exec, regExp, and string extend the core types for the test harness and the unit tests, which were their only users, so they move to test/src/harness. The json and xml extensions had no users outside the two tools and are removed with them, and the tests for both are trimmed to the core functions that remain. The help unit test built its fixture by calling the generator's C functions directly, which was the last thing outside the tools that depended on them. It now runs build.py help-data to write the packed help and loads that, so it still runs against the current declarations without needing the generated C, which may not have been written when the test runs. The manual page wrapping is a port of the wrapper the binary uses for its own help, so the manual page and pgbackrest help break lines the same way, including where a word longer than a line is left unbroken. libyaml is no longer needed to build pgBackRest since the only thing that read yaml was the C generator. It is dropped from the development container and from every CI platform, and meson no longer looks for it. Code generation needs nothing compiled first either, so a test run no longer builds a generator before it can select the tests. The common source list is merged into the pgbackrest source list since it existed only so the generator could be built from the same files, and the entries that were out of alphabetical order are put in order while the list is being rewritten anyway. Every file the generator writes is byte identical to what the C wrote, for the project declarations and for the documentation's own. The documentation is checked the same way: the HTML rendered on this branch is byte identical to the HTML rendered on integration, every file of it, which exercises the whole pipeline on the real content rather than only the four files the tool writes. Every module in both libraries is declared in define.yaml and has full line and branch coverage. The coverage list is keyed by file rather than by module name, since the same name in two languages is two files and one of them would have silently lost its requirement while both existed. TestError becomes ToolError, since it is the error type for three tools now rather than for the harness alone.
301 lines
13 KiB
Python
301 lines
13 KiB
Python
"""Build Command and Configuration Reference.
|
|
|
|
Renders the reference documents from the same declarations the binary is generated from, so what is documented is what the code does
|
|
rather than a second description of it that has to be kept in step.
|
|
|
|
The configuration reference documents every option that can be set in the configuration file, grouped by the section it belongs to. The
|
|
command reference documents every command and, under each, the options that command takes, grouped by what they are about. An option
|
|
therefore appears in both, described once and rendered twice."""
|
|
|
|
####################################################################################################################################
|
|
from common.xml import xml_document_new, xml_node_add, xml_node_attribute_set, xml_node_child_add, xml_node_content_add
|
|
from config.parse import DEFAULT_TYPE_DYNAMIC, OPT_TYPE_BOOLEAN, SECTION_COMMAND_LINE
|
|
|
|
# Role a user gives an option, as opposed to a role a command starts for itself
|
|
_CMD_ROLE_MAIN = "main"
|
|
|
|
# Section an option is documented under when it has none of its own
|
|
_SECTION_DEFAULT = "general"
|
|
|
|
# Sections that mean the same thing in the command reference as in the configuration reference. Anything else is about the command
|
|
# itself and is grouped under one heading there.
|
|
_SECTION_COMMAND_LIST = ("general", "log", "maintainer", "repository", "stanza")
|
|
|
|
_SECTION_COMMAND = "command"
|
|
|
|
|
|
####################################################################################################################################
|
|
def _find(item_list, name):
|
|
"""Find an entry in a list by name, or None when it is not there."""
|
|
|
|
for item in item_list or []:
|
|
if item.name == name:
|
|
return item
|
|
|
|
return None
|
|
|
|
|
|
####################################################################################################################################
|
|
def _default_block(block_list, opt_cfg, default):
|
|
"""Describe the default of an option, which may be a value per value of the option it depends on."""
|
|
|
|
# A dynamic default is worked out when the command runs, so what it depends on is described rather than a value
|
|
if opt_cfg.default_type == DEFAULT_TYPE_DYNAMIC:
|
|
block_list.append("default: [path of executed pgbackrest binary]")
|
|
elif default is not None:
|
|
if default.value is not None:
|
|
# A boolean is written the way it is given on the command line rather than the way it is declared
|
|
if opt_cfg.type == OPT_TYPE_BOOLEAN:
|
|
block_list.append("default: %s" % ("y" if default.value == "true" else "n"))
|
|
else:
|
|
block_list.append("default: %s" % default.value)
|
|
else:
|
|
block_list.append("default (depending on %s):" % opt_cfg.depend.option.name)
|
|
|
|
for map in default.map_list:
|
|
block_list.append(" %s - %s" % (map.map, map.value))
|
|
|
|
block_list.append("")
|
|
|
|
|
|
####################################################################################################################################
|
|
def _allow_range_block(block_list, opt_cfg):
|
|
"""Describe the range an option allows, which may be a range per value of the option it depends on."""
|
|
|
|
if opt_cfg.allow_range.map_list is not None:
|
|
block_list.append("allow range (depending on %s):" % opt_cfg.depend.option.name)
|
|
|
|
for map in opt_cfg.allow_range.map_list:
|
|
block_list.append(" %s - [%s, %s]" % (map.map, map.min, map.max))
|
|
|
|
block_list.append("")
|
|
else:
|
|
block_list.append("allowed: [%s, %s]" % (opt_cfg.allow_range.min, opt_cfg.allow_range.max))
|
|
|
|
|
|
####################################################################################################################################
|
|
def _example_block(block_list, opt_cfg, opt_hlp, command):
|
|
"""Show how an option is given, which is on the command line for a command and in the configuration file otherwise."""
|
|
|
|
# An option in a group is indexed, and the example shows the first index since that is what a reader will start with
|
|
option = opt_cfg.name
|
|
|
|
if opt_cfg.group is not None:
|
|
option = "%s1%s" % (opt_cfg.group, opt_cfg.name[len(opt_cfg.group) :])
|
|
|
|
output = "example: "
|
|
|
|
for example_idx, example in enumerate(opt_hlp.example_list):
|
|
if command:
|
|
if example_idx != 0:
|
|
output += " "
|
|
|
|
output += "--"
|
|
|
|
# A boolean is given by naming it, so turning it off is naming the negation of it
|
|
if opt_cfg.type == OPT_TYPE_BOOLEAN and example == "n":
|
|
output += "no-"
|
|
# In the configuration file each example is a line of its own, since that is how it would be written
|
|
elif example_idx != 0:
|
|
block_list.append(output)
|
|
output = "example: "
|
|
|
|
output += option
|
|
|
|
if not command or opt_cfg.type != OPT_TYPE_BOOLEAN:
|
|
output += "=%s" % example
|
|
|
|
block_list.append(output)
|
|
|
|
|
|
####################################################################################################################################
|
|
def _option_render(xml_section, opt_cmd_cfg, opt_cfg, opt_hlp):
|
|
"""Render one option, either as a command takes it or as the configuration file sets it."""
|
|
|
|
xml_option = xml_node_add(xml_section, "section")
|
|
xml_title = xml_node_add(xml_option, "title")
|
|
|
|
xml_node_attribute_set(xml_option, "id", "option-%s" % opt_hlp.name)
|
|
xml_node_content_add(xml_title, "%s Option (" % opt_hlp.title)
|
|
xml_node_content_add(xml_node_add(xml_title, "id"), "--%s" % opt_hlp.name)
|
|
xml_node_content_add(xml_title, ")")
|
|
xml_node_child_add(xml_node_add(xml_option, "p"), opt_hlp.summary)
|
|
|
|
if opt_cfg.beta:
|
|
xml_node_content_add(xml_node_add(xml_option, "p"), "FOR BETA TESTING ONLY. DO NOT USE IN PRODUCTION.")
|
|
|
|
xml_node_child_add(xml_option, opt_hlp.description)
|
|
|
|
# What a reader needs to use the option, which is rendered as one block below the description
|
|
block_list = []
|
|
default = (
|
|
opt_cmd_cfg.default_value if opt_cmd_cfg is not None and opt_cmd_cfg.default_value is not None else opt_cfg.default_value
|
|
)
|
|
|
|
_default_block(block_list, opt_cfg, default)
|
|
|
|
if opt_cfg.allow_range is not None:
|
|
_allow_range_block(block_list, opt_cfg)
|
|
|
|
if opt_hlp.example_list is not None:
|
|
_example_block(block_list, opt_cfg, opt_hlp, opt_cmd_cfg is not None)
|
|
|
|
if len(block_list) > 0:
|
|
xml_node_content_add(xml_node_add(xml_option, "code-block"), "\n".join(block_list).strip())
|
|
|
|
# Names the option can still be given by, less any that is the option name itself since that is not a deprecated name
|
|
if opt_cfg.deprecate_list is not None:
|
|
deprecate = ""
|
|
|
|
for entry in opt_cfg.deprecate_list:
|
|
if entry.name != opt_cfg.name:
|
|
deprecate += "%s %s" % ("," if deprecate != "" else "", entry.name)
|
|
|
|
if deprecate != "":
|
|
xml_node_content_add(
|
|
xml_node_add(xml_option, "p"),
|
|
"Deprecated Name%s:%s" % ("s" if len(opt_cfg.deprecate_list) > 1 else "", deprecate),
|
|
)
|
|
|
|
|
|
####################################################################################################################################
|
|
def _document_new(title, description, introduction):
|
|
"""Build a reference document, which begins with what it is and an introduction to it."""
|
|
|
|
result = xml_document_new("doc", dtd_name="doc", dtd_file="doc.dtd")
|
|
|
|
xml_node_attribute_set(result.root, "title", "{[project]}")
|
|
xml_node_attribute_set(result.root, "subtitle", title)
|
|
xml_node_attribute_set(result.root, "toc", "y")
|
|
|
|
xml_node_content_add(xml_node_add(result.root, "description"), description)
|
|
|
|
xml_intro = xml_node_add(result.root, "section")
|
|
|
|
xml_node_attribute_set(xml_intro, "id", "introduction")
|
|
xml_node_content_add(xml_node_add(xml_intro, "title"), "Introduction")
|
|
xml_node_child_add(xml_intro, introduction)
|
|
|
|
return result
|
|
|
|
|
|
####################################################################################################################################
|
|
def reference_configuration_render(bld_cfg, bld_hlp):
|
|
"""Render the configuration reference, i.e. every option that can be set in the configuration file."""
|
|
|
|
result = _document_new(bld_hlp.opt_title, bld_hlp.opt_description, bld_hlp.opt_introduction)
|
|
|
|
for section in bld_hlp.sct_list:
|
|
xml_section = xml_node_add(result.root, "section")
|
|
|
|
xml_node_attribute_set(xml_section, "id", "section-%s" % section.id)
|
|
xml_node_content_add(xml_node_add(xml_section, "title"), "%s Options" % section.name)
|
|
xml_node_child_add(xml_section, section.introduction)
|
|
|
|
for opt_hlp in bld_hlp.opt_list:
|
|
opt_cfg = _find(bld_cfg.opt_list, opt_hlp.name)
|
|
|
|
# Skip an option documented under another section, one that cannot be set in the configuration file at all, and one that
|
|
# is internal since a reader has no use for it
|
|
if (opt_hlp.section or _SECTION_DEFAULT) != section.id:
|
|
continue
|
|
|
|
if opt_cfg.section == SECTION_COMMAND_LINE or opt_cfg.internal:
|
|
continue
|
|
|
|
_option_render(xml_section, None, opt_cfg, opt_hlp)
|
|
|
|
return result
|
|
|
|
|
|
####################################################################################################################################
|
|
def _command_section(section):
|
|
"""The section an option is grouped under in the command reference.
|
|
|
|
A section that means the same thing everywhere keeps its name. Anything else is about the command itself, so it is grouped under one
|
|
heading rather than a heading per command."""
|
|
|
|
if section is None:
|
|
return _SECTION_DEFAULT
|
|
|
|
return section if section in _SECTION_COMMAND_LIST else _SECTION_COMMAND
|
|
|
|
|
|
####################################################################################################################################
|
|
class _CommandOption:
|
|
"""An option as one command takes it, i.e. the option, what the command says about it, and the help to render."""
|
|
|
|
def __init__(self, opt_cfg, opt_cmd_cfg, opt_hlp, section):
|
|
self.opt_cfg = opt_cfg
|
|
self.opt_cmd_cfg = opt_cmd_cfg
|
|
self.opt_hlp = opt_hlp
|
|
self.section = section
|
|
|
|
|
|
####################################################################################################################################
|
|
def _command_option_list(cmd_cfg, cmd_hlp, bld_cfg, bld_hlp):
|
|
"""The options a command takes, with the help to render for each."""
|
|
|
|
result = []
|
|
|
|
for opt_cfg in bld_cfg.opt_list:
|
|
# Skip an option a reader has no use for or should not be told about
|
|
if opt_cfg.internal or opt_cfg.secure:
|
|
continue
|
|
|
|
# Skip an option this command does not take, or takes only for a role the user does not start
|
|
opt_cmd_cfg = _find(opt_cfg.cmd_list, cmd_cfg.name)
|
|
|
|
if opt_cmd_cfg is None or opt_cmd_cfg.internal or _CMD_ROLE_MAIN not in opt_cmd_cfg.role_list:
|
|
continue
|
|
|
|
# Help a command documents differently wins over the help the option has of its own
|
|
opt_hlp = _find(cmd_hlp.opt_list, opt_cfg.name)
|
|
section = _SECTION_COMMAND if opt_hlp is not None else None
|
|
|
|
if opt_hlp is None:
|
|
opt_hlp = _find(bld_hlp.opt_list, opt_cfg.name)
|
|
section = opt_hlp.section
|
|
|
|
result.append(_CommandOption(opt_cfg, opt_cmd_cfg, opt_hlp, _command_section(section)))
|
|
|
|
return result
|
|
|
|
|
|
####################################################################################################################################
|
|
def reference_command_render(bld_cfg, bld_hlp):
|
|
"""Render the command reference, i.e. every command and the options it takes."""
|
|
|
|
result = _document_new(bld_hlp.cmd_title, bld_hlp.cmd_description, bld_hlp.cmd_introduction)
|
|
|
|
for cmd_hlp in bld_hlp.cmd_list:
|
|
cmd_cfg = _find(bld_cfg.cmd_list, cmd_hlp.name)
|
|
|
|
# Skip a command a reader has no use for
|
|
if cmd_cfg.internal:
|
|
continue
|
|
|
|
xml_section = xml_node_add(result.root, "section")
|
|
xml_title = xml_node_add(xml_section, "title")
|
|
|
|
xml_node_attribute_set(xml_section, "id", "command-%s" % cmd_hlp.name)
|
|
xml_node_content_add(xml_title, "%s Command (" % cmd_hlp.title)
|
|
xml_node_content_add(xml_node_add(xml_title, "id"), cmd_hlp.name)
|
|
xml_node_content_add(xml_title, ")")
|
|
xml_node_child_add(xml_section, cmd_hlp.description)
|
|
|
|
opt_list = _command_option_list(cmd_cfg, cmd_hlp, bld_cfg, bld_hlp)
|
|
|
|
for section in sorted({opt.section for opt in opt_list}):
|
|
xml_category = xml_node_add(xml_section, "section")
|
|
|
|
xml_node_attribute_set(xml_category, "id", "category-%s" % section)
|
|
xml_node_attribute_set(xml_category, "toc", "n")
|
|
xml_node_content_add(xml_node_add(xml_category, "title"), "%s Options" % (section[:1].upper() + section[1:]))
|
|
|
|
for opt in opt_list:
|
|
if opt.section == section:
|
|
_option_render(xml_category, opt.opt_cmd_cfg, opt.opt_cfg, opt.opt_hlp)
|
|
|
|
return result
|