* Set the delegated field in Lexicon config to bypass subdomain resolution (#9821)
The Lexicon-based DNS plugins use a mechanism to determine which actual segment of the input domain is actually the DNS zone in which the DNS-01 challenge has to be initiated (eg. `subdomain.domain.com` or `domain.com` for input `subdomain.domain.com`): they tries recursively to configure Lexicon and initiate authentication from the most specific to most generic domain segment, and select the first segment where Lexicon stop erroring out.
This mechanism broke with #9746 because now the plugins call Lexicon client instead of the underlying providers, and the client makes guess on the actual domain requested. Typically for `subdomain.domain.com` it will actually try to authenticate against `domain.com`, and so the mechanism above does not work anymore.
This PR fixes the issue by using the `delegated` field in Lexicon config each time the plugin needs it. This field is designed for this kind of purpose: it will instruct Lexicon what is the actual DNS zone domain instead of guessing it.
I tested the change with one of my OVH account. The expected behavior is re-established and the plugin is able to test `subdomain.domain.com` then `domain.com` as before.
Fixes#9791Fixes#9818
(cherry picked from commit cf4f07d17e)
* add changelog entry for 9821 (#9822)
(cherry picked from commit 7bb85f8440)
---------
Co-authored-by: Adrien Ferrand <adferrand@users.noreply.github.com>
* helpful: fix handling of abbreviated ConfigArgparse arguments (#9796)
* helpful: fix handling of abbreviated ConfigArgparse arguments
ConfigArgparse allows for "abbreviated" arguments, i.e. just the prefix
of an argument, but it doesn't set the argument sources in these cases.
This commit checks for those cases and sets the sources appropriately.
* failing to find an action raises an error instead of logging
* Update changelog
* Add handling for short arguments, fix equals sign handling
These were silently being dropped before, possibly leading to instances
of `NamespaceConfig.set_by_user()` returning false negatives.
(cherry picked from commit 11e17ef77b)
* Fix finish_release.py (#9800)
* response is value
* rename vars
(cherry picked from commit a96fb4b6ce)
* Merge pull request #9762 from certbot/docs/yaml-config
Add YAML files for Readthedocs requirements
(cherry picked from commit 44046c70c3)
* Update Lexicon requirements to stabilize certbot-dns-ovh behavior (#9802)
* Update minimum Lexicon version required for certbot-dns-ovh
* Add types
* FIx mypy
* Fix lint
* Fix BOTH lint and mypy
(cherry picked from commit 5cf5f36f19)
* simplify code (#9807)
(cherry picked from commit 6f7b5ab1cd)
* Include linting fixes from 8a95c03
---------
Co-authored-by: Will Greenberg <willg@eff.org>
Co-authored-by: Alexis <alexis@eff.org>
Co-authored-by: Adrien Ferrand <adferrand@users.noreply.github.com>
* Refactor Lexicon-based DNS plugins and upgrade minimal version of Lexicon
* Relax filterwarning to comply with envs where boto3 is not installed
* Update pinned dependencies
* Use our previous method to deprecate part of modules
* Safe import internally
* Add changelog
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Migrate entrypoint logic from pkg_resources to importlib.metadata
* Usage of importlib_metadata up to Python 3.9 to align API behavior to Python 3.10
---------
Co-authored-by: Adrien Ferrand <adrien.ferrand@amadeus.com>
Co-authored-by: Adrien Ferrand <adrien.ferrand@arteris.com>
* Migrate pkg_resources API related to resources to importlib_resources
* Fix lint and mypy + pin lexicon
* Update filterwarnings
* Update oldest tests requirements
* Update pinned dependencies
* Fix for modern versions of python
* Fix assets load in nginx integration tests
* Fix a warning
* Isolate static generation from importlib.resource into a private function
---------
Co-authored-by: Adrien Ferrand <adrien.ferrand@amadeus.com>
* dns-google: fix condition to don't use private dns zones
* update MD
* Fix condition
* fix condition
* update testdata
* fix identation
* update tests
* update changelog
* Update dns_google.py
* add test for split horizon dns google
* add dnsName to managed zones
* update quickstart and remove os import
* simplify theme use
* list sphinx_rtd_theme as extension
Our docs builds failed last night, presumably because #9754 updated `sphinx_rtd_theme` which changed some unknown thing.
Looking into it, our usage of this project was very unconventional. Following the code comment I deleted in this PR to https://docs.readthedocs.io/en/stable/faq.html#i-want-to-use-the-read-the-docs-theme-locally, simple instructions are given to put the following in your `conf.py` file:
```
extensions = [
...
'sphinx_rtd_theme',
]
html_theme = "sphinx_rtd_theme"
```
I did this instead of the more complicated logic we were using and all builds passed locally. I also triggered a build on readthedocs with these changes which also passed.
This takes care of the dependabot alerts those with access can see at https://github.com/certbot/certbot/security/dependabot.
Pinning back `cython` is needed because without it, our full test suite will fail when trying to build `pyyaml` on ARM systems.
* Do not call deprecated datetime.utcnow() and datetime.utcfromtimestamp()
* Ignore DeprecationWarnings from importing dependencies
$ python3 -Wdefault
Python 3.12.0b4 (main, Jul 12 2023, 00:00:00) [GCC 13.1.1 20230614 (Red Hat 13.1.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
/usr/lib/python3.12/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
>>> import pytz
/usr/lib/python3.12/site-packages/pytz/tzinfo.py:27: DeprecationWarning: datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.fromtimestamp(timestamp, datetime.UTC).
_epoch = datetime.utcfromtimestamp(0)
* Used pytz.UTC consistently for clarity
* repin current
* repin oldest
* csr must have version set to zero
* only set PIP_USE_PEP517 for macOS
* experiment with brew update git failure workaround
* Revert change to NamespaceConfig's constructor
NamespaceConfig's argument sources dict is now set with a method,
and raises a runtime error if one isn't set when set_by_user() is
called.
* Actually update CHANGELOG to reflect the set_by_user changes
* linter appeasement
* configuration: update docs, add test
This test ensures that calling `set_by_user` without an initialized
sources dict raises a RuntimeError.
* Rewrite helpful_test to appease the linter
* Use public interface to access argparse sources dict
* HelpfulParser builds ArgumentSources dict, stores it in NamespaceConfig
After arguments/config files/user prompted input have been parsed, we
build a mapping of Namespace options to an ArgumentSource value. These
generally come from argparse's builtin "source_to_settings" dict, but
we also add a source value representing dynamic values set at runtime.
This dict is then passed to NamespaceConfig, which can then be queried
directly or via the "set_by_user" method, which replaces the global
"set_by_cli" and "option_was_set" functions.
* Use NamespaceConfig.set_by_user instead of set_by_cli/option_was_set
This involves passing the NamespaceConfig around to more functions
than before, removes the need for most of the global state shenanigans
needed by set_by_cli and friends.
* Set runtime config values on the NamespaceConfig object
This'll correctly mark them as being "runtime" values in the
ArgumentSources dict
* Bump oldest configargparse version
We need a version that has get_source_to_settings_dict()
* Add more cli unit tests, use ArgumentSource.DEFAULT by default
One of the tests revealed that ConfigArgParse's source dict excludes
arguments it considers unimportant/irrelevant. We now mark all arguments
as having a DEFAULT source by default, and update them otherwise.
* Mark more argument sources as RUNTIME
* Removes some redundant helpful_test.py, moves one to cli_test.py
We were already testing most of these cases in cli_test.py, only
with a more complete HelpfulArgumentParser setup. And since the hsts/no-hsts
test was manually performing the kind of argument adding that cli
already does out of the box, I figured the cli tests were a more natural
place for it.
* appease the linter
* Various fixups from review
* Add windows compatability fix
* Add test ensuring relevant_values behaves properly
* Build sources dict in a more predictable manner
The dict is now built in a defined order: first defaults, then config
files, then env vars, then command line args. This way we eliminate the
possibility of undefined behavior if configargparse puts an arg's entry
in multiple source dicts.
* remove superfluous update to sources dict
* remove duplicate constant defines, resolve circular import situation
* letstest: -ubuntu18.04 +centos9stream +debian11
* letstest: username for centos 9 stream is ec2-user
This is mentioned on https://centos.org/download/aws-images/
* ensure mod_ssl is installed
in centos 9 stream, apache has to be restarted after mod_ssl is
installed, or the snakeoil certificates will not be present and
apache won't start.
this also removes nghttp2 being installed as the relevant bug
is long fixed.
* dns-rfc2136: add test coverage for PR #9672
* fix compatibility with oldest dnspython
* rename test to be more descriptive
Co-authored-by: ohemorange <ebportnoy@gmail.com>
---------
Co-authored-by: ohemorange <ebportnoy@gmail.com>
This is, to my knowledge, an entirely inconsequential PR to add support for entirely novel challenge types.
Presently in the [`challb_to_achall` function](399b932a86/certbot/certbot/_internal/auth_handler.py (L367)) if the challenge type is not of a type known to certbot an error is thrown. This check is mostly pointless as an authenticator would not request a challenge unknown to it. This check does however forbid any plugins from supporting entirely novel challenges not of the key authorisation form.
* support unknown ACME challenge types
* add to changelog
* update tests
---------
Co-authored-by: Brad Warren <bmw@eff.org>
* remove pointless paragraph about --server and wildcards
* docs: update help text for --dry-run and --staging
* docs: update "Changing the ACME Server" for --dry-run
* add note about webserver reloads
