Pebble 2.5.1 supports OCSP stapling, so we can finally replace all boulder tests/harnesses with the much simpler pebble setup.
Closes#9898
* Remove unused `--acme-server` argument
Since this argument is never set and always defaults to 'pebble', just
remove it to simplify assumptions about which test server's being used.
* Remove boulder option from integration tests
Now that pebble supports all of our test cases, we can move off of
the much more complicated boulder test harness.
* pebble_artifacts: bump to latest pebble release
* pebble_artifacts: fix download path
* certbot-ci: unzip pebble assets
* CI: rip out windows tests/jobs
* tox.ini: rm outdated Windows comment
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* ci: rm redundant integration test
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* acme_server: raise error if proxy and http-01 port are both set
* acme_server: rm vestigial preterimate commands stuff
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Azure recently dropped the `docker-compose` standalone executable (aka
docker-compose v1), and since it's not receiving updates anymore, let's
get with the times and update to v2 as well.
* Migrate pkg_resources API related to resources to importlib_resources
* Fix lint and mypy + pin lexicon
* Update filterwarnings
* Update oldest tests requirements
* Update pinned dependencies
* Fix for modern versions of python
* Fix assets load in nginx integration tests
* Fix a warning
* Isolate static generation from importlib.resource into a private function
---------
Co-authored-by: Adrien Ferrand <adrien.ferrand@amadeus.com>
* Do not call deprecated datetime.utcnow() and datetime.utcfromtimestamp()
* Ignore DeprecationWarnings from importing dependencies
$ python3 -Wdefault
Python 3.12.0b4 (main, Jul 12 2023, 00:00:00) [GCC 13.1.1 20230614 (Red Hat 13.1.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
/usr/lib/python3.12/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
>>> import pytz
/usr/lib/python3.12/site-packages/pytz/tzinfo.py:27: DeprecationWarning: datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.fromtimestamp(timestamp, datetime.UTC).
_epoch = datetime.utcfromtimestamp(0)
* Used pytz.UTC consistently for clarity
* repin current
* repin oldest
* csr must have version set to zero
* only set PIP_USE_PEP517 for macOS
* experiment with brew update git failure workaround
* add some missing types
* install pkg-config
* install pkg-config for docker too
* add pkg-config to plugins
* pkg-config when cryptography may need to be built
* deps cleanup
* more comments
* more tweaks
I want to use isort as part of https://github.com/certbot/certbot/issues/9572 because I want to do it programmatically, however, I felt like the config needed to be tweaked a bit due to it not understanding what is and is not our own code.
This PR updates the isort config so it recognizes our own modules and runs `isort .` from the root of the repo to update everything.
* update isort config
* run "isort ."
* Add command to update config files without issuing/renewing cert
* toss up a vague untested skeleton
* remove duplicated code
* set certname in config
* consistent name, no zope
* import copy
* reconsitute is in renewal
* import renewal
* import cli
* fix lint errors
* call choose_configurator_plugins for its side effect of writing to config
* Set certonly in choose config plugins as we do for renew
* rewrite by piggybacking on existing side effects of a dry run instead
* do not allow domains to be set while reconfiguring
* remove unused cert_manager.reconfigure
* remove unused imports
* Add comments and messages
* add cli information
* start adding tests
* remove test code
* get certname before setting up plugins
* get plugin from lineage if not set on cli
* import copy
* always reconstitute
* only load cert once
* add error message
* improve comment
* mock everything out for tests
* test functionality is working!
* add tests for adding and modifying hooks
* test that we don't modify the config if the dry run fails
* improve documentation
* add webroot to reconfigure common options
* lint and clean up intermediate artifacts
* mock validate_hooks for windows
* print success message with updated parameters
* Improve success message
* add message for no changes have been made
* improve changed message to show before as well
* syntax
* Add changes will apply at the next renewal message
* lint
* lint really likes dict.items() for some reason
* run the deploy hook
* turn off dry run to test deploy hook
* patch list_hooks call for tests
* factor out reporting results code
* Remove reporting of which values were changed
* add flag to run deploy hook despite doing a dry run, and recommend setting that to yes when running reconfigure and modifying the deploy hook
* missing () around multi-line string
* test if the two dicts are equal instead of finding the actual changes, thus avoiding having to deal with webroot_map being a list
* refer to --deploy-hook instead of deploy hook
* use renewal configuration instead of configuration information
* mention that the deploy hook will use the active cert not the test one
* disable lint and remove new from language asking about running a deploy hook
* pluralize run deploy hook(s)
* Add test for reporting results when there is a webroot map
* update changelog
* Update error message about modifying domains on the certificate
* update changelog
* Add basic integration tests
* Just set -a rather than redoing the whole testing infrastructure
* used webroot in integration test since it's already installed
* file contents are accessed twice now
---------
Co-authored-by: Alex Zorin <alex@zorin.au>
* certbot-ci: boulder will now only supports port 80 for http-01
* forgot to actually use the http_01_port argument
* print the port the proxy listens on
* try allow binding to privileged ports
Based on my design [here](https://docs.google.com/document/d/1jGh_bZPnrhi96KzuIcyCJfnudl4m3pRPGkiK4fTo8e4/edit?usp=sharing).
Fixes https://github.com/certbot/certbot/issues/4634 and https://github.com/certbot/certbot/issues/4635.
- [x] Deprecate `NamespaceConfig.csr_dir`,`NamespaceConfig.key_dir`, ~~`constants.CSR_DIR` and `constants.KEY_DIR`~~. (`constants` is `_internal` so we can just delete it eventually).
- [x] Update `certbot.crypto_util.generate_csr` and `.generate_key` to make `csr_dir` and `key_dir` optional, respectively.
- [x] Change `certbot._internal.client.Client.obtain_certificate` to no longer include `csr_dir` and `key_dir` to the `.generate_csr` and `.generate_key` calls, respectively.
- Automatically delete unwanted lineage items:
- [x] In `certbot._internal.storage.RenewableCert`, add a function to truncate the lineage history according to the criteria (keep the current and the 5 prior certificates).
- [x] Add a test suite for `truncate`
- [x] In `certbot._internal.renewal.renew_cert`, call the lineage truncation function after the symlinks have been updated for the renewal.
* Stop writing new files to /csr and /keys
* storage: add lineage truncation
* remove unused code
* deprecate keys_dir and csr_dir
* update CHANGELOG
* just keep 5 prior certificates, dont be clever with expiry
* docs: remove reference to /archive and /keys
* filter {csr,key}_dir deprecations directly in tests
* deps: remove pyjwt dependency
* pinning: strip extras from dependencies
`poetry export` outputs in requirements.txt format, which is now
apparently producing "dep[extra]==...". We are using this output
as the constraints file for pip and pip's new resolver does not
permit extras in the constraints file.
This change filters out the extras specifiers.
* repin current dependencies
* fix new pylint complaints
* silence lint about distutils.version
We have already deprecated the function and it'll be removed in
2.0.
* docs: set sphinx language to 'en'
this is emitting a warning and failing the build
* Revert "pinning: strip extras from dependencies"
This reverts commit 11268fd231.
* pin poetry back to avoid extras issue
* repin
* fix new mypy complaints in acme/
* certbot-ci: fix challtestsrv address for boulder-v2
The port is no longer exposed on the Docker host.
* vary the challtestsrv URL by acme server
* fix mypy
* fix comment
Co-authored-by: ohemorange <ebportnoy@gmail.com>
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* improve handling and ux of unexpected key type migration
* update unit tests
* update integration tests
* if --cert-name and --key-type are set, dont prompt
* Add support for revoking ecdsa keys without --cert-name.
Co-Authored-By: commonism <commonism@users.noreply.github.com>
* Move alg to acme_client.ClientNetwork instantiating in acme_from_config_key
* Fix argument for RS256/ES256
* Support also ES384 and ES512 signing algorithms.
* Work in progress
* Work in progress
* Work in progress
* Work in progress
* Fix issues around nullability of VirtualHost.path, may discuss that during review
* Work in progress
* Fix remaining types
* Various lint fixes
* Reconfigure tox and mypy to disallow untyped defs globally
* Cleanup compatibility tests
* Use cast for unused v2 logic
* Improve types
* Remove unused comment
* Fix coverage
* Better types
* Fix another type
* Update certbot-apache/certbot_apache/_internal/apacheparser.py
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot-apache/certbot_apache/_internal/assertions.py
Co-authored-by: alexzorin <alex@zor.io>
* Fix type
* Various fixes
* Refactor imports
* Keep naming convention consistent on TypeVars
* Improve types
* Improve types
* Remove remaining Sequence[str] in the project
Co-authored-by: alexzorin <alex@zor.io>
* Add generic methods to save some casts, and fix lint
* Update current and oldest pinning
* Fix classes
* Remove some todos thanks to josepy 1.11.0
* Cleanup some useless pylint disable
* Finish complete typing
* Better TypeVar names
* Upgrade pinning and fix some typing errors
* Use protocol
* Fix types in apache
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes https://github.com/certbot/certbot/issues/9058.
The changes to the CI config are equivalent to the ones made in https://github.com/certbot/certbot/pull/8460.
Other than ignoring some warnings raised by botocore, the main additional work that had to be done here was switching away from using `distutils.version.LooseVersion` since the entire `distutils` module was deprecated in Python 3.10. To do that, I took a few different approaches:
* If the version strings being parsed are from Python packages such as Certbot or setuptools, I switched to using [pkg_resources.parse_version](https://setuptools.pypa.io/en/latest/pkg_resources.html#parsing-utilities) from `setuptools`. This functionality has been available since [setuptools 8.0 from 2014](https://setuptools.pypa.io/en/latest/history.html#id865).
* If the version strings being parsed are not from Python packages, I added code equivalent to `distutils.version.LooseVersion` in `certbot.util.parse_loose_version`.
* The code for `CERTBOT_PIP_NO_BINARY` can be completely removed since that variable isn't used or referenced anywhere in this repo.
* add python 3.10 support
* make some version changes
* don't use looseversion in setup.py
* switch to pkg_resources
* deprecate get_strict_version
* fix route53 tests
* remove unused CERTBOT_PIP_NO_BINARY code
* stop using distutils in letstest
* add unit tests
* more changelog entries
Fixes#8899
This PR removes the pinning upper limit of mypy currently set to <0.900 and adds the required types-* stub packages to make recent versions of mypy work.
* Unpin mypy
* Improve type in TempHandler
* Add types
While bumping pinned packages in #8928, we came across a new version of pylint (2.9.3). Upgrading to this version requires some changes to Certbot's code, which is what this change is about.
* pylint: upgrade pinned verson and fix new lints
* maxsplit should be 1, not -1, for rsplit
This is part of https://github.com/certbot/certbot/issues/8782. I took it on now because the currently pinned version of `pylint` doesn't work with newer versions of `poetry` which I wanted to upgrade as part of https://github.com/certbot/certbot/issues/8787.
To say a bit more about the specific changes in this PR:
* Newer versions of `pylint` complain if `Popen` isn't used as a context manager. Instead of making this change, I switched to using `subprocess.run` which is simpler and [recommended in the Python docs](https://docs.python.org/3/library/subprocess.html#using-the-subprocess-module). I also disabled this check in a few places where no longer using `Popen` would require significant refactoring.
* The deleted code in `certbot/certbot/_internal/renewal.py` is cruft since https://github.com/certbot/certbot/pull/8685.
* The unused argument to `enable_mod` in the Apache plugin is used in some over the override classes that subclass that class.
* unpin pylint and repin dependencies
* disable raise-missing-from
* disable wrong-input-order
* remove unused code
* misc lint fixes
* remove unused import
* various lint fixes
