* letstest: -ubuntu18.04 +centos9stream +debian11
* letstest: username for centos 9 stream is ec2-user
This is mentioned on https://centos.org/download/aws-images/
* ensure mod_ssl is installed
in centos 9 stream, apache has to be restarted after mod_ssl is
installed, or the snakeoil certificates will not be present and
apache won't start.
this also removes nghttp2 being installed as the relevant bug
is long fixed.
* add some missing types
* install pkg-config
* install pkg-config for docker too
* add pkg-config to plugins
* pkg-config when cryptography may need to be built
* deps cleanup
* more comments
* more tweaks
I want to use isort as part of https://github.com/certbot/certbot/issues/9572 because I want to do it programmatically, however, I felt like the config needed to be tweaked a bit due to it not understanding what is and is not our own code.
This PR updates the isort config so it recognizes our own modules and runs `isort .` from the root of the repo to update everything.
* update isort config
* run "isort ."
I think test_apache2.sh still has value as it allows us to test our Apache plugin with the Apache layouts found on different OSes. Unfortunately, many of the OSes we're currently testing against don't have Python 3.7+ packaged yet we still support these OSes through things like snap where we bundle our own version of Python.
To allow us to continue testing on these OSes, I switched to installing Python through pyenv. I also took the opportunity to clean up the scripts, removing a lot of code, failing more quickly, and simplifying failure logic in test_apache2.sh.
The reason I want to do this is many of the targets of `test_sdists.sh` use Python 3.6 which [has reached its EOL](https://www.python.org/dev/peps/pep-0494/#lifespan). We could instead just stop running the test on these systems or install a newer version of Python 3 outside of OS packaging, but instead I decided to look into why we have these tests to begin with.
I introduced these tests many years ago in https://github.com/certbot/certbot/pull/4089 as a fix for https://github.com/certbot/certbot/issues/4044. Essentially the problem was the way packagers ran tests and the way we ran tests were slightly different. This difference could cause test failures when distros tried to run tests on our packages.
Since I did this, [we've switched to telling packagers to run tests using `pytest` like we do](5e76669c50/certbot/docs/packaging.rst (notes-for-package-maintainers)) and we've greatly reduced our reliance on OS packaging through things like `snap`.
Because of this, I think we should stop running this test, reducing our reliance on the heavy "test farm tests", and simplifying our CI pipeline. I think future problems here is quite unlikely and even if we have them, it should only affect tests on our non-primary distribution mechanisms which I think is a very minor concern.
When reviewing this PR, it's probably worth noting that I just replaced `targets.yaml` with the contents of `apache2_targets.yaml` since the Apache 2 tests are the only runs we're running with this change.
Fixes https://github.com/certbot/certbot/issues/9058.
The changes to the CI config are equivalent to the ones made in https://github.com/certbot/certbot/pull/8460.
Other than ignoring some warnings raised by botocore, the main additional work that had to be done here was switching away from using `distutils.version.LooseVersion` since the entire `distutils` module was deprecated in Python 3.10. To do that, I took a few different approaches:
* If the version strings being parsed are from Python packages such as Certbot or setuptools, I switched to using [pkg_resources.parse_version](https://setuptools.pypa.io/en/latest/pkg_resources.html#parsing-utilities) from `setuptools`. This functionality has been available since [setuptools 8.0 from 2014](https://setuptools.pypa.io/en/latest/history.html#id865).
* If the version strings being parsed are not from Python packages, I added code equivalent to `distutils.version.LooseVersion` in `certbot.util.parse_loose_version`.
* The code for `CERTBOT_PIP_NO_BINARY` can be completely removed since that variable isn't used or referenced anywhere in this repo.
* add python 3.10 support
* make some version changes
* don't use looseversion in setup.py
* switch to pkg_resources
* deprecate get_strict_version
* fix route53 tests
* remove unused CERTBOT_PIP_NO_BINARY code
* stop using distutils in letstest
* add unit tests
* more changelog entries
Apache test farm tests started failing last night due to a change in pyenv. See https://dev.azure.com/certbot/certbot/_build/results?buildId=3948&view=logs&j=f67c2a39-2c4f-5190-915f-6f32a7a4306f&t=96f0f394-f513-5158-f5e7-a26e55aeadbf&l=26943.
I managed to fix that in d94f20f8b7, however, the OSes the tests were failing on were Debian 9 and Ubuntu 16.04. [Debian 9 reached its end-of-life in July 2020](https://wiki.debian.org/DebianReleases) and [Ubuntu 16.04 reached its end of standard support in April 2021](https://wiki.ubuntu.com/Releases). As shown at the same links, Debian 9 still has support from the LTS team and Ubuntu 16.04 has ESM support. Do we still want to support either of these OSes?
If so, we can use the commit I linked in the first sentence of the last paragraph, but I think supporting the OSes through their standard support is good enough. The Certbot team has enough on their plate and especially when the OSes are so old that we can't even use their packaged version of Python anymore which complicates our tests, I think we can just drop support and move on.
I don't have a strong opinion here though so if someone else does, let me know what you'd like to see or make the PR yourself based on the changes in my linked commit and I'll merge it.
You can see the tests passing with this change at https://dev.azure.com/certbot/certbot/_build/results?buildId=3955&view=results.
* Remove apache tests on old OSes
* remove unused pyenv code
Fixes https://github.com/certbot/certbot/issues/8781.
This PR makes our test farm tests into a normal package so it and its dependencies can be tracked and installed like our other packages.
Other noteworthy changes in this PR:
* Rather than continuing to place logs in your CWD, they're placed in a temporary directory that is printed to the terminal.
* `tests/letstest/auto_targets.yaml` was deleted rather than renamed because the file is no longer used.
* make a letstest package
* remove deleted deps
* fix letstest install
* add __init__.py
* call main
* Explicitly mention activating venv
* rerename file
* fix version.py path
* clarify "this"
* Use >= instead of caret requirement
