* Remove cast for jose.fields.
https://github.com/certbot/certbot/pull/9073 references this.
* Some of them can't be removed, though.
* Fix josepy type hints of json
* Increase josepy pinning version.
Note that the repin scripts have not been used.
* Run repin scripts.
* Fix constraints
* Remove Windows 2016 environment, generate 64 bit installer
* Add note to changelog
* Use win_amd64 as installer suffix
* Bump PYTHON_BITNESS to 64
* Require 64 bit Windows for the installer_build job
* Update certbot install path
* update windows test name
* Base installer suffix on PYTHON_BITNESS again
* Update changelog to request users uninstall old version
* Revert setuptools-rust pin
This was a temporary workaround to fix
https://github.com/certbot/certbot/issues/9111, but it looks like the
the issue resolved itself
* Make mypy happy
There was an unused ignore statement, and Validator.certificate was
unnecessarily casting strings as bytes for an X509 digest method.
* Pin setuptools-rust to prevent build-dep hiccups in the future
* Add generic methods to save some casts, and fix lint
* Update current and oldest pinning
* Fix classes
* Remove some todos thanks to josepy 1.11.0
* Cleanup some useless pylint disable
* Finish complete typing
* Better TypeVar names
* Upgrade pinning and fix some typing errors
* Use protocol
* Fix types in apache
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes https://github.com/certbot/certbot/issues/9058.
The changes to the CI config are equivalent to the ones made in https://github.com/certbot/certbot/pull/8460.
Other than ignoring some warnings raised by botocore, the main additional work that had to be done here was switching away from using `distutils.version.LooseVersion` since the entire `distutils` module was deprecated in Python 3.10. To do that, I took a few different approaches:
* If the version strings being parsed are from Python packages such as Certbot or setuptools, I switched to using [pkg_resources.parse_version](https://setuptools.pypa.io/en/latest/pkg_resources.html#parsing-utilities) from `setuptools`. This functionality has been available since [setuptools 8.0 from 2014](https://setuptools.pypa.io/en/latest/history.html#id865).
* If the version strings being parsed are not from Python packages, I added code equivalent to `distutils.version.LooseVersion` in `certbot.util.parse_loose_version`.
* The code for `CERTBOT_PIP_NO_BINARY` can be completely removed since that variable isn't used or referenced anywhere in this repo.
* add python 3.10 support
* make some version changes
* don't use looseversion in setup.py
* switch to pkg_resources
* deprecate get_strict_version
* fix route53 tests
* remove unused CERTBOT_PIP_NO_BINARY code
* stop using distutils in letstest
* add unit tests
* more changelog entries
This PR upgrades the pinned version of the dependencies. Version `1.9.0` of josepy is used so errors related to JWK serialization with EC keys (see https://github.com/certbot/josepy/issues/109) are fixed for Certbot.
This PR removes all zope dependencies from plugins configuration.
It also lets Sphinx upgrade to the next major version by removing the plugin dedicated to zope interfaces documentation. As a consequence, the deprecated zope interfaces are not documented anymore.
* Cleanup zope dependencies in plugins and upgrade sphinx
* Update pinnings
Fixes#8899
This PR removes the pinning upper limit of mypy currently set to <0.900 and adds the required types-* stub packages to make recent versions of mypy work.
* Unpin mypy
* Improve type in TempHandler
* Add types
[Snapcraft 5.0](https://forum.snapcraft.io/t/release-notes-snapcraft-5-0/25751) implemented creating build IDs based on the project's contents instead of the directory path in https://github.com/snapcore/snapcraft/pull/3554. This is a feature we initially wanted, but it broke our workaround added in https://github.com/certbot/certbot/pull/8719. Our workaround is broken because now that the build ID is based on the project's contents, copying the project to a temporary directory has no effect.
This PR removes the workaround from https://github.com/certbot/certbot/pull/8719 and instead constructs a random build ID that it provides to snapcraft. This provides us with even more randomness to avoid build ID conflicts while avoiding having to copy the project to a temporary directory before every build.
* improve-remote-build
* use lowercase letters
This PR is a new approach for fixing #8732 based on the discussions occurred in the first PR #8877.
This PR upgrades python-augeas to the latest version, and avoids tests failure of Windows because of this upgrade. To do so it leverages the [tox multi-platform feature](https://tox.readthedocs.io/en/latest/example/platform.html) and modifications to `tools/venv.py` in order to not install and not test `certbot-apache` on Windows.
* Unpin python-augeas and upgrade current pinnings
* Do not install certbot-apache in Windows dev environments
* Introduce tox specific win packages + remove certbot compatibility on windows
* Add libaugeas to sphinx build
* Redefine lint and mypy targets
* Keep the lint and mypy environments
While bumping pinned packages in #8928, we came across a new version of pylint (2.9.3). Upgrading to this version requires some changes to Certbot's code, which is what this change is about.
* pylint: upgrade pinned verson and fix new lints
* maxsplit should be 1, not -1, for rsplit
* use poetry 1.2.0a1
* pin pip normally
* use normal constraints file with pipstrap
* remove unused STRIP_HASHES var
* Check for old poetry versions
* keep pip, setuptools, and wheel pinned in oldest
* remove strip hashes
* pin back pip
* fix new lint error
This is part of https://github.com/certbot/certbot/issues/8782. I took it on now because the currently pinned version of `pylint` doesn't work with newer versions of `poetry` which I wanted to upgrade as part of https://github.com/certbot/certbot/issues/8787.
To say a bit more about the specific changes in this PR:
* Newer versions of `pylint` complain if `Popen` isn't used as a context manager. Instead of making this change, I switched to using `subprocess.run` which is simpler and [recommended in the Python docs](https://docs.python.org/3/library/subprocess.html#using-the-subprocess-module). I also disabled this check in a few places where no longer using `Popen` would require significant refactoring.
* The deleted code in `certbot/certbot/_internal/renewal.py` is cruft since https://github.com/certbot/certbot/pull/8685.
* The unused argument to `enable_mod` in the Apache plugin is used in some over the override classes that subclass that class.
* unpin pylint and repin dependencies
* disable raise-missing-from
* disable wrong-input-order
* remove unused code
* misc lint fixes
* remove unused import
* various lint fixes
I think we should use our `pip_install*` scripts wherever we can and I'm not quite sure yet if I'd call `repoze.sphinx.autointerface` unmaintained.
* use pip_install_editable
* update sphinx comment
Since Saturday the CI pipeline is failing due to several Sphinx errors. See https://dev.azure.com/certbot/certbot/_build/results?buildId=3928&view=logs&j=d74e04fe-9740-597d-e9fa-1d0400037dfd&t=dde413a4-f24c-59a0-9684-e33d79f9aa02
First, the build of certbot-dns-google is failing because of a particular configuration. It seems that this configuration has been written here to activate the support of the RST instruction `.. code-block:: json` in documentation. However, it does not seem to be necessary for a similar situation in certbot-dns-route53 documentation. So let's try to remove it and fix the Sphinx builds.
Second, Sphinx builds were not pinning dependencies, so Sphinx 4.x (that has been released yesterday) started to be used in the pipeline. Sadly this new version is not compatible with the plugin `repoze.sphinx.autointerface`, used to extract documentation from `zope.interface`. So I fixed the pinning and also explicitly pin Sphinx to 3.5.x for now.
Technically speaking the second action is sufficient to fix the first error, but I keep the dedicated solution because it improves the documentation in my opinion.
This situation could be fixed by not requiring `repoze.sphinx.autointerface`, but this is possible only if we remove `zope.interface` from Certbot. Luckily I started the work few days ago ;).
* Remove explicit lexer call in certbot-dns-google doc builds.
* Write a valid JSON file in the documentation
* Apply constraints to sphinx build environments
* Pin Sphinx to 3.5.4
* Update dependencies
* Pin traitlets
Fixes https://github.com/certbot/certbot/issues/8781.
This PR makes our test farm tests into a normal package so it and its dependencies can be tracked and installed like our other packages.
Other noteworthy changes in this PR:
* Rather than continuing to place logs in your CWD, they're placed in a temporary directory that is printed to the terminal.
* `tests/letstest/auto_targets.yaml` was deleted rather than renamed because the file is no longer used.
* make a letstest package
* remove deleted deps
* fix letstest install
* add __init__.py
* call main
* Explicitly mention activating venv
* rerename file
* fix version.py path
* clarify "this"
* Use >= instead of caret requirement
Fixes#8802.
Also removed the unused `kgs` cruft while I was here, since it's leftover from the [initial release commit](3c08b512c3) and I'm pretty sure we don't use that anymore.
I think this PR improves tools/snap/build_remote.py's output in a number of ways such as:
* Logs of snap builds were being deleted because they weren't being copied out of the temporary directory added in https://github.com/certbot/certbot/pull/8719.
* The lock should now always be acquired before printing output when multiple processes are running which helps prevent processes mixing their output with each other.
* Output is never buffered which ensures that repeated calls to `print` from the same process while it holds the output lock is kept together.
* The case where we printed output about the "chroot problem" and stopped retrying the build has been deleted because with the fix in https://github.com/certbot/certbot/pull/8719, we should be able to recover in this case.
* If the build failed for any reason, we dump as much output about the problem as we can. I think most times we won't need to read this output, but I personally prefer it being there in case we want it for some reason. Due to this change, I also simplified `_build_snap` and `_dump_results` a bit since `_build_snap` handles printing logs as needed.
* print more output
* lock when printing output
* clarify purpose of lock
* preserve logfiles
* python better
* consistently flush output
* remove workspaces dict
* rename variable
* remove unused variable
* don't use all which exits early
* fix typo
Fixes#8773
I took option 2 from the issue mentionned above (importing `typing-extensions` on dev dependencies) to avoid modifying certbot runtime requirements given that what needs to be added is useful for mypy only.
I did not change the Python version used to execute the linting and mypy on the standard tests, given that the tox `docker_dev` target already checks if the development environment is working for Python < 3.8.
Fixes#8425
This PR upgrades mypy to the latest version available, 0.812.
Given the advanced type inference capabilities provided by this newer version, this PRs also fixes various type inconsistencies that are now detected. Here are the non obvious changes done to fix types:
* typing in mixins has been solved using `Protocol` classes, as recommended by mypy (https://mypy.readthedocs.io/en/latest/more_types.html#mixin-classes, https://mypy.readthedocs.io/en/stable/protocols.html)
* `cast` when we are playing with `Union` types
This PR also disables the strict optional checks that have been enable by default in recent versions of mypy. Once this PR is merged, I will create an issue to study how these checks can be enabled.
`typing.Protocol` is available only since Python 3.8. To keep compatibility with Python 3.6, I try to import the class `Protocol` from `typing`, and fallback to assign `object` to `Protocol` if that fails. This way the code is working with all versions of Python, but the mypy check can be run only with Python 3.8+ because it needs the protocol feature. As a consequence, tox runs mypy under Python 3.8.
Alternatives are:
* importing `typing_extensions`, that proposes backport of newest typing features to Python 3.6, but this implies to add a dependency to Certbot just to run mypy
* redesign the concerned classes to not use mixins, or use them differently, but this implies to modify the code itself even if there is nothing wrong with it and it is just a matter of instructing mypy to understand in which context the mixins can be used
* ignoring type for these classes with `# type: ignore` but we loose the benefit of mypy for them
* Upgrade mypy
* First step for acme
* Cast for the rescue
* Fixing types for certbot
* Fix typing for certbot-nginx
* Finalize type fixes, configure no optional strict check for mypy in tox
* Align requirements
* Isort
* Pylint
* Protocol for python 3.6
* Use Python 3.9 for mypy, make code compatible with Python 3.8<
* Pylint and mypy
* Pragma no cover
* Pythonic NotImplemented constant
* More type definitions
* Add comments
* Simplify typing logic
* Use vararg tuple
* Relax constraints on mypy
* Add more type
* Do not silence error if target is not defined
* Conditionally import Protocol for type checking only
* Clean up imports
* Add comments
* Align python version linting with mypy and coverage
* Just ignore types in an unused module
* Add comments
* Fix lint
