We observed recently several unexpected behavior during the execution of snap jobs in Azure. In particular it seems that `snapcraft remote-build` is tending to reattach to the latest builds on Launchpad triggered by the nightly builds on master, independently from the actual branch, status of the code, or targeted architectures.
Primarily if the builds on Launchpad are stalled for some reason, it blocks effectively any other Azure snap jobs until someone manually cancel the builds on Launchpad. Secondarily it means that the outcome of the builds may be inconsistent, because they can be the result of a build for the master source even if you are on a PR that modifieds these sources (including `snapcraft.yaml`).
After digging in `snapcraft` source code, I realized that the signature computed to understand if a build should be resumed, is not based one some hashes against the snapcraft working directory content, but is simply a hash of the working directory absolute path *itself*. It means that every builds triggered from the working directory `/my/path/certbot` for instance, are recognized as the same unique build on Launchpad side, and may be resumed if they already exist, and so independently from the source code, `snapcraft.yaml` or targeted archs.
For the record, relevant parts in `snapcraft` source code:
82024d3748/snapcraft/project/_project.py (L44)82024d3748/snapcraft/project/_project.py (L86-L89)82024d3748/snapcraft/cli/remote.py (L128-L132)
This PR makes effectively the resume build mechanism effectively a noop by moving the source code first in a temporary directory with random name before running `snapcraft remote-build`. This way the signature is never the same and builds are always recognized as brand new builds.
* Invalidate snapcraft remote-build cache by using a temporary workspace.
* Capture one more state in the build
* Precise the certificate naming convention mechanism in a note.
* Add certificate name convention in user guide, refer to it in compatibility page.
* Update certbot/docs/compatibility.rst
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot/docs/using.rst
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot/docs/using.rst
Co-authored-by: alexzorin <alex@zor.io>
* Improve the note about naming conventions
Co-authored-by: alexzorin <alex@zor.io>
While working on #8640, I realized that there were some hidden circular dependencies in certbot._internal.cli package. Then cerbot could break if the order of these imports changes.
This PR fixes that and apply isort on top of the result.
* Kill snapcraft build when a "Chroot problem" is encountered
* Display specific helper for "Chroot problem" status and cancel retry mechanism in this case.
* Isolate build tmp directories
* Configure XDG_CACHE_HOME
* Kill snapcraftctl with chroot problem is encountered
Fixes#8427
This PR converts the Python 2 types hints into Python 3 types annotations. I have used the project https://github.com/ilevkivskyi/com2ann which has been designed for that specific purpose and did that very well.
The only remaining things to do were to fix broken type hints that became wrong code after migration, and to fix lines too long with the new syntax.
* Raw execution of com2ann
* Fixing broken type annotations
* Cleanup imports
There are still some left, but the `modification_check` test fails. Some are still in `tools`, and they can probably be removed as well. `with_statement` was introduced officially in Python 2.5, so there's really old stuff in the code base.
Fixes https://github.com/certbot/certbot/issues/8690.
After this PR, we'll let the release script make its automated changes to certbot-auto as part of the 1.14.0 release and then never make any code changes to certbot-auto ever again!
* disable upgrades on debian
* update test_leauto_upgrades.sh
* update changelog
* revoke: try determine the server automatically
When revoking via --cert-name, use the server from the lineage (unless
overriden by the CLI).
* RenewableCert.storage might be None
* guard against an empty lineage server
* nginx: authenticate all matching vhosts for HTTP01
Previously, the nginx authenticator would set up the HTTP-01 challenge
response on a single HTTP vhost which matched the challenge domain.
The nginx authenticator will now set the challenge response on every
vhost which matches the challenge domain, including duplicates and HTTPS
vhosts.
This makes the authenticator usable behind a CDN where all origin
traffic is performed over HTTPS and also makes the authenticator work
more reliably against "invalid" nginx configurations, such as those
where there are duplicate vhosts.
* some typos
* dont authenticate the same vhost twice
One vhost may appear in both the HTTP and HTTPS vhost lists. Use a set()
to avoid trying to mod the same vhost twice.
* fix type annotations
* rewrite changelog entry
Fixing #8634. It's my first time contributing to this repository, if there's something wrong please let me know.
Before this fix
```
$ python3 extract_changelog.py 1.12.0
...
### Fixed
* Fixed the apache component on openSUSE Tumbleweed which no longer provides
an apache2ctl symlink and uses apachectl instead.
* Fixed a typo in `certbot/crypto_util.py` causing an error upon attempting `secp521r1` key generation
More details about these changes can be found on our GitHub repo.
```
After this fix
```
$ python3 extract_changelog.py 1.12.0
...
### Fixed
* Fixed the apache component on openSUSE Tumbleweed which no longer provides
an apache2ctl symlink and uses apachectl instead.
* Fixed a typo in `certbot/crypto_util.py` causing an error upon attempting `secp521r1` key generation
More details about these changes can be found on our GitHub repo.
```
There is some code in [`_paths_parser`](ae3ed200c0/certbot/certbot/_internal/cli/paths_parser.py (L17-L34)) which has the effect of varying the value type of `config.cert_path` and `config.key_path` based on the CLI verb. When the verb is `revoke`, the type is a tuple `(path: str, contents: bytes)`, otherwise it is a single `str` representing the file path. (I wasn't able to find a written reason as to why it works this way).
This commit removes that special `revoke` case and ensures it is always a `str`.
Why change it now?
I am trying to write some changes and there's some code in `cert_manager` which only works if the verb is `revoke`, you hack `config.cert_path` to be a tuple beforehand, or you [(not actually in `master`) try sniff for the value type](49911afaa6/certbot/certbot/_internal/cert_manager.py (L224-L227)). I have a bad feeling about such workarounds. I would prefer to just make these variables simpler to use, but I'm open to opinions.
In addition to the test suites, I've manually tested `revoke` (including by `--key-path`) and `install`. Are there other places I may have missed?
Unblocks #8636 and #8671.
* docs: rewrite "Revoking certificates"
- `--cert-name` is supported since a long time ago
- `--delete-after-revoke` is default
- Mention that non-default `--server` must be specified
- Document difference between acme key/cert key revocation methods
- Reshuffle text to keep more important things earlier
* minor edits
* remove revocation note
* remove "preauthorization" revocation method
* rewrite deletion note
Fixes#8680.
We seem to have no existing testing code anywhere in this vicinity, so figured I'd get this up quickly then work on that. Manual tests (renew staging certificate, should allow it; renew non-staging cert as staging, should error) passed.
* Remove check for 'fake' in issuer name when renewing certs
* Change fake issuer name to make sure we're not relying on it anywhere
This PR deprecates the certbot-auto specific CLI flags, in the perspective of removing them in a future release as said in #8483.
* Deprecate certbot-auto specific flags
* Update changelog
* Clean tests
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Since Ubuntu 18.04 there is python3-certbot-apache which should be the recommended version.
The Debian package names should probably be updated accordingly.
Fixes https://github.com/certbot/certbot/issues/8494.
I left the `six` dependency pinned in `tests/letstest/requirements.txt` and `tools/oldest_constraints.txt` because `six` is still a transitive dependency with our current pinnings.
The extra moving around of imports is due to me using `isort` to help me keep dependencies in sorted order after replacing imports of `six`.
* remove some six usage in acme
* remove six from acme
* remove six.add_metaclass usage
* fix six.moves.zip
* fix six.moves.builtins.open
* six.moves server fixes
* 's/six\.moves\.range/range/g'
* stop using six.moves.xrange
* fix urllib imports
* s/six\.binary_type/bytes/g
* s/six\.string_types/str/g
* 's/six\.text_type/str/g'
* fix six.iteritems usage
* fix itervalues usage
* switch from six.StringIO to io.StringIO
* remove six imports
* misc fixes
* stop using six.reload_module
* no six.PY2
* rip out six
* keep six pinned in oldest constraints
* fix log_test.py
* update changelog
* Update cli.ini
Sharing back some extended examples I desired, did not find, and derived on my own
* Update cli.ini
Alex,
ok - simplified as requested
Matt
* Update cli.ini
removed trailing quote on line 32
* Update certbot/examples/cli.ini
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot/examples/cli.ini
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot/examples/cli.ini
Co-authored-by: alexzorin <alex@zor.io>
* remove stray newline
Co-authored-by: alexzorin <alex@zor.io>
Fixes https://github.com/certbot/certbot/issues/7913.
I only added the deprecation warning to `certbot.tests.util` because that's the only place where I think someone could be using the `mock` module through our API.
* remove external mock from acme
* update Certbot's mock usage
* remove mock dependency in plugins
* remove external mock from compatibility test
* add changelog entry
