Fixes#7268
I removed the reference to automatically selecting which ACME protocol we use, since at some point we'll want to rip out the non-spec-compliant ACMEv1 code.
Python 2 is going to get harder and harder to install locally so I don't think we should assume/require devs to have it installed.
This PR builds on #7905 so our developer guide only has people use Python 3.
Part of #7886.
This PR conditionally installs `mock` in `certbot-dns-*/setup.py` based on setuptools version and python version, when possible. It then updates the tests to use `unittest.mock` when `mock` isn't available.
* Do not require mock in Python 3 in certbot-dns modules
* update changelog
* error when trying to build wheels with old setuptools
* add type: ignores
Part of #7886.
This PR conditionally installs mock in `apache/setup.py` based on setuptools version and python version, when possible. It then updates `apache` tests to use `unittest.mock` when `mock` isn't available.
* Conditionally install mock in apache
* error out on newer python and older setuptools
* error when trying to build wheels with old setuptools
* use unittest.mock when third-party mock isn't available in apache, with no cover and type ignore
This PR is exactly the same as #7895, but know we know a little bit more about what was going on with `mypy`.
Part of #7886.
This PR conditionally installs mock in `certbot/setup.py` based on setuptools version and python version, when possible. It then updates `certbot` tests to use `unittest.mock` when `mock` isn't available.
* Conditionally install mock in certbot
* use unittest.mock when third-party mock isn't available in certbot
* Add type:ignores because of https://github.com/python/mypy/issues/1153
* error out on newer python and older setuptools
* error when trying to build wheels with old setuptools
Part of #7886.
This PR conditionally installs mock in `acme/setup.py` based on setuptools version and python version, when possible. It then updates `acme` tests to use `unittest.mock` when `mock` isn't available.
Now with `type: ignore` as appropriate. Once the "future steps" of #7886 are finished, and mypy is on Python 3, the `pragma no cover`s and `type ignore`s will be gone.
* Conditionally install mock in acme
* error out on newer python and older setuptools
* error when trying to build wheels with old setuptools
* use unittest.mock when third-party mock isn't available in acme, with no cover and type ignore
This PR fixes the Travis failures that can be seen https://travis-ci.com/certbot/certbot/builds/160258644. Running the tests locally, it looks like Ubuntu has started shutting down the 19.04 repos which makes sense as this release has been EOL'd. See https://wiki.ubuntu.com/Releases.
I have the full suite including the test farm tests running at https://travis-ci.com/github/certbot/certbot/builds/160269969 with this change.
The issue of adding 19.10 to our test farm tests is tracked by #7851. I think that issue is important and it's in our current milestone, but I'd personally rather get our tests passing for now and try to expand them to run on other systems later.
* Revert "Do not require mock in Python 3 in certbot module (#7895)"
This reverts commit 77871ba71c.
* Revert "Do not require mock in Python 3 in acme module (#7894)"
This reverts commit cd0acf5dcc.
Part of #7886.
This PR conditionally installs mock in `certbot/setup.py` based on setuptools version and python version, when possible. It then updates `certbot` tests to use `unittest.mock` when `mock` isn't available.
* Conditionally install mock in certbot
* use unittest.mock when third-party mock isn't available in certbot
* Add type:ignores because of https://github.com/python/mypy/issues/1153
* error when trying to build wheels with old setuptools
Part of #7886.
This PR conditionally installs mock in acme/setup.py based on setuptools version and python version, when possible. It then updates acme tests to use unittest.mock when mock isn't available.
* Conditionally install mock in acme
* use unittest.mock when third-party mock isn't available in acme
* error when trying to build wheels with old setuptools
* Fix dangerous default argument
* Remove unused imports
* Remove unnecessary comprehension
* Use literal syntax to create data structure
* Use literal syntax instead of function calls to create data structure
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Fixes#7857.
* stop using urllib2 in test farm tests
* use six for urllib instead
* remove fabric lcd usage
* correct lcd removal
* remove fabric cd
* convert some remote calls to v2
* move more cxns to v2
* get run working with prefix
* get sudo commands working
* remove final fabric v1 references including local
* update requirements and README
* add new venv to gitignore
* update version used in travis
* remove deploy_script unused kwargs
* fix killboulder implementation so I can test creating a new boulder server
* hardcode the gopath due to broken env manamagement in fabric2
* Update letstest readme
* move the comment about hardcoding the ggopath
* catch BaseException instead of Exception
* work around fabric #2007
* use connections as context managers to ensure they're closed
* remove reference to virtualenv
Translate a proxy specified by an environment variable ("http_proxy"
or "HTTP_PROXY") into options recognized by "openssl ocsp". Support
is limited to HTTP proxies which don't require authentication.
Fixes#6150
Fixes#7875 .
After [this comment](https://github.com/certbot/certbot/issues/7875#issuecomment-608145208) and evaluating the options, I opted to go with `stricttextualmsg`, as required by RFC 8555. Reasoning is that the ACME v1 code path (via OpenSSL) produces a `fullchain_pem` which satisfies `stricttextualmsg`, so we don't need to be more generous than that.
One downside of the `re` approach is that it doesn't seem capable of capturing repeating group matches. As a result, it matches each certificate individually, silently passing over any data in between the encapsulation boundaries, such as explanatory text, which is prohibited by RFC 8555.
It would be ideal to raise an error when encountering such a non-conformant chain, but we'd need to create a mini-parser to do it, I think.
* Fix fullchain parsing for CRLF chains.
fullchain parsing now works in two passes:
1. A first pass which is generous with what it accepts - basically
preeb(CERTIFICATE)+anything+posteb(CERTIFICATE). This determines
the boundaries for each certificate.
2. A second pass which normalizes (by parsing and re-encoding) each
certificate found in the first pass.
* typo in docstring
* remove redundant group in regex
* can't use assertRaisesRegex until py27 is gone
* acme: socket timeout for HTTP standalone servers
Adds a default 30 second timeout to the StreamRequestHandler for clients
connecting to standalone HTTP-01 servers. This should prevent most cases
of an idle client connection from preventing the standalone server from
shutting down.
Fixes#7386
* use idiomatic kwargs default value
* move HTTP01Server lower to fix mypy forward ref.
* fix test crash on macOS due to socket double-close
* maybe its not an OSError?
* disable coverage check on useless branch
Fixes#7594.
Removes the code asking interactively if the user would like to add a redirect.
* Remove interactive redirect ask
* display.enhancements is no longer used, so remove it.
* update changelog
* remove references to removed display.enhancements
* add redirect_default flag to enhance_config to conditionally set default for redirect value
* Update default in help text.
* Load apacheconfig dependency, gate behind flag
* Bump apacheconfig dependency to latest version and install dev version of apache for coverage tests
* Move augeasnode_test tests to more generic parsernode_test
* Revert "Move augeasnode_test tests to more generic parsernode_test"
This reverts commit 6bb986ef78.
* Mock AugeasNode into DualNode's place, and run augeasnode tests exclusively on AugeasNode
* Don't calculate coverage for skeleton functions
* clean up helper function in augeasnode_test
Fixes#7350.
This PR changes the parsed modules from a `set` to a `dict`, with the filepath argument as the value. Accordingly, after calling `enable_mod` to enable `ssl_module`, modules now need to be re-parsed, so call `reset_modules`.
* Add mechanism for selecting apache config file, based on work done in #7191.
* Check OpenSSL version
* Remove os imports
* debian override still needs os
* Reformat remaining apache tests with modules dict syntax
* Clean up more apache tests
* Switch from property to method for openssl and add tests for coverage.
* Sometimes the dict location will be None in which case we should in fact return None
* warn thoroughly and consistently in openssl_version function
* update tests for new warnings
* read file as bytes, and factor out the open for testing
* normalize ssl_module_location path to account for being relative to server root
* Use byte literals in a python 2 and 3 compatible way
* string does need to be a literal
* patch builtins open
* add debug, remove space
* Add test to check if OpenSSL detection is working on different systems
* fix relative test location for cwd
* put </IfModule> on its own line in test case
* Revert test file to status in master.
* Call augeas load before reparsing modules to pick up the changes
* fix grep, tail, and mod_ssl location on centos
* strip the trailing whitespace from fedora
* just use LooseVersion in test
* call apache2ctl on debian systems
* Use sudo for apache2ctl command
* add check to make sure we're getting a version
* Add boolean so we don't warn on debian/ubuntu before trying to enable mod_ssl
* Reduce warnings while testing by setting mock _openssl_version.
* Make sure we're not throwing away any unwritten changes to the config
* test last warning case for coverage
* text changes for clarity
This PR builds on #7657 and cleans up additional unnecessary pylint comments and some stray comments referring to pylint: disable comments that have been deleted that I didn't notice in my review of that PR.
* Remove stray pylint link.
* Cleanup more pylint comments
* Cleanup magic_typing imports
* Remove unneeded pylint: enable comments
This PR is an alternative to #7125.
Instead of disabling the strict mode on Pebble, this PR fixes the JWS payloads regarding RFC 8555 to be compliant, and allow certbot to work with Pebble v2.1.0+.
* Fix acme compliance to RFC 8555.
* Working mixin
* Activate back pebble strict mode
* Use mixin for type
* Update dependencies
* Fix also in fields_to_partial_json
* Update pebble
* Add changelog
This PR is the first part of work described in #6724.
It reintroduces the tls-alpn-01 challenge in `acme` module, that was introduced by #5894 and reverted by #6100. The reason it was removed in the past is because some tests showed that with `1.0.2` branch of OpenSSL, the self-signed certificate containing the authorization key is sent to the requester even if the ALPN protocol `acme-tls/1` was not declared as supported by the requester during the TLS handshake.
However recent discussions lead to the conclusion that this behavior was not a security issue, because first it is coherent with the behavior with servers that do not support ALPN at all, and second it cannot make a tls-alpn-01 challenge be validated in this kind of corner case.
On top of the original modifications given by #5894, I merged the code to be up-to-date with our `master`, and fixed tests to match recent evolution about not displaying the `keyAuthorization` in the deserialized JSON form of an ACME challenge.
I also move the logic to verify if ALPN is available on the current system, and so that the tls-alpn-01 challenge can be used, to a dedicated static function `is_available` in `acme.challenge.TLSALPN01`. This function is used in the related tests to skip them, and will be used in the future from Certbot plugins to trigger or not the logic related to tls-alpn-01, depending on the OpenSSL version available to Python.
* Reimplement TLS-ALPN-01 challenge and standalone TLS-ALPN server from #5894.
* Setup a class method to check if tls-alpn-01 is supported.
* Add potential missing parameter in validation for tls-alpn
* Improve comments
* Make a class private
* Handle old versions of openssl that do not terminate the handshake when they should do.
* Add changelog
* Explicitly close the TLS connection by the book.
* Remove unused exception
* Fix lint
Fixes#7835
I had to mock out `get_serial_from_cert` to keep a test from failing, because `cert_path` was mocked itself in `test_report_human_readable`.
Also, I kept the same style for the serial number as the recent Let's Encrypt e-mail: lowercase hexadecimal without a `0x` prefix and without colons every 2 chars. Shouldn't be a problem to change the format if required.
Fixes#5484
This PRs makes Certbot expose two new environment variables in the auth and cleanup hooks of the `manual` plugin:
* `CERTBOT_REMAINING_CHALLENGES` contains the number of challenges that remain after the current one (so it equals to 0 when the script is called for the last challenge)
* `CERTBOT_ALL_DOMAINS` contains a comma-separated list of all domains concerned by a challenge for the current certificate
With these variables, an hook script can know when it is run for the last time, and then trigger appropriate finalizers for all challenges that have been executed. This will be particularly useful for certificates with a lot of domains validated with DNS-01 challenges: instead of waiting on each hook execution to check that the relevant DNS TXT entry has been inserted, these waits can be avoided thanks to the latest hook verifying all domains in one run.
* Inject environment variables in manual scripts about remaining challenges
* Adapt tests
* Less variables and less lines
* Update manual.py
* Update manual_test.py
* Add documentation
* Add changelog
