[As discussed in Mattermost](https://opensource.eff.org/eff-open-source/pl/yhtp4qu4zpfczm5wxmzxhndrto), our Apache test farm tests are failing because the CA certificate in the old version of boulder we have pinned expired over the weekend. This PR fixes that by running a local Pebble instance instead of an external boulder instance.
* switch from external boulder to local pebble
* add --http-01-port to run_acme_server
This PR is an alternative to #7125.
Instead of disabling the strict mode on Pebble, this PR fixes the JWS payloads regarding RFC 8555 to be compliant, and allow certbot to work with Pebble v2.1.0+.
* Fix acme compliance to RFC 8555.
* Working mixin
* Activate back pebble strict mode
* Use mixin for type
* Update dependencies
* Fix also in fields_to_partial_json
* Update pebble
* Add changelog
* Fix account_tests
* Fix hook executable test
* Remove the temporary decorator @broken_on_windows
* Fix util_test
* No broken unit test on Windows anymore
* More elegant mock
* Fix context manager
* Fix lint
* Fix mypy
* Adapt coverage
* Corrections
* Fix lint
* Adapt coverage
* Update certbot/tests/compat/filesystem_test.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update util_test.py
* Fix pylint
* Forbid os.access
* Update os_test.py
* Update os.py
* Fix lint
* Update filesystem.py
* Update filesystem.py
* Update filesystem.py
* Update os.py
* Start fixing tests
* Platform independent hooks
* Fix probe fd close
* Add broken_on_windows for integration tests
* Fix a lot of tests
* Use a python hook script, to prepare cross-platform
* New approach to be compliant with Linux and Windows on hook scripts
* New tests fixed
* Test for permissions on Windows
* Permissions comparison for Windows
* No broken tests in certbot core anymore
* Change mode
* Specific config for appveyor
* Use forked pebble for now
* Various fixes
* Assert file permissions for world on private keys
* Clean code
* Fix several things
* Add integration target
* Optimize integration env
* Re-enable all AppVeyor envs
* Use again official pebble
* Update pebble_artifacts.py
* Set PYTEST_ADDOPTS silently
* Update appveyor.yml
* Pin pywin32 for tests, give a minimal requirement for certbot.
* Remove injection of nginx in PATH
* Clean debug code
* Various cleanup, ensure to remove workspace after tests
* Update tox target
* Improve assertions. Control the keyword echoed in hooks
* Fix for virtualenv on Python 3.7.4 for Windows
* Update certbot-ci/certbot_integration_tests/certbot_tests/assertions.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Add conditionally pywin in certbot-ci like in certbot
* Implement a logic, miss the private key of pebble
* Complete process
* Fix nginx cert path
* Check conditionnally docker
* Update gitignore, fix apacheconftest
* Full object
* Carriage return
* Work in progress
* Move to official v2.1.0 of pebble
* Fix name
* Update acme_server.py
* Link things together with new version of pebble
* Plug the logic to tests
* Update config
* Reinitiate config
* Add OCSP config to pebble
* Working.
* Simplify logic
* Clean code
* Use forked pebble for now
# Conflicts:
# certbot-ci/certbot_integration_tests/utils/pebble_artifacts.py
* Move full logic of mock at the acme server config
* Continue work
* Finish fixing the date parsing
* Update module name
* Use again official pebble
* Activate mock OCSP server
* Clean code
* Update pebble_artifacts.py
* Remove OCSP stale test
* Add executable permissions
* Clean code
* Update setup.py
* Simplify code
* On-demand import of pebble_ocsp_server
* Revert "Remove OCSP stale test"
This reverts commit 2e4c985b42.
# Conflicts:
# certbot-ci/certbot_integration_tests/utils/misc.py
* Fix for virtualenv on Python 3.7.4 for Windows
* Update acme_server.py
This PR is a part of the actions necessary to make Certbot-CI work on Windows, in order to execute the integration tests on this platform.
Following #7156, this PR changes how the integration tests are setup against Pebble to not need Docker anymore.
As a reminder, one can check #7156 and letsencrypt/pebble#240 to see the rationale about why using Docker is a problem to run the integration tests on Windows.
Basically, this PR executes directly Pebble using its executable, since it is build using Go, and Go produces self-contained executable that can run without any installation on Linux and on Windows. During the integration tests setup, Certbot-CI will get the Pebble (and Challtestsrv) executables for the defined target version on the GitHub releases. The binaries are persisted on the filesystem, so it is not needed to download them again on the second integration tests execution. Nonetheless, we are talking about 20MB of executables.
Since the setup needs to hold a state, I also took this occasion to refactor the acme_server, in order to use on object oriented approach and improve the readability/maintainability.
Once this PR and #7156 are merged, Docker will not be needed anymore for the main integration tests usecase, that is to use Pebble.
* Complete process
* Fix nginx cert path
* Check conditionnally docker
* Update gitignore, fix apacheconftest
* Full object
* Carriage return
* Move to official v2.1.0 of pebble
* Fix name
* Update acme_server.py
* Relaunch CI
* Update certbot-ci/certbot_integration_tests/utils/acme_server.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot-ci/certbot_integration_tests/utils/acme_server.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update docstring
* Update documentation
* Configure a stdout to ACMEServer
* Map all process through defined stdout
* Remove unused variable
* Handle using signals
* Use failsafe entering context
* Remove failsafe rmtree, that is not needed anymore
