Fixes#5490.
There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup.
To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST).
After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet.
The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem.
Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here.
* break lockstep between packages
* Use per package requirements files
* add local oldest requirements files
* update tox.ini
* work with dev0 versions
* Install requirements in separate step.
* don't error when we don't have requirements
* install latest packages in editable mode
* Update .travis.yml
* Add reminder comments
* move dev to requirements
* request acme[dev]
* Update pip_install documentation
* Drop support for EOL Python 2.6
* Use more helpful assertIn/NotIn instead of assertTrue/False
* Drop support for EOL Python 3.3
* Remove redundant Python 3.3 code
* Restore code for RHEL 6 and virtualenv for Py2.7
* Revert pipstrap.py to upstream
* Merge py26_packages and non_py26_packages into all_packages
* Revert changes to *-auto in root
* Update by calling letsencrypt-auto-source/build.py
* Revert permissions for pipstrap.py
* Make leauto_upgrades do a real upgrade
* Cleanup vars and output
* Sleep until the server is ready
* add simple_http_server.py
* Use a randomly assigned port
* s/realpath/readlink
* wait for server before getting port
* s/localhost/all interfaces
* Add tools/merge_requirements.py
* Revert "Fix oldest tests by pinning Google DNS deps (#5000)"
This reverts commit f68fba2be2.
* Add tools/oldest_constraints.txt
* Remove oldest constraints from tox.ini
* Rename dev constraints file
* Update tools/pip_install.sh
* Update install_and_test.sh
* Fix pip_install.sh
* Don't cat when you can cp
* Add ng-httpsclient to dev constraints for oldest tests
* Bump tested setuptools version
* Update dev_constraints comment
* Better document oldest dependencies
* test against oldest versions we say we require
* Update dev constraints
* Properly handle empty lines
* Update constraints gen in pip_install
* Remove duplicated zope.component
* Reduce pyasn1-modules dependency
* Remove blank line
* pin back google-api-python-client
* pin back uritemplate
* pin josepy for oldest tests
* Undo changes to install_and_test.sh
* Update install_and_test.sh description
* use split instead of partition
pip install generates a lot of lines of output that make it harder to see what
tox is running in general. This adds the -q flag to pip install.
At the same time, add `set -x` in install_and_test.sh and pip_install.sh so they
echo the commands they are running. This makes it a little clearer what's going
on in tests.
I didn't put `set -x` at the top or in the shebang, because moving it lower lets
us avoid echoing some of the messy if/then setup statements in these scripts,
which focussed attention on the pip install command.
* Use pipstrap to install a good version of pip
* Use pytest in cb-auto tests
* Remove nose usage in auto_test.py
* remove nose dev dep
* use pytest in test_tests
* Use pytest in tox
* Update dev dependency pinnings
* remove nose multiprocess lines
* Use pytest for coverage
* Use older py and pytest for old python versions
* Add test for Error.__str__
* pin pytest in oldest test
* Fix tests for DNS-DO plugin on py26
* Work around bug for Python 3.3
* Clarify dockerfile comments
* Add tools/pip_constraints.txt to pin all Python dependencies
* Use tools/pip_constraints.txt in tools/pip_install.sh
* Install dnsmadeeasy extras in dnsmadeeasy plugin
Introduce a plugin that automates the process of completing a dns-01 challenge by creating, and subsequently removing, TXT records using RFC 2136 Dynamic Updates (a.k.a. nsupdate).
This plugin has been tested with BIND, but may work with other RFC 2136-compatible DNS servers, such as PowerDNS.
Implement an Authenticator which can fulfill a dns-01 challenge using
the LuaDNS API. Applicable only for domains using LuaDNS for DNS.
Testing Done:
* `tox -e py27`
* `tox -e lint`
* Manual testing:
* Used `certbot certonly --dns-luadns -d`, specifying a
credentials file as a command line argument. Verified that a
certificate was successfully obtained without user interaction.
* Negative testing:
* Path to non-existent credentials file.
* Credentials file with unsafe permissions (644).
* Path to credentials file without an email.
* Path to credentials file with an invalid email.
* Path to credentials file without a token.
* Path to credentials file with an invalid token.
* Domain name not registered to LuaDNS account.
Implement an Authenticator which can fulfill a dns-01 challenge using
the DNS Made Easy API. Applicable only for domains using DNS Made Easy.
Testing Done:
* `tox -e py27`
* `tox -e lint`
* Manual testing:
(`http://api.sandbox.dnsmadeeasy.com/V2.0` used as the
`api_endpoint` for all manual testing)
* Used `certbot certonly --dns-dnsmadeeasy -d`, specifying a
credentials file as a command line argument. Verified that a
certificate was successfully obtained without user interaction.
* Negative testing:
* Path to non-existent credentials file.
* Credentials file with unsafe permissions (644).
* Path to credentials file with an invalid API key.
* Path to credentials file with a malformed API key.
* Path to credentials file with an invalid Secret key.
* Path to credentials file with a malformed Secret key.
* Domain name not registered to DNS Made Easy account.
This change renames certbot-route53 to certbot-dns-route53 and updates
the package's setup.py file to maintain backwards compatibility.
Testing Done:
* Run `certbot` with `-a certbot-route53:auth`, verify the plugin runs.
* Run `certbot` with `--dns-route53`, verify the plugin runs.
* Remove dependency on git from pip_install.sh.
Using git allowed this file to continue to work even if it was moved to another
directory. This slight increase in robustness wasn't worth it though as it
broke our development Dockerfile (see #4703), the certbot website's Dockerfile
(see certbot/website#226), and our test farm tests (see
certbot/tests/letstest/scripts/test_apache2.sh for an example that calls
tools/venv.sh without installing git). Rather than continuing to find and patch
these things, let's just allow this script to fail if it's moved rather than
propagating the git dependency all over the place.
* Add readlink.py.
This is the equivalent of `readlink -f` on many Linux systems. This is useful
as there are often differences in readlink on different platforms.
* Use readlink.py in pip_install.sh.
This allows us to work around differences in readlink on macOS.
This change refactors the release script to handle subpackages which are
not bundled as a part of cerbot-auto.
The script now allows developers to define subpackages as either being
included in certbot-auto, or not.
The script then uses one of three sets of subpackages for each operation:
* The version number is updated for all non-certbot subpackages
(and certbot itself is handled separately)
* sdists and wheels are created for all non-certbot subpackages
(and certbot itself is handled separately)
* Testing is performed for all subpackages
* Hashes are pinned for certbot-auto subpackages (including certbot)
Implement an Authenticator which can fulfill a dns-01 challenge using
the NS1 DNS API. Applicable only for domains using NS1 DNS.
Testing Done:
* `tox -e py27`
* `tox -e lint`
* Manual testing:
* Used `certbot certonly --dns-nsone -d`, specifying a
credentials file as a command line argument. Verified that a
certificate was successfully obtained without user interaction.
* Used `certbot certonly --dns-nsone -d`, without specifying a
credentials file as a command line argument. Verified that the
user was prompted and that a certificate was successfully
obtained.
* Used `certbot certonly -d`. Verified that the user was prompted for
a credentials file after selecting dnsimple interactively and that
a certificate was successfully obtained.
* Used `certbot renew --force-renewal`. Verified that certificates
were renewed without user interaction.
* Negative testing:
* Path to non-existent credentials file.
* Credentials file with unsafe permissions (644).
* Path to credentials file with an invalid token.
* Path to credentials file without a token.
* Domain name not registered to NS1 account.
Implement an Authenticator which can fulfill a dns-01 challenge using
the DNSimple DNS API. Applicable only for domains using DNSimple DNS.
Testing Done:
* `tox -e py27`
* `tox -e lint`
* Manual testing:
* Used `certbot certonly --dns-dnsimple -d`, specifying a
credentials file as a command line argument. Verified that a
certificate was successfully obtained without user interaction.
* Used `certbot certonly --dns-dnsimple -d`, without specifying a
credentials file as a command line argument. Verified that the
user was prompted and that a certificate was successfully
obtained.
* Used `certbot certonly -d`. Verified that the user was prompted for
a credentials file after selecting dnsimple interactively and that
a certificate was successfully obtained.
* Used `certbot renew --force-renewal`. Verified that certificates
were renewed without user interaction.
* Negative testing:
* Path to non-existent credentials file.
* Credentials file with unsafe permissions (644).
* Path to credentials file with an invalid token.
* Path to credentials file without a token.
* Domain name not registered to DNSimple account.
Implement an Authenticator which can fulfill a dns-01 challenge using
the CloudXNS DNS API. Applicable only for domains using CloudXNS DNS.
Testing Done:
* `tox -e py27`
* `tox -e lint`
* Manual testing:
* Used `certbot certonly --dns-cloudxns -d`, specifying a
credentials file as a command line argument. Verified that a
certificate was successfully obtained without user interaction.
* Used `certbot certonly --dns-cloudxns -d`, without specifying a
credentials file as a command line argument. Verified that the
user was prompted and that a certificate was successfully
obtained.
* Used `certbot certonly -d`. Verified that the user was prompted for
a credentials file after selecting cloudxns interactively and that
a certificate was successfully obtained.
* Used `certbot renew --force-renewal`. Verified that certificates
were renewed without user interaction.
* Negative testing:
* Path to non-existent credentials file.
* Credentials file with unsafe permissions (644).
* Domain name not registered to CloudXNS account.
* Add an account deactivate utility script.
This is handy if you created an account with a tool other than Certbot, and want
to deactivate the account.
* Move deactivate.py to tools.
* Add test for ConflictError.
* Fix lint error.
* Document how to set server.
Implement an Authenticator which can fulfill a dns-01 challenge using
the Google Cloud DNS API. Applicable only for domains using Google Cloud
DNS for DNS.
Testing Done:
* `tox -e py27`
* `tox -e lint`
* Manual testing:
* Used `certbot certonly --dns-google -d`, specifying a credentials
file as a command line argument. Verified that a certificate was
successfully obtained without user interaction.
* Used `certbot certonly --dns-google -d`, without specifying a
credentials file as a command line argument. Verified that the
user was prompted and that a certificate was successfully
obtained.
* Used `certbot certonly -d`. Verified that the user was prompted for
a credentials file after selecting google interactively and that
a certificate was successfully obtained.
* Used `certbot renew --force-renewal`. Verified that certificates
were renewed without user interaction.
* Negative testing:
* Path to non-existent credentials file.
* Credentials file with unsafe permissions (644).
* Domain name not registered to Google Cloud Platform account.
Implement an Authenticator which can fulfill a dns-01 challenge using the
DigitalOcean API. Applicable only for domains using DigitalOcean for DNS.
Testing Done:
* `tox -e py27`
* `tox -e lint`
* Manual testing:
* Used `certbot certonly --dns-digitalocean -d`, specifying a
credentials file as a command line argument. Verified that a
certificate was successfully obtained without user interaction.
* Used `certbot certonly --dns-digitalocean -d`, without specifying a
credentials file as a command line argument. Verified that the user
was prompted and that a certificate was successfully obtained.
* Used `certbot certonly -d`. Verified that the user was prompted for
a credentials file after selecting digitalocean interactively and
that a certificate was successfully obtained.
* Used `certbot renew --force-renewal`. Verified that certificates
were renewed without user interaction.
* Negative testing:
* Path to non-existent credentials file.
* Credentials file with unsafe permissions (644).
* Credentials file missing token.
* Credentials file with blank token.
* Credentials file with incorrect token.
* Domain name not registered to DigitalOcean account.
* Revert "Pin python-augeas version to avoid error with 1.0.0 (#4422)"
This reverts commit 1c51ae2588.
* make dependency-requirements
* separate certbot and dependency requirements
* fix build.py
* update hashin comment
* simplify release pinning
* separate letsencrypt dependency
* pin hashes in venv
* error out when bad things happen
* use pinned dependencies in tox
* Revert "pin hashes in venv"
This reverts commit 1cd38a9e50.
* use pip_install.sh in venv_common
* quote pip install args
* bump mock version
Implement an Authenticator which can fulfill a dns-01 challenge using the
Cloudflare API. Applicable only for domains using Cloudflare for DNS.
Testing Done:
* `tox -e py27`
* `tox -e lint`
* Manual testing:
* Used `certbot certonly --dns-cloudflare -d`, specifying a
credentials file as a command line argument. Verified that a
certificate was successfully obtained without user interaction.
* Used `certbot certonly --dns-cloudflare -d`, without specifying a
credentials file as a command line argument. Verified that the user
was prompted and that a certificate was successfully obtained.
* Used `certbot certonly -d`. Verified that the user was prompted for
a credentials file after selecting cloudflare interactively and
that a certificate was successfully obtained.
* Used `certbot renew --force-renewal`. Verified that certificates
were renewed without user interaction.
* Negative testing:
* Path to non-existent credentials file.
* Credentials file with unsafe permissions (644).
* Credentials file missing e-mail address.
* Credentials file with blank API key.
* Credentials file with incorrect e-mail address.
* Credentials file with malformed API key.
* Credentials file with invalid API key.
* Domain name not registered to Cloudflare account.
