* Move acme tests to tests/ directory outside of acme module
* Fix call to messages_test in client_test
* Move test_util.py and testdata/ into tests/
* Update manifest to package tests
* Exclude pycache and .py[cod]
* Refactor tests out of module for certbot-dns-cloudflare
* Refactor tests out of module for certbot-dns-cloudxns
* Refactor tests out of module for certbot-dns-digitalocean
* Refactor tests out of module for certbot-dns-dnsimple
* Refactor tests out of module for certbot-dns-dnsmadeeasy
* Refactor tests out of module for certbot-dns-gehirn
* Refactor tests out of module for certbot-dns-google
* Refactor tests out of module for certbot-dns-linode
* Refactor tests out of module for certbot-dns-luadns
* Refactor tests out of module for certbot-dns-nsone
* Refactor tests out of module for certbot-dns-ovh
* Refactor tests out of module for certbot-dns-rfc2136
* Refactor tests out of module for certbot-dns-sakuracloud
* Refactor tests out of module for certbot-dns-route53
* Move certbot-dns-google testdata/ under tests/
* Use pytest for dns plugins
* Exclude pycache and .py[cod]
Clean up some places missed by #7544.
Found this when running test farm tests. They were working as of 5d90544, and I will truly shocked if subsequent changes (all to the windows installer) made them stop working.
* Release script needs to target new CHANGELOG location
* Clean up various other CHANGELOG path references
* Update windows paths for new certbot location
* Add certbot to packages list for windows installer
* Update pull_request_template.md
* Remove line breaks
Github seems to be keeping the line breaks rather than ignoring them, making it be formatted weirdly, so remove them.
Part of #5775.
* Create _internal folder certbot-nginx
* Move configurator.py to _internal
* Move constants.py to _internal
* Move display_ops.py to _internal
* Move http_01.py to _internal
* Move nginxparser.py to _internal
* Move obj.py to _internal
* Move parser_obj.py to _internal
* Move parser.py to _internal
* Update location and references for tls_configs
* exclude parser_obj from coverage
Summary of changes in this PR:
- Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below).
- Make repo root README symlink to `certbot` README.
- Pull tests outside of the distributed module.
- Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery.
- Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments.
- Replace mentions of `.` with `certbot` when referring to packages to install, usually editably.
- Clean up some unused code around executing tests in a different directory.
- Create public shim around main and make that the entry point.
New directory structure summary:
```
repo root ("certbot", probably, but for clarity all files I mention are relative to here)
├── certbot
│ ├── setup.py
│ ├── certbot
│ │ ├── __init__.py
│ │ ├── achallenges.py
│ │ ├── _internal
│ │ │ ├── __init__.py
│ │ │ ├── account.py
│ │ │ ├── ...
│ │ ├── ...
│ ├── tests
│ │ ├── account_test.py
│ │ ├── display
│ │ │ ├── __init__.py
│ │ │ ├── ...
│ │ ├── ... # note no __init__.py at this level
│ ├── ...
├── acme
│ ├── ...
├── certbot-apache
│ ├── ...
├── ...
```
* refactor certbot/ and certbot/tests/ to use the same structure as the other packages
* git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g"
* git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g"
* git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g"
* Remove replacement of certbot into . in install_and_test.py
* copy license back out to main folder
* remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/
* Move README back into main folder, and make the version inside certbot/ a symlink
* symlink certbot READMEs the other way around
* move testdata into the public api certbot zone
* update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests
* certbot version has been bumped down a directory level
* make certbot tests directory not a package and import sibling as module
* Remove unused script cruft
* change . to certbot in test_sdists
* remove outdated comment referencing a command that doesn't work
* Install instructions should reference an existing file
* update file paths in Dockerfile
* some package named in tox.ini were manually specified, change those to certbot
* new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code
* remove other instance of pyargs
* fix up some references in _release.sh by searching for ' . ' and manual check
* another stray . in tox.ini
* fix paths in tools/_release.sh
* Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper
* Create public shim around main and make that the entry point
* without pyargs, tests cannot be run from an empty directory
* Remove cruft for running certbot directly from main
* Have main shim take real arg
* add docs/api file for main, and fix up main comment
* Update certbot/docs/install.rst
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Fix comments in readthedocs requirements files to refer to current package
* Update .[docs] reference in contributing.rst
* Move plugins tests to certbot tests directory
* add certbot tests to MANIFEST.in so packagers can run python setup.py test
* move examples directory inside certbot/
* Move CHANGELOG into certbot, and create a top-level symlink
* Remove unused sys and logging from main shim
* nginx http01 test no longer relies on certbot plugins common test
Part of #5775. We don't use these docs anywhere, so delete them.
Removes:
- `certbot-nginx/readthedocs.org.requirements.txt`
- `certbot-nginx/docs/` folder
- docs include in `MANIFEST.in`
- docs dependencies in `setup.py`
* Remove unused nginx docs
* Add changelog entry about the removal
Part of #5775. We don't use these docs anywhere, so delete them.
Removes:
- `certbot-compatibility-test/readthedocs.org.requirements.txt`
- `certbot-compatibility-test/docs/` folder
- docs include in `MANIFEST.in`
- docs dependencies in `setup.py`
Part of #5775. We don't use these docs anywhere, so delete them.
Removes:
- `certbot-apache/readthedocs.org.requirements.txt`
- `certbot-apache/docs/` folder
- docs include in `MANIFEST.in`
- docs dependencies in `setup.py`
Fixes#7184.
I updated #7358 to track the issue of unpinning all of these dependencies.
* pin back configargparse
* Pin back zope packages.
* update deps
* Add changelog entry.
* run build.py
When you try to run this script, it crashes with:
```
standard_init_linux.go:211: exec user process caused "exec format error"
```
This is caused by the script being written to have the contents:
```
\
#!/bin/sh
set -e
...
```
This fixes the problem by removing the slash and moving the shebang to the first line of the string.
Part of #5775. Methodology similar to #7528. Also refactors NGINX test util to use certbot.tests.util.ConfigTestCase.
* refactor nginx tests to no longer rely on certbot.configuration internals
* Move configuration.py to _internal
While coding for #7536, I ran into another issue. It appears that Certbot logs generated during the scheduled task execution have wrong permissions that make them almost unusable: they do not have an owner, and their ACL contains nonsense values (non existant accounts name).
The class `logging.handler.RotatingFileHandler` is responsible for these logs, and become mad when it is in a Python process run under a scheduled task owned by `SYSTEM`. This is precisely our case here.
This PR avoids (but not fix) the issue, by changing the owner of the scheduled task from `SYSTEM` to the `Administrators` group, that appears to work fine.
* Use Administrators group instead of SYSTEM to run the certbot renew task
Turned out that the scheduled task that runs `certbot renew` twice a day, is failing. Without any kind of log of course, otherwise it would not be fun.
It can be revealed by opening a powershell under the `NT AUTHORITY\SYSTEM` account, under which the scheduled task is run. Under theses circumstances, the bug is revealed: Certbot breaks when trying to invoke `certbot.compat.filesystem._get_current_user()`. Indeed the logic there implied to call `win32api.GetUserNameEx(win32api.NameSamCompatible)` and this function does not return always a useful value.
For normal account, it will be typically `DOMAIN_OR_MACHINE_NAME\YOUR_USER_NAME` (e.g. `My Machine\Adrien Ferrand`). But for the account `NT AUTHORITY\SYSTEM`, it will return `MACHINE_NAME\DOMAIN$`, which is a nonsense and makes fail the resolution of the actual SID of the account at the end of `_get_current_user()`.
This PR fixes this behavior by using an explicit construction of the account name that works both for normal users and `SYSTEM`.
* Use a different way to resolve current user account, that works both for normal users and SYSTEM.
* Add a comment to run Certbot under NT AUTHORITY\SYSTEM
Fixes#7548.
This PR udpdates installation instructions to get rid of python2 and certbot-auto in the how-to explaining the Certbot development environment setup.
Instead, Python 3 is used, and appropriate instructions for APT and RPM based distributions are provided.
* Don't call core constants from nginx plugin
* Move constants.py to _internal/
* Move ENHANCEMENTS from now-internal constants to public plugins.enhancements
* Update display.enhancements.ask from its 2015 comment
* Move display/completer.py to _internal/
* Move display/dummy_readline.py to _internal/
* Move display/enhancements.py to _internal/
* Create __init__.py in _internal/display
* Create _internal package for Certbot's non-public modules
* Move account.py to _internal
* Move auth_handler.py to _internal
* Move cert_manager.py to _internal
* Move client.py to _internal
* Move error_handler.py to _internal
* Move lock.py to _internal
* Move main.py to _internal
* Move notify.py to _internal
* Move ocsp.py to _internal
* Move renewal.py to _internal
* Move reporter.py to _internal
* Move storage.py to _internal
* Move updater.py to _internal
* update apache and nginx oldest requirements
* Keep the lock file as certbot.lock
* nginx oldest tests still need to rely on newer certbot
* python doesn't have good dependency resolution, so specify the transitive dependency
* update required minimum versions in nginx setup.py
In Travis, the full test suite doesn't run on PRs for point release branches, just on commits for them. I think this behavior makes sense because what we actually want to test before a point release is the exact commit we want to release after any squashing/merging has been done. This PR modifies Azure to match this behavior.
After this PR lands, I need to update the tests required to pass on GitHub.
This PR uses pipstrap to bootstrap the venv used to build Windows installers. This effectively pin all build dependencies, since pynsist is already installed through pip_install.py script.
* Use pipstrap
* Pin also NSIS version
