* breaking(dovecot): ship a self-reliant Dovecot configuration
DMS imported the config files of Debian's Dovecot package at build time
and patched them with `sed` at startup. For the Dovecot Community Edition
(CE) it even downloaded Debian's packages only to extract their config.
The result depended on whatever the packages shipped, and every feature
toggle was an `!include` line that a script had to comment or uncomment.
Provide the complete configuration in the repository instead:
- `target/dovecot/dovecot.conf` replaces the `10-*`, `15-*`, `20-*` and
`90-*` files as well as the `*.inc` files. Everything that is not set
there uses Dovecot's built-in defaults.
- `target/dovecot/features/` holds the optional parts (passwd-file, LDAP
and OAuth2 authentication, quotas, Rspamd learning). Startup copies the
ones that are enabled into the empty `/etc/dovecot/conf.d/`, which
`dovecot.conf` includes before `local.conf` (`dovecot.cf`).
- The scripts only substitute ENV-derived values now; the include
toggling and the second `sed` per feature are gone.
- Drop the Debian config import and the CE-only download from the build.
Breaking: files that users replaced, bind-mounted or patched in
`/etc/dovecot/conf.d/` no longer exist (see the changelog for the
mapping). Debian's deprecated ManageSieve listener on port 2000 and its
`fts_autoindex*` defaults are gone as well.
Add tests for the removed port 2000 and for a valid `doveconf -n` with the
protocols enabled according to ENV, and adjust the existing tests and
fixtures to the new file layout. Rewrite the Dovecot override docs.
Closes#4841
* fix(dovecot): load enabled features before user supplied config
Features that DMS enables were copied to `conf.d/`, so Dovecot read them
in file name order together with user drop-ins. A user file such as
`90-custom.conf` was read before `quota.conf` or `auth-ldap.conf`, and the
feature silently overrode the user's settings.
- copy enabled features to `/etc/dovecot/features.d/` instead, which
`dovecot.conf` includes before `conf.d/*.conf`
- include the master user passdb before `conf.d/*.conf` as well, so a
drop-in can override `auth_master_user_separator`
- create `features.d/auth-ldap.conf` with mode 640 (group `dovecot`), as
it contains the LDAP bind password
- link the changelog entries to the PR, document the always-active master
passdb, and adjust docs and tests to the new layout
- test that a `conf.d/` file overrides the quota feature, and that
`auth-ldap.conf` is not world-readable
Since the Dovecot 2.4 migration (v16), `imapsieve_from Junk` fires for
every copy out of `Junk`, and `learn-ham.sieve` piped each one to
`rspamc learn_ham`. Deleting a message from `Junk` therefore trained the
classifier to treat it as ham.
The script now stops when the destination is `Junk` itself or a mailbox
with the `\Trash` special-use attribute, as in the upstream Dovecot 2.4
spam reporting example. Ham is learned for moves to any other folder.
Fixes#4831
Signed-off-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Since the Dovecot 2.4 migration, DMS registered one `sieve_script` block
per script it generates. Any other script placed in
`/usr/lib/dovecot/sieve-global/{before,after}/` (e.g. via
`user-patches.sh`) compiled but was silently ignored, unlike up to v15
where `sieve_before` / `sieve_after` pointed at the directories.
Register both directories as `before` / `after` script storages in
`90-sieve.conf` instead. Pigeonhole runs every `*.sieve` file in them in
file name order, which matches the previous execution order of the
scripts DMS generates itself (`50-`, `60-` and `70-` prefixes). The
per-file `sieve_script` blocks are removed.
- docs: describe the drop-in directories in the Sieve page
- tests: add a regression test with a script added via `user-patches.sh`
- changelog: add a `Fixed` entry with upgrade notes
Closes#4847
Postfix LDAP configs are now generated at startup to `/etc/postfix/ldap/<KIND>.cf`
(`users`, `groups`, `aliases`, `domains`, `senders`) from these sources, later ones take precedence:
1. Defaults (`version = 3`, `result_attribute = mail`, `senders`: `mail, uid`).
2. `LDAP_<SETTING>` ENV for all configs.
3. The custom config `ldap-<KIND>.cf` from the DMS config volume (now also supported for `senders`).
4. `POSTFIX_<KIND>_<SETTING>` ENV for a single config.
A config without `query_filter` is not generated and its lookup table is not configured
(including `smtpd_sender_login_maps` with `SPOOF_PROTECTION=1`). Previously all lookup
tables were always configured, as the default configs shipped by the image always existed.
Breaking:
- `LDAP_QUERY_FILTER_{USER,GROUP,ALIAS,DOMAIN,SENDERS}` are renamed to
`POSTFIX_{USERS,GROUPS,ALIASES,DOMAINS,SENDERS}_QUERY_FILTER` (an error is logged when the old ENV is set).
- `/etc/postfix/ldap-*.cf` moved to `/etc/postfix/ldap/*.cf`, the default configs (with default query filters) were removed.
Reworked from the original "Config Templates" proposal: Dovecot (already reworked for Dovecot 2.4)
and SASLAuthd are unchanged, and no template files or additional dependencies (`zenv`, `envsubst`) are required.
Signed-off-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
- Bump to [`jaq` v2 release](https://github.com/01mf02/jaq/releases/tag/v2.0.0), artifact naming convention changed.
- Tidied up the changelog a little bit unrelated to this `jaq` update.
- Fixed a typo with an `rspamd.sh` comment + minor revision to the comment.
* docs: Better document DMS volumes
* docs: Remove any mention of `ONE_DIR` ENV
* chore: Remove `ONE_DIR` ENV from scripts
Only `ONE_DIR=0` has any effect. As the actual feature is now dependent upon the `/var/mail-state` location existing.
It is advised not mounting anything there instead if wanting to avoid runtime state consolidation.
* docs: Adjust link ref convention
This is more search friendly / organized to find references to all DMS volumes.
* lint: Ensure final newline is present
VSCode by default excludes this if the last line rendered is removed (rendered as a separate blank line).
A separate setting can enforce adding the final newline upon save regardless.
This is a more explicit reminder for any future contributors that get thrown off by the usage of `sed` here and may be inclined to change it.
Add a link to reference a comment where it's already been explored what the alternative `sed` invocations available are.
* added check for Rspamd DKIM on startup
The newly added function `__rspamd__check_dkim_permissions` performs a
check on DKIM private key files. This is useful to prevent issues
like #3621 in the future. The function is deliberately kept simple and
may not catch every single misconfiguration in terms of permissions and
ownership, but it should be quite accurate.
Please note that the Rspamd setup does NOT change at all, and the checks
will not abort the setup in case they fail. A simple warning is emmited.
* add more documentation to Rspamd functions
* Apply suggestions from code review
* improve `__do_as_rspamd_user`
* rework check similar to review suggestion
see https://github.com/docker-mailserver/docker-mailserver/pull/3627#discussion_r1388697547
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* outsource Rspamd ENVs into explicit helper
This will allow us to uniformly source the helper and get the values
from everywhere consistently. This is more than desirable since we will
be using these values not only for the Rspamd setup, but also for DKIM
management and during change-detection.
* integrate Rspamd into changedetection
We outsource one more function to reside in the helper script for Rspamd
so that we can call this function from the Rspamd setup and from the
changedetection functionality too.
* realize deprecation of old commands file for Rspamd
THIS IS A BREAKING CHANGE!
This change realizes the log message: "Using old file location now
(deprecated) - this will prevent startup in v13.0.0" Startup will now
fail.
* added '--force' option to Rspamd DKIM script
* use new helper to get ENVs for Rspamd in DKIM script
* remove the need for linking directories
This was unnecessary, as explained in
https://github.com/docker-mailserver/docker-mailserver/pull/3597#discussion_r1369413599
* Apply suggestions from code review
review by @polarathene
* apply more review feedback from @polarathene
- <https://github.com/docker-mailserver/docker-mailserver/pull/3599#discussion_r1370885519>
- <https://github.com/docker-mailserver/docker-mailserver/pull/3599#discussion_r1370904201>
* update documentation
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Instead of using `etc/rspamd/override.d/dkim_signing.conf`, we will now
be using `/tmp/docker-mailserver/rspamd/override.d/dkim_signing.conf`.
The new location is persisted (and linked again during startup) and
hence better suited.
* simplify `_setup_logrotate`
* adjust Rspamd's log file and improve it's management
* add information to docs about Rspamd log
* update log query helper to allow another file location
* bail in case `LOGROTATE_INTERVAL` is invalid
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
The new function can
1. update/append
2. update/prepend
3. initialize if non-existent
options in `/etc/postfix/main.cf` in a safe and secure manner. When the
container is improperly restarted, the option is not applied twice.
---
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* adjust learning of ham
See #3333
When moving a mail from the Junk folder to the Trash folder, the mail
previously classified as ham due to the wildcard match of `*`. Because
the syntax does not allow for negation, we can only change the behavior
in a way that mails are learned as ham when they are moved into `INBOX`
from `Junk`. This is reasonable though.
* adjust tests accordingly
* adjust docs accordingly
* add sanity check for Clam size & adjusted MaxScanSize
The second part is of special importance! See
<https://askubuntu.com/a/1448525>, which explains that the maximum scan
size is important as well. We previously just set the maximum file size,
which actually is pretty insecure as we silently not scan mile bigger
than `MaxScanSize`. This is corrected now.
* add SlamAV size configuration to Rspamd