`ENABLE_RSPAMD` now defaults to `1`, while `ENABLE_AMAVIS`,
`ENABLE_OPENDKIM`, `ENABLE_OPENDMARC` and `ENABLE_POLICYD_SPF` default
to `0`. `ENABLE_RSPAMD_VALKEY` follows `ENABLE_RSPAMD`.
DMS refuses to start when OpenDKIM keys exist in the config volume and
`ENABLE_OPENDKIM` is unset, so an upgrade does not silently stop signing
outbound mail.
Tests that relied on the old defaults now set the legacy ENV
explicitly. `default_security_stack.bats` covers the new defaults and
the startup refusal.
Closes#4364
* breaking(dovecot): ship a self-reliant Dovecot configuration
DMS imported the config files of Debian's Dovecot package at build time
and patched them with `sed` at startup. For the Dovecot Community Edition
(CE) it even downloaded Debian's packages only to extract their config.
The result depended on whatever the packages shipped, and every feature
toggle was an `!include` line that a script had to comment or uncomment.
Provide the complete configuration in the repository instead:
- `target/dovecot/dovecot.conf` replaces the `10-*`, `15-*`, `20-*` and
`90-*` files as well as the `*.inc` files. Everything that is not set
there uses Dovecot's built-in defaults.
- `target/dovecot/features/` holds the optional parts (passwd-file, LDAP
and OAuth2 authentication, quotas, Rspamd learning). Startup copies the
ones that are enabled into the empty `/etc/dovecot/conf.d/`, which
`dovecot.conf` includes before `local.conf` (`dovecot.cf`).
- The scripts only substitute ENV-derived values now; the include
toggling and the second `sed` per feature are gone.
- Drop the Debian config import and the CE-only download from the build.
Breaking: files that users replaced, bind-mounted or patched in
`/etc/dovecot/conf.d/` no longer exist (see the changelog for the
mapping). Debian's deprecated ManageSieve listener on port 2000 and its
`fts_autoindex*` defaults are gone as well.
Add tests for the removed port 2000 and for a valid `doveconf -n` with the
protocols enabled according to ENV, and adjust the existing tests and
fixtures to the new file layout. Rewrite the Dovecot override docs.
Closes#4841
* fix(dovecot): load enabled features before user supplied config
Features that DMS enables were copied to `conf.d/`, so Dovecot read them
in file name order together with user drop-ins. A user file such as
`90-custom.conf` was read before `quota.conf` or `auth-ldap.conf`, and the
feature silently overrode the user's settings.
- copy enabled features to `/etc/dovecot/features.d/` instead, which
`dovecot.conf` includes before `conf.d/*.conf`
- include the master user passdb before `conf.d/*.conf` as well, so a
drop-in can override `auth_master_user_separator`
- create `features.d/auth-ldap.conf` with mode 640 (group `dovecot`), as
it contains the LDAP bind password
- link the changelog entries to the PR, document the always-active master
passdb, and adjust docs and tests to the new layout
- test that a `conf.d/` file overrides the quota feature, and that
`auth-ldap.conf` is not world-readable
* fix(postfix): deliver mail to accounts on a domain with a catch-all alias
Postfix resolves `virtual_alias_maps` before `virtual_mailbox_maps`, so a
catch-all alias (`@example.com`) in `postfix-virtual.cf` also captured the
mail of every account on that domain. The documented workaround (an alias
to itself, `addr addr`) cannot be created with `setup alias add`, as the
CLI rejects an alias that is also an account.
Add that alias for each account of a catch-all domain to the generated
`/etc/postfix/virtual`, at startup and when `postfix-accounts.cf` changes.
`postfix-virtual.cf` is left untouched, so the CLI guard stays, the Dovecot
quota dummy accounts are unaffected, and entries added by hand are kept.
Only applies to `ACCOUNT_PROVISIONER=FILE`.
Also fix the overlap checks of `setup alias add` and `setup email add`:
they used a prefix `grep`, so `user@example.com` was rejected when
`user@example.com.au` existed. They now use `_db_has_entry_with_key`,
which compares the whole address.
Closes#4856
Since the Dovecot 2.4 migration (v16), `imapsieve_from Junk` fires for
every copy out of `Junk`, and `learn-ham.sieve` piped each one to
`rspamc learn_ham`. Deleting a message from `Junk` therefore trained the
classifier to treat it as ham.
The script now stops when the destination is `Junk` itself or a mailbox
with the `\Trash` special-use attribute, as in the upstream Dovecot 2.4
spam reporting example. Ham is learned for moves to any other folder.
Fixes#4831
Signed-off-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
`_setup_docker_permit` always parsed an IPv4 address from
`NETWORK_INTERFACE` and aborted startup when there was none, even for the
default `PERMIT_DOCKER=none`, which does not need it. DMS therefore
failed to start in IPv6-only networks.
Replace `PERMIT_DOCKER` with `DMS_TRUSTED_NETWORKS`, which no longer
implies Docker, carries the `DMS_` prefix and supports IPv6:
- `none` (default): trust no client
- `host`, `subnet`: delegate to Postfix `mynetworks_style`, so no
`ip addr` parsing is needed
- a list of CIDR ranges (IPv4 and IPv6, separated by spaces or commas)
The ranges Postfix derives or the user provides are normalized (lower
case, compressed IPv6, host bits masked) before they are added to the
OpenDKIM and OpenDMARC trust lists, as both only match that text form.
An invalid value now stops the startup instead of being cleared with a
warning.
`NETWORK_INTERFACE` is removed together with `helpers/network.sh`.
`PERMIT_DOCKER` and `NETWORK_INTERFACE` log an error when still set and
are ignored, so no client is trusted until the user migrates:
- `container` => `host`
- `host`, `connected-networks` => `subnet`
- `network` => `172.16.0.0/12` (or the exact range of the network)
- docs: document `DMS_TRUSTED_NETWORKS` with a migration table and update
all pages and demo setups that used the old variables
- tests: add `trusted_networks.bats` (no IPv4 address, `host`, CIDR list,
invalid value, legacy variables), rename `permit_docker.bats` and
migrate all tests to the new variable
- changelog: add `Breaking` and `Fixed` entries
Closes#4634Closes#3478
Since the Dovecot 2.4 migration, DMS registered one `sieve_script` block
per script it generates. Any other script placed in
`/usr/lib/dovecot/sieve-global/{before,after}/` (e.g. via
`user-patches.sh`) compiled but was silently ignored, unlike up to v15
where `sieve_before` / `sieve_after` pointed at the directories.
Register both directories as `before` / `after` script storages in
`90-sieve.conf` instead. Pigeonhole runs every `*.sieve` file in them in
file name order, which matches the previous execution order of the
scripts DMS generates itself (`50-`, `60-` and `70-` prefixes). The
per-file `sieve_script` blocks are removed.
- docs: describe the drop-in directories in the Sieve page
- tests: add a regression test with a script added via `user-patches.sh`
- changelog: add a `Fixed` entry with upgrade notes
Closes#4847
* chore: install swaks from the Debian package
Debian Trixie ships the same upstream version (20240103.0) that was
downloaded from the GitHub release, so the download is no longer needed
and the package receives updates from Debian.
* docs(changelog): link the pull request
---------
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* fix(build): use Debian's Dovecot config layout with the CE packages
The Dovecot CE repo ships a minimal config layout that DMS does not
expect: no 15-lda.conf, 90-fts.conf or 20-pop3.conf, a 2.3-style
20-managesieve.conf, and a dovecot.conf without the commented
protocols/listen lines or the local.conf include. The startup scripts
then fail to enable LMTP, so Postfix cannot deliver to Dovecot, and a
custom dovecot.cf is ignored.
When DOVECOT_COMMUNITY_REPO=1, install the config files from Debian's
own packages of the same release, as Debian's postinst does. Also
tolerate the missing 90-fts-flatcurve.conf, which only Debian ships,
as the build otherwise fails.
* fix(build): install ssl-cert explicitly
Dovecot and Postfix default to the snakeoil cert from the ssl-cert
package. Debian's dovecot-core depends on ssl-cert, but the Dovecot CE
package does not, so with DOVECOT_COMMUNITY_REPO=1 the cert is missing
and Dovecot fails to start when no SSL_TYPE is set.
* fix(dovecot): disable attachment detection keywords explicitly
Dovecot 2.4.2 changed the default of mail_attachment_detection_options
to add-flags for maildir and dbox. This per-driver default is applied
even with dovecot_config_version = 2.4.0, so LMTP adds a
$HasNoAttachment keyword and maildir stores new mail in cur/ instead of
new/. Set it explicitly to keep the 2.4.0/2.4.1 behaviour. This is a
no-op with Debian's Dovecot 2.4.1.
* tests: make the letsencrypt doveconf check work with Dovecot 2.4.2+
Querying `doveconf ssl_server` prints nothing with Dovecot 2.4.5, and
querying the full setting names prints empty values with 2.4.1. Use
`doveconf -n -P`, which prints the same lines on both.
* docs: add changelog entries for the Dovecot CE build fixes
* docs: link the changelog entries to #4834
* fix(build): exclude the CE repo instead of matching deb.debian.org
Matching the deb.debian.org host made builds behind a different Debian
mirror (Artifactory, apt-cacher-ng, regional mirrors) fail to find the
Debian version of dovecot-core. There are only two sources, so exclude
repo.dovecot.org instead.
* docs: describe the attachment detection setting as a deliberate choice
Mail in cur/ without the S flag is still valid maildir and still
unread, so the explicit empty value is for parity with the Debian
build, not a bug fix. Say so in the config comment and the changelog,
and mention how to enable the feature.
---------
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* fix(rspamd): pin Rspamd to 4.1.5 and install it from a verified .deb
The Rspamd apt repo only lists the newest release, so every image build
and rebuild picked up whatever Rspamd had just released. Download the
pinned release from the repo's pool instead and verify its sha256 for
amd64 and arm64. 4.1.5 is the version in Debian testing (forky).
* docs(changelog): link the pull request
Postfix LDAP configs are now generated at startup to `/etc/postfix/ldap/<KIND>.cf`
(`users`, `groups`, `aliases`, `domains`, `senders`) from these sources, later ones take precedence:
1. Defaults (`version = 3`, `result_attribute = mail`, `senders`: `mail, uid`).
2. `LDAP_<SETTING>` ENV for all configs.
3. The custom config `ldap-<KIND>.cf` from the DMS config volume (now also supported for `senders`).
4. `POSTFIX_<KIND>_<SETTING>` ENV for a single config.
A config without `query_filter` is not generated and its lookup table is not configured
(including `smtpd_sender_login_maps` with `SPOOF_PROTECTION=1`). Previously all lookup
tables were always configured, as the default configs shipped by the image always existed.
Breaking:
- `LDAP_QUERY_FILTER_{USER,GROUP,ALIAS,DOMAIN,SENDERS}` are renamed to
`POSTFIX_{USERS,GROUPS,ALIASES,DOMAINS,SENDERS}_QUERY_FILTER` (an error is logged when the old ENV is set).
- `/etc/postfix/ldap-*.cf` moved to `/etc/postfix/ldap/*.cf`, the default configs (with default query filters) were removed.
Reworked from the original "Config Templates" proposal: Dovecot (already reworked for Dovecot 2.4)
and SASLAuthd are unchanged, and no template files or additional dependencies (`zenv`, `envsubst`) are required.
Signed-off-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* fix: Helper `_replace_by_env_in_file()` support delimiter parameter
Minor change to add support for usage with `saslauthd.conf` which differs with `:` for it's key/value delimiter.
Also adopts the`_escape_for_sed()` method, instead of the inline sed pattern (_which for some reason escaped `=` but not `(` + `)`, thus buggy if ever matching on input with those tokens_).
`_escape_for_sed()` likewise wasn't escaping for `(` + `)` or even `|`, which are required for proper escape support if using `sed -E` / `sed -r` with this method. Additionally added escaping support for `&` replacement segment token, which seems valid. Increased verbosity to better grok pattern matching expression, clarified escaping concern with `sed` delimiter since the project is not consistent there.
* chore: Preserve K/V white-space of original source
No need to check for a non-empty value to prepend a space (_since an empty value is used via sed anyway?_).
Can also be a bit DRY with the sed pattern, matching the key + delimiter (_and all white-space before/after the delimiter until the value_), then capture that for the replacement left-side value while only actually swapping the value for the ENV input value.
Should be an improvement, unless there is a scenario that would differ between `` and ` ` as valid value assignments?
* chore: Adopt `sedfile` + case-insensitive pattern matching
Both should be improvements for usage
Co-authored-by: Casper <casperklein@users.noreply.github.com>
* fix: Harden `_replace_by_env_in_file()` and update tests
- Split each ENV line at the first `=` only, `IFS='=' read` dropped a trailing `=` from values (e.g. base64 padded passwords).
- Use `/` as the `sed` delimiter, as it is the only delimiter escaped by `_escape_for_sed()` (values with `#` broke the expression).
- Revert `sedfile` to `sed -i`: most prefixed ENV have no matching key, so an unchanged file is expected and `sedfile` would log spurious errors.
- Drop the unused key/value delimiter parameter.
- Avoid trailing white-space when the value is empty.
- Tests: Expect the retained white-space, add edge-cases for `sed` tokens and a trailing `=`, compare LDAP settings white-space insensitive.
Signed-off-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
---------
Signed-off-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* chore: bump `jaq` from 2.3.0 to 3.0.0
* chore: bump `step-cli` from 0.28.7 to 0.30.2
* chore: formatting
---------
Signed-off-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* fix: Enable change detection support for `ACCOUNT_PROVISIONER=LDAP`
* fix(test/rspamd_full): Explicitly wait for the `rspamd_proxy` port to be available before testing
- Ensure catch-all alias entries (eg: `@example.test`) are also skipped. Adding a dummy account for catch-all will not match a `quota-status` query to Dovecot by Postfix for a recipient.
- When there are multiple addresses provided, they will now be iterated through by the `,` delimiter, instead of as a single value that fails. This way the first valid aliased address to a local Dovecot mailbox account will now be used for the dummy alias account.
- The common logic for extracting the quota user attribute is now split out to a common function call.