This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 12.3.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Replace the remaining Knowledge Base links with their MariaDB
Documentation equivalents, including the 838 URLs in the help tables.
Only URLs change in fill_help_tables.sql.
Merging upward: fill_help_tables.sql conflicts at 10.11->11.4 and
11.8->12.3. At those two merges keep the target branch's version,
since 11.4, 11.8 and 12.3 each carry their own URL fix. From 12.3 to
13.1 and 13.1 to main it merges cleanly; take the incoming change.
.github/pull_request_template.md is deleted in 12.3; keep the deletion.
Replace the remaining Knowledge Base links with their MariaDB
Documentation equivalents, including the 838 URLs in the help tables.
Only URLs change in fill_help_tables.sql.
Merging upward: fill_help_tables.sql conflicts at 10.11->11.4 and
11.8->12.3. At those two merges keep the target branch's version,
since 11.4, 11.8 and 12.3 each carry their own URL fix. From 12.3 to
13.1 and 13.1 to main it merges cleanly; take the incoming change.
.github/pull_request_template.md is deleted in 12.3; keep the deletion.
Replace the remaining Knowledge Base links with their MariaDB
Documentation equivalents, including the 838 URLs in the help tables.
Only URLs change in fill_help_tables.sql.
Merging upward: fill_help_tables.sql conflicts at 10.11->11.4 and
11.8->12.3. At those two merges keep the target branch's version,
since 11.4, 11.8 and 12.3 each carry their own URL fix. From 12.3 to
13.1 and 13.1 to main it merges cleanly; take the incoming change.
.github/pull_request_template.md is deleted in 12.3; keep the deletion.
mariadb-upgrade silently dropped EXECUTE grants on sys schema stored
functions and procedures. The sys schema install scripts reinstalled
every routine with DROP FUNCTION/PROCEDURE IF EXISTS followed by
CREATE. DROP cascades to delete the routine's rows in
mysql.procs_priv, so any EXECUTE grant a DBA had issued on e.g.
sys.table_exists or sys.quote_identifier was lost every time
mariadb-upgrade reinstalled the sys schema, even though the routine
itself came back unchanged.
Fix: replace DROP ... IF EXISTS + CREATE with CREATE OR REPLACE in
all 53 sys_schema function/procedure files and in the two templates
(templates/function.sql, templates/procedure.sql) so future routines
follow the same pattern. CREATE OR REPLACE PROCEDURE/FUNCTION goes
through sp_drop_routine_internal(), which only deletes the
mysql.proc row and never reaches sp_revoke_privileges() (that is
only called from the explicit DROP PROCEDURE/FUNCTION statement),
so mysql.procs_priv is left untouched and existing grants survive.
This mirrors the pattern already used by sys schema views
(CREATE OR REPLACE ... VIEW, since MDEV-9077), which never had this
problem.
Four of the converted files (functions/format_path.sql,
functions/ps_is_account_enabled_57.sql,
procedures/ps_setup_reset_to_default.sql,
procedures/ps_trace_thread_57.sql) are not referenced by
scripts/sys_schema/CMakeLists.txt; they were converted anyway for
consistency and have no behavioural effect.
As a side effect, a pre-existing UDF whose name collides with a sys
routine name is no longer destroyed before the reinstall fails:
DROP FUNCTION IF EXISTS resolved the UDF namespace first, silently
dropping the UDF and then failing on ER_SP_ALREADY_EXISTS anyway;
CREATE OR REPLACE fails immediately on ER_UDF_EXISTS with the UDF
intact.
scripts/maria_add_gis_sp.sql.in and the sys_config triggers were
deliberately left untouched: the GIS procedures are only
re-installed at bootstrap time (mariadb-upgrade instead patches
their definer in place via UPDATE), and triggers carry no
procs_priv rows, so neither is on the code path this bug is about.
Added mysql-test/main/mysql_upgrade_sys_routine_grants.test, which
grants EXECUTE on sys.table_exists and sys.quote_identifier,
overwrites both routine bodies with a marker to prove the upgrade
actually reinstalls them (rather than the sys schema install being
skipped), runs mariadb-upgrade, and checks both that the grants
survived and that the real routine bodies came back.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Issue: mariadb-backup SST unconditionally passes socat's "commonname="
option; some socat builds don't register it, so parseopts() rejects
it as unknown regardless of value, breaking all SSL-encrypted SST.
Solution: probe the socat binary once for commonname support and
drop the option when unsupported.
Backported the fix done by Pekka Lampio for mariaDB 11.4 in PR
https://github.com/mariadb-corporation/codership-mariadb-server/pull/543
The PR has a fix for wsrep_sst_rsync script and new mtr test:
galera_3nodes.galera_mdev_36677" to check that the rsync SST method of Galera
works correctly also when the joiner node store InnoDB log files in a dedicated
directory separate from the data dictionary
Note: merging this PR to 11.4 may not be fully functional as there are other
changes in the rsync SST script. Take a look at the original 11.4 PR when merging.
With log_bin=ON a transaction is committed via two-phase commit (the
binary log is the second participant), so it passes through the InnoDB
XA-prepare state. While a donor is held in BLOCK_COMMIT for a mariabackup
backup, its parallel appliers (wsrep_slave_threads > 1) leave one or more
such writesets prepared-but-not-yet-committed, and the snapshot captures
them. On a freshly SST'd joiner nothing resolves these prepared
transactions: binlog crash recovery does not run (the joiner has no in-use
binlog to recover from), and the wsrep continuity-based commit is inactive
because wsrep_emulate_bin_log is FALSE when log_bin is ON. The leftover
prepared transactions then abort startup with "Found <N> prepared
transactions!". Note this does not depend on the prepared set being
non-contiguous - even a contiguous run aborts, because nothing commits
or rolls it back.
Rollback these transactions in xarecover_handlerton(). If rollback fails
flag error to cause unireg_abort().
After a mariabackup SST the joiner could fail with
ER_GTID_STRICT_OUT_OF_ORDER (error 1950)
while re-binlogging transactions received over IST.
The cause is that the binary log copied from the donor carries a
Gtid_list whose position can be ahead of the storage-engine snapshot
because getting snapshot and binlog are no mutually atomic, so transactions
can be present in the copied binlog that are not committed in the copied
engine snapshot.
After the SST the joiner reports the (committed) engine position to the
cluster, IST resends those transactions, and re-binlogging them under
gtid_strict_mode=ON collides with the ahead Gtid_list -> error 1950.
(MDEV-34483 made the engine snapshot stop short of the binlog, which is
what exposed this.)
The copied binary log carries no transactions the joiner needs - only a
Gtid_list - so instead of shipping and then having to truncate/reconcile
it, the joiner now starts a fresh binary log and seeds its GTID position
from the storage-engine checkpoint during recovery. That checkpoint is
the committed cluster position, i.e. exactly where IST resumes, so the
joiner's binary log stays in lockstep with the rest of the cluster and
no out-of-order GTID can occur.
This works for both wsrep_gtid_mode settings; only the binlog domain of
the cluster stream differs:
- wsrep_gtid_mode=ON : wsrep_gtid_domain_id (cluster writes are
re-tagged to it), which is the domain stored in the checkpoint;
- wsrep_gtid_mode=OFF: gtid_domain_id (cluster writes keep the node's
configured domain).
Async-replica positions (mysql.gtid_slave_pos) are part of the engine
snapshot and survive the SST unchanged, so a Galera node can still serve
as an async master or replica across the SST.
This commit:
- sql/log.cc: adds wsrep_seed_binlog_gtid_state(), called from
do_binlog_recovery() when the joiner has no binary log, seeding the
binlog GTID state for the cluster domain to the SE checkpoint position.
- scripts/wsrep_sst_mariabackup.sh: no longer moves the donor's binary
log into place on the joiner.
- extra/mariabackup: backward compatibility: keep shipping binlog file
in SST but
- on donor fix the race between rotation and shipping so that the file
shipped is the one that had been rotated;
- on joiner discard shipped binlog in favour of one generated by
wsrep_seed_binlog_gtid_state().
- sql/wsrep_sst.cc: logs the position actually adopted from storage
(the authoritative post-SST position) rather than the script-reported
one.
- sql/handler.cc: downgrades the "Discovered discontinuity in recovered
wsrep transaction XIDs" message in wsrep_order_and_check_continuity()
from warning to debug level. With parallel appliers a snapshot
routinely captures prepared XIDs that are not contiguous with the
engine checkpoint, so this is normal during SST recovery and of no
value in regular operation; the transactions past the checkpoint are
re-delivered by the cluster (IST/SST) regardless.
- Adds an MDEV-38147 MTR test reproducing the issue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* don't create mariadb-wsrep-new-cluster in the mariadbd-writable path,
the server should not be able to poison the environment with OUTFILE.
Create it in /run
* As in /run it must be deleted by root, let galera_new_cluster delete
it, not the service
* wsrep-start-position cannot be created by root, so avoid a file
for it at all
Assisted-By: Claude:claude-5-opus
Issue:
When ssl-mode required encryption but the means to perform it was missing,
the SST scripts silently fell back to a cleartext transfer:
- wsrep_sst_rsync: ran over plain TCP when the 'stunnel' binary was absent.
- wsrep_sst_mariabackup: socat used a cleartext socket when ssl-mode was set
but no usable cert/key was found (encrypt stayed 0).
Solution:
Abort the SST instead of falling back to an unencrypted transfer when
ssl-mode is not DISABLED but encryption cannot be set up:
- wsrep_sst_rsync: derive the implicit ssl-mode from the SSL config even
when stunnel is absent, then abort with ENOENT if ssl-mode is active
and the stunnel binary is not found.
- wsrep_sst_mariabackup: after reading the SSL configuration, abort with
EINVAL if ssl-mode is not DISABLED but encrypt resolved to 0 (no usable
cert/key).
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Issue:
The mariabackup SST script enabled a peer certificate name check
for any ssl-mode starting with VERIFY, so VERIFY_CA behaved like
VERIFY_IDENTITY, unlike the rsync script.
Solution:
For VERIFY_CA verify the chain only - drop the socat
"commonname" check in mariabackup and gate the rsync checkHost/checkIP
on VERIFY_IDENTITY.
Issue:
Verify_ca_matches_cert ran "openssl verify -CAfile <ca> <cert>", which
treats only the first cert in the file as the target. For a leaf+intermediate
bundle against a root-only CA, openssl cannot reach the root, so the check
aborts the SST (exit 22).
Solution:
Pass the cert file as -untrusted so bundled intermediates complete
the chain to the CA. Trust anchor is unchanged (still -CAfile only).
Issue: The SST TLS options are not role-specific, so a node cannot
present a different certificate as donor (TLS client) than as joiner
(TLS server), which some environments require.
Solution: In the shared check_server_ssl_config, add role-specific
options that override the generic ones: ssl-server-{ca,capath,cert,key}
for the joiner and ssl-client-{ca,capath,cert,key} for the donor.
Applies to both mariabackup and rsync SST.
An appropriately privileged user (with SUPER privileges) could
execute shell commands as the uid of the mariadbd process
because the values of the system variable wsrep_sst_auth,
which can be modified at runtime, were not properly
sanitized when used to construct a shell command.
Combined rsync and mariabackup test cases and added
test case for incorrect values for wsrep_sst_auth.
The upgrade SQL assumes the functions will resolve.
With a non-default path this isn't true and results in:
ERROR 1305 (42000) at line 3211: FUNCTION extract_schema_from_file_name does not exist
Resolve by setting PATH = 'CURRENT_SCHEMA' for the upgrade.
Idea thanks to Sergei Golubchik
Issue:
wsrep_sst_rsync.sh interpolated WSREP_SST_OPT_REMOTE_USER and
WSREP_SST_OPT_REMOTE_PSWD verbatim. Because both values originate from
the joiner side of the SST request, a newline in either could splice
an extra directive into the donor-written stunnel.conf (silently
downgrading peer-cert verification) or an extra line into the rsync
magic file. MDEV-39413 had introduced safe() for the same threat class
in wsrep_sst_mariabackup but did not extend it to the rsync script.
Solution:
Routing the rsync interpolations through safe() closes the gap, and
extending safe() to also reject tab and newline ensures multi-line
values cannot survive into a config-file heredoc.
scripts/fill_help_tables.sql from
https://github.com/mariadb-corporation/mariadb-docs/ as of
3fb208fcdfed06beb52ef9e305017de2387ff012
Other URL changes are taken from the HTTP redirection URL
from the old KB.
Some test cases where impacted:
* ps_missed_cmds{,_bin_prot} - used a topic_id of 0 that is
now used, used MAX_INT instead.
* range_notembedded used the help data as a test case. As its not
a stable table data, replaces with static data. Test case valid
by ensuring the ANALYZE EXPLAIN and optimizer trace had the same
structure. Also valided the MDEV-24739/MDEV-25069 bug fixes from
where the test case originaled, when reverted, exhibited the
previous behavour before the bug fixes.
