1
0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Commit Graph

3352 Commits

Author SHA1 Message Date
Marko Mäkelä
46b25c3730 MDEV-41166 --backup --innodb-log-checkpoint-now may copy too much
xtrabackup_backup_func(): Request for a checkpoint synchronously
so that recv_sys.find_checkpoint() will observe the effect.

Reviewed by: Thirunarayanan Balathandayuthapani

(cherry picked from commit 98970415ed)
2026-09-21 14:55:35 +03:00
Oleksandr Byelkin
b140f801e6 Merge branch '13.0' into 13.1 2026-09-04 12:21:02 +02:00
Oleksandr Byelkin
71dd635f68 Merge tag '12.3' into 13.0 2026-09-04 12:14:15 +02:00
Oleksandr Byelkin
f860f2a937 Merge branch '11.8' into 12.3 2026-08-20 08:27:14 +02:00
Oleksandr Byelkin
3b8db45403 Merge branch '11.4' into 11.8 2026-08-19 11:28:01 +02:00
Oleksandr Byelkin
170b1d7073 Merge branch '10.11' into 11.4 2026-08-19 08:40:05 +02:00
Alexey Yurchenko
aba2caa893 MDEV-40179 Found N prepared transactions after mariabackup SST
With log_bin=ON a transaction is committed via two-phase commit (the
binary log is the second participant), so it passes through the InnoDB
XA-prepare state. While a donor is held in BLOCK_COMMIT for a mariabackup
backup, its parallel appliers (wsrep_slave_threads > 1) leave one or more
such writesets prepared-but-not-yet-committed, and the snapshot captures
them. On a freshly SST'd joiner nothing resolves these prepared
transactions: binlog crash recovery does not run (the joiner has no in-use
binlog to recover from), and the wsrep continuity-based commit is inactive
because wsrep_emulate_bin_log is FALSE when log_bin is ON. The leftover
prepared transactions then abort startup with "Found <N> prepared
transactions!". Note this does not depend on the prepared set being
non-contiguous - even a contiguous run aborts, because nothing commits
or rolls it back.

Rollback these transactions in xarecover_handlerton(). If rollback fails
flag error to cause unireg_abort().
2026-08-10 08:04:50 +03:00
Alexey Yurchenko
5785e9cbc2 MDEV-38147 error 1950 after mariabackup SST with gtid_strict_mode=ON
After a mariabackup SST the joiner could fail with

  ER_GTID_STRICT_OUT_OF_ORDER (error 1950)

while re-binlogging transactions received over IST.

The cause is that the binary log copied from the donor carries a
Gtid_list whose position can be ahead of the storage-engine snapshot
because getting snapshot and binlog are no mutually atomic, so transactions
can be present in the copied binlog that are not committed in the copied
engine snapshot.
After the SST the joiner reports the (committed) engine position to the
cluster, IST resends those transactions, and re-binlogging them under
gtid_strict_mode=ON collides with the ahead Gtid_list -> error 1950.
(MDEV-34483 made the engine snapshot stop short of the binlog, which is
what exposed this.)

The copied binary log carries no transactions the joiner needs - only a
Gtid_list - so instead of shipping and then having to truncate/reconcile
it, the joiner now starts a fresh binary log and seeds its GTID position
from the storage-engine checkpoint during recovery. That checkpoint is
the committed cluster position, i.e. exactly where IST resumes, so the
joiner's binary log stays in lockstep with the rest of the cluster and
no out-of-order GTID can occur.

This works for both wsrep_gtid_mode settings; only the binlog domain of
the cluster stream differs:

  - wsrep_gtid_mode=ON : wsrep_gtid_domain_id (cluster writes are
    re-tagged to it), which is the domain stored in the checkpoint;
  - wsrep_gtid_mode=OFF: gtid_domain_id (cluster writes keep the node's
    configured domain).

Async-replica positions (mysql.gtid_slave_pos) are part of the engine
snapshot and survive the SST unchanged, so a Galera node can still serve
as an async master or replica across the SST.

This commit:
 - sql/log.cc: adds wsrep_seed_binlog_gtid_state(), called from
   do_binlog_recovery() when the joiner has no binary log, seeding the
   binlog GTID state for the cluster domain to the SE checkpoint position.
 - scripts/wsrep_sst_mariabackup.sh: no longer moves the donor's binary
   log into place on the joiner.
 - extra/mariabackup: backward compatibility: keep shipping binlog file
   in SST but
   - on donor fix the race between rotation and shipping so that the file
     shipped is the one that had been rotated;
   - on joiner discard shipped binlog in favour of one generated by
     wsrep_seed_binlog_gtid_state().
 - sql/wsrep_sst.cc: logs the position actually adopted from storage
   (the authoritative post-SST position) rather than the script-reported
   one.
 - sql/handler.cc: downgrades the "Discovered discontinuity in recovered
   wsrep transaction XIDs" message in wsrep_order_and_check_continuity()
   from warning to debug level. With parallel appliers a snapshot
   routinely captures prepared XIDs that are not contiguous with the
   engine checkpoint, so this is normal during SST recovery and of no
   value in regular operation; the transactions past the checkpoint are
   re-delivered by the cluster (IST/SST) regardless.
 - Adds an MDEV-38147 MTR test reproducing the issue.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-10 08:04:50 +03:00
forkfun
b8dbc264f0 Merge branch '13.0' into 'main' 2026-08-03 13:42:39 +02:00
forkfun
1a4bb1bbfd Merge branch '12.3' into '13.0' 2026-07-30 16:15:40 +02:00
forkfun
33240f3c96 Merge branch '11.8' into '12.3' 2026-07-30 11:41:55 +02:00
forkfun
eaca80bf32 Merge branch '11.4' into '11.8' 2026-07-29 12:27:46 +02:00
Thirunarayanan Balathandayuthapani
41cdc5630e MDEV-39468 Mariabackup: redo log copier stalls and fails with misleading error message
Problem:
========
During backup process, log copier thread stalls at lsn and fails with
the misleading "Was only able to copy log from X to Y, not Z; try increasing
innodb_log_file_size", even when the redo was intact and the log far
from full.

Problem is that copier advances recv_sys.lsn only on CRC-validated mtr,
but it parses the redo at the server is concurrently writing to it.
InnoDB rewrites the current partial write block repeatedly as
mini-transaction fills it, so an mmap read (or) pread of a page
being written can observe a torn/intermediate image of the tail block.
If that tail block happens to parse as a longer mtr which is still valid,
has a correct CRC then recv_sys.lsn, recv_sys.offset gets advanced wrongly
and points to middle of the mini-transaction. In that case, later
parse return GOT_EOF always and copier thread never reaches target
and fails

Solution:
=========
Poll the server's durably-flushed LSN (status var Innodb_lsn_flushed) and
limit the redo log copier based on it.

backup_log_parse(): parses one mtr and if it exceeds the limit
rolls back recv_sys.lsn/offset and reports GOT_EOF; the copier
re-parses those bytes on a later pass once the fence has
advanced past them. Used on both the mmap and buffered paths.

log_copying_thread(): opens its own connection (it runs
concurrently with the main thread, which owns mysql_connection) and refreshes
the max_limit for parsing of redo log in each pass.

In the final backup phase (BACKUP STAGE BLOCK_COMMIT) it gates
on the exact target max(metadata_last_lsn, metadata_to_lsn).
so the last partial block is copied in full up to the target.

The stall diagnostic / retry spin is suppressed on a reached_parse_limit wait
so a normal "caught up, wait" is not misreported as drift.

get_log_flushed_lsn(): Added to get log_flushed_lsn from SHOW STATUS outout;
2026-07-28 17:48:06 +05:30
forkfun
08eeb6a417 Merge branch '10.11' into '11.4' 2026-07-28 13:14:38 +02:00
Daniel Black
07213c0475 MDEV-39113 MSAN/ADDR addr2line stack resolver detrimental
MSAN/ASAN test environment, the addr2line was so high in memory
utilization that it was the pick of the OOM killer to resolve the OOM
situation. Once this occurred there wasn't a saved core or gdb backtrace
of the core to resolve the issue.

To resolve this, make stack-trace default to 0 (off) for the addr2line
base implementation under memory sanitizer and address sanitizer.

MariaDB-backup also forces the enabling of stack-trace. Disabling this
unconditionally reduces the risk of a user operational impact if a
lengthy stack trace starting in a mariadb-backup critical locked period.

The mysqld--help test now excludes the stack-trace as its result is
environment dependant. The "Defaults to..." output for suppressed
variables, currently only stack-trace, is excluded.

Since thread-stack is an excluded varable, the ubsan/asan exclusions
from commits dfa6fba959 and dfa6fba959 aren't required.
2026-07-28 13:55:51 +10:00
Mohammad Tafzeel Shams
01a321fad5 MDEV-40504: Fix memory leak in mariabackup incremental copy
ibx_copy_incremental_over_full() : Replace die() with
proper error handling to avoid memory leaks when RocksDB
backup directory operations fail.
2026-07-27 11:23:28 +05:30
Daniel Black
d3afdd162b MDEV-35738 mariadb build -fsanitize=pointer-compare
invalid pointer pairs are when the length/memory of one string are
intermixed with another.

For comp_err, the end null pointer was compared to soffset within
my_strtoll10. As we didn't need the end position a NULL arg option
was compatble.

For uca-dump, Address Sanitizer raises invalid pointer pair because
argv options (opt) don't have an obvious correlation of having a start
at opt_X.length, even though the lstrncmp makes this true.

The DBUG_ASSERTS of strmov (added MDEV-11752) where incompatible with
pointer-compare. Replaced strmov with static inline version in
m_string.h using memmove that allows overlaps, and being inline allows
the uneeded parts of the implementation to be optimized away.
2026-07-25 13:44:37 +10:00
Michal Schorm
7322a6656a MDEV-39169 Replace deprecated network functions in resolveip
Replace inet_aton/inet_addr/gethostbyname/gethostbyaddr with
their modern POSIX equivalents inet_pton/getaddrinfo/getnameinfo.
This adds IPv6 support to resolveip and removes dependency on the
deprecated h_errno global.

Handle IPv6 unspecified address (::) as Null-IP-Addr, consistent
with existing IPv4 INADDR_ANY handling for 0.0.0.0.

Add MTR test covering IPv4/IPv6 reverse lookups, special
addresses, multiple arguments, mixed address types, and
invalid hostname handling.

Co-Authored-By: Claude AI <noreply@anthropic.com>
2026-07-16 14:53:33 +03:00
forkfun
867d16cea7 Merge branch '11.4' into '11.8' 2026-06-23 12:52:39 +02:00
forkfun
bbd558c386 Merge branch '10.11' into '11.4' 2026-06-22 12:26:09 +02:00
Oleksandr Byelkin
0d59c434fa Merge branch '10.6' into 10.11 2026-06-21 15:32:20 +02:00
Kristian Nielsen
37964c65fa Merge 12.3 to 13.0
This patch to mysqldump is only merged partially:

  commit 666ff1db36
  Author: Monty <monty@mariadb.org>
  Date:   Fri May 15 16:39:35 2026 +0300

      MDEV-39519 Change mariadb-dump to use SHOW REPLICA STATUS instead of SHOW SLAVE STATUS

The functionality for --dump-slave cannot be merged as the underlying
functionality was removed in this commit:

  commit 21c940d599
  Author: ParadoxV5 <paradox.ver5@gmail.com>
  Date:   Thu Dec 4 21:22:20 2025 -0700

      MDEV-37146 mariadb-dump: Replace SHOW SLAVE STATUS usage with information_schema.SLAVE_STATUS

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-16 12:24:32 +02:00
Kristian Nielsen
fcc5fc8916 Merge 11.8 to 12.3
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 16:53:03 +02:00
Kristian Nielsen
420e28ce28 Merge 11.4 to 11.8
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 15:15:10 +02:00
Kristian Nielsen
85ec122c15 Merge 10.11 to 11.4
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 12:50:45 +02:00
Marko Mäkelä
3a29e4fa8c MDEV-39971: generate_option_list may invoke pclose(nullptr)
call_mariadbd(): Remove the redundant and potentially harmful call
of my_pclose(f) when my_popen() failed. At least in some versions
of GNU libc the function pclose(3) is declared with
__attribute__((nonnull)).

Fixes up commit 7828fb475b (MDEV-32745)
2026-06-10 17:40:05 +10:00
Otto Kekäläinen
87825d38d7 Fix Lintian flagged spelling in MariaDB Server repository
Fixes following Lintian nags:

    X: mariadb-plugin-connect: spelling-error-in-binary Exeption Exception [usr/lib/mysql/plugin/ha_connect.so]
    X: mariadb-client-core: spelling-error-in-binary specifiying specifying [usr/bin/mariadb-migrate-config-file]
2026-06-03 10:35:17 +10:00
Yuchen Pei
33d0ab084c MDEV-39789 Fix compiling without perfschema 2026-05-29 14:33:48 +10:00
Oleksandr Byelkin
9f98f82b14 Merge branch '11.8' into 12.3 2026-05-26 21:15:58 +02:00
Sergei Golubchik
6b35f1748d Merge branch '11.4' into 11.8 2026-05-23 21:43:50 +02:00
Sergei Golubchik
b4a80422bf Merge branch '10.11' into 11.4 2026-05-23 16:59:15 +02:00
Sergei Golubchik
197f92bee0 Merge branch '10.6' into 10.11 2026-05-23 16:55:50 +02:00
Oleksandr Byelkin
f572addcc1 Merge branch '12.3' into 13.0 2026-05-22 09:52:05 +02:00
Oleksandr Byelkin
3f388a1cce Merge branch '11.8' into 12.3 2026-05-22 09:27:18 +02:00
Sergei Golubchik
a343e60d2f Merge branch '11.4' into 11.8 2026-05-21 20:17:03 +02:00
Sergei Golubchik
5000962deb Merge branch '10.11' into 11.4 2026-05-21 16:54:28 +02:00
Sergei Golubchik
4b2289fec6 Merge branch '10.6' into 10.11 2026-05-21 16:19:19 +02:00
Sergei Golubchik
1648170c5c MDEV-39565 missing filename check in mariadb-backup --decompress
check for tablename-safe characters in backed up table files
2026-05-20 22:49:20 +02:00
Marko Mäkelä
b89cb5d5c7 Merge mariadb-10.11.17 into 10.11 2026-05-19 08:12:14 +03:00
Daniel Black
8923afa5c2 MDEV-39584: update help sql tables and replace KB links
scripts/fill_help_tables.sql from
https://github.com/mariadb-corporation/mariadb-docs/ as of
3fb208fcdfed06beb52ef9e305017de2387ff012

Other URL changes are taken from the HTTP redirection URL
from the old KB.

Some test cases where impacted:
* ps_missed_cmds{,_bin_prot} - used a topic_id of 0 that is
 now used, used MAX_INT instead.

* range_notembedded used the help data as a test case. As its not
  a stable table data, replaces with static data. Test case valid
  by ensuring the ANALYZE EXPLAIN and optimizer trace had the same
  structure. Also valided the MDEV-24739/MDEV-25069 bug fixes from
  where the test case originaled, when reverted, exhibited the
  previous behavour before the bug fixes.
2026-05-15 13:30:21 +10:00
Oleksandr Byelkin
9cb70f0220 Merge branch '11.4' into 11.8 2026-05-08 08:11:03 +02:00
Oleksandr Byelkin
7598e4e95d Merge branch '10.11' into 11.4 2026-05-08 08:00:01 +02:00
Oleksandr Byelkin
6828ff7390 Merge branch '10.6' into 10.11 2026-05-08 07:38:22 +02:00
Marko Mäkelä
3394a9d114 MDEV-37949: Implement innodb_log_archive
The InnoDB write-ahead log file in the old innodb_log_archive=OFF
format is named ib_logfile0, pre-allocated to innodb_log_file_size and
written as a ring buffer. This is good for write performance and space
management, but unsuitable for arbitrary point-in-time recovery or for
facilitating efficient incremental backup.

innodb_log_archive=ON: A new format where InnoDB will create and
preallocate files ib_%016x.log, instead of writing a circular file
ib_logfile0. Each file will be pre-allocated to innodb_log_file_size.
Once a log fills up, we will create and pre-allocate another log file,
to which log records will be written.  Upon the completion of the first
log checkpoint in a recently created log file, the old log file will
be marked read-only, signaling that there will be no further writes to
that file, and that the file may safely be moved to long-term storage.

The file name includes the log sequence number (LSN) at file offset
12288 (log_t::START_OFFSET). Each checkpoint is identified by storing
a 64-bit big-endian offset into an optional sequence of FILE_MODIFY
records followed by a FILE_CHECKPOINT record, between 12288 and the
end of the file.

The innodb_encrypt_log format is identified by storing the encryption
information at the start of the log file. The first 64-bit value will
be 1, which is an invalid checkpoint offset. Each
innodb_log_archive=ON log must use the same encryption parameters.
Changing innodb_encrypt_log or related parameters is only possible by
setting innodb_log_archive=OFF and restarting the server, which will
permanently lose the history of the archived log.

The maximum number of log checkpoints that the innodb_log_archive=ON
file header can represent is limited to 12288/8=1536 when using
innodb_encrypt_log=OFF. If we run out of slots in a log file, each
subsequently completed checkpoint in that log file will overwrite the
last slot in the checkpoint header, until we switch to the next log.

innodb_log_recovery_start: The checkpoint LSN to start recovery from.
This will be useful when recovering from an archived log. This is useful
for restoring an incremental backup (applying InnoDB log files that were
copied since the previous restore).

innodb_log_recovery_target: The requested LSN to end recovery at.
When this is set, all persistent InnoDB tables will be read-only, and
no writes to the log are allowed. The intended purpose of this setting
is to prepare an incremental backup, as well as to allow data
retrieval as of a particular logical point of time.

Setting innodb_log_recovery_target>0 is much like setting
innodb_read_only=ON, with the exception that the data files may be
written to by crash recovery, and locking reads will conflict with any
incomplete transactions as necessary, and all transaction isolation
levels will work normally (not hard-wired to READ UNCOMMITTED).

srv_read_only_mode: When this is set (innodb_read_only=ON), also
recv_sys.rpo (innodb_log_recovery_target) will be set to the current LSN.
This ensures that it will suffice to check only one of these variables
when blocking writes to persistent tables.

The status variable innodb_lsn_archived will reflect the LSN
since when a complete InnoDB log archive is available. Its initial
value will be that of the new parameter innodb_log_archive_start.
If that variable is 0 (the default), the innodb_lsn_archived will
be recovered from the available log files. If innodb_log_archive=OFF,
innodb_lsn_archived will be adjusted to the latest checkpoint every
time a log checkpoint is executed. If innodb_log_archive=ON, the value
should not change.

SET GLOBAL innodb_log_archive=!@@GLOBAL.innodb_log_archive will take
effect as soon as possible, possibly after a log checkpoint has been
completed. The log file will be renamed between ib_logfile0 and
ib_%016x.log as appropriate.

When innodb_log_archive=ON, the setting SET GLOBAL innodb_log_file_size
will affect subsequently created log files when the file that is being
currently written is running out. If we are switching log files exactly
at the same time, then a somewhat misleading error message
"innodb_log_file_size change is already in progress" will be issued.

no_checkpoint_prepare.inc: A new file, to prepare for subsequent
inclusion of no_checkpoint_end.inc. We will invoke the server to
parse the log and to determine the latest checkpoint.

All --suite=encryption tests that use innodb_encrypt_log
will be skipped for innodb_log_archive=ON, because enabling
or disabling encryption on the log is not possible without
temporarily setting innodb_log_archive=OFF and restarting
the server. The idea is to add the following arguments to an
invocation of mysql-test/mtr:

--mysqld=--loose-innodb-log-archive \
--mysqld=--loose-innodb-log-recovery-start=12288 \
--mysqld=--loose-innodb-log-file-mmap=OFF \
--skip-test=mariabackup

Alternatively, specify --mysqld=--loose-innodb-log-file-mmap=ON
to cover both code paths.

The mariabackup test suite must be skipped when using the
innodb_log_archive=ON format, because mariadb-backup will only
support the old ib_logfile0 format (innodb_log_archive=OFF).

A number of tests would fail when the parameter
innodb_log_recovery_start=12288 is present, which is forcing
recovery to start from the beginning of the history
(the database creation). The affected tests have been adjusted with
explicit --innodb-log-recovery-start=0 to override that:

(0) Some injected corruption may be "healed" by replaying the log
from the beginning. Some tests expect an empty buffer pool after
a restart, with no page I/O due to crash recovery.
(1) Any test that sets innodb_read_only=ON would fail with an error
message that the setting prevents crash recovery, unless
innodb_log_recovery_start=0.
(2) Any test that changes innodb_undo_tablespaces would fail in crash
recovery, because crash recovery assumes that the undo tablespace ID
that is available from the undo* files corresponds with the start of
the log. This is an unforunate design bug which we cannot fix easily.

log_sys.first_lsn: The start of the current log file, to be consulted
in log_t::write_checkpoint() when renaming files.

log_sys.archived_lsn: New field: The value of innodb_lsn_archived.

log_sys.end_lsn: New field: The log_sys.get_lsn() when the latest
checkpoint was initiated. That is, the start LSN of a possibly empty
sequence of FILE_MODIFY records followed by FILE_CHECKPOINT.

log_sys.resize_target: The value of innodb_log_file_size that will be
used for creating the next archive log file once the current file (of
log_sys.file_size) fills up.

log_sys.archive: New field: The value of innodb_log_archive.

log_sys.next_checkpoint_no: Widen to uint16_t. There may be up to
12288/8=1536 checkpoints in the header.

log_sys.log: If innodb_log_archive=ON, this file handle will be kept
open also in the PMEM code path.

log_sys.resize_log: If innodb_log_archive=ON, we may have two log
files open both during normal operation and when parsing the log. This
will store the other handle (old or new file).

log_sys.resize_buf: In the memory-mapped code path, this will point
to the file resize_log when innodb_log_archive=ON.

recv_sys.log_archive: All innodb_log_archive=ON files that will be
considered in recovery.

recv_sys.was_archive: A flag indicating that an innodb_log_archive=ON
file is in innodb_log_archive=OFF format.

log_sys.is_pmem, log_t::is_mmap_writeable(): A new predicate.
If is_mmap_writeable(), we assert and guarantee buf_size == capacity().

log_t::checkpoint_age_max(): Return the maximum checkpoint age.
When innodb_log_archive=ON the limit is innodb_log_file_size larger.

log_t::archive_new_write(): Create and allocate a new log file, and
write the outstanding data to both the current and the new file, or
only to the new file, until write_checkpoint() completes the first
checkpoint in the new file. If the log file would overflow,
increase innodb_log_file_size to the next multiple of 4096 bytes.

log_t::archive_create(bool): Create and allocate an archive log file.
On POSIX, ensure that we get the exact correct file size by invoking
ftruncate() on POSIX, because os_file_set_size() would only
truncate files on Microsoft Windows.  Remember the file handle
of the current log in resize_log, so that write_checkpoint() will be
able to make it read-only.

log_t::archived_mmap_switch_prepare(): Invoke archive_create() unless
another thread already did it.

log_t::archived_mmap_switch_complete(): Switch to the buffer that was
created in log_t::archived_mmap_switch_prepare(). Remember the old
log_sys.buf in log_sys.checkpoint_buf, to be unmapped in
log_t::write_checkpoint().

buf_flush_archive_create(): Create a spare archive log file if needed.

buf_flush_page_cleaner(): Invoke buf_flush_archive_create() about
one second after a checkpoint where we switched archived log files.
This allows any writer threads to resume activity after we completed
the previous checkpoint.

log_t::write_checkpoint(): Allow an old checkpoint to be completed in
the old log file even after a new one has been created. If we are
writing the first checkpoint in a new log file, we will mark the old
log file read-only. We will also update log_sys.first_lsn unless it
was already updated in ARCHIVED_MMAP code path. In that code path,
there is the special case where log_sys.resize_buf == nullptr and
log_sys.checkpoint_buf points to log_sys.resize_log (the old log file
that is about to be made read-only). In this case, log_sys.first_lsn
will already point to the start of the current log_sys.log, even
though the switch has not been fully completed yet.
Try to preallocate the next archive file if needed, with the goal that
when we need the file it will already be ready for use.

log_t::write_checkpoint(), log_checkpoint_low(), log_checkpoint(),
buf_flush_sync_for_checkpoint(): Return the LSN on which to invoke
archive_create(), or 0 if no spare archive log file needs to be created.

log_t::header_rewrite(my_bool): Rewrite the log file header before or
after renaming the log file, and write a message about the change,
so that there will be a chance to recover in case the server is being
killed during this operation.  The recovery of the last ib_%016%.log
does tolerate also the ib_logfile0 format.

log_t::set_archive(my_bool,THD): Implement SET GLOBAL innodb_log_archive.
An error will be returned if non-archived SET GLOBAL innodb_log_file_size
(log file resizing) is in progress. Wait for checkpoint if necessary.
The current log file will be renamed to either ib_logfile0 or
ib_%016x.log, as appropriate. In SET GLOBAL innodb_log_archive=OFF,
we trigger a write-ahead of the log if necessary, to prevent overrun.

log_t::archive_rename(): Rename an archived log to ib_logfile0 on recovery
in case there had been a crash during set_archive().

log_t::archive_set_size(): A new function, to ensure that
log_sys.resize_target is set on startup.

log_checkpoint_low(): Do not prevent a checkpoint at the start of a file.
We want the first innodb_log_archive=ON file to start with a checkpoint.

log_t::create(lsn_t): Initialize last_checkpoint_lsn. Initialize the
log header as specified by log_sys.archive (innodb_log_archive).

log_write_buf(): Add the parameter max_length, the file wrap limit.

log_write_up_to(), mtr_t::commit_log_release<bool mmap=true>():
If we are switching log files, invoke buf_flush_ahead(lsn, false)
to ensure that a log checkpoint will be completed in the new file.

mtr_t::finish_writer(): Specialize for innodb_log_archive=ON.

mtr_t::commit_file(): Ensure that log archive rotation will complete.

log_t::append_prepare<log_t::ARCHIVED_MMAP>(): Special case.

log_t::append_prepare(): Remove the call to set_check_for_checkpoint()
that was duplicating logic in log_close().

log_t::get_path(): Get the name of the current log file.

log_t::get_circular_path(size_t): Get the path name of a circular file.
Replaces get_log_file_path().

log_t::get_archive_path(lsn_t): Return a name of an archived log file.

log_t::append_archive_name(): Append the archive log file name
to a path string.

log_t::checkpoint_margin(): Replaces log_checkpoint_margin().
If a new archived log file has been created, wait for the
first checkpoint in that file. Revise the formula when
log_sys.need_checkpoint needs to hold. When innodb_log_archive=ON,
we can allow the checkpoint to be older by innodb_log_file_size.

log_close(): Relax the log_sys.need_checkpoint condition
for innodb_log_archive=ON.

srv_log_rebuild_if_needed(): Never rebuild if innodb_log_archive=ON.
The setting innodb_log_file_size will affect the creation of
subsequent log files. The parameter innodb_encrypt_log cannot be
changed while the log is in the innodb_log_archive=ON format.

log_t::attach(), log_mmap(): Add the parameter log_access,
to distinguish memory-mapped or read-only access.

log_t::attach(): When disabling innodb_log_file_mmap, read
checkpoint_buf from the last innodb_log_archive=ON file.

log_t::clear_mmap(): Clear the tail of the checkpoint buffer
if is_mmap_writeable().

log_t::set_recovered(): Invoke clear_mmap(), and restore the
log buffer to the correct position.

recv_sys_t::apply(): Let log_t::clear_mmap() enable log writes.

log_file_is_zero(): Check if a log file starts with NUL bytes
(is a preallocated file).

recv_sys_t::find_checkpoint(): Find and remember the checkpoint position
in the last file when innodb_log_recovery_start points to an older file.
When innodb_log_file_mmap=OFF, restore log_sys.checkpoint_buf from
the latest log file. If the last archive log file is actually
in innodb_log_archive=OFF format despite being named ib_%016.log,
try to recover it in that format. If the circular ib_logfile0 is missing,
determine the oldest archived log file with contiguous LSN.
If innodb_log_archive=ON, refuse to start if ib_logfile0 exists.
Open non-last non-preallocated archived log file in read/write mode.

recv_sys_t::find_checkpoint_archived(): Validate each checkpoint in
the current file header, and by default aim to recover from the last
valid one. Terminate the search if the last validated checkpoint
spanned two files. If innodb_log_recovery_start has been specified,
attempt to validate it even if there is no such information stored
in the checkpoint header.

log_parse_file(): Do not invoke fil_name_process() during
recv_sys_t::find_checkpoint_archived(), when we tolerate FILE_MODIFY
records while looking for a FILE_CHECKPOINT record.

recv_scan_log(): Invoke log_t::archived_switch_recovery() upon
reaching the end of the current archived log file.

log_t::archived_switch_recovery_prepare(): Make use of
recv_sys.log_archive and open all but the last file read-only.

log_t::archived_switch_recovery(): Switch files in the pread() code path.

log_t::archived_mmap_switch_recovery_complete(): Switch files in the
memory-mapped code path.

recv_warp: A pointer wrapper for memory-mapped parsing that spans two
archive log files.

recv_sys_t::parse_mmap(): Use recv_warp for innodb_log_archive=ON.

recv_sys_t::parse(): Tweak some logic for innodb_log_archive=ON.

log_t::set_recovered_checkpoint(): Set the checkpoint on recovery.
Updates also the end_lsn.

log_t::set_recovered_lsn(): Also update flush_lock and write_lock,
to ensure that log_write_up_to() will be a no-op. Invoke
unstash_archive_file() in case there was a garbage (pre-allocated)
file at the end which was not parsed at all.

log_t::persist(): Even if the flushed_to_disk_lsn does not change,
we may want to reset the write_lsn_offset.
2026-05-05 13:57:50 +03:00
Marko Mäkelä
e13d89949c Merge 12.3 into main 2026-05-05 13:14:42 +03:00
Sergei Golubchik
04e0901077 MDEV-32745 followup for 7828fb475b
* add a test for new --cat_file feature
* fix off-by-one error in --cat_file lines limit
* fix broken INCLUDE_DIRECTORIES in extra/
* fix typos
* remove mtr-specific workaround from the tool, solve it in the test
* remove dead code
* fixed memory leak in mariadb-migration-config-file [client-server] section
* fixed memory leak in mariadbd --character_set_server=xxx
  (execution time for main.mariadb-migrate-config-file went down
  from 150s to 2s)
* moved tool header to the tool dir
* avoid tripple-initialization of plugins
* disable tool builds in ASAN builds (because of RocksDB)
* compilation failure on x86 (signedness)
2026-05-04 18:21:26 +02:00
Jan Lindström
f6c02899fe MDEV-39483 : Hang on mariabackup SST
This regression was caused by commit 794b1d0ec5
Binlog-in-engine: New binlog implementation integrated in InnoDB.
Mariabackup request BACKUP STAGE BLOCK_COMMIT MDL-lock
using m_bs_con connection. Because we have wsrep,
write_galera_info is called using mysql_connection.
Note that m_bs_con and mysql_connection are different
connections. In write_galera_info write_current_binlog_file
is called and FLUSH BINLOG LOGS is executed. In reload_acl_and_cache
MDL_BACKUP_START MDL-lock is requested. Because we already
have conflicting MDL-lock for BLOCK_COMMIT in different THD
it has to wait. This wait ends on timeout and backup fails
causing mariabackup SST to fail and node will not join the cluster.

Fixed by using same connection for write_galera_info as for
BACKUP STAGE BLOCK_COMMIT i.e. m_bs_con.
2026-05-04 11:07:20 +03:00
Monty
f860cb0f64 Added pcre2-8 for embedded server 2026-05-01 15:14:06 +03:00
Michael Widenius
7828fb475b MDEV-32745 Add a simple MySQL to MariaDB upgrade helper
The tool is named mariadb-migrate-config-file.
The main purpose of the tool is to change MySQL option
files to work both for MySQL and MariaDB.
There are options to do the changes in the options file inline,
or at-end-of-file. One can also remove or comment unknown options.

The list of supported options is generated compile time from
mariadbd --help. All server options, including compiled plugins, are
supported.

The bulk of the code comes from Väinö.
Monty has updated it with a lot of extra options.
Wlad helped with cmake integration

Other things:
- Fixed a memory leak in sql_plugin.cc
- plugin-load will now in case of errors try to load all given plugins
  before aborted
- If silent-startup is used, plugin-load will not give errors for
  plugins it cannot load or warnings about plugin marturity level.
- my_rm_tree() will now delete symlinks, not the actual file, if
  MY_NOSYMLINK flag is used.
- my_stat() will now give data for symlink if MY_NOSYMLINKS is used.
- Added 'number of lines' option to mysqltest --cat_file

@Authors: Väinö Mäkelä <vaino.makela@iki.fi>,monty@mariadb.com
2026-05-01 15:14:06 +03:00
Sergei Golubchik
6060eec596 MDEV-34570 mariabackup prepare fails with data-at-rest encryption and "-u root" option
rewrite mariadb-backup "early" option parsing to use my_getopt

+ proper handling of values separated from the option by a space (not =)
+ case insensitive and -/_ insensitive comparison
- multiple --defaults-group don't work
- multiple --login-path don't work
- --incremental-dir overwrites --target-dir, not "whatever comes first"
2026-04-29 10:29:17 +02:00