The server_audit_timestamp_format %f specifier always rendered zero
microseconds. The server downcast the precise time to seconds before
passing it to audit plugins, and the plugin then re-fetched the time
itself at write time.
Pass the server's high-resolution time through the audit API instead:
- extend mysql_event_general with general_time_microseconds (added in
MYSQL_AUDIT_INTERFACE_VERSION 0x0304), keeping general_time in
seconds for backward compatibility
- the server_audit plugin uses event->general_time_microseconds for
query log entries instead of re-fetching the time at write time
Connection and table events carry no timestamp in the audit API, so the
plugin keeps taking the time at event time for those entries.
With log_bin=ON a transaction is committed via two-phase commit (the
binary log is the second participant), so it passes through the InnoDB
XA-prepare state. While a donor is held in BLOCK_COMMIT for a mariabackup
backup, its parallel appliers (wsrep_slave_threads > 1) leave one or more
such writesets prepared-but-not-yet-committed, and the snapshot captures
them. On a freshly SST'd joiner nothing resolves these prepared
transactions: binlog crash recovery does not run (the joiner has no in-use
binlog to recover from), and the wsrep continuity-based commit is inactive
because wsrep_emulate_bin_log is FALSE when log_bin is ON. The leftover
prepared transactions then abort startup with "Found <N> prepared
transactions!". Note this does not depend on the prepared set being
non-contiguous - even a contiguous run aborts, because nothing commits
or rolls it back.
Rollback these transactions in xarecover_handlerton(). If rollback fails
flag error to cause unireg_abort().
After a mariabackup SST the joiner could fail with
ER_GTID_STRICT_OUT_OF_ORDER (error 1950)
while re-binlogging transactions received over IST.
The cause is that the binary log copied from the donor carries a
Gtid_list whose position can be ahead of the storage-engine snapshot
because getting snapshot and binlog are no mutually atomic, so transactions
can be present in the copied binlog that are not committed in the copied
engine snapshot.
After the SST the joiner reports the (committed) engine position to the
cluster, IST resends those transactions, and re-binlogging them under
gtid_strict_mode=ON collides with the ahead Gtid_list -> error 1950.
(MDEV-34483 made the engine snapshot stop short of the binlog, which is
what exposed this.)
The copied binary log carries no transactions the joiner needs - only a
Gtid_list - so instead of shipping and then having to truncate/reconcile
it, the joiner now starts a fresh binary log and seeds its GTID position
from the storage-engine checkpoint during recovery. That checkpoint is
the committed cluster position, i.e. exactly where IST resumes, so the
joiner's binary log stays in lockstep with the rest of the cluster and
no out-of-order GTID can occur.
This works for both wsrep_gtid_mode settings; only the binlog domain of
the cluster stream differs:
- wsrep_gtid_mode=ON : wsrep_gtid_domain_id (cluster writes are
re-tagged to it), which is the domain stored in the checkpoint;
- wsrep_gtid_mode=OFF: gtid_domain_id (cluster writes keep the node's
configured domain).
Async-replica positions (mysql.gtid_slave_pos) are part of the engine
snapshot and survive the SST unchanged, so a Galera node can still serve
as an async master or replica across the SST.
This commit:
- sql/log.cc: adds wsrep_seed_binlog_gtid_state(), called from
do_binlog_recovery() when the joiner has no binary log, seeding the
binlog GTID state for the cluster domain to the SE checkpoint position.
- scripts/wsrep_sst_mariabackup.sh: no longer moves the donor's binary
log into place on the joiner.
- extra/mariabackup: backward compatibility: keep shipping binlog file
in SST but
- on donor fix the race between rotation and shipping so that the file
shipped is the one that had been rotated;
- on joiner discard shipped binlog in favour of one generated by
wsrep_seed_binlog_gtid_state().
- sql/wsrep_sst.cc: logs the position actually adopted from storage
(the authoritative post-SST position) rather than the script-reported
one.
- sql/handler.cc: downgrades the "Discovered discontinuity in recovered
wsrep transaction XIDs" message in wsrep_order_and_check_continuity()
from warning to debug level. With parallel appliers a snapshot
routinely captures prepared XIDs that are not contiguous with the
engine checkpoint, so this is normal during SST recovery and of no
value in regular operation; the transactions past the checkpoint are
re-delivered by the cluster (IST/SST) regardless.
- Adds an MDEV-38147 MTR test reproducing the issue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Protect all READ_ONLY sysvars from run-time changes.
Put them into a separate section and use my_virtual_mem_protect()
to make this section read-only before the server starts accepting
connections.
Verify that they're all protected in the sys_var constructor.
One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables
On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup
Assisted-By: Claude:claude-4.8-opus
This patch to mysqldump is only merged partially:
commit 666ff1db36
Author: Monty <monty@mariadb.org>
Date: Fri May 15 16:39:35 2026 +0300
MDEV-39519 Change mariadb-dump to use SHOW REPLICA STATUS instead of SHOW SLAVE STATUS
The functionality for --dump-slave cannot be merged as the underlying
functionality was removed in this commit:
commit 21c940d599
Author: ParadoxV5 <paradox.ver5@gmail.com>
Date: Thu Dec 4 21:22:20 2025 -0700
MDEV-37146 mariadb-dump: Replace SHOW SLAVE STATUS usage with information_schema.SLAVE_STATUS
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
- Change thd->backup_commit_lock from MDL_request* to MDL_ticket*,
and convert related call sites in handler.cc, log.cc, sql_class.cc,
and xa.cc to use MDL_ACQUIRE_LOCK.
- Convert reload_acl_and_cache() in sql_reload.cc, purge_master_logs()
and reset_master() in sql_repl.cc to MDL_ACQUIRE_LOCK, holding the
ticket in a local MDL_ticket* (these functions used a local
MDL_request originally and should not touch thd->backup_commit_lock).
- Convert acquire_lock() in partition_info.cc to MDL_ACQUIRE_LOCK.
- Add MDL_REQUEST_LIST_ADD() helper for enqueuing lock requests into
MDL_request_list, and convert call sites in sql_base.cc and sp.cc.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Don't attempt to rotate any old-style binlog files if trying to set
@@binlog_checksum. The @@binlog_checksum variable is unused with the
new binlog, and there are no old-style binlog files to rotate.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
`DBUG_EXECUTE_IF` is a macro that uses
the infamous `do {…} while(0)` “idiom”.
This detail causes the `break` in the code block to terminate the
`while(0)` behind the macro instead of the `for (;;)` outside.
Consequently, this fault simulation does not terminate the normal flow,
which in turn overwrites the simulated fault with a non-fault outcome.
Since this fault simulation is the first step of the loop,
this commit fixes the fault injection by moving it before the loop.
Remove the unscoped #pragma GCC diagnostic ignored
"-Wdeprecated-declarations" from include/violite.h and replace all
sprintf/vsprintf calls with snprintf/vsnprintf across the codebase.
Where possible, pass actual buffer sizes as function parameters
instead of using hardcoded constants or recomputing strlen().
Notable API changes:
- get_date(): added to_len parameter for buffer size
- nice_time(), end_timer() in client/mysql.cc: added buff_size
- generate_new_name(): added name_size parameter
- calc_md5(): added buffer_size parameter (fixes off-by-one)
- MD5_HASH_TO_STRING macro: added _size parameter
- Added MYSQL_UDF_MAX_RESULT_LENGTH constant in mysql_com.h
Also fixed a pre-existing bug in sql/rpl_mi.cc where the ellipsis
"..." was written to the wrong buffer (dbuff instead of buff).
Vendored code is excluded: extra/readline, extra/wolfssl,
wsrep-lib, libmariadb, storage/mroonga/vendor, zlib.
Remove function checks that are guaranteed by C89/C99 standards to
reduce the excessive time spent during CMake configuration step. Also,
clean up related compatibility code.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
When adding in variable type binlog_row_event_fragment_threshold, it
was made a ulong to mimick max_allowed_packet and
slave_max_allowed_packet variables (as they are intertwined). However,
using ulong is bad practice as it has a different type depending on the
architecture. This is shown by test sys_vars.sysvars_server_notembedded
where the variable shows as `BIGINT UNSIGNED` on 64-bit systems, and
`INT UNSIGNED` on 32-bit systems.
This patch updates the variable to be uint (32-bit), as there is a
fundamental limitation of 4GB on event size, so the new
Partial_rows_log_event event couldn't exceed that size anyway.
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
This patch solves two problems:
1. Rows log events cannot be transmitted to the slave if their
size exceeds slave_max_packet_size (max 1GB at the time of
writing this patch, i.e. MariaDB 12.3)
2. Rows log events cannot be binlogged if they are larger than
4GB because the common binlog event header field event_len is
32-bits.
This patch adds support for fragmenting large Rows_log_events
through a new event type, Partial_rows_log_event. When any given
instantiation of a Rows_log_event (e.g. Write_rows_log_event, etc)
is too large to be sent to a replica (i.e. larger than the value
slave_max_allowed_packet, as configured on a replica), then the rows
event must be fragmented into sub-events (i.e.
Partial_rows_log_events), so the event can be transmitted to the
replica. The replica will then take the content of each of these
Partial_rows_log_events, and join them together into a large
Rows_log_event to be executed as normal. Partial_rows_log_events
are written to the binary log sequentially, and the replica assembles
the events in the order they are binlogged.
To control the size of each Partial_rows_log_event, a new system
variable is added: binlog_row_event_fragment_threshold. All
Partial_rows_log_events within the same group will have this size,
except for the last, which will take up the remaining length of the
underlying Rows_log_event.
Each Partial_rows_log_event stores its sequence number (seq_no) in the
overall series of fragments, the total number of fragments needed to
re-assemble the Rows_log_event (total_fragments), a uchar for flags for
embedding extra data, and any additional data as specified by the
flags. Currently, only the first event in a grouping will have
additional data: it will set the first bit in the flags
(FL_ORIG_EVENT_SIZE) to indicate it will be storing the total size of
the underlying Rows_log_event.
The cached Rows_log_event data is fragmented into
Partial_rows_log_events as follows. The primary will still generate a
Rows_log_event to write to the binlog; however, during the actual
writing process, the raw data of the rows event is split into
fragments, each covering some continuous section of the rows data. A
Partial_rows_log_event is created for each continuous section, and the
Partial_rows_log_events are written sequentially in-place of the
too-large Rows_log_event. The original data to be fragmented will
include a header and data header; however, will not include a checksum,
as each Partial_rows_log_event will have a checksum for validation, as
well as a sequence_number and total number of fragments to ensure all
fragments are present.
The re-assembly and execution of the original Rows_log_event on the
replica happens in Partial_rows_log_event::do_apply_event(). The rgi
is extended with a memory buffer that holds all data for the
original Rows_log_event. As each Partial_rows_log_event is
ingested, its Rows_log_event content is appended to this memory
buffer. Once the last fragment has added its content, a new
Rows_log_event is created using that buffer, and executed.
A new error message is added to indicate that the slave has received an
invalid stream of Partial_rows_log_events:
ER_PARTIAL_ROWS_LOG_EVENT_BAD_STREAM.
Note this commit only adds the server logic for fragmentic and
assembling events, the client logic (mysqlbinlog) is in the next
commit.
Alternative designs considered were:
1. Alternative 1: Change the master-slave communication protocol
such that the master would send events in chunks of size
slave_max_allowed_packet. Though this is still a valid idea,
and would solve the first problem described in this commit
message, this would still leave the limitation that
Rows_log_events could not exceed 4GB. Eventually, this change
should still be addressed (e.g. in MDEV-37853), and for users
of binlog-in-engine (MDEV-34705) which supports out-of-band
binlogging of events, this MDEV-32570 work will be superseded.
2. Alternative 2: Create a generic “Container_log_event” with the
intention to embed various other types of event data for
various purposes, with flags that describe the purpose of a
given container. This seemed overboard, as there is already a
generic Log_event framework that provides the necessary
abstractions to fragment/reassemble events without adding in
extra abstractions.
3. Alternative 3: Add a flag to Rows_log_event with semantics to
overwrite/correct the event_len field of the common event
header to use a 64-bit field stored in the data_header of the
Rows_log_event; and also do alternative 1, so the master would
send the large (> 4GB) rows event in chunks. This approach
would add too much complexity (changing both the binlogging
and transport layer); as well as introduce inconsistency to
the event definition (event_len and next_event_position would
no longer have consistent meanings).
Reviewed-by: Hemant Dangi <hemant.dangi@mariadb.com>
Acked-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
Implement an improved binlog implementation that is integrated into
the storage engine. The new implementation is enabled with the
--binlog-storage-engine option. Initially the InnoDB storage engine
implements the binlog.
Integrating the binlog in the storage engine improves performance,
since it makes the InnoDB redo log the single source of truth and
avoids the need for expensive two-phase commit between binlog and
engine. It also makes it possible to disable durability (set
--innodb-flush-log-at-trx-commit=0) to further improve performance,
while still preserving the ability to recover the binlog and database
into a consistent state after a crash.
The new binlog implementation also greatly improves the internal
design and implementation of the binlog, and enables future
enhancements for replication.
This is a squash of the original 11.4-based patch series.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
When ddl log recovery needs to binlog during the crash recovery, the
GTID was binlogged without the required "ddl" marker. This caused wrong
behaviour on the slave when using parallel replication.
Fixed by explicitly marking the "current statement" as DDL when binlogging
in ddl log crash recovery.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This patch fixes that ALTER TABLE can call wakeup_subsequent_commits() too
early and allow following event groups to commit out-of-order in parallel
replication. Fixed by calling suspend_subsequent_commits() at the start
of the ALTER.
Could be seen as an assertion:
!tmp_gco->next_gco || tmp_gco->last_sub_id > sub_id
(Normally this is prevented because an ALTER TABLE will run in its own GCO,
and thus no following event groups can even start; however the missing DDL
mark caused by MDEV-38429 made this visible. And calling
wakeup_subsequent_commits() too early is wrong in any case).
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
compiling mariadb from a git tree
Fixed sprintf deprecation warnings compiling on MacOSX.
Replaced some sprintf calls with equivalent snprintf calls,
enough so that "normal" compile on MacOSX (as documented
in the docs) completes without warnings.
Two transactions could binlog their completions in opposite to how it
is done in Engine. That is is rare situations ROLLBACK in Engine of
the dependency parent transaction could be scheduled by the
transaction before its binlogging. That give a follower dependency
child one get binlogged ahead of the parent.
For fixing this bug its necessary to ensure the binlogging phase is
always first one in the internal one-phase rollback protocol.
The commit combines
1. a code polishing piece over a part of MDEV-21117 that
made binlog handlerton always commit first in no-2pc cases and
2. the same rule now applies to the rollback.
An added test demonstrates how the child could otherwise reach binlog
before its parent.
Two transactions could binlog their completions in opposite to how it
is done in Engine. That is is rare situations ROLLBACK in Engine of
the dependency parent transaction could be scheduled by the
transaction before its binlogging. That give a follower dependency
child one get binlogged ahead of the parent.
For fixing this bug its necessary to ensure the binlogging phase is
always first one in the internal one-phase rollback protocol.
The commit makes sure the binlog handlerton always rollbacks as first
handlerton in no-2pc cases.
An added test demonstrates how the child could otherwise reach binlog
before its parent.
The binary log could be corrupted when committing a large transaction
(i.e. one whose data exceeds the binlog_cache_size limit and spills
into a tmp file) in binlog_format=row if the server's --tmp-dir is
full. The corruption that happens is only the GTID of the errored
transaction would be written into the binary log, without any
body/finalizing events. This would happen because the content of the
transaction wasn't flushed at the proper time, and the transaction's
binlog cache data was not durable while trying to copy the content
from the binlog cache file into the binary log itself. While switching
the tmp file from a WRITE_CACHE to a READ_CACHE, the server would see
there is still data to flush in the cache, and first try to flush it.
This is not a valid time to flush that data to the temporary file
though, as:
1. The GTID event has already been written directly to the binary
log. So if this flushing fails, it leaves the binary log in a
corrupted state.
2. This is done during group commit, and will slow down other
concurrent transactions, which are otherwise ready to commit.
This patch fixes these issues by ensuring all transaction data is
fully flushed to its temporary file (if used) before starting any
critical paths, i.e. in binlog_flush_cache(). Note that if the binlog
cache is solely in-memory, this flush-to-temporary-file is skipped.
Reviewed-by: Andrei Elkin <andrei.elkin@mariadb.com>
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
This bug happens when using galera, max_tmp_session_space_usage and a
transaction that uses temporary file of a size between 'binlog-cache-size'
(default 32K) and 64K followed by a change user command.
In the case of a transaction of size between 32k and 64k, the server don't
truncate the binary log file on disk to speed up things, which is fine.
The bug was that change_user() reset the tmp_space_used variable and the
next time the temporary_file was truncated, we got a negative value for
tmp_space_used, which caused the error "Local temporary space limit
reached".
Fixed by not resetting tmp_space_used in init(), which is called by
change_user().
Other things
- Truncate binary log cache files when change_user() is called. This
makes the new users session not depending on log file sizes by previous
user.
- Some new ASSERT and DBUG_ENTER
- Fixed typo in DBUG output in ma_pagecache.c
`LOG_EVENT_OFFSET` is a duplicate of `EVENT_TYPE_OFFSET`
and `BIN_LOG_HEADER_SIZE` depending on the context.
While there, centralize `BIN_LOG_HEADER_SIZE` to `sql/rpl_constants.h`
(previously from `sql/unireg.h` and `client/mysqlbinlog.cc`)
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
- Removed duplicate words, like "the the" and "to to"
- Removed duplicate lines (one double sort line found in mysql.cc)
- Fixed some typos found while searching for duplicate words.
Command used to find duplicate words:
egrep -rI "\s([a-zA-Z]+)\s+\1\s" | grep -v param
Thanks to Artjoms Rimdjonoks for the command and pointing out the
spelling errors.
Ensure that Annotate_rows is always written direct after GTID information,
before any table_map events.
Before this patch, the following problems existed when mixing
transactional and not transactional tables in the same statement:
- Annotate rows could be written after row events or in the next GTID
event.
- See rpl_row_mixing_engines
- Annotate_rows was not always written to binary log in case of error
with a transactional table (rolled back) but a not transactional
table was updated.
- See sp_trans_log, binlog_row_mix_innodb_myisam
Fixed by writing the Annotate_rows event into the non transactional
cache if there are not transactional tables used. If not, write the
event into the transactional cache.
