Null bytes conflict with strings’ null-termination,
\while newline bytes are delimiters in the `multi-master.info` file.
While filename-unsafe chars are canonicalized when mapping names
to filenames, these two still cause trouble in other places.
Therefore, this commit extends `check_master_connection_name()`
to also reject names containing these chars.
This check function is now also used for system variable basenames
(e.g., in `@@replicate_do_db`), which (probably harmlessly)
did not have the existing `MAX_CONNECTION_NAME` check.
my_win_sopen() only checked the length of the raw path it was given, but
the server opens table files via paths relative to its data directory,
so a short relative path can still overflow MAX_PATH once
resolved against a long enough datadir. That case was misreported as
ER_BAD_DB_ERROR instead of ER_CANT_CREATE_TABLE/ENAMETOOLONG.
Resolve the path via GetFullPathName() before deciding whether the path
was too long, factored into resolved_path_too_long(). A failed resolve
is only treated as too-long when GetLastError() is
ERROR_FILENAME_EXCED_RANGE, so unrelated resolution failures keep
mapping to the original CreateFile error.
Also fix the global errtable mapping of ERROR_FILENAME_EXCED_RANGE: it
was mapped to ENOENT, so any Win32 call that got that error code
directly (not via ERROR_PATH_NOT_FOUND) was misreported the same way.
Assisted-by: Claude:claude-sonnet-5
Between two executions of a prepared statement, Item_sp::cleanup() freed
the stored function's memory root but left the arena's free list pointing
into it. That list is filled when the function call returns and the active
arena is restored.
On the next execution Item_sum_sp::clear() walked the stale list and
destroyed already freed items.
Free the items before the memory they live in. The sequence is now
in Item_sp::free_call_ctx(), so the three places that repeat it
cannot drift apart again.
Also reported as: https://github.com/mariadb-corporation/galera/issues/613.
THD::cleanup() cleared THD::wsrep_client_thread. Besides connection
close, THD::cleanup() is also called from THD::change_user() when
handling COM_CHANGE_USER and COM_RESET_CONNECTION. THD::init() does
not set the flag back, and the only place that sets it is
thd_prepare_connection(). After either command the session was no
longer treated as a wsrep client connection: WSREP_CLIENT() returned
false for the rest of the session.
As a result wsrep_sync_wait was silently ignored, so causal reads
could return stale data. Such connections were also skipped by
wsrep_close_client_connections() and
wsrep_wait_committing_connections_close(). Connection pools issue
COM_RESET_CONNECTION whenever a connection is reused, so in practice
most pooled sessions were affected.
Fix by moving the reset of wsrep_client_thread from THD::cleanup()
to end_connection(), after wsrep_close().
The galera.galera_change_user test is extended to do a causal read
on a fresh connection, after COM_CHANGE_USER and after
COM_RESET_CONNECTION. Each read must increase the provider's
wsrep_causal_reads status counter.
innodb.innodb-virtual-columns-debug contained only the MDEV-17005 test
case. It is about InnoDB virtual column templates, so it belongs with
the other InnoDB virtual column debug tests in gcol.innodb_virtual_debug.
Rationale
Adding a separate test file for every single bugfix tends to grow the
number of MTR tests by tens of thousands (as there are already 40k
MDEV tickets). Each test file carries a fixed overhead that does not
depend on its size: before and after every test MTR runs
check-testcase to compare the server state, checks the error log for
warnings, and may restart the server when the options differ from the
previous test.
For tiny test files this overhead dominates the time of the test
itself, so the total time of MTR runs on buildbot and in local testing
grows considerably.
Besides that, every test is at least two files (.test and .result,
often also .opt or .combinations). Tens of thousands of extra small
files bloat the test directories, which slows down MTR test collection
and any filesystem traversal of the source tree, and they enlarge the
git index and tree objects, which makes git status, checkout, clone
and merges between branches slower.
The test sporadically fails because DELETE does not reach the
ib_open_after_dict_open sync point:
SET debug_sync= "now WAIT_FOR delete_open";
+Warnings:
+Warning 1639 debug sync point wait timed out
SELECT a FROM t1;
a
-NULL
-NULL
The sync point is in ha_innobase::open(), which is called only when a
new TABLE is created from the share. After ALTER TABLE ... ALGORITHM=COPY
no TABLE for t1 is cached, but t1 has indexed virtual columns, so the
purge thread opens it via open_purge_table() and on close_thread_tables()
leaves the TABLE in the table cache. If that happens before DELETE, the
DELETE reuses the cached TABLE, does not call ha_innobase::open() and
completes without signalling delete_open.
The fix waits for purge with innodb_max_purge_lag_wait=0 and then does
FLUSH TABLES t1 before arming the sync point, so DELETE always opens a
new TABLE.
TABLE_ROWS in INFORMATION_SCHEMA.PARTITIONS for InnoDB is
dict_table_t::stat_n_rows. Each INSERT puts the record into the
clustered index and only afterwards increments stat_n_rows without
any latch. The second INSERT into a fresh partition schedules a
background persistent stats recalc, which is not throttled because
stats_last_recalc is still 0. If the recalc scans the leaf page
after the next INSERT has added its record but before that INSERT
increments stat_n_rows, the recalc stores the exact count and the
increment then adds one more, so p2 shows 4 rows instead of 3.
The fix disables innodb_stats_auto_recalc for the duration of
inc/vcol_partition.inc in the InnoDB variant of the test, so
stat_n_rows is maintained only by the per-row increments and is
exact.
The test sporadically fails with ER_LOCK_WAIT_TIMEOUT on DROP TABLE t1
of the Bug#53676 test case (see MDEV-29566).
mysqltest: At line 119: query 'DROP TABLE t1' failed:
ER_LOCK_WAIT_TIMEOUT (1205): Lock wait timeout exceeded; try restarting transaction
The DROP TABLE is executed by con2 with lock_wait_timeout=0. The INSERT
puts two rows into each partition, which exceeds the auto recalc
threshold of persistent statistics, so the partitions are added to the
recalc pool. The statistics background thread opens the partitions with
a shared MDL on t1, and if it does so at the moment of DROP TABLE, the
MDL request fails instantly.
The fix creates the table with STATS_PERSISTENT=0. The test case is not
about statistics; this also avoids locking mysql.innodb_table_stats and
mysql.innodb_index_stats in ha_innobase::delete_table() when the failed
ALTER TABLE ... ADD PARTITION rolls back the new partitions.
Item_func_match::master points at an equal MATCH item that owns the
shared ft_handler. setup_ftfuncs() sets it only when it is still unset,
and cleanup() never reset it.
A mergeable view is merged once. On re-execution mysql_derived_prepare()
returns early because TABLE_LIST::merged is set, so the view's own MATCH
item is never re-fixed and keeps the NULL table left by cleanup().
init_ftfuncs() skips unfixed items in ftfunc_list, but init_search()
follows master without that check and dereferenced the NULL table.
Reset master in cleanup(), after the ft_handler ownership check that
reads it, so the link is rebuilt from scratch on every execution.
backup_log_ddl() built its log record in a fixed-size stack buffer,
but add_name_to_buffer() can expand each identifier character to 5
bytes when re-encoding it into my_charset_filename, so a RENAME TABLE
with long, special-character names overflowed the buffer.
Fixed by building the record in a growable String, converting names
directly into it, and setting backup_log_error instead of writing a
partial record if an allocation fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tests test_proxy_header_connect_errors_reset() and
test_proxy_header_host_denied_not_counted() rely on their client IPs
(192.0.2.x test IPs, per RFC 5737) failing reverse DNS lookup permanently,
which is what a real, working resolver reports for them. Linux's resolver
isn't so RFC-compliant, when it has no route to any nameserver at all: it
reports EAI_AGAIN (temporary) instead, which is deliberately excluded from
connect-error accounting to avoid blocking hosts during a DNS outage.
That silently defeats the max_connect_errors check these tests exercise.
Fix by forcing the deterministic "permanent failure" outcome via the
existing getnameinfo_error_noname debug instrumentation, same as its
sibling tests. Debug-only, like those siblings, since the workaround
needs DBUG_EXECUTE_IF.
Assisted-By: Claude Sonnet 5 <noreply@anthropic.com>
DATE_FORMAT calculated its maximum result length solely from the format
string. Formatting a negative TIME also prepends a minus sign, so the
cursor protocol trusted an undersized result and truncated -01 to -0.
Account for the possible sign in format_length() for TIME_FORMAT() and
DATE_FORMAT() with a native TIME argument. Enable the existing cursor
protocol regression cases for binary, latin1, and utf32 result character
sets.
The test fails on random secret with backslash.
The random handlersocket_plain_secret is generated from the '0'..'z'
range, which includes the backslash. The test masked the secret with
--replace_result $HS_SECRET, and mysqltest parses --replace_result
arguments with backslash escapes, so the search string lost the
backslash and did not match the output. The unmasked secret then
appeared in the result (about 19% of runs).
The fix masks the secret in SQL: IF(variable_name LIKE
'handlersocket_plain_secret%', 'HS_SECRET', variable_value), which does
not depend on the secret value.
Problem: Executing queries that require virtual/generated column
evaluation during filesort trigger debug assertions due to missing
columns in read_set. Release builds return wrong results: rows that do
not match the WHERE, or missing rows that do.
Cause: find_all_keys() temporarily assigns TABLE::tmp_set as both
read_set and write_set (keyread does the same). InnoDB calls
TABLE::update_virtual_field() in the middle of the
scan (row_sel_sec_rec_is_for_clust_rec() ->
innobase_get_computed_value()) when it checks a secondary index record
against an older row version, a delete-marked record, or at READ
UNCOMMITTED. TABLE::update_virtual_field() calls
bitmap_clear_all(&tmp_set) before evaluating virtual column
dependencies. Since all three pointers share the same underlying
bitmap buffer, clearing tmp_set also clears the active
read_set/write_set, causing required columns to appear missing during
execution and triggering the assertion.
Fix: The fix uses a local bitmap instead of tmp_set for the virtual
column dependency walk in TABLE::update_virtual_field(), so the active
read_set/write_set is not touched.
Test: vcol.vcol_keys_innodb covers READ UNCOMMITTED (MDEV-36896) and a
concurrent uncommitted update under REPEATABLE READ (MDEV-41365).
Duplicated by MDEV-41365.
With save_merged false, Item_equal::merge_into_list() merges this
object's Item_equal instances into another, linking *this object's
items into the target. However this allowed callers to free an
item via one list but later read it through the other; a case of
use-after-free.
If save_merged is false then delete the items from the source.
dynamic_column_decimal_read() derived intg/frac from an unbounded
var-uint and only checked scale>precision in int domain, so a crafted
value could still overflow decimal_bin_size()'s uint16 parameters,
tripping its scale<=precision assert (an out-of-bounds array read in
release builds).
Bound intg/frac against DECIMAL_MAX_POSSIBLE_PRECISION before use, and
reject the var-uint decoder's error signal, as the sibling
dynamic_column_string_read() already does. Validate the decoded
ulonglong before narrowing it to int, so a value wrapping around on
narrowing cannot bypass the bound.
A multi-byte PERIOD name at the 64-character limit (192 bytes) filled
the name buffer exactly, leaving no room for the '_N' suffix appended
when generating a unique name for the implicit CHECK constraint.
Truncate on a character boundary, but only once a suffix is actually
needed (mirroring make_unique_key_name), so a non-colliding name is
never altered.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CONCAT uses String::alloc() when its result has zero allocated capacity.
Such a string can still reference external data, which alloc() discards.
Remove the zero-capacity special case so the existing growth calculation
and String::realloc() preserve the prefix for all strings. Explain why
realloc() is required and cover both JSON_TYPE() and GET_FORMAT() in the
10.11 section of main.func_concat.
Issue: wsrep_notify.sh builds the mysql client invocation as text with
values wrapped in '...', then re-parses it via eval. A configured
password (or SSL path) containing a single quote breaks out of that
quoting, splitting into extra arguments; the unquoted SSL_PARAM splice
also let ssl_* values undergo pathname (glob) expansion before eval
ever ran.
Solution: escape embedded single quotes before interpolating USER,
PSWD, HOST and the ssl_* values into the eval'd string, and quote the
SSL_PARAM splice so it isn't glob/word-split before eval runs. Applied
identically to the std_data test copies, and added a galera regression
test reproducing the reported password-splitting.
There were two compatible copies of kill_galera.inc and which one a test got
depended on its suite, so tests outside the galera suite still used the copy
with the non-portable "kill -9". Neither waited for the process to exit: both
ended in wait_until_disconnected.inc, which only says the server stopped
serving the connection, while the process can hold its pid file, ports and
data directory for a while after that. Tests that restart the node or read
its grastate right away worked around it with a --sleep.
Keep one copy, under include, and wait for the process with a kill(0) probe.
mysqld does not remove its pid file when killed by a signal, so
wait_until_no_pidfile.inc cannot be used here, but my_safe_process is in
waitpid() on it and reaps it as soon as it exits. Drop the seven --sleeps
that only waited for the process, and the six wait_until_disconnected.inc
calls the include already does.
Waiting for the rest of the cluster to notice is a separate thing which the
include cannot do for the caller, so the tests polling wsrep_cluster_size or
wsrep_cluster_status on a surviving node keep doing that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test polled INFORMATION_SCHEMA.PROCESSLIST for one of three thread states
to know that its two TOI DDLs were parked. None of them is reliable:
"acquiring total order isolation" is overwritten by THD_STAGE_INFO() two lines
after it is set and never reaches the process list, and while waiting for a
metadata lock the state alternates once per second between "Waiting for table
metadata lock" and the stage restored when the wait is left, which for a
statement that already holds total order isolation is "Waiting for TOI DDL"
and was not accepted. Both DDLs are in that state here, and the wait that
requires the two to match at the same poll is the one that timed out.
Use the mdl_acquire_lock_wait and wsrep_before_toi_begin debug sync points
instead. The test needs a debug build from now on.
On timeout wait_condition_with_debug_and_kill.inc killed the node but did not
abort the test, which then ran UNLOCK TABLES against the dead server and
failed with an unrelated error that hid the timeout. Abort after the kill.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
What was wrong:
The former wrapping cases in the "Overflow edge cases" section of
mysqltest_expression_evaluation.test had been moved to the end of the
file, so the section no longer showed them.
How it is fixed:
Put the five cases back in their original places in that section. They
now die, so each is followed by an expected-error check that runs a
nested mysqltest instance. The duplicates at the end of the file are
removed.
How it is tested:
main.mysqltest_expression_evaluation, main.mysqltest and
main.mysqltest_string_functions pass, also with --ps-protocol and with
a UBSAN-built mariadb-test client. This changes only the test files.
What was wrong:
The mysqltest expression evaluator did signed arithmetic without
checking that the result fits in long long. Boundary expressions
triggered undefined behavior or returned mathematically wrong wrapped
values, e.g. -9223372036854775808 / -1 gave -9223372036854775808, and
unsigned 18446744073709551615 + 1 gave 0.
How it is fixed:
Check unary minus, +, -, *, / and abs() before evaluating and die() with
an "Evaluation error: ... overflow" message when the result is not
representable in 64 bits. Unsigned +, - and * are checked the same way;
an unsigned difference that is negative but fits in long long is
returned as a signed value. A negative signed operand mixed with an
unsigned one is rejected instead of being silently converted. Valid
boundary results are kept: -9223372036854775808 % -1 = 0 and
-4611686018427387904 * 2 = -9223372036854775808. Unary minus on an
unsigned value still wraps, as conv(-~0, ...) in
main.mysqltest_string_functions relies on it.
How it is tested:
main.mysqltest_expression_evaluation: the former wrapping cases now
expect an error, and signed and unsigned overflow cases were added.
main.mysqltest, main.mysqltest_string_functions and the other tests
using $() pass, also with --ps-protocol.
Clear OPTION_NOT_AUTOCOMMIT/OPTION_BEGIN before opening the table below,
not after. A storage engine may register itself into the "all"
transaction as part of the table open/lock (e.g. InnoDB's
external_lock()), depending on those bits. Clearing them only after the
table is open is too late: the engine has already registered into "all"
using the still-set bits, and since record_gtid is meant to be a standalone
autocommit-style write, nothing will later issue the matching "all"-level
commit to clear that registration and the performance-schema transaction
handle, and it leaks into whatever runs on this THD next.
OpenSSL 3 escapes '/' and '+' in X509_NAME_oneline() output; OpenSSL 1.1
and WolfSSL don't. A REQUIRE ISSUER/SUBJECT grant from one library can
stop matching after switching to another.
Default comparison stays strcmp(). old_mode=X509_LENIENT_COMPARE opts
into falling back to an escape-aware compare, applied to whichever side
the currently-linked library's own escaping affects, at the cost of
reopening the ambiguity a crafted certificate could exploit to
impersonate another identity.
Adds regression tests against a real certificate with an ambiguous CN
Assisted-by: Claude Sonnet 5 <noreply@anthropic.com>
Follow-up to 6c391b9205.
SHOW CREATE DATABASE always appends "/*!40100 DEFAULT CHARACTER SET
... */", but the default charset differs by version (latin1 on 11.4,
utf8mb4 on 11.8+). Strip it instead of matching a specific value,
since it is not the subject for the fix.
Also switch to evalp: same execution, but logs "$var" instead of the
substituted value, so the replace_regex/disable_query_log hacks go away.
MDEV-38920-evs-config-warn checks that there is a warning about bad
configuration values and they are not accepted.
MDEV-38920-install-timer-expired reproduces 'install timer expired'
situation.
Both tests require fixed Galera library to pass (4.29) hence will be
skipped when run with older versions.
Commit b68e29a9c6 explicitly disabled use of SSL encryption in SST
by setting ssl-mode=DISABLED in the top configuration files.
This test is a backward compatibility test so it relies on the
deduction of ssl-mode from the presence of tkey and tcert params
in [sst] section. Unset ssl-mode in config to allow to derive it
from the presence of tkey and tcert.
JSON_KEYS after modifying character set name/collation
Analysis:
Buffer overflow crashes and empty key duplication in check_key_in_list.
Fix:
Checking result buffer validity prevents segmentation faults on empty
strings and malformed inputs while preserving correct key matching behavior.
JSON_KEYS after modifying character set name/collation
Analysis:
Since the length of string is 0, accessing out of boundry memory,
leads to crash
Fix:
If string length is empty, return success.
Bug 1: put_dbopt() used strmov() to copy schema_comment into a fixed
DATABASE_COMMENT_MAXLEN+1 buffer. validate_comment_length() only
truncates comment->length in non-strict sql_mode, leaving comment->str
NUL-terminated at its original (unbounded) length. strmov() copies
until the source NUL, ignoring the truncated length, overflowing the
destination buffer for long comments.
The fix uses strmake() bounded by comment->length instead, matching
the LEX_CSTRING contract (length is authoritative, str need not be
NUL-terminated at length).
Bug 2: write_db_opt() used strxnmov() to copy the full un-truncated
comment until it ran out of buffer space mid-string with no trailing
newline. Which made load_db_opt() silently discard the whole
unterminated "comment=" line on the next restart, losing the comment
entirely instead of just truncating it.
The fix bounds the comment copy into db.opt by the already-validated
comment->length via strmake(), instead of relying on the source
string's own NUL terminator, matching the put_dbopt() fix.
Bug 3: validate_comment_length() only runs on a COMMENT clause given
in the current statement. ALTER DATABASE without one instead pulls
the existing comment off disk via load_db_opt(), which never bounded
it. That unvalidated length then reached write_db_opt()'s own
comment= copy into its stack buffer, so a legacy or hand-edited
db.opt with an overlong comment= line overflowed it on ALTER DATABASE.
The fix: load_db_opt() now clamps the parsed comment to
DATABASE_COMMENT_MAXLEN right when it reads the "comment=" line, so
every consumer (put_dbopt(), write_db_opt()'s ALTER path) always sees
an already-bounded value.
The clamp itself must truncate by bytes, not characters:
Well_formed_prefix()'s LEX_CSTRING overload takes a character count,
but DATABASE_COMMENT_MAXLEN sizes the buffers in bytes.
Bug 4: write_db_opt() still trusted validate_comment_length() to bound
a directly-given COMMENT to DATABASE_COMMENT_MAXLEN bytes, but it only
bounds it to DATABASE_COMMENT_MAXLEN *characters* -- so a multi-byte
comment could still overflow the same fixed buffers.
The fix: write_db_opt() clamps schema_comment to
DATABASE_COMMENT_MAXLEN bytes itself, right after
validate_comment_length() returns.
Problem:
When CONNECT engine pushes a WHERE clause down to an external table,
it writes the filter into the work area, without checking how much
space is left. A large string literal in the WHERE clause can overflow
the work area allocated by the engine. For ex: if connect_work_size is
set to 4MB and the string literal in the WHERE clause is larger than
4MB, it can grow past the allocated work area.
Fix:
Track the space left in the work area and check it before writing. If
the filter doesn't fit, drop it instead of writing past the buffer.
Problem:
=======
When innodb_encryption_rotation_iops=0, an encryption thread
could be waiting on fil_crypt_iops_cond in fil_crypt_alloc_iops().
fil_crypt_set_thread_cnt() lowers srv_n_fil_crypt_threads and
broadcasts only fil_crypt_thread_cond, so that the waiting thread
never re-evaluates should_shutdown() and never exits.
Solution:
=========
fil_crypt_set_thread_cnt(): Broadcast fil_crypt_iops_cond as well,
so a thread waiting for IOPS wakes up and sees should_shutdown(),
exits.
Replace the two recorded reads of Slave_received_heartbeats with an
assertion that the counter is nonzero.
The counter advances once per heartbeat period for as long as the
connection is running. The test waited for it to reach 2 and then
read it again in a separate query, so a heartbeat arriving between
those two queries recorded an unexpected value.
The same wait timed out when the counter was already past 2 at the
first poll, so it now accepts any value at or above the target.
Defer the host-privileged/host-blocked check for a PROXY-header-derived
address until after the client's SSL handshake completes, instead of
sending it immediately in clear text.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Problem:
=======
ha_innobase_inplace_ctx::~ha_innobase_inplace_ctx() runs, whenever
ctx->instant_table is set and frees the old_v_cols exist also.
old_v_cols and old_n_v_cols are captured in the constructor as
prebuilt_arg->table->v_cols and n_v_cols, i.e. an alias of the live
table's own virtual columns, not a copy. By the time this destructor
runs, old_table->v_cols is either still that same array. If the
failure happened before ctx->instant_column() ever ran like during
prepare_inplace_alter_table_dict() or failure happened during the
commit phase innobase_instant_try(). In both cases old_v_cols is
the table's current, live v_cols array, so this loop destructs
dict_v_col_t objects that are still in use.
Solution:
========
ha_innobase_inplace_ctx::~ha_innobase_inplace_ctx(): Destruct
instant_table->v_cols[], not old_v_cols[]. instant_table is the
independently allocated dict_table_t that prepare_instant() built;
It owns its own v_cols array, whose dict_v_col_t::v_indexes must
be destructed before dict_mem_table_free() reclaims instant_table's memory.
Problem:
=======
For fulltext index, row_create_index_for_mysql() calls
fts_create_index_tables(). If creating FTS auxiliary table fails,
error handling performs trx->rollback() of the dictionary
transaction. Rollback removes the parent table from
dictionary cache and frees it. After that,
convert_error_code_to_mysql() reads table->flags after table->heap.
This leads to read of freed memory.
Solution:
========
create_index(): Read table->flags into a local variable before
calling row_create_index_for_mysql()
They were not necessary, just try to run as service, and fallback to
command line.
Add some diagnostics - unexpected errors from StartServiceCtrlDispatcher
and RegisterServiceCtrlHandler are now reported to Windows event log.
Also use authoritative service name, returned as first argument
in svc_main by service control manager.
fil_space_t::freed_ranges is a range_set guarded by
freed_range_mutex, and every mutator takes that lock except one:
the branch of mtr_t::commit() that frees pages for an unlogged
mini-transaction on the temporary tablespace (the common case,
since temp-space pages never set m_modifications). That path
called fil_space_t::add_free_range() directly, racing with any
other locked mutator of the same std::set - most notably
fil_space_t::flush_freed(), which the page cleaner invokes for
every tablespace while resizing the buffer pool. The race
corrupts the range_set's underlying tree and crashes the server.
Take freed_range_mutex around the loop, matching every other
caller (process_freed_pages(), flush_freed(),
clear_freed_ranges()).
Added innodb.temp_truncate_freed_race, which reproduces the
crash with a configurable probability by racing concurrent
temporary-table churn against a continuously resizing buffer
pool across several rounds against the same server.
fil_name_process(): Treat FILE_CREATE in the same way as FILE_MODIFY
that led to a FIL_LOAD_DEFER return. Remove the parameter lsn,
and return file_name_t& in which the caller may assign create_lsn
when processing a FILE_CREATE record.
deferred_spaces.reinit_all(): Never create anything for deleted
tablespaces. Doing so could cause a legitimate file to be deleted
if files are being deleted and re-created with the same name.
deferred_space.create(): Remove some duplicated code. Missing
tablespace files will be created in fil_node_open_file_low()
starting with
commit 759e3523e3 (MDEV-38026).
deferred_spaces::item::lsn: Remove. Starting with
commit 37d8577aee (MDEV-40728)
each FILE_ record is parsed only once.
recv_sys_t::parse_store_if_exists(): Tell the caller to skip
tablespaces for which both FILE_CREATE and FILE_DELETE was parsed.
This improves performance, not correctness.
recv_validate_tablespace(): Avoid duplicated tablespace lookup
and remove a redundant deferred_spaces.add(); fil_name_process
already keeps deferred_spaces in sync with recv_spaces.
fil_space_t::rename(): If !log, assert !replace and that
the target file name does not exist.
os_file_rename_func(): Do not check that the target path
does not exist. This is already checked by every caller.
This fixes a debug assertion failure that could otherwise
occur when recovering from a crash in
fil_space_t::rename() between the write of the FILE_RENAME
and the actual rename.
Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Saahil Alam
xtrabackup_backup_func(): Request for a checkpoint synchronously
so that recv_sys.find_checkpoint() will observe the effect.
Reviewed by: Thirunarayanan Balathandayuthapani
Problem:
========
A query on INFORMATION_SCHEMA.INNODB_SYS_TABLES crashes when
SYS_TABLES contains a record that was inserted by a transaction
which has not been committed. This can happen after a
crash while a CREATE TABLE was in progress, if the server is
restarted with innodb_force_recovery=4 or greater,
because trx_rollback_recovered() is then skipped and the
recovered transaction remains ACTIVE.
dict_sys_tables_rec_read() returns READ_NOT_FOUND for such a
record, and dict_load_table_low() returns that as success with no
error message and setting *table to nullptr.
i_s_sys_tables_fill_table() checks only the error
message and passes the nullptr table to i_s_dict_fill_sys_tables(),
which dereferences it.
Solution:
========
i_s_sys_tables_fill_table(): Skip the SYS_TABLES record when
dict_load_table_low() reports success but returns no table, because
such a record is not visible.
Error handling in sst_prepare_mysqldump() could overwrite
the error code from malloc/sprintf with status received later from
mysql_thread_create() call. This let the function report success
after an allocation failure, without setting *addr_out,
breaking the caller's ability to correctly free the allocated
address buffer.
This commit fixes the error handling in sst_prepare_mysqldump so
that each error path frees its own allocation. Thread create uses
separate status variable.
`UPDATE ticket SET is_valid=0 WHERE ticket_id IN
(SELECT ticket_id FROM ticket WHERE booking_id=1)` acquires S next-key
and gap locks on the FK secondary index `fk_ticket_2_booking`, causing
deadlocks between concurrent transactions operating on non-overlapping
`booking_id` values. The expected behavior — matching
`UPDATE ... WHERE ticket_id IN (1,2)` (explicit PK list) — is to only
lock the PRIMARY key records being updated.
**Root cause**: the optimizer converts the `IN(SELECT)` to a semi-join,
which creates two table references for the same table. The SQL layer
then processes the statement as `SQLCOM_UPDATE_MULTI`. In
`ha_innobase::store_lock()`, the read-only (subquery) table handle
receives `TL_READ`, but the existing `LOCK_NONE` optimization for DML
read tables was gated on `isolation_level <= READ_COMMITTED` and did
not include `SQLCOM_UPDATE_MULTI` at all. In `REPEATABLE READ` (the
default), the subquery's secondary index scan therefore acquired
`LOCK_S` next-key locks, including gap locks spanning into adjacent key
ranges.
**Fix**: add `SQLCOM_UPDATE_MULTI` to the `LOCK_NONE` (consistent read)
condition in `store_lock()`:
1. For `TL_READ` (binlog off or ROW format): use consistent read at
all isolation levels below `SERIALIZABLE`. This is safe because:
- The MVCC snapshot is stable within a transaction in
`REPEATABLE READ`
- Rows being modified still acquire X locks via the write-side
table handle (`F_WRLCK` / `LOCK_X`)
`SERIALIZABLE` is excluded: the `::external_lock()` upgrade of
`LOCK_NONE` to `LOCK_S` applies only to non-autocommit
transactions, so an autocommit multi-table UPDATE would otherwise
read its read-side tables via a consistent read at `SERIALIZABLE`,
weakening its guarantees.
2. For `TL_READ_NO_INSERT` (statement-based binlog): extend the
existing `READ COMMITTED` optimization to also cover
`SQLCOM_UPDATE_MULTI`
The fix is deliberately limited to `SQLCOM_UPDATE_MULTI` (not
`SQLCOM_UPDATE`) because single-table UPDATE with scalar subqueries on
other tables traditionally uses S locks to block concurrent reads of the
subquery table, and broadening the change there would alter observable
behavior for existing applications.
**Lock count before/after** (semi-join UPDATE on `booking_id=1`):
- Before: 5 lock structs, 5 row locks (IS + S×3 on FK index + IX +
X×2 on PK)
- After: 2 lock structs, 2 row locks (IX + X×2 on PK) — identical to
explicit `WHERE ticket_id IN (1,2)`
**Tests**:
`update_subquery_fk_gap_lock.test` — verifies that concurrent
`UPDATE...IN(SELECT)` operations on non-overlapping `booking_id` values
do not block each other. Two connections each update their own
`booking_id`'s tickets via semi-join subquery, then insert new tickets
for their `booking_id`. Both operations must succeed without lock wait
timeout, confirming that the spurious FK index gap locks are eliminated.
`update_subquery_fk_gap_lock_isolation.test` — ACID isolation regression
tests (requires ROW binlog format) verifying that replacing S locks with
consistent read (`LOCK_NONE`) on the semi-join read table does not cause
isolation violations at `REPEATABLE READ`:
1. **Phantom insert**: T1 updates `booking_id=1` tickets via semi-join,
T2 concurrently inserts a new ticket with `booking_id=1`. T2 must
not block (no S gap locks). T1's MVCC snapshot does not see the new
row, so T1 updates only the 2 original tickets. Correct under
snapshot isolation.
2. **Concurrent UPDATE on filter column**: T1 updates `booking_id=1`
tickets, T2 changes `ticket_id=2`'s `booking_id` from 1 to 2. Both
need X lock on `ticket_id=2`'s PK record — serialized via PK X
lock alone, proving FK index S locks are unnecessary.
3. **Concurrent DELETE**: T1 updates `booking_id=1` tickets, T2 deletes
`ticket_id=1`. Both need PK X lock — serialized, no isolation
violation.
4. **SERIALIZABLE S locks**: At `SERIALIZABLE`, the read-side table
keeps `LOCK_S`, so T2's INSERT into the subquery range must block
with `ER_LOCK_WAIT_TIMEOUT`. Confirms the fix does not weaken
`SERIALIZABLE` guarantees.
5. **Multi-table UPDATE on different tables**: T1 does
`UPDATE t_write JOIN t_read ... SET t_write.val = t_read.val`, T2
concurrently modifies `t_read`. T2 must not block (T1 holds no S
locks on `t_read`). T1's snapshot values are used. Correct under
snapshot isolation.
6. **MVCC snapshot consistency**: T1 establishes snapshot, T2 modifies
`t_read`, T1 does multi-table UPDATE using snapshot then re-reads.
Re-read must return the same values the UPDATE used, confirming
MVCC snapshot stability at `REPEATABLE READ`.
7. **SERIALIZABLE with autocommit**: the `::external_lock()` upgrade
of `LOCK_NONE` to `LOCK_S` does not apply to autocommit statements,
so `store_lock()` itself must not choose `LOCK_NONE` at
`SERIALIZABLE`. An autocommit `UPDATE...IN(SELECT)` stalled
mid-scan on a PK X lock must hold S next-key locks on the FK index
entries it has already scanned: an INSERT whose FK index entry
falls inside that range must block with `ER_LOCK_WAIT_TIMEOUT`.
`mdev-14846.test` depends on the multi-table UPDATEs' read-only
tables being read with S locks to produce the deadlock whose error
handling it verifies. Both transactions taking part in that deadlock
now run at `SERIALIZABLE`, where the read-only tables keep their S
locks, preserving the original deadlock scenario and its
`ER_LOCK_DEADLOCK` expectation unchanged. Tying this to the binary
log format instead would not work everywhere: the embedded server
has no binary log, so its read side always gets `TL_READ`.
Verified via git grep that this file has zero references anywhere
in the codebase on the 10.11 branch.
Ran mysql-test-run.pl --suite=binlog: 96.97% pass rate, with one
pre-existing unrelated failure (binlog.binlog_mysqlbinlog-cp932) -
a known Windows console/CP932 encoding environment issue, also
observed identically on main.
bur_pool_t::size_in_bytes_max_default: Define as 0 also on
ThreadSanitizer. The symbol __SANITIZE_THREAD__ is predefined
starting with Clang 22 or GCC 7 when building with -fsanitize=thread.
my_large_malloc() attempted large pages whenever --large-pages was
enabled, silently rounding the size up and reporting it back via an
in/out parameter. ut_malloc_dontdump() never passed that adjusted
size on to its own callers (the InnoDB redo log buffer and
recv_sys_t::tmp_buf), so freeing later used the original, smaller
size, causing the reported "faux memory leak".
Only the buffer pool and the MyISAM/Aria key caches are documented
to benefit from large pages. Everything else that ended up calling
my_large_malloc() only wanted its "do not dump to core" property and
picked up large pages as an undocumented side effect; those buffers
are also small and sequentially accessed, so they would have gained
little from large pages anyway.
Add MY_TRY_LARGE_PAGES: my_large_malloc() and my_large_virtual_alloc()
now only attempt large pages when a caller passes this flag, instead
of always trying whenever the global option is set. Only the buffer
pool and the key caches pass it. The redo log buffer, tmp_buf, and
row0log.cc's crypt buffers no longer request large pages at all,
which removes the size-rounding bug for them without touching that
code.
my_large_virtual_alloc()'s fallback (no usable large page size) must
also return read-write memory right away, like the Windows large-pages
fallback already does, since my_virtual_mem_commit() is a no-op for
MY_TRY_LARGE_PAGES. my_large_pages_flag is now set once, in
my_init_large_pages(), and never changed thereafter, on any platform.
Both my_virtual_mem_commit() and my_virtual_mem_decommit() are now
no-ops, aside from accounting, whenever large pages are requested.
Also fix a broken mtr suppression regex in main.large_pages that
would fail the test on Windows.
EXCHANGE PARTITION fails with ERROR 1736 (Tables have different definitions)
when tables contain generated columns with AND/OR conditions, even when the
expressions are logically equivalent. This occurs because when expressions are
re-parsed (e.g., via CREATE TABLE ... LIKE), the order of arguments in AND/OR
conditions may change, but the comparison was order-sensitive.
The Item_cond::eq() method was implemented to perform set-based comparison
for commutative AND/OR operations. The set-based comparison algorithm ensures
that two Item_cond expressions are considered equal if they contain the same set
of equivalent arguments, regardless of order.
Added comprehensive test case covering:
- Generated columns with OR conditions
- Generated columns with AND conditions
- Multiple generated columns with different AND/OR combinations
- Nested AND/OR conditions
The fix allows EXCHANGE PARTITION to succeed when expressions are logically
equivalent but have different argument ordering, which is correct behavior
since AND/OR operations are commutative.
MDEV-38757 Limit unordered vcol condition comparison to EXCHANGE PARTITION
Review follow-up to the previous commit, which made Item_cond::eq()
compare AND/OR argument lists as sets for every caller. That changed
equality semantics globally and broke main.derived_cond_pushdown, where
conditions that eq() started reporting as equal were dropped from
attached_condition. Reordering AND/OR operands also changes evaluation
order, which is observable when operands are functions, so the relaxed
comparison must not be the default.
Item::Eq_config gains an unordered_conditions flag, defaulting to false,
next to the existing binary_cmp and omit_table_names flags.
Item_cond::eq() compares its argument lists as sets only when that flag
is set, and otherwise reports two distinct Item_cond objects as unequal,
as it did before this patch series. The flag is threaded through
Virtual_column_info::is_equal() and a new mysql_compare_tables()
parameter, and only Sql_cmd_alter_table_exchange_partition passes it as
true, so the relaxed comparison stays confined to the EXCHANGE PARTITION
metadata check.
The set comparison tests containment in both directions rather than
comparing element counts, so an expression also matches a form that
repeats one of its terms, for example
col1 > 10 and col2 < 100 or col3 > 50
col3 > 50 or col1 > 10 and col2 < 100 or col3 > 50
Test 5 of parts.partition_exchange_generated_columns covers that case.
