Regression from MDEV-32745 (7828fb475b), which guarded the
plugin-load my_error() calls with opt_silent_startup. That option is
a lifetime global, set once at startup and never reset, so the guard
suppressed the SQL error for the whole server lifetime, not just
during startup. Runtime operations (INSTALL PLUGIN, CREATE FUNCTION
... SONAME) then skipped my_error(), never set the diagnostics area
and wrongly succeeded - e.g. main.ps's "call proc_1()" no longer
failed with ER_CANT_OPEN_LIBRARY.
Startup callers pass MYF(ME_ERROR_LOG); runtime callers pass MYF(0).
Gate the silencing on that flag via silent_plugin_startup() so it
applies only to the startup error-log path, and runtime errors always
reach the client.
No new test case: the runtime failure path is already covered by
existing tests (e.g. main.ps's ER_CANT_OPEN_LIBRARY check). The
regression stayed invisible only because stock MTR does not start
servers with --silent-startup. A dedicated test would have to restart
the server with --silent-startup solely to assert that a startup-only
option does not affect runtime, which adds little over the restored
invariant.
When updating replicate_do_db and slave_parallel_mode, the handlers
checked !base_name->length instead of !base_name->str to determine
whether to fall back to @@SESSION.default_master_connection.
Because an explicit blank qualifier (``) has length 0 with a non-NULL
pointer, SET statements using empty backticks mistakenly updated
default_master_connection rather than the unnamed connection.
Fixed by checking !base_name->str instead of !base_name->length.
Also fixed SELECT @@GLOBAL.slave_parallel_mode, which had the same
issue when reading the variable.
Renamed the regression test to sys_vars_default_master to reflect
the broader coverage and merged the test cases of
MDEV-40296 and MDEV-40297.
Reviewed-by: ParadoxV5 <paradox.ver5@gmail.com>
The test reads innodb_buffer_pool_resize_status while the
SET GLOBAL innodb_buffer_pool_size thread is blocked at
buf_pool_shrink_before_wakeup and expects nothing withdrawn yet:
Withdrawing blocks. (505/505).
But buf_pool_t::resize() sets n_blocks_to_withdraw and
first_to_withdraw and releases buf_pool.mutex before that sync
point. From then on other threads withdraw blocks: allocate() from
any page allocation, buf_LRU_block_free_non_file_page() on eviction,
and the page cleaner LRU batch which evicts pages from the withdrawn
area while to_withdraw is nonzero. So the first counter can be
anything down to 0:
Withdrawing blocks. (0/506).
The fix masks the first counter in replace_regex, so the result
does not depend on how many blocks were already withdrawn.
This does not weaken the test: it still checks that the shrink is
reported while in progress. How many blocks are already withdrawn at
that moment depends only on timing.
initialize counters when constructing a new PFS_program object
The test fails because a newly created routine can get a performance
schema slot freed by a routine dropped in an earlier test. The counters
of a reused slot are not reset, so the new routine shows the old
routine's statistics.
Problem: btr_pcur_t::restore_position() accepts a stored position
optimistically when the block at the stored descriptor address has the
same page id and the same buf_block_t::modify_clock. When
buf_pool_t::resize() shrinks the buffer pool, the withdrawn block
descriptors are freed. When it grows the buffer pool again, the new
descriptors at the same addresses start with modify_clock=0. A cursor
that was stored before the shrink can then find a different instance of
the page with an equal modify_clock, and it restores to a record that
is no longer correct. In a release build, an UPDATE can change a row
that does not match its WHERE condition and skip the rows that match.
Fix: Add buf_pool_t::withdrawn_modify_clock. buf_pool_t::shrink() sets
it to the maximum modify_clock of the blocks that it withdraws.
buf_pool_t::resize() sets the initial modify_clock of each block that
it adds to one more than this. A relocation in shrink() increments only
the modify_clock of the destination block, so a withdrawn block can
keep the value that a cursor stored. Thus the modify_clock at a block
descriptor address never repeats an earlier value, and a stale
optimistic restore fails. The new member uses a hole before
buf_pool_t::flush_list_mutex, so the layout of the other members does
not change.
The test innodb.buf_pool_resize_pcur_restore reads a page into the
first new block after a grow, and makes a stored cursor for a read and
for an UPDATE restore its position across a page reorganization, a
shrink and a grow.
MyCTX used an EVP_CIPHER_CTX in a stack buffer instead of allocating it
with EVP_CIPHER_CTX_new(). OpenSSL 4.0 rejects a context that was not
created that way for ciphers that use an IV: EVP_CipherInit_ex() fails
with "invalid iv length", so AES_ENCRYPT() and KDF() in CBC/CTR/GCM
modes return NULL. ECB has no IV and still works, which is why only the
non-ECB modes broke.
Allocate the context with EVP_CIPHER_CTX_new()/EVP_CIPHER_CTX_free(), and
on an init error free and null it, so a later update()/finish() from a
caller that ignored the error is a safe no-op, not a double free. Remove
check_openssl_compatibility() and the EVP_CIPHER_CTX_SIZE and
EVP_CIPHER_CTX_init macros.
my_md5() used the same stack-buffer-sized-by-EVP_MD_CTX_SIZE hack, whose
only guard was check_openssl_compatibility(). MD5_CTX is a concrete struct
in both OpenSSL and WolfSSL, so use it directly with MD5_Init/Update/Final
on a caller-owned context, like my_sha.inl does for SHA*_CTX; this removes
the EVP_MD_CTX hack and EVP_MD_CTX_SIZE. The exception is OpenSSL 1.0.x,
which aborts on the low-level MD5 API in FIPS mode (MDEV-7788): there, and
only there, keep the EVP path with the non-FIPS flag. 1.1.0 removed the
FIPS Object Module and 3.0+ low-level MD5 bypasses the provider, so the
concrete MD5_CTX is correct on every other version. SHA is unaffected, it
already used concrete SHA*_CTX structs.
Verified against OpenSSL 4.0.1: the mysys aes-t test fails on the
CBC/CTR/GCM cases with the stack buffer and passes with
EVP_CIPHER_CTX_new(). The 1.0.x MD5 path verified against a FIPS build of
OpenSSL 1.0.2u: the EVP path computes MD5, the low-level path aborts.
No visible performance degradation: the extra allocation costs
~12 ns/call (WolfSSL) and ~30 ns (OpenSSL) on Windows at a 30-byte
payload, nothing at 16 KB, and nothing on Linux/glibc; sysbench OLTP
over encrypted tables and redo log is unchanged.
Tests main.tlsv13 and main.ssl_crl: OpenSSL 4.0.3 reports TLS alerts as
"tls alert ..." (was "ssl/tls alert ..."), accept both.
Assisted-by: Claude:claude-opus-4-8
parse_v2_header() required only 25 (TCPv4) / 49 (TCPv6) bytes, but then
reads the source port at hdr[25] / hdr[49]. A truncated header could make
it read uninitialized stack.
Require the whole address block: 28 bytes for TCPv4 and 52 for TCPv6.
Also, the declared header length was trusted without checking how many
bytes vio_read() returned. Reject short reads. The spec requires the
sender to send the header at once, and allows the receiver to drop the
connection on a partial header, so a single vio_read() is enough.
The v1 parser had the same problem on EOF: vio_read() returning 0 was
taken for a byte read, so the uninitialized buffer was parsed. Require
exactly one byte.
Extend test_proxy_header_tcp with a header cut short before the port,
and add test_proxy_header_short for a header shorter than it declares.
Light refactoring, to avoid repetition: added connect_tcp() and
init_v2_header(), which are now shared by several proxy header tests.
Assisted-by: Claude:claude-sonnet-5-5
After MDEV-39261 purge opens TABLE via open_purge_table() for any
undo record of a table with indexed virtual columns, including the
insert undo record. If purge does that after FLUSH TABLES, the TABLE
is put back into the table cache. Then DELETE from con1 reuses the
cached TABLE, skips open_table_from_share() and never hits the
TABLE_after_field_clone sync point. DELETE completes immediately,
wait_condition times out and SELECT returns an empty set.
The fix waits for purge to complete before FLUSH TABLES.
versioning.partition (MDEV-41543) and main.grant2 (MDEV-41099) print
row_end of the current row. Since MDEV-32188 in 11.5 the max TIMESTAMP
is 2106-02-07 06:28:15.999999 on 64-bit and 2038-01-19 03:14:07.999999
on 32-bit, so the results recorded on 64-bit fail on 32-bit.
The fix prints CURRENT instead of row_end of the current row in
versioning.partition (using current_row_ts() from common.inc) and
replaces row_end with # in main.grant2, where only the privilege to
read it is tested.
Affected since 11.8. The fix is committed to the originating 10.11 to
minimize conflicts.
Event_job_data::execute() pointed thd->security_ctx at a local
Security_context. A concurrent PROCESSLIST scan captures that pointer
without a lock and can read it after execute() returns.
Make the Security_context a member of Event_job_data, which lives in
Event_worker_thread::run() until after thd is deleted.
Add main.events_debug test
The problem with the test was that it restarted node 2 by writing
"try:--galera-unknown-option" into the expect file. However, MariaDB mtr has
never supported a "try" command. Instead mtr skipped any expect file line not
starting with "restart", and node 2 was not started with the unknown option
(which was the idea in the test logic). The test only slept and then restarted
node 2 normally.
MDEV-40023 made mtr report unknown expect file commands as errors,
so the test now fails in 13.1 and later.
The fix is to start node 2 with the unknown option directly with --exec.
In cluster, the unknown option is found only after the state transfer has started.
The server should interrupt the state transfer and abort.
Galera version 26.4.25 or later is needed for graceful shutdown, and this is now
required by the test.
A locking read with SKIP LOCKED through a secondary index calls
row_sel_get_clust_rec_for_mysql() to lock the clustered index record.
On a lock conflict, that function returned DB_LOCK_WAIT without
setting the output record, and row_search_mvcc() then called
lock_trx_handle_wait() to cancel the wait. If the conflicting
transaction had committed in the meantime, the waiting lock was
already granted and lock_trx_handle_wait() returned DB_SUCCESS.
row_search_mvcc() then found clust_rec == NULL for a locking read:
debug builds failed an assertion, and release builds skipped a row
whose lock the transaction now held.
Row_sel_get_clust_rec_for_mysql::operator()(): Call
lock_trx_handle_wait() here instead of in row_search_mvcc(). When
the lock was granted meanwhile, return the clustered index record
and store the cursor position as for any granted lock, like
row_search_mvcc() does with "goto locks_ok" for the record that it
is positioned on.
XmlDB was copied with strcpy/strcat into a 64-byte stack buffer to
build the XPath. Tabname used the same buffer with snprintf, silently
truncating the XPath.
Build the XPath in a buffer allocated by its real length.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`@@rpl_semi_sync_slave_enabled` can be changed without stopping
replication, because it takes effect when a connection is established.
But when it takes effect, it applies to *all* replication connections.
This means that starting an async connection will switch any ongoing
semi-sync connections to async as well, and vice versa, which will
cause those connections to fail to replicate in the wrong mode.
This fix resolves this oversight by changing the value application
to be specific to only the establishing connection (technically,
moving the singleton’s field `Repl_semi_sync_slave::m_slave_enabled`
to the instance variable `Master_info::semi_sync_enabled`).
This design builds toward MDEV-40941, which proposes migrating
`@@rpl_semi_sync_slave_enabled` to a proper per-connection config.
Note, `Master_info::semi_sync_reply_enabled` must remain separate,
because it only disables ACK replying in async slave fallback,
whereas the semi-sync slave setting also controls whether the IO
thread should expect additional semi-sync flags in event packets.
To maintain backward compatibility until MDEV-40941,
the status variable `Rpl_semi_sync_slave_status` is kept showing
the applied `@@rpl_semi_sync_slave_enabled` for the most recent
START SLAVE in any connection; that is, not yet made per-connection.
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
DuckDB has no row locks and its parser rejects locking clauses, so
SELECT ... FOR UPDATE on a DuckDB table failed with ER_GET_ERRMSG
("Parser Error: SELECT locking clause is not supported!") coming from
the embedded DuckDB.
Check for locking reads in ha_duckdb::external_lock() and raise
ER_NOT_SUPPORTED_YET from the storage engine instead. The check only
applies to SQLCOM_SELECT, so INSERT ... SELECT, UPDATE and DELETE are
not affected.
A WHERE conjunct that calls rand() but doesn't refer to a column, such
as rand() < 0, is checked on the last non-const top level base table
in the join order. A join of only const tables has no such table and
returns at most one row, so the conjunct is checked once before the
join starts, together with the conditions on columns of an outer
query.
When every top level base table is const and materialized semi-joins
follow them, the join order also has no non-const top level base
table, but only_const_tables() is false. The conjunct was never
checked, and such a query gave wrong results.
The check before the join now applies whenever the join order has no
non-const top level base table, which is the same condition under
which no table in the join order receives the conjunct. Such a join
also returns at most one row.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A join of only const tables stored the WHERE conjuncts that refer to a
column of an outer query or call rand() in pseudo_bits_cond. The
executor read pseudo_bits_cond only when the query needed no temporary
table, and read outer_ref_cond otherwise. A window function or
DISTINCT with ROLLUP needs a temporary table, so those conjuncts were
never checked, and the query returned rows that the WHERE clause
rejects.
pseudo_bits_cond came from the fix for LP bug 908269 in 2012. For a
join of only const tables, that fix also collected conjuncts that call
rand(), and it stored them in a new field without stating a reason.
The code that filled the two fields and the code that read them chose
between them by the same test. The tests have differed since
MDEV-8646 in 2016, which added the temporary table condition to the
executor's test. Later changes only display the field, in EXPLAIN
FORMAT=JSON (MDEV-19714) and in the optimizer trace (MDEV-21095). In
MDEV-19714, its author wrote that it was not clear why pseudo_bits_cond
was needed in addition to outer_ref_cond.
pseudo_bits_cond is removed, and outer_ref_cond holds these conjuncts
for every join. The executor checks it on both paths, so where the
conjuncts are stored no longer depends on the executor's test.
Conjuncts that call rand() are still collected only for a join of only
const tables. EXPLAIN FORMAT=JSON shows these conjuncts as
outer_ref_condition instead of pseudo_bits_condition.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a versioned table stores its history in partitions, an implicit
query (no FOR SYSTEM_TIME) needs only current rows, which all live in
the current partition. vers_setup_conds() no longer adds a row_end
condition for such a table, and prune_partitions() restricts
read_partitions to the current partition instead.
read_partitions is reset to all partitions every statement in
open_table(), so the restriction must be applied on each execution.
prune_partitions() runs on each execution, also for merged derived
tables and views, where vers_setup_conds() is not called again. Both
functions guard with !vers_conditions.was_set(). In prune_partitions()
!is_set() would not do: set_all() sets type (flips is_set()) but not
orig_type (was_set()), so is_set() would be true on prepared-statement /
stored-procedure re-executions and skip the restriction, leaving history
partitions readable - rows matched then grow on each execute/call.
A query whose time comes from @@system_versioning_asof must still read
history, so the restriction is also skipped when from_sysvar is set.
init_from_sysvar() sets this new flag rather than orig_type, so
was_set(), print() and the write lock check still see only an explicit
FOR SYSTEM_TIME. Like orig_type, the flag persists to prepared-statement
re-execution.
EXPLAIN no longer shows "Using where" for such a query: correctness now
comes from partition selection rather than a row_end filter.
With sql_safe_updates, DELETE without WHERE on such a table is now
rejected even with LIMIT, as for a non-versioned table. With an index
on row_end, UPDATE or DELETE without WHERE (UPDATE also without LIMIT)
is now rejected too: it was accepted as a range scan on row_end. A
versioned table without SYSTEM_TIME partitions keeps the old behavior
(MDEV-41393).
read_partitions is reset to all partitions (or the PARTITION() set) once
per statement in open_table() (set_partition_bitmaps()), and
prune_partitions() only intersects it with lock_partitions and the pruned
set instead of rebuilding it.
mark_all_partitions_as_used() is dropped.
Pure refactoring, results unchanged. It lets a restriction set on
read_partitions before pruning survive it - needed when read_partitions
is pre-restricted, e.g. a versioned table whose history is partitioned
reads only the current partition.
prune_partitions() built the used-partition set directly in
read_partitions: it cleared it, let find_used_partitions*() fill it, and
in the index-merge case used it as scratch too - both buffer and result.
The patch adds PART_PRUNE_PARAM::parts_bitmap and builds the return
into it instead. read_partitions is produced from it only at the end:
copied when the condition was analyzed (IMPOSSIBLE included,
parts_bitmap left empty), or set to all locked partitions via
mark_all_partitions_as_used() when nothing could be pruned.
Pure refactoring: read_partitions ends up identical on every path. It
decouples the pruning scratch from read_partitions, so a later commit can
switch the final bitmap_copy() to bitmap_intersect() - letting pruning
narrow a pre-set read_partitions (e.g. a versioned table selecting only
its current partition) instead of overwriting it.
The test checks that sys.innodb_buffer_stats_by_table reports old
pages (pages_old) after a server restart. InnoDB creates the old
sublist of the buffer pool LRU list only when the list holds at least
BUF_LRU_OLD_MIN_LEN (512) pages. After the restart, most of the pages
in the list come from the buffer pool load at startup. This load runs
asynchronously, and the server accepts connections before it
completes. If the test runs the check first, the list can be shorter
than 512 pages, no page is old, and the result shows 0 instead of 1.
The option --innodb-buffer-pool-load-at-startup=0 disables the load.
This is the extreme case of the race: the check always runs before
any page of the dump is loaded. With this option, the test fails every
time with innodb_page_size=4k and 16k.
Wait for the buffer pool load to complete before the check, as other
tests do.
Without passing through the compile and linker flags, options
like -stdlib=libc++ can create ABI differences between ducksdb
extensions and what the server links too.
The jemalloc has C files so CMAKE_C_FLAGS is passed through too.
The warnings were in RocksDB (not MyRocks) and looked like this:
rocksdb/db/db_impl/db_impl.cc: In member function ‘virtual rocksdb::Status
rocksdb::DBImpl::FlushWAL(bool)’:
rocksdb/db/db_impl/db_impl.cc:1329:23: warning: redundant move in return
statement [-Wredundant-move]
1329 | return std::move(io_s);
| ~~~~~~~~~^~~~~~
slave_close_thread_tables() unconditionally called
trans_commit_stmt()/trans_rollback_stmt(), which assert
!thd->in_sub_stmt. A BINLOG statement with malformed base64 payload
executed from an AFTER INSERT trigger hits this: on decode failure,
mysql_client_binlog_statement() sets thd->is_error() and calls
slave_close_thread_tables() from within the trigger's sub-statement.
The fix refuses BINLOG in stored routines, triggers, events and
anonymous blocks with ER_SP_BADSTATEMENT at parse time, like LOCK,
HANDLER etc. Dynamic SQL (EXECUTE IMMEDIATE, EXECUTE) run from a
routine bypasses the parser check, so mysql_client_binlog_statement()
checks spcont as well: in a stored procedure in_sub_stmt is 0, but the
decoded events assert !spcont in reset_for_next_command().
1. BINLOG statement executed from a trigger, SF or SP is disabled
by the patch:
Row events (Rows_log_event) open their target table via a
one-shot check that only fires at the top of a fresh statement.
Inside a trigger, the table is never opened this way. It may be
fixed by reusing query_tables, but:
- find_locked_table() matches only by table name -- it could return
the TABLE instance the enclosing statement is actively writing
through, not an idle one. Reusing it would require pre-saving its
state: record[0]/bitmaps/handler/etc. (and would bypass
ER_CANT_UPDATE_USED_TABLE_IN_SF_OR_TRG)
- set_stmt_row_injection()/set_time() calls would mutate thd->lex,
which at that point is main_lex -- shared with the enclosing
statement, not something safe to touch.
Statement events (Query_log_event) run the embedded query via
mysql_parse(): it bundles lex_start(), reset_for_next_command() and
parse_sql() as one unit meant for a genuinely new top-level statement,
not a one-off nested parse.
Calling parse_sql() directly instead avoids that, but then we own
everything mysql_parse() was doing for us: a private LEX and a
private Query_arena (or allocations land on main_lex/whatever arena
is currently active, shared with the enclosing statement), plus
calling mysql_execute_command() ourselves afterwards.
In any case, DML for query_tables cannot be done due to
ER_CANT_UPDATE_USED_TABLE_IN_SF_OR_TRG reasons explained above.
2. PS for BINLOG statement still works and needs a leak fix for
statement events:
A BINLOG statement decoding to a Query_log_event, executed via
PREPARE/EXECUTE, leaked its nested query's allocations onto the PS's
own persistent arena: thd->stmt_arena pointed at it while mysql_parse()
ran the decoded query. Second EXECUTE asserted on ROOT_FLAG_READ_ONLY,
since PROTECT_STATEMENT_MEMROOT marks that arena read-only after a
successful execution.
The fix redirects thd->stmt_arena to thd itself for the duration of
the event application, so stmt_arena->is_conventional() reads
true and activate_stmt_arena_if_needed() (called e.g. from
save_leaf_tables()) never redirects allocations to the PS's arena in
the first place.
Harmless for what it's protecting: leaf_tables_exec is normally cached
on the persistent arena so a repeatedly-executed statement's
SELECT_LEX doesn't rebuild it every time, but our SELECT_LEX is torn
down and reparsed fresh (due to mysql_parse() semantics) on every
EXECUTE, so there's nothing to cache here regardless of which arena is
used.
GetJVM() only looked for %JAVA_HOME%\bin\client\jvm.dll, and for the
RuntimeLib registry value of the "Java Runtime Environment" key. Since
JDK 9 there is no "client" JVM, jvm.dll is in bin\server, and the JDK
installers write JavaHome, but not RuntimeLib, under JavaSoft\JDK.
Look for jvm.dll the way Java installations are laid out. Prefer
bin\server over bin\client, in the JDK and in its jre subdirectory.
Without connect_jvm_path (which stays exclusive, as before), try
JAVA_HOME first, then the JavaSoft\{JDK|JRE} registry keys:
CurrentVersion, then the RuntimeLib or JavaHome value of that version.
Keep the first meaningful LoadLibrary or registry error (e.g. a 32/64
bit mismatch) rather than the "not found" of the last probe, reject
over-long paths instead of truncating them, accept REG_EXPAND_SZ
values, ignore an empty JAVA_HOME, and remove a stray printf to stdout.
Tested manually with Oracle JDK 23: with JAVA_HOME unset (registry),
with JAVA_HOME set, and with an invalid JAVA_HOME falling back to the
registry.
Assisted-By: Claude:claude-5.5-sonnet
SET GLOBAL event_scheduler=1 executed by a connection during shutdown
could start the scheduler thread after close_connections() called
Events::stop(). Nothing stops this thread and close_connections()
waits for it forever.
Now it hangs instead of a crash since MDEV-30260 fix: it moved
deinit() from close_connections() to clean_up(), so the scheduler is
not freed under the running thread anymore.
The fix makes Event_scheduler::start() do nothing when abort_loop is
set. abort_loop is set before close_connections() and is checked under
LOCK_scheduler_state, which stop() also takes, so the scheduler is
either started before stop() and stopped by it, or not started at all.
No error needed: the server is shutting down.
This SQL script failed:
SET NAMES latin1 COLLATE latin1_swedish_ci;
CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1;
SET NAMES big5 COLLATE big5_chinese_ci;
SELECT * FROM v1 WHERE c1='y';
with the following error:
ERROR 1267 (HY000): Illegal mix of collations
(latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE)
for operation '='
Note, latin1_swedish_ci and big5_chinese_ci are used here as examples.
The error also happened with different collation combinations.
Fix main idea:
If two collations have equal comparison rules (known as "tailoring")
on a given character repertoire,
like latin1_swedish_ci and big5_chinese_ci on ASCII letters,
then the "Illegal mix of collations" error can be avoided in
a comparison operator. We can choose any of the sides as the operation
effective collation - the result will be equal.
This optimization is not applied when at least one side has an explicit
COLLATE clause. Two explicit COLLATE clauses in one comparison are
already illegal when the character sets are the same, so for
consistency this stays illegal when the character sets differ too.
Most important details:
- Splitting enum_repertoire_t into smaller subsets,
for better repertoire granularity.
A variable holding a repertoire value can now have multiple
MY_REPERTOIRE_XXX flags set.
This patch implements detecting tailoring equality
on these repertoires:
* MY_REPERTOIRE_ASCII_DIGITS - [0..9]
* MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'.
It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and
MY_REPERTOIRE_ASCII_DOT. They were a part of
MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values
of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed
(223 and 255).
* MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9].
* MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore
* MY_REPERTOIRE_ASCII - the entire range U+0000..U+007F
- Adding a new virtual function "tailoring" in my_collation_handler_st.
It returns the tailoring on the given repertoire for the given
collation.
If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0},
it means the illegal mix optimization cannot be used
for this collation on the given repertoire.
If these calls:
tr1= cs1->coll->tailoring(cs1, some_repertoire);
tr2= cs2->coll->tailoring(cs2, some_repertoire);
return both non-NULL results and tr1.str==tr2.str,
then these collations are equal on the given repertoire
and are mutually replaceable for a comparison operator,
so "Illegal mix of collations" can be avoided.
- Tailoring strings are shared constants my_tailoring_str_*
(strings/strings_def.h). Tailorings are compared by the string
pointer, so tailoring() implementations return strings only from
these constants. The strings do not depend on PAD/NOPAD.
- my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit
character sets no longer stops at the first bad byte sequence with
the repertoire found so far, which could be MY_REPERTOIRE_NONE
(compatible with everything). It returns MY_REPERTOIRE_EXTENDED
for such strings. A unit test was added into unittest/strings.
- Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set
can store all ASCII characters (it is false for swe7).
It is used in Item_func_conv_charset to calculate "safe", and in
left_is_algorithmically_simpler() to prefer other character sets
to those where ASCII strings cannot be converted safely.
- DBUG_ASSERTs were added into mysys/charset.c, to check that
the "tailoring" method is set in collation handlers when a collation
is added: compiled-in, from ctype-extra.c, or loaded from Index.xml.
- Adding a new function my_collations_equal_on_repertoire(cs1, cs2,
repertoire) in strings/. It tells if two collations are equal on
the repertoire, taking tailoring() results and PAD/NOPAD and the UCA
version into account. Callers do not compare tailoring() results
directly, so the way tailorings are compared (currently by the pointer
to the shared constant string) is private to strings/ and can be
extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30.
- Adding a new method DTCollation::aggregate_by_tailoring().
Collations with different MY_CS_NOPAD are never compatible.
UCA collations of different versions are not compatible on
the repertoire with ASCII punctuation, as the UCA version
affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT
and CIRCUMFLEX ACCENT after PERCENT SIGN).
- DTCollation::aggregate() now makes the repertoire of the result cover
the repertoires of both sides in all successful branches.
Before, set(dt) replaced the repertoire with the repertoire of the
winner side only, so the result could have a too narrow repertoire,
e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the
value '1.5' has a punctuation character. With the more granular
repertoires this could wrongly allow mixing collations which are
equal on ALNUM but different on punctuation.
- Adding a new flag MY_COLL_ALLOW_BY_TAILORING.
It indicates to DTCollation::aggregate() that the illegal
mix optimization by repertoire can be used in the given context.
MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING.
Note, only comparison operators pass this flag.
Functions returning a string result do not pass this flag,
because in operations like CONCAT(a,b) we still need to evaluate
precisely the collation of the result - we cannot just choose
a collation of one of the sides (even if they are compatible
on the given repertoire). This also applies to ExtractValue()
and UpdateXML(), which now aggregate their arguments without
this flag.
- As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of
bits rather than a single bit, the way to detect "is only ASCII"
repertoires has changed in the code.
For example:
// repertoire *IS* ascii
if (repertoire == MY_REPERTOIRE_ASCII)
has changed in multiple places in the code to
// repertoire *HAS* only ascii characters
if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII))
The new function my_repertoire_is_subset_of() in m_ctype.h
is used for this purpose in C code. In C++ code
DTCollation::repertoire_is_subset_of() is used, e.g.:
if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII))
- Repertoire of some expressions was adjusted to the new meaning:
* Item_null now has MY_REPERTOIRE_NONE (was ASCII).
* MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30
(was EXTENDED).
* Lex_string_with_metadata_st::repertoire(cs) now scans the string
contents to detect the actual repertoire.
* HEX() now has MY_REPERTOIRE_ASCII_ALNUM.
* DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale
which is actually used: the explicit third argument, or
@@lc_time_names. It was always @@lc_time_names before.
A non-constant locale argument is assumed to be non-ASCII.
* QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY,
JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra
characters they put into the result (quotes, separators, padding,
brackets).
* LOWER() and UPPER() add the letters of both cases to the repertoire,
because the result can contain letters of the opposite case.
In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an
ASCII letter can be converted to a non-ASCII one (I -> dotless i).
Unicode collations are detected by the new member
casefold_info_st::can_convert_to_non_ascii_on_casefolding,
simple 8bit collations by to_lower['I'] and to_upper['i'].
- Lex_string_with_metadata_st::repertoire(cs) now scans a string literal
for any byte oriented character set (mbminlen==1), not only for
ASCII based ones. It was assumed to have the full repertoire for swe7,
which is not ASCII based. The scan is done by Unicode code points, so
swe7 strings with digits and letters get a narrow repertoire, while
e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII.
Therefore swe7_swedish_ci can be mixed with other collations now,
in the same way as other collations with MY_CS_IDENT_CASEUP_CI.
The test func_time was adjusted: a swe7 literal ' ull' with ASCII
characters is now safe to convert, the error is still tested for '[ull'.
- cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets
the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS,
as it sorts the minus and the dot differently. Both were verified
with ctype_digits_verify.
- The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL),
so my_charset_is_ascii_based() is true for tis620.
It was false for tis620, although tis620 is ASCII compatible.
Therefore tis620 string literals were always considered to have
a non-ASCII repertoire, and the repertoire optimizations did not work
for them. Now ASCII-only tis620 literals are detected as ASCII.
This changes the result of subselect_extra_no_semijoin
from an error to success.
- tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case
and then compare by the code point on the entire ASCII range
(the Thai specific rules affect only non-ASCII characters).
A new function my_tailoring_ascii_casedn_ci() returns tailorings
for such collations on ALNUM, IDENT and ASCII. For example, the
underscore sorts before the letters, unlike in the caseup tailorings.
- New flags were added for CHARSET_INFO::state
* MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations.
It means that this collation has no irregularities on the ASCII
repertoire, and converts lower case letters to upper case ones,
so the underscore sorts after the letters (with an upper to lower
case conversion it would sort before the letters).
* MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations.
It means that this collation has no irregularities on the IDENT
repertoire (but can have irregularities say on punctuation).
* MY_CS_ASCII_STD_UCA - for UCA collations.
It means that a UCA collation does not reorder ASCII characters.
* MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits
0..9 are sorted in the code point order and are not tailored, and,
for PAD collations, the space sorts before the digits. Strings of
digits are compared in the same way in all such collations, even if
they have irregularities on letters (latin7, cp866, czech, danish,
turkish UCA, etc.), or differ by case sensitivity. It is detected
from sort_order for 8bit collations, and from the tailoring rules
for UCA collations. Collations with MY_CS_DIGITS_STD still must have
the same PAD/NOPAD attribute.
* MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means
MY_CS_DIGITS_STD, and additionally the minus sorts before the dot,
which sorts before the digits (and the space sorts before the minus
in PAD collations). Strings like '-10.5' are compared in the same
way in all such collations. It is detected from sort_order for simple
8bit collations (my_coll_init_simple()), and from the tailoring rules
for UCA collations. For the multi-byte collations big5_chinese_ci,
gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared
by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD
and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for
compiled collations (conf_to_src.c prints them), and the detection is
skipped for a collation which already has any of the repertoire flags
(MY_CS_REPERTOIRE_FLAGS).
- strings/conf_to_src.c was modified to detect and print
MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags.
- strings/ctype-extra.c was regenerated with new flags.
- 8bit collations are not given MY_CS_ASCII_CASEUP_CI and
MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort
before the digits. A shorter string is padded with spaces for
comparison, so the position of the space matters even for strings
without spaces ("ab" is compared to "abc" as "ab " to "abc").
A test collation latin1_space_after_letters_ci was added to
mysql-test/std_data/ldml/ for this. ctype-extra.c does not change.
- The initializer of my_collation_cs_handler in ctype-utf8.c
(utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match
my_collation_handler_st: the missing get_id and get_collation_name
members were added, so that eq_collation was not initialized in their
place, and the new member "tailoring" is now set to my_tailoring_none.
- Results of the existing tests func_str, func_test, ps, view
and subselect_sj changed from "Illegal mix of collations" to success.
In these tests ASCII-only strings were compared using collations
with equal tailoring on ASCII, e.g. latin1_swedish_ci with
latin2_general_ci (func_str, ps), koi8r_general_ci with
latin1_swedish_ci (func_test), latin1_general_ci with
latin1_swedish_ci on letters and digits (view), cp932_japanese_ci
with latin1_swedish_ci (subselect_sj).
Such comparisons are now allowed.
Cases with collations that are still incompatible on ASCII,
e.g. latin7_general_ci, were added to the tests and still fail.
- Adding a number of MTR tests in
plugin/func_test/mysql-test/func_test/.
They display a tailoring by collation name and repertoire as
returned by:
cs->coll->tailoring(cs, some_repertoire)
A dynamically linked plugin function collation_tailoring() was added
for the purpose of these tests. Tailorings do not depend on PAD/NOPAD
and on the UCA version, so the function shows "[nopad]" for NOPAD
collations, and "[version X.Y.Z]" for UCA collations on the ASCII
repertoire. The version is formatted by a new class UCAVersion,
which uses a new method CharBuffer::append_uint8().
- Adding the test plugin/func_test/mysql-test/func_test/
ctype_caseup_ci_verify.test. It checks the order of ASCII characters
for all collations declared "caseup CI" on the ASCII or IDENT range.
- Adding the test plugin/func_test/mysql-test/func_test/
collation_tailoring_args.test. It checks the arguments of the plugin
function collation_tailoring(): the repertoire passed by name or by
number, and the NULL results for unknown names.
- Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test.
It displays the tailorings of collations loaded at runtime from
Index.xml: simple 8bit collations (flags detected from the weights)
and Unicode collations (LDML rules which do or do not reorder ASCII).
- Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test
They display two-dimensional charts showing which collations
are compatible on which repertoires.
Tests for DATE_FORMAT() with an explicit locale were added to
ctype_big5_tailoring.test, ctype_cp932_tailoring.test and
ctype_latin5_tailoring.test, which also has the chart for latin5.
- Adding a number of MTR tests in the form of the originally
reported script for various collations:
SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...';
The repeated blocks of queries are shared through
mysql-test/include/
ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc.
- Adding tests to ctype_cp932.test checking that ExtractValue() and
UpdateXML() still raise "Illegal mix of collations".
Problem: JSON_OVERLAPS treats unequal nested arrays as equal when only
one parser reaches the array end. Requiring both ends also exposes a
cleanup issue: scanning ']' moves stack_p to the parent, so skipping
the current level consumes parent elements. This misses the common [2]
in [[1],[2]] and [[],[2]], and can skip unequal recursive elements.
Fix: Require both parsers to reach JST_ARRAY_END. Remember their array
levels before comparing and skip to those saved levels after completed
comparisons and mismatches, preserving parent siblings. Explain why the
saved levels are needed. Add regression coverage for both argument
orders, unequal lengths, empty arrays, recursive comparisons, type
mismatches, array order and a later common object member.
get_clone() returns NULL on out-of-memory, but two of its callers in
prep_alter_part_table() dereferenced the result without a check, so the
server crashed:
changed INTERVAL of SYSTEM_TIME partitioning
ADD PARTITION under LOCK TABLES
get_clone() itself did not check the allocation of Vers_part_info and
dereferenced it.
handle_alter_part_error() did not check the clone either. The clone only
kept the DDL log entries after the table is closed. In the branch where
DDL log recovery fails, write_log_completed() released the entries
through lpt->part_info, which the table close may have freed, and
release_log_entries() released the same entries again through the
clone. That branch is not reachable now, as ddl_log_execute_entry()
always returns false.
The fix checks the allocation of Vers_part_info in get_clone() and the
result of get_clone() in prep_alter_part_table() like the other callers
there do. handle_alter_part_error() keeps a copy of DDL_LOG_STATE
instead of the clone and passes it to write_log_completed() and
release_log_entries(), which now take DDL_LOG_STATE. The redundant
release_log_entries() after write_log_completed() is removed.
ALTER TABLE ... PARTITION BY SYSTEM_TIME with a changed INTERVAL or
STARTS on a table already partitioned by SYSTEM_TIME took the fast
ALTER path, which only rewrites the table definition. The history rows
stayed where the old layout put them, so after the ALTER they were in
wrong partitions. An unchanged interval took the copy path instead.
MDEV-29873 replaced memcmp() of the intervals, which is true when they
differ, with interval_t::operator==, which is true when they are equal,
and lost the negation.
The copy path was also broken: it assigned the whole vers_info of the
new partitioning spec to the cloned partition info, now_part and
hist_part included. The ALTER does not list partitions, so now_part
became NULL and check_partition_info() failed with ER_VERS_WRONG_PARTS
for a table with explicit partitions (MDEV-33370 test case).
The fix negates the comparison again (now with interval_t::operator!=),
so a changed interval rebuilds the table and redistributes the history.
It copies only interval, limit and auto_hist from the new spec and keeps
now_part and hist_part of the existing partitions.
Without STARTS the start of the interval defaults to the current time
rounded down to the INTERVAL unit. So ALTER with the same INTERVAL and
no STARTS usually gets a different start and now rebuilds the table.
Vers_part_info::operator=, which copied now_part and hist_part and had
no other callers, is deleted.
Fast ALTER was also chosen when the same statement changed the table
definition, e.g. ADD COLUMN with the same INTERVAL and STARTS, or DROP
COLUMN with LIMIT. It wrote the new .frm, but the partitions kept the
old record layout. Now any change of the table definition
(alter_info->flags) takes the copy path.
PURGE BINARY LOGS could end with a warning instead of deleting files, as
the test below shows when run on the base of this commit. The warning
names a wrong reason. It follows an automatic purge that was refused by
the slave guard, slave_connections_needed_for_purge: the automatic purge
does not delete a binlog while it might still be needed by a slave, not
necessarily a connected one, that is while fewer than the variable's
number of slaves are connected.
Since MDEV-34504 the manual PURGE is meant to ignore the guard.
Yet it still protects a binlog that a slave is reading.
Technically this was possible because the automatic and the manual purge
share a code path, can_purge_log(), which cached the refusal and replayed
it, under the reason of the first arm below, to any later purge:
if (is_active(name) ||
(!is_relay_log && waiting_for_slave_to_change_binlog &&
purge_sending_new_binlog_file == sending_new_binlog_file &&
!strcmp(name, purge_binlog_name)))
reason= "it is the current active binlog";
So "active" had to be read as "active or busy". The fix removes this
duality too: the two arms are now separate and have their own reasons.
The main issue is fixed by narrowing the busy arm with an additional
`!interactive &&` conjunct, so that a manual PURGE never consults the
cached refusal.
Running the test below on an unfixed 11.4 server (source tree at commit
42038e1145) fails with "Result length mismatch" and, in part 2, this
diff against the recorded result:
PURGE BINARY LOGS TO 'master-bin.000006';
+Warnings:
+Note 1375 Binary log 'master-bin.000003' is not purged because it is the current active binlog
SHOW WARNINGS;
Level Code Message
+Note 1375 Binary log 'master-bin.000003' is not purged because it is the current active binlog
show binary logs;
Log_name File_size
+master-bin.000003 #
+master-bin.000004 #
+master-bin.000005 #
master-bin.000006 #
master-bin.000007 #
master-bin.000008 #
Test binlog.binlog_purge_stale_refusal_cache: the test sets
slave_connections_needed_for_purge=1, binlog_expire_logs_seconds=0 and
max_binlog_total_size=0 first and restores them at the end. It starts
with RESET MASTER so that the binlog names, 000003 and so on, do not depend
on the tests run before it in the same mtr environment. Part 1 is a control with
nothing cached; part 2 makes an automatic purge refuse 000003 (the files
must be older than binlog_expire_logs_seconds, hence --real_sleep 2 before
the rotation) and then runs PURGE BINARY LOGS TO 'master-bin.000006',
which must remove 000003..000005.
As a workaround, a PURGE could work if the following settings are made
before it, in this order:
SET GLOBAL binlog_expire_logs_seconds=0, slave_connections_needed_for_purge=0;
The second setting invalidates the cached refusal. The first one keeps
the purge that this setting triggers from refusing and caching again.
Note that both settings change the server's policy, not only this one
PURGE: the time-based expiry and the slave guard are off until they are
restored.
1) take into account TMP_TABLE_ALL_COLUMNS when
we are modifying agg_item->result_field
2) remove unused now "bool materialized_subquery;"
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test sent
ALTER TABLE t1 ADD COLUMN f3 INTEGER; INSERT INTO t1 VALUES (NULL, 10000, 10000);
from node_2 as a single --send. Nothing ordered that INSERT against the
CREATE UNIQUE INDEX issued from node_1, so the second DDL could reach node_2
while the INSERT was still executing there and BF-abort it. The automatic
autocommit retry then replicated with last_seen_seqno behind the seqno of the
DDL and failed certification against its exclusive key on test.t1. As
wsrep_retry_autocommit defaults to 1 there was no second retry and the client
got ER_LOCK_DEADLOCK.
Send only the ALTER TABLE and execute the INSERT after both DDL statements
have completed on both nodes.
Waiting for the second DDL by polling INFORMATION_SCHEMA.PROCESSLIST did not
prove that it had been ordered either: wsrep_TOI_begin() sets the
"Waiting to execute in isolation" stage before it replicates the writeset, so
that state can be observed before the DDL has reached total order. Wait on
node_2 for the new index to appear instead, which happens only once the DDL
has been ordered behind the parked ALTER TABLE, and record on node_1 that
neither DDL has been applied there while the first one is still parked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Problem: JSON_TABLE maps JSON numbers [0,1] to BIT(1) values [1,1].
The scalar helper passes decimal text to BIT's binary-string store,
so the byte for '0' is 48 and is clipped to 1.
Fix: Add Field::store_numeric() for number literals serialized as text,
including fractions and exponents, and use it for JSON_TABLE numbers.
BIT parses these literals as rounded unsigned integers and applies its
existing range checks. Other fields retain their existing conversion;
JSON strings and booleans retain their existing storage paths.
Preserve the existing distinction between conversion warnings and JSON
structural errors, so numeric overflow does not invoke ON ERROR.
Extend json.json_table to cover BIT widths, unsigned boundaries,
fractions, exponents, boolean/string/null inputs, other field types,
conversion warnings and ON ERROR responses.
Assisted-by: OpenAI Codex:gpt-6
Issue:
generate-ssl-certs.sh is a /bin/sh script but uses process substitution
(<(printf ...)) for -extfile, which fails with plain sh.
Solution:
Write the extensions to a file in gcadir/ and pass it to -extfile.
Problem: SUM(DISTINCT b'1100') returns 9 instead of 12. The hybrid
hex/bit literal handler uses the default DOUBLE temporary field, so
its byte length becomes a DOUBLE(M,0) display width and clips the
numeric value.
Fix: Override numeric DISTINCT field construction for hybrid hex and
bit literals to allocate an unsigned Field_longlong directly. Preserve
argument-based field selection for other types so DISTINCT retains the
input expression's precision and scale, including decimal AVG inputs.
Extend main.sum_distinct to cover bit and hex literals, zero, byte and
unsigned 64-bit boundaries, duplicates, AVG, COUNT and decimal scale.
Recreated-by: Georgi "Joro" Kodinov <joro@mariadb.org>
cherry-pick commits 9ecec1f730, 24fd8c7856, 3e97105975
* make it easier to add more old-mode values
* update deprecation.h to match changes in versioning
