Replaces std::function / std::optional / std::unordered_map /
std::unordered_set / std::string_view and template-heavy code in the
master.info / relay-log.info persistence layer with plain MariaDB
idioms.
- On-disk file format unchanged.
- SQL syntrax unchanged
- No notable test changes, except those for which new error messages was
added and output from SHOW VARIABLES.
- master_* CLI options moved from inline globals in
rpl_master_info_file.h to Sys_var_* in sys_vars.cc (READ_ONLY,
AUTO_SET on use_gtid and heartbeat_period). master_heartbeat_period
is Sys_var_double (seconds); master_use_gtid is Sys_var_enum. Global
storage sits outside HAVE_REPLICATION so libmariadbd links.
- This adds the rpl_master_info variables to SHOW VARIABLES
- Value classes are no longer templates. Persistent renamed to Value;
Optional_* classes renamed (Uint_value_with_default, Path_value,
etc.). has_value flag in the base; is_default()
non-virtual. Multi-line method bodies moved to new rpl_info_file.cc
and rpl_master_info_file.cc.
- VALUE_LIST (Mem_fn array) and VALUE_MAP (std::unordered_map)
replaced by per-instance Value *const value_list[] / value_map[]
arrays + a static TYPELIB populated at first construction from each
Value's variable_name. seen-set dedup is now a uint32 bitmask.
- Default handling is not anymore done with using special values.
Setting a value to default is now done by calling set_default(),
which sets the has_value member to 0 and stores the default value
in the variable.
When reading a value, one does not have to check if the value is
a default value.
This makes the interface cleaner and easier to manage.
- CHANGE MASTER TO parser no longer builds std::function lambdas:
LEX_MASTER_INFO carries mi_used_options / mi_used_default bitmasks
plus plain value fields; change_master() applies them via bit-test +
direct copy or set_default(). std::optional<T> and trilean are gone
from the value class API.
- Heartbeat_period_value::load_from uses my_strtod() instead of
my_decimal for the on-disk format. from_decimal kept for CHANGE
MASTER.
- RocksDB #if __cplusplus < 201703L guard at the top of
rpl_master_info_file.h dropped; header now compiles in C++11.
cstdint dropped; uint32_t / UINT32_MAX in touched code replaced by
MariaDB's uint32 / UINT_MAX32. include/my_global.h gains
INT_BUFFER_SIZE next to LONGLONG_BUFFER_SIZE.
- Pointer-style throughout (T * not T &). Multi-line inline functions
used in more than one place moved to .cc files per style.
New features:
- SHOW VARIABLES now shows the master default values.
- If there is an wrong value for a row in a master.info file the
corresponding variable name and the value is now written to the
error log.
Bug fixes uncovered by the refactor:
- master_ssl_verify_server_cert had a pre-existing inconsistency with
security implications: the documented default and the Sys_var_mybool
default are TRUE (validate the master's certificate), but the OLD
init_ssl_config() set the per-Master_info field to FALSE on RESET
SLAVE ALL. Concretely: after RESET SLAVE ALL followed by a bare
CHANGE MASTER TO (no explicit MASTER_SSL_VERIFY_SERVER_CERT), the
slave would connect with MASTER_SSL=1 enabled but with certificate
validation silently disabled.
Fixed by not resetting master_ssl_verify_server_cert in
RESET SLAVE ALL;
- As some replication setups sets master_ssl_verify_server_cert= 0
while other does not do that, the expected value used in
check-testcase.inc "Master_SSL_Verify_Server_Cert No" does not make
sense. The value of this variable is now ignored by
check-testcase.inc
- The old code would fail to read a master info file if there was a new
variable longer than 23 characters. This is now extended to 80.
- Removed warning on RESET SLAVE that "RESET SLAVE is implicitly changing"
the value of 'Using_Gtid' from 'No' to 'Slave_Pos'" as this change did
not really happen. This error was given if the master could support
GTID but the user had specified directly or indirectly to not use GTID.
Other things
- Size of mariadb (optimize build) shrunk with 1,408,009 bytes
(Based on 'dec' from the size command)
- Size of code in sql directory did not change notable.
- Reworked Master_use_gtid_value to remember users last choice and use
this as default when connecting to slave.
Replace mi->using_gtid, with was a pointer to master_use_gtid, with
two functions:
- using_gtid() returns the current mode, for the code that needs the
value itself (SHOW SLAVE STATUS and performance schema output,
comparisons against CURRENT_POS and SLAVE_POS).
- is_gtid_supported() tells whether the connection uses gtid at all.
The many callers that compared the mode against USE_GTID_NO with
== or != now call this function instead of testing the enum value.
- Master_use_gtid_value::supported() did the same test as
is_gtid_supported() and is removed
- Removed some operators in rpl_master_info_file.* that hid
that functions was used to read and update values.
Behavior changes:
- The last used value in CHANGE MASTER for MASTER_USE_GTID will be
remembered for future CHANGE MASTER for the same connection. This is
true even if RESET SLAVE is called.
- If Using git is not used with change master but log_file_name, pos,
relay_log_name or relay_log_pos is specified this is treated as
if Using_git=NO would have been given. (Old behavior, but not
documented).
--auto-master-use-gtid does nothing. The default value of
master_use_gtid is by default SlavePos that will automatically
change to NO if the master does not support GTID. This already
handles what auto-master-use-gtid tries to do.
- CHANGE MASTER's MASTER_HEARTBEAT_PERIOD now works like other MASTER
options: When set to a value, this value is kept until value is
changed or set to DEFAULT. The DEFAULT is
@global.master_heartbeat_period if this is set as startup, if not
then @global.slave_net_timeout/2 is used.
Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.
Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:
CREATE PROCEDURE p1()
BEGIN
-- case 1: DEFAULT clause
DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK
-- case 2: SP variable assignment statement
DECLARE spvar2 INT;
SET spvar2= f1_with_ps(); -- OK
END;
- Only assignments to SP variables works for now:
* SET spvar= func_with_ps(); -- OK
* SET @uvar= func_with_ps(); -- Error
- Only bare function calls are supported for now. Using a function in
an expression does not make it PS-safe yet:
SET v= f1()+0;
- The parser now does not reject PS statements in stored functions.
PS applicability in stored functions is now detected at run time.
Note, PS statements in triggers are still prohibited by the parser.
- Functions with PS do not acquire MDL locks on tables, and no MDL is
taken on the routines themselves either. They work like procedures in
terms of table opening and routine locking: a concurrent DROP FUNCTION
can complete while such a function is executing.
- Functions with PS are not replicated as a single `SELECT f1()` call.
They are replicated per-statement, like procedures.
Helper changes:
- Changing the return result for LEX::sp_variable_declarations_init()
from void to bool to catch errors in the caller properly.
Misc:
- This patch incorporates fixes for the following bugs found during debugging:
MDEV-40224,MDEV-40225,MDEV-40226,MDEV-40227,MDEV-40240,
MDEV-40285,MDEV-40288,MDEV-40315,MDEV-40318,MDEV-40890,MDEV-40900,
MDEV-40901,MDEV-40913,MDEV-40914,MDEV-41013,MDEV-41015,MDEV-41019
Assisted-by: Claude - reviews and minor clean-ups
Move the initialization of context recorder, and replay after
run_set_statement_if_requested() is invoked in the
mysql_execute_command() in sql_parse.cc
This allows to record and save both Optimizer Context and Trace with
straightforward command sequence:
set optimizer_record_context=1, optimizer_trace=1;
explain select ... ;
select context from information_schema.optimizer_context;
select trace from information_schema.optimizer_trace;
- Introduce init_optimizer_context_recorder_if_needed() which is called
at query start. We n longer create the recorder lazily.
- Recorder functions no longer take MEM_ROOT to store data on as an
argument. The recorder knows the MEM_ROOT it should use.
This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is controlled
by optimizer_replay_context, and points to an user defined variable name,
that has the context information.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index, cost of reading an index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records, along with their costs are also
loaded from the trace.
The trace context which is in JSON format is firstly set into the
user defined session variable, and this variable name is set to
optimizer_replay_context system variable.
Later, when a user issues a query, the contents of the json context is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.
The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.
Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
This feature stores the basic stats of the base tables that are used in
a query, to the optimizer trace. This feature is also controlled by
optimizer_record_context, and is not enabled by default. The stats
such as num_of_records present in the table, indexes if present
then their names, along with the average number of records_per_key
with in each index are dumped to the trace. Additionally, stats from
range analysis of the queries are also dumped into the trace
The approach taken here is to extend the existing function
store_tables_context_in_trace() and add new dump_table_stats_to_trace()
to opt_trace_ddl_info.cc. For storing the range analysis stats into the
trace, they are first recorded in the hash defined in
Optimizer_Stats_Context_Recorder, and is used later while dumping the
stats into the trace from dump_table_stats_to_trace().
Several new tests are also added to opt_trace_store_stats.test
Binding DEFAULT to a LIMIT ? placeholder via EXECUTE ... USING is never
rejected at bind time. unit->set_limit() later evaluates it and sets
ER_INVALID_DEFAULT_PARAM in the Diagnostics_area, but SQLCOM_UPDATE (and
SQLCOM_UPDATE_MULTI, SQLCOM_INSERT_SELECT/REPLACE_SELECT) proceeded into
open_tables() regardless. A concurrent metadata change (e.g. ALTER TABLE
FORCE) then triggers Reprepare_observer::report_error(), which tries to
set ER_NEED_REPREPARE on an already-set DA and hits the assert.
Add the same thd->is_error() check after unit->set_limit() (as in
SQLCOM_DELETE), aborting before open_tables() is reached.
The "is it my own thread" check compared the login user name
(Security_context::user) and ignored the host, so u1@localhost could see
and kill threads of u1@'127.0.0.1'.
Compare the authenticated account instead:
* all comparisons are done in sctx->is_priv_user() now
* change user_matches() to priv_user_matches(), which uses is_priv_user()
* use it in KILL, KILL USER, SHOW PROCESSLIST, I_S.PROCESSLIST,
COM_PROCESS_INFO and SHOW EXPLAIN/ANALYZE FOR.
* all the remaining places use is_priv_user() directly instead
of doing strcmp: SHOW GRANTS, SHOW CREATE PROCEDURE, I_S.VIEWS,
the DEFINER clause, optimizer trace and change_security_context().
Assisted-by: Claude:claude-5-opus
WITH GRANT OPTION is not a privilege in the standard sense,
so whenever INFORMATION_SCHEMA visibility dictates "any privilege"
this does not include WITH GRANT OPTION, even though GRANT_ACL
is a privilege in MariaDB.
Assisted-By: Claude:claude-4.6-sonnet
check_stack_overrun() was compiled out under ASAN, so a deeply nested
expression recursed in Item_func::fix_fields() until the stack was
exhausted and the server crashed instead of reporting
ER_STACK_OVERRUN_NEED_MORE.
Since MDEV-34533 (Monty) the stack usage seems to be accounted correctly
under ASAN via my_get_stack_pointer(), so the check seems to works there
too.
Fix:
Remove the #ifndef __SANITIZE_ADDRESS__ guard from check_stack_overrun()
Add a test for ER_STACK_OVERRUN_NEED_MORE
SQLCOM_REVOKE_ALL needs it because SQLCOM_REVOKE does.
SQLCOM_DROP_ROLE needs it because SQLCOM_DROP_USER does.
SQLCOM_CREATE_SERVER / SQLCOM_ALTER_SERVER / SQLCOM_DROP_SERVER
need it because SQLCOM_CREATE_USER / etc do.
introduce a dedicated enum type for sql_command_flags
to simplify debugging:
(gdb) p sql_command_flags[SQLCOM_SELECT]
$2 = (CF_REEXECUTION_FRAGILE | CF_CAN_GENERATE_ROW_EVENTS | CF_PREOPEN_TMP_TABLES | CF_CAN_BE_EXPLAINED | CF_OPTIMIZER_TRACE)
This patch implements support for implicit query block (QB) names in
optimizer hints, allowing hints to reference query blocks and tables
within derived tables, views and CTEs without requiring explicit
QB_NAME hints.
Examples.
-- Addressing a table inside a derived table using implicit QB name
select /*+ no_index(t1@dt) */ *
from (select * from t1 where a > 10) as DT;
-- this is an equivalent to:
select /*+ no_index(t1@dt) */ * from
(select /*+ qb_name(dt)*/ * from t1 where a > 10) as DT;
-- Addressing a query block corresponding to the derived table
select /*+ no_bnl(@dt) */ *
from (select * from t1, t2 where t.1.a > t2.a) as DT;
-- View
create view v1 as select * from t1 where a > 10 and b > 100;
-- referencing a table inside a view by implicit QB name:
select /*+ index_merge(t1@v1 idx_a, idx_b) */ *
from v1, t2 where v1.a = t2.a;
-- equivalent to:
create view v1 as select /*+ qb_name(qb_v1) */ *
from t1 where a > 10 and b > 100;
select /*+ index_merge(t1@qb_v1 idx_a, idx_b) */ *
from v1, t2 where v1.a = t2.a;
-- CTE
with aless100 as (select a from t1 where b <100)
select /*+ index(t1@aless100) */ * from aless100;
-- equivalent to:
with aless100 as (select /*+ qb_name(aless100) */ a from t1 where b <100)
select /*+ index(t1@aless100) */ * from aless100;
Limitations:
- Only SELECT statements support implicit QB names. DML operations
(UPDATE, DELETE, INSERT) only support explicit QB names
Originally two MDL_EXCLUSIVE are used for pure alias of derived table
and its schema which is empty db name. It is concurrency performance
overhead and there is no obvious reason why such MDL should exist.
MDEV-33985 changed pure alias check from `db.str` to `db.length` and
that allowed `empty_c_string` to be pure alias too. It is important
for MDEV-33985 fix with sequences. Now such kind of pure alias skips
MDL initialization and thus derived table fails falsely on BACKUP
namespace as it is default namespace for 0-filled structure (see TODO
comments).
The fix a) keeps MDL_NOT_INITIALIZED for uninitialized MDL and that
was the consequence of MDL_INTENTION_EXCLUSIVE value 0 (see TODO
comment).
And b) does not update MDL type for pure alias when view is
initialized, so it is MDL_NOT_INITIALIZED. `lock_table_names()` code
naturally skips such kind of MDL with (mdl_request.type <
MDL_SHARED_UPGRADABLE) condition when processing tables list.
Optimization in st_select_lex_unit::prepare() removes ORDER BY for
certain subqueries. That excludes ORDER BY items from being fixed, but
sl->window_funcs still contains window function items and those
related to optimized out ORDER BY are unfixed. The error about missing
window spec is thrown when the item is fixed. Hence we get redundant
processing of window function items without checking window spec
existence.
The fix removes the related window function items when ORDER BY is
optimized out. ORDER accumulates window_funcs at parser stage which
are then removed from SELECT_LEX::window_funcs. The fix also updates
similar optimization in mysql_make_view().
SHOW CREATE SERVER was accessible to any user with a global
privilege, without requiring FEDERATED ADMIN. Add a privilege
check before mysql_show_create_server() so that only users
with FEDERATED ADMIN can view server definitions.
