This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is controlled
by optimizer_replay_context, and points to an user defined variable name,
that has the context information.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index, cost of reading an index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records, along with their costs are also
loaded from the trace.
The trace context which is in JSON format is firstly set into the
user defined session variable, and this variable name is set to
optimizer_replay_context system variable.
Later, when a user issues a query, the contents of the json context is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.
The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.
Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
When innodb_compression_algorithm is set to a non-zlib algorithm (e.g.
lz4) and the provider plugin is loaded from mysql.plugin rather than
command line, InnoDB failed to start because it checked for the provider
at plugin initialization time, before plugin_load() reads mysql.plugin.
Fix:
* InnoDB returns HA_ERR_RETRY_INIT when the compression provider is
missing.
* In sql_plugin.cc, the retry loop is changed to not reap until
mysql.plugin has been loaded
Assisted-By: Claude:claude-4.6-sonnet
* add a test for new --cat_file feature
* fix off-by-one error in --cat_file lines limit
* fix broken INCLUDE_DIRECTORIES in extra/
* fix typos
* remove mtr-specific workaround from the tool, solve it in the test
* remove dead code
* fixed memory leak in mariadb-migration-config-file [client-server] section
* fixed memory leak in mariadbd --character_set_server=xxx
(execution time for main.mariadb-migrate-config-file went down
from 150s to 2s)
* moved tool header to the tool dir
* avoid tripple-initialization of plugins
* disable tool builds in ASAN builds (because of RocksDB)
* compilation failure on x86 (signedness)
The tool is named mariadb-migrate-config-file.
The main purpose of the tool is to change MySQL option
files to work both for MySQL and MariaDB.
There are options to do the changes in the options file inline,
or at-end-of-file. One can also remove or comment unknown options.
The list of supported options is generated compile time from
mariadbd --help. All server options, including compiled plugins, are
supported.
The bulk of the code comes from Väinö.
Monty has updated it with a lot of extra options.
Wlad helped with cmake integration
Other things:
- Fixed a memory leak in sql_plugin.cc
- plugin-load will now in case of errors try to load all given plugins
before aborted
- If silent-startup is used, plugin-load will not give errors for
plugins it cannot load or warnings about plugin marturity level.
- my_rm_tree() will now delete symlinks, not the actual file, if
MY_NOSYMLINK flag is used.
- my_stat() will now give data for symlink if MY_NOSYMLINKS is used.
- Added 'number of lines' option to mysqltest --cat_file
@Authors: Väinö Mäkelä <vaino.makela@iki.fi>,monty@mariadb.com
This saves 10 calls to pthread_mutex_lock when constructing the
initial OK packet for the default session_track_system_variables which
has 6 variables
Also added a testcase showing there's no need to worry about wildcard
session_track_system_variables as it causes construction of empty
string in the initial OK packet.
the "Test that bad value for plugin enum option is rejected correctly"
needed multiple fixes:
1. don't set plugin-dir based on $MYSQLTEST_VARDIR, all plugins are
in var/plugins, but $MYSQLTEST_VARDIR is var/1/, var/2/, etc if
--parallel is used (that is, practically always), thus the ha_example.so
cannot be loaded, because cannot be found. Test fails with
"unknown option --plugin-example-enum" as the plugin is not loaded
2. force --plugin-maturity=experimental, otherwise even if not parallel
the plugin will fail to load because of low maturity, test still
fails with "unknown option --plugin-example-enum"
3. don't specify .so extension explicitly otherwise the plugin still
doesn't load on windows, even if paths and maturity are fixed
4. set --plugin-example=FORCE otherwise plugin fails to load after reading
--plugin-example-enum-var=noexist because of unknown enum value,
the server ignores the failure and starts normally. the test hangs.
5. This needs the fix in sql_plugin.cc to detect that the plugin is
forced even when some options failed to parse. It used to consider
plugin forced only if all options parsed correctly, which was wrong.
Now the test passes, testing what it was supposed to test - failure
to parse an enum value of a plugin option.
Without these fixes the test hanged as in 4 when run on the main branch
in non-prarallel (e.g. one test only) mode.
Every Wsrep system thread should run with READ_COMMITTED transaction
isolation level to prevent issues caused by InnoDB gap locks.
The exception is statement-based replication for appliers, where
REPEATABLE_READ is required by the server code. To account for that,
set the isolation level before every applied event. It won't affect
an already running transaction, but allows to handle both statement-
and row-based replications accordingly.
However, the problem might arise with the mixed replication format.
Apart from that, there was a separate issue with applier thread vars:
wsrep_plugins_post_init() would overwrite thd->variables for every
applier thread and forget to restore proper default isolation level.
Then, upon every server transaction termination,
trans_reset_one_shot_statistics() would set thread's isolation
level to the one stored in thd->variables, thus spoiling the isolation
level value for appliers.
Apparently, there was a separate issue with applier thread variables:
wsrep_plugins_post_init() would overwrite thd->variables for every
applier thread and forget to restore proper read only context.
Then, upon every server transaction termination,
trans_reset_one_shot_statistics() would set thread's read only context
to the one stored in thd->variables, thus spoiling the read only
context value for appliers.
Relaxed check, only number of columns and the PK.
Enough to avoid crashes, but doesn't break upgrades and migration
from MySQL as in MDEV-37777.
Added checks everywhere. (flush/create/alter/drop server)
Check mysql.plugin table too.
* rpl.rpl_system_versioning_partitions updated for MDEV-32188
* innodb.row_size_error_log_warnings_3 changed error for MDEV-33658
(checks are done in a different order)
There were too many C casts rather than C++ casts here.
Started a partial conversion covering within-file scoped
changes.
Suggested by Brandon Nesterenko
In plugins, use the correct resolver for ULONG and ULONGLONG
types.
InnoDB has a UINT type as evidenced by "Unknown variable type code 0x182
in plugin 'InnoDB'." so the implementation for UNSIGNED INT was added.
Any InnoDB mtr test that changes lock_wait_timeout,
spider_param_force_commit and some MyRocks unsigned thread server
variables can verify this change is correct.
Reviewer: Brandon Nesterenko
This commit updates default memory allocations size used with MEM_ROOT
objects to minimize the number of calls to malloc().
Changes:
- Updated MEM_ROOT block sizes in sql_const.h
- Updated MALLOC_OVERHEAD to also take into account the extra memory
allocated by my_malloc()
- Updated init_alloc_root() to only take MALLOC_OVERHEAD into account as
buffer size, not MALLOC_OVERHEAD + sizeof(USED_MEM).
- Reset mem_root->first_block_usage if and only if first block was used.
- Increase MEM_ROOT buffers sized used by my_load_defaults, plugin_init,
Create_tmp_table, allocate_table_share, TABLE and TABLE_SHARE.
This decreases number of malloc calls during queries.
- Use a small buffer for THD->main_mem_root in THD::THD. This avoids
multiple malloc() call for new connections.
I tried the above changes on a complex select query with 12 tables.
The following shows the number of extra allocations that where used
to increase the size of the MEM_ROOT buffers.
Original code:
- Connection to MariaDB: 9 allocations
- First query run: 146 allocations
- Second query run: 24 allocations
Max memory allocated for thd when using with heap table: 61,262,408
Max memory allocated for thd when using Aria tmp table: 419,464
After changes:
Connection to MariaDB: 0 allocations
- First run: 25 allocations
- Second run: 7 allocations
Max memory allocated for thd when using with heap table: 61,347,424
Max memory allocated for thd when using Aria table: 529,168
The new code uses slightly more memory, but avoids memory fragmentation
and is slightly faster thanks to much fewer calls to malloc().
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
