This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 13.1.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 12.3.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
1. Built-in plugins (plugin_init):
A non-ASCII builtin name is a developer error. Added a check
using my_string_repertoire() that prints an error via
print_init_failed_error() and returns 1 to abort server startup.
2. Dynamically loaded plugins (plugin_add):
Added a pre-validation loop before the registration loop. If any
plugin name in the .so is non-ASCII, the entire .so is rejected
before any plugin is registered, preventing partial state where
valid plugins from the same .so get left INACTIVE in the hash.
Added test plugins and MTR tests to verify both rejection of
non-ASCII named plugins and successful loading of ASCII named plugins.
Bug: tp_foreach() found an engine in hton2plugin[] and passed the
result of plugin_lock() to plugin_hton() without a check. While
reap_plugins() deinitializes an uninstalled engine (PLUGIN_IS_DYING),
the slot stays set until the end of ha_finalize_handlerton(), and
plugin_lock() returns NULL. The server crashed in plugin_hton(), for
example in RESET MASTER through ha_commit_checkpoint_request().
This is a regression from aed5928207, which changed plugin_foreach()
to tp_foreach() and lost the PLUGIN_IS_READY state mask.
Fix: add plugin_lock_ready(), which locks only a PLUGIN_IS_READY plugin
and reports under LOCK_plugin whether a failed lock was for a READY
plugin (out of memory in debug builds). tp_foreach() skips a plugin that
is not READY and returns an error for a READY plugin that cannot be
locked. An uninstalled but busy engine (PLUGIN_IS_DELETED) is not
visited, as with plugin_foreach() before.
The test uses a DEBUG_SYNC point in ha_finalize_handlerton() to run
RESET MASTER while an engine is deinitialized.
Assisted-by: https://mariadb.org/governance/governance-ai-policy/
This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is controlled
by optimizer_replay_context, and points to an user defined variable name,
that has the context information.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index, cost of reading an index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records, along with their costs are also
loaded from the trace.
The trace context which is in JSON format is firstly set into the
user defined session variable, and this variable name is set to
optimizer_replay_context system variable.
Later, when a user issues a query, the contents of the json context is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.
The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.
Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
Protect all READ_ONLY sysvars from run-time changes.
Put them into a separate section and use my_virtual_mem_protect()
to make this section read-only before the server starts accepting
connections.
Verify that they're all protected in the sys_var constructor.
One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables
On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup
Assisted-By: Claude:claude-4.8-opus
When innodb_compression_algorithm is set to a non-zlib algorithm (e.g.
lz4) and the provider plugin is loaded from mysql.plugin rather than
command line, InnoDB failed to start because it checked for the provider
at plugin initialization time, before plugin_load() reads mysql.plugin.
Fix:
* InnoDB returns HA_ERR_RETRY_INIT when the compression provider is
missing.
* In sql_plugin.cc, the retry loop is changed to not reap until
mysql.plugin has been loaded
Assisted-By: Claude:claude-4.6-sonnet
* add a test for new --cat_file feature
* fix off-by-one error in --cat_file lines limit
* fix broken INCLUDE_DIRECTORIES in extra/
* fix typos
* remove mtr-specific workaround from the tool, solve it in the test
* remove dead code
* fixed memory leak in mariadb-migration-config-file [client-server] section
* fixed memory leak in mariadbd --character_set_server=xxx
(execution time for main.mariadb-migrate-config-file went down
from 150s to 2s)
* moved tool header to the tool dir
* avoid tripple-initialization of plugins
* disable tool builds in ASAN builds (because of RocksDB)
* compilation failure on x86 (signedness)
The tool is named mariadb-migrate-config-file.
The main purpose of the tool is to change MySQL option
files to work both for MySQL and MariaDB.
There are options to do the changes in the options file inline,
or at-end-of-file. One can also remove or comment unknown options.
The list of supported options is generated compile time from
mariadbd --help. All server options, including compiled plugins, are
supported.
The bulk of the code comes from Väinö.
Monty has updated it with a lot of extra options.
Wlad helped with cmake integration
Other things:
- Fixed a memory leak in sql_plugin.cc
- plugin-load will now in case of errors try to load all given plugins
before aborted
- If silent-startup is used, plugin-load will not give errors for
plugins it cannot load or warnings about plugin marturity level.
- my_rm_tree() will now delete symlinks, not the actual file, if
MY_NOSYMLINK flag is used.
- my_stat() will now give data for symlink if MY_NOSYMLINKS is used.
- Added 'number of lines' option to mysqltest --cat_file
@Authors: Väinö Mäkelä <vaino.makela@iki.fi>,monty@mariadb.com
This saves 10 calls to pthread_mutex_lock when constructing the
initial OK packet for the default session_track_system_variables which
has 6 variables
Also added a testcase showing there's no need to worry about wildcard
session_track_system_variables as it causes construction of empty
string in the initial OK packet.
the "Test that bad value for plugin enum option is rejected correctly"
needed multiple fixes:
1. don't set plugin-dir based on $MYSQLTEST_VARDIR, all plugins are
in var/plugins, but $MYSQLTEST_VARDIR is var/1/, var/2/, etc if
--parallel is used (that is, practically always), thus the ha_example.so
cannot be loaded, because cannot be found. Test fails with
"unknown option --plugin-example-enum" as the plugin is not loaded
2. force --plugin-maturity=experimental, otherwise even if not parallel
the plugin will fail to load because of low maturity, test still
fails with "unknown option --plugin-example-enum"
3. don't specify .so extension explicitly otherwise the plugin still
doesn't load on windows, even if paths and maturity are fixed
4. set --plugin-example=FORCE otherwise plugin fails to load after reading
--plugin-example-enum-var=noexist because of unknown enum value,
the server ignores the failure and starts normally. the test hangs.
5. This needs the fix in sql_plugin.cc to detect that the plugin is
forced even when some options failed to parse. It used to consider
plugin forced only if all options parsed correctly, which was wrong.
Now the test passes, testing what it was supposed to test - failure
to parse an enum value of a plugin option.
Without these fixes the test hanged as in 4 when run on the main branch
in non-prarallel (e.g. one test only) mode.
Every Wsrep system thread should run with READ_COMMITTED transaction
isolation level to prevent issues caused by InnoDB gap locks.
The exception is statement-based replication for appliers, where
REPEATABLE_READ is required by the server code. To account for that,
set the isolation level before every applied event. It won't affect
an already running transaction, but allows to handle both statement-
and row-based replications accordingly.
However, the problem might arise with the mixed replication format.
Apart from that, there was a separate issue with applier thread vars:
wsrep_plugins_post_init() would overwrite thd->variables for every
applier thread and forget to restore proper default isolation level.
Then, upon every server transaction termination,
trans_reset_one_shot_statistics() would set thread's isolation
level to the one stored in thd->variables, thus spoiling the isolation
level value for appliers.
Apparently, there was a separate issue with applier thread variables:
wsrep_plugins_post_init() would overwrite thd->variables for every
applier thread and forget to restore proper read only context.
Then, upon every server transaction termination,
trans_reset_one_shot_statistics() would set thread's read only context
to the one stored in thd->variables, thus spoiling the read only
context value for appliers.
Relaxed check, only number of columns and the PK.
Enough to avoid crashes, but doesn't break upgrades and migration
from MySQL as in MDEV-37777.
Added checks everywhere. (flush/create/alter/drop server)
Check mysql.plugin table too.
