Protect all READ_ONLY sysvars from run-time changes.
Put them into a separate section and use my_virtual_mem_protect()
to make this section read-only before the server starts accepting
connections.
Verify that they're all protected in the sys_var constructor.
One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables
On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup
Assisted-By: Claude:claude-4.8-opus
When innodb_compression_algorithm is set to a non-zlib algorithm (e.g.
lz4) and the provider plugin is loaded from mysql.plugin rather than
command line, InnoDB failed to start because it checked for the provider
at plugin initialization time, before plugin_load() reads mysql.plugin.
Fix:
* InnoDB returns HA_ERR_RETRY_INIT when the compression provider is
missing.
* In sql_plugin.cc, the retry loop is changed to not reap until
mysql.plugin has been loaded
Assisted-By: Claude:claude-4.6-sonnet
the "Test that bad value for plugin enum option is rejected correctly"
needed multiple fixes:
1. don't set plugin-dir based on $MYSQLTEST_VARDIR, all plugins are
in var/plugins, but $MYSQLTEST_VARDIR is var/1/, var/2/, etc if
--parallel is used (that is, practically always), thus the ha_example.so
cannot be loaded, because cannot be found. Test fails with
"unknown option --plugin-example-enum" as the plugin is not loaded
2. force --plugin-maturity=experimental, otherwise even if not parallel
the plugin will fail to load because of low maturity, test still
fails with "unknown option --plugin-example-enum"
3. don't specify .so extension explicitly otherwise the plugin still
doesn't load on windows, even if paths and maturity are fixed
4. set --plugin-example=FORCE otherwise plugin fails to load after reading
--plugin-example-enum-var=noexist because of unknown enum value,
the server ignores the failure and starts normally. the test hangs.
5. This needs the fix in sql_plugin.cc to detect that the plugin is
forced even when some options failed to parse. It used to consider
plugin forced only if all options parsed correctly, which was wrong.
Now the test passes, testing what it was supposed to test - failure
to parse an enum value of a plugin option.
Without these fixes the test hanged as in 4 when run on the main branch
in non-prarallel (e.g. one test only) mode.
Every Wsrep system thread should run with READ_COMMITTED transaction
isolation level to prevent issues caused by InnoDB gap locks.
The exception is statement-based replication for appliers, where
REPEATABLE_READ is required by the server code. To account for that,
set the isolation level before every applied event. It won't affect
an already running transaction, but allows to handle both statement-
and row-based replications accordingly.
However, the problem might arise with the mixed replication format.
Apart from that, there was a separate issue with applier thread vars:
wsrep_plugins_post_init() would overwrite thd->variables for every
applier thread and forget to restore proper default isolation level.
Then, upon every server transaction termination,
trans_reset_one_shot_statistics() would set thread's isolation
level to the one stored in thd->variables, thus spoiling the isolation
level value for appliers.
Apparently, there was a separate issue with applier thread variables:
wsrep_plugins_post_init() would overwrite thd->variables for every
applier thread and forget to restore proper read only context.
Then, upon every server transaction termination,
trans_reset_one_shot_statistics() would set thread's read only context
to the one stored in thd->variables, thus spoiling the read only
context value for appliers.
Relaxed check, only number of columns and the PK.
Enough to avoid crashes, but doesn't break upgrades and migration
from MySQL as in MDEV-37777.
Added checks everywhere. (flush/create/alter/drop server)
Check mysql.plugin table too.
There were too many C casts rather than C++ casts here.
Started a partial conversion covering within-file scoped
changes.
Suggested by Brandon Nesterenko
In plugins, use the correct resolver for ULONG and ULONGLONG
types.
InnoDB has a UINT type as evidenced by "Unknown variable type code 0x182
in plugin 'InnoDB'." so the implementation for UNSIGNED INT was added.
Any InnoDB mtr test that changes lock_wait_timeout,
spider_param_force_commit and some MyRocks unsigned thread server
variables can verify this change is correct.
Reviewer: Brandon Nesterenko
This commit updates default memory allocations size used with MEM_ROOT
objects to minimize the number of calls to malloc().
Changes:
- Updated MEM_ROOT block sizes in sql_const.h
- Updated MALLOC_OVERHEAD to also take into account the extra memory
allocated by my_malloc()
- Updated init_alloc_root() to only take MALLOC_OVERHEAD into account as
buffer size, not MALLOC_OVERHEAD + sizeof(USED_MEM).
- Reset mem_root->first_block_usage if and only if first block was used.
- Increase MEM_ROOT buffers sized used by my_load_defaults, plugin_init,
Create_tmp_table, allocate_table_share, TABLE and TABLE_SHARE.
This decreases number of malloc calls during queries.
- Use a small buffer for THD->main_mem_root in THD::THD. This avoids
multiple malloc() call for new connections.
I tried the above changes on a complex select query with 12 tables.
The following shows the number of extra allocations that where used
to increase the size of the MEM_ROOT buffers.
Original code:
- Connection to MariaDB: 9 allocations
- First query run: 146 allocations
- Second query run: 24 allocations
Max memory allocated for thd when using with heap table: 61,262,408
Max memory allocated for thd when using Aria tmp table: 419,464
After changes:
Connection to MariaDB: 0 allocations
- First run: 25 allocations
- Second run: 7 allocations
Max memory allocated for thd when using with heap table: 61,347,424
Max memory allocated for thd when using Aria table: 529,168
The new code uses slightly more memory, but avoids memory fragmentation
and is slightly faster thanks to much fewer calls to malloc().
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
Change the type of my_hash_get_key to:
1) Return const
2) Change the context parameter to be const void*
Also fix casting in hash adjacent areas.
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
That PR uncovered countless issues on `my_snprintf` uses.
This commit backports a squashed subset of their fixes.
(Excludes previous parts #3485 and #3493)
The problem was that when using clang + asan, we do not get a correct value
for the thread stack as some local variables are not allocated at the
normal stack.
It looks like that for example clang 18.1.3, when compiling with
-O2 -fsanitize=addressan it puts local variables and things allocated by
alloca() in other areas than on the stack.
The following code shows the issue
Thread 6 "mariadbd" hit Breakpoint 3, do_handle_one_connection
(connect=0x5080000027b8,
put_in_cache=<optimized out>) at sql/sql_connect.cc:1399
THD *thd;
1399 thd->thread_stack= (char*) &thd;
(gdb) p &thd
(THD **) 0x7fffedee7060
(gdb) p $sp
(void *) 0x7fffef4e7bc0
The address of thd is 24M away from the stack pointer
(gdb) info reg
...
rsp 0x7fffef4e7bc0 0x7fffef4e7bc0
...
r13 0x7fffedee7060 140737185214560
r13 is pointing to the address of the thd. Probably some kind of
"local stack" used by the sanitizer
I have verified this with gdb on a recursive call that calls alloca()
in a loop. In this case all objects was stored in a local heap,
not on the stack.
To solve this issue in a portable way, I have added two functions:
my_get_stack_pointer() returns the address of the current stack pointer.
The code is using asm instructions for intel 32/64 bit, powerpc,
arm 32/64 bit and sparc 32/64 bit.
Supported compilers are gcc, clang and MSVC.
For MSVC 64 bit we are using _AddressOfReturnAddress()
As a fallback for other compilers/arch we use the address of a local
variable.
my_get_stack_bounds() that will return the address of the base stack
and stack size using pthread_attr_getstack() or NtCurrentTed() with
fallback to using the address of a local variable and user provided
stack size.
Server changes are:
- Moving setting of thread_stack to THD::store_globals() using
my_get_stack_bounds().
- Removing setting of thd->thread_stack, except in functions that
allocates a lot on the stack before calling store_globals(). When
using estimates for stack start, we reduce stack_size with
MY_STACK_SAFE_MARGIN (8192) to take into account the stack used
before calling store_globals().
I also added a unittest, stack_allocation-t, to verify the new code.
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
