OpenSSL 3 escapes '/' and '+' in X509_NAME_oneline() output; OpenSSL 1.1
and WolfSSL don't. A REQUIRE ISSUER/SUBJECT grant from one library can
stop matching after switching to another.
Default comparison stays strcmp(). old_mode=X509_LENIENT_COMPARE opts
into falling back to an escape-aware compare, applied to whichever side
the currently-linked library's own escaping affects, at the cost of
reopening the ambiguity a crafted certificate could exploit to
impersonate another identity.
Adds regression tests against a real certificate with an ambiguous CN
Assisted-by: Claude Sonnet 5 <noreply@anthropic.com>
The "is it my own thread" check compared the login user name
(Security_context::user) and ignored the host, so u1@localhost could see
and kill threads of u1@'127.0.0.1'.
Compare the authenticated account instead:
* all comparisons are done in sctx->is_priv_user() now
* change user_matches() to priv_user_matches(), which uses is_priv_user()
* use it in KILL, KILL USER, SHOW PROCESSLIST, I_S.PROCESSLIST,
COM_PROCESS_INFO and SHOW EXPLAIN/ANALYZE FOR.
* all the remaining places use is_priv_user() directly instead
of doing strcmp: SHOW GRANTS, SHOW CREATE PROCEDURE, I_S.VIEWS,
the DEFINER clause, optimizer trace and change_security_context().
Assisted-by: Claude:claude-5-opus
user_var_entry objects and their names have a connection lifetime,
they exist until the connection ends (or is reset), and then they're
all deleted at once. This is exactly the use case for MEM_ROOT,
let's store them there.
Additionally, let's set MY_ROOT_USE_VMEM flag to keep this memroot
off the general heap where user_var_entry values are stored
and where heap buffer overflows can happen.
The latter makes memory allocations for the MEM_ROOT about 10x more
expensive, so let's always start with an empty memroot (= zero overhead
if no user variables are used) and on THD cleanup let's retain one
memroot block (= zero overhead if the next connection takes THD from the
cache and uses user variables up to one block size).
Assisted-By: Claude:claude-4.8-opus
On THD::reset, typecast the result of query_start_sec_part() to
tv_usec in a more portable way. Previously, we assumed that the
left hand side was always long-compatible but that isn't the case
on mac.
GROUP_CONCAT is limited by group_concat_max_len,
but also, as a string function it must respect max_allowed_packet.
Let's introduce THD::gconcat_max_len() helper to simplify checks.
Also:
* make max group_concat_max_len value the same as max max_allowed_packet
* use the same MY_MIN((ulonglong) ..., UINT_MAX32) in
Item_func_json_objectagg as in Item_func_group_concat
* use overflow-safe type for lengths in Item_func_quote
(MAX_MAX_ALLOWED_PACKET is 1G so uint cannot overflow yet, but it's
a fragile assumption)
The json_engine killed_ptr was of type uchar however the enum
in the server is dependant on the architecture. On big-endian
architectures like IBM Z, retreiving a uchar* retreived the
unmodifed part of the THD->kill enum location and was always 0.
As C++11 allows enums to inherit a type, used uint32_t as the base
class of killed_state type in the server and used uint32_t in the
json library.
reload_acl_and_cache required an expression change to avoid
the compile error:
sql/sql_reload.cc:472:24: error: enumerated and non-enumerated
type in conditional expression [-Werror=enum-conversion]
472 | return result || (thd ? thd->killed : 0);
Added the kill_ptr assignment the following functions can be
killed:
* Item_func_json_depth::val_int
* Item_func_json_type::val_str
* Item_func_json_length::val_int
Item_func_json_array_append::val_str, check_killed() only applied
to json_error: label and not return_null.
Item_func_json_overlaps::val_bool ensure check_killed() is called.
Item_func_json_format::val_str(), corresponding to the SQL,
json_compact, json_detailed, json_loose - add debug instrumentation
for the func_json_notembedded test.
MDEV-26726 (fcd345de48) added a json_pause_execution sync point
however the test case used debug_max_statement_time, which doesn't
exist. Fix the sync point name in the test.
Routine name resolution performed on a temporary memory root
during execution. So on the second execution LEX::spname members
pointed to a cleared memory.
Fixing to peform the resolution to parse time, like the CALL statement does.
During the execution time LEX::spname members now stay untouched.
When executing an atomic sequence of RENAME operations, such as:
RENAME TABLE t1 TO t2, t3 TO t4, ...
any failure in the sequence triggers a rollback of previously completed
renames to preserve atomicity.
However, when an error occurs, `my_error()` is invoked immediately, which
sets the `thd->is_error()` flag. This premature flag setting causes
the rollback logic to misinterpret the thread state, leading to incorrect
reversion behavior and assertion failures.
To address this, the errors are now not emitted immediately but captured
and postponed. A new class `Postponed_error_handler` is introduced for
this purpose. Only after all operations are completed (including
possible DDL reversion), the captured errors are emitted.
Oracle mode has no priority between set-operators but the current
implementation with disable_index_if_needed() can not work with
arbitrary mix of distinct/non-distinct set-operators as the algorithm
is tied to the properly wrapped version of expression. Index can be
disabled only once and cannot be reenabled afterwards.
To adapt Oracle mode for the aforementioned implementation we
prioritize each set operator with parens in the order of
appearance. So the expression:
S1 op1 S2 op2 S3 op3 S4
will be rewritten like this:
(((S1 op1 S2) op2 S3) op3 S4).
The rewritten expression is valid for both Oracle and default mode and
renders the same result. Prioritizing is done with
create_priority_nest() but it was tweaked to work with derived to be
the first operand. Originally create_priority_nest() could wrap only
second operand, like this:
S1 op1 S2 op2 S3 -> S1 op1 (S2 op2 S3).
Now this behaviour is kept by non-NULL attach_to argument. With NULL
attach_to argument it produces this transformation:
S1 op1 S2 -> (S1 op1 S2), but to be strict it will look like this:
SELECT .. FROM (S1 op1 S2).
This expression is treated as the first one in the select list, so
nothing to "attach to".
On importance of not making fake_select_lex after
create_priority_nest() in parsed_select_expr_start(). This will fail
nastily in Protocol::valid_handler() check in store_longlong() for
EXPLAIN. This happens because select_options are inherited from newly
made fake_select_lex but without SELECT_DESCRIBE flag which is meant
to be assigned by mysql_explain_union() but union_needs_tmp_table() is
false (because now top-level unit has no set-operator, it is just
select from derived), so SELECT_DESCRIBE assignment is
skipped. Depending on that flag send_result_set_metadata() is
conditionally executed in JOIN::exec_inner(). And originally it is
done via select_describe() branch for top-level select when there is
no set operator (it is select from derived as explained above), not
via direct send_result_set_metadata().
On IS_OR_WAS_ORACLE. Sql_mode_save_for_frm_handling removes MODE_ORACLE
when view is registered or opened. We cannot return it back as this
conflicts with MDEV-12478 (tested by compat/oracle.func_concat). So we
are temporarily adding WAS_ORACLE flag instead. WAS_MODE_ORACLE is for
testing both WAS_ORACLE and MODE_ORACLE simultaneously.
Setting a name for the the THD::security_ctx:user field for wsrep applier threads.
With this, the audit log events related to wsrep applying will be written in the
audit log.
Using user name <wsrep_applier> for wsrep appliers. This is for having identical
look with async replication, which uses: <replication_slave> for audit logging.
Using same approach as async replication to replace the security_ctx user name
with "system user" for processlist output.
Commit has also a mtr test galera.MDEV-35511, to verify wsrep applier audit logging.
The test does not install/uninstall audit log plugin, but loads the audit log plugin
before the test. This is because uninstalling the audit log plugin gives a warning
saying that plugin is busy and uninstall will be delayed until server shutdown.
This anomaly must be because of the applier thread being active audit logger.
Same problem with plugin uninstall happens also with async relication workers.
If plugn would remain installed, the post test sanity check complains of mismatching
state of pre and post test states, and test execution would fail for this.
don't reload stored routines in the middle of the execution
of a routine. we don't want different iterations of a loop
to see diffefent definitions
For this: remember Cversion in THD on the first sp cache lookup,
after that only compare versions with this value not with Cversion.
The test failed as the Jira ticket describes because of insufficient
main transaction isolation from an out-of-band dynamic "new" transaction.
Specifically the replicated transaction's temporary tables became
accessible to the "new" transaction.
The fixes ensure the "new" transaction can't reach any piece of
replication transaction execution context, the temporary table list
including.
This is safe as by so far start_new_trans class instances
need not any replication info, as it must've been anticipated by design.
- Removed duplicate words, like "the the" and "to to"
- Removed duplicate lines (one double sort line found in mysql.cc)
- Fixed some typos found while searching for duplicate words.
Command used to find duplicate words:
egrep -rI "\s([a-zA-Z]+)\s+\1\s" | grep -v param
Thanks to Artjoms Rimdjonoks for the command and pointing out the
spelling errors.
Ensure that Annotate_rows is always written direct after GTID information,
before any table_map events.
Before this patch, the following problems existed when mixing
transactional and not transactional tables in the same statement:
- Annotate rows could be written after row events or in the next GTID
event.
- See rpl_row_mixing_engines
- Annotate_rows was not always written to binary log in case of error
with a transactional table (rolled back) but a not transactional
table was updated.
- See sp_trans_log, binlog_row_mix_innodb_myisam
Fixed by writing the Annotate_rows event into the non transactional
cache if there are not transactional tables used. If not, write the
event into the transactional cache.
See also MDEV-30046.
Idempotent write_row works same as REPLACE: if there is a duplicating
record in the table, then it will be deleted and re-inserted, with the
same update optimization.
The code in Rows:log_event::write_row was basically copy-pasted from
write_record.
What's done:
REPLACE operation was unified across replication and sql. It is now
representred as a Write_record class, that holds the whole state, and allows
re-using some resources in between the row writes.
Replace, IODKU and single insert implementations are split across different
methods, reluting in a much cleaner code.
The entry point is preserved as a single Write_record::write_record() call.
The implementation to call is chosen on the constructor stage.
This allowed several optimizations to be done:
1. The table key list is not iterated for every row. We find last unique key in
the order of checking once and preserve it across the rows. See last_uniq_key().
2. ib_handler::referenced_by_foreign_key acquires a global lock. This call was
done per row as well. Not all the table config that allows optimized replace is
folded into a single boolean field can_optimize. All the fields to check are
even stored in a single register on a 64-bit platform.
3. DUP_REPLACE and DUP_UPDATE cases now have one less level of indirection
4. modified_non_trans_tables is checked and set only when it's really needed.
5. Obsolete bitmap manipulations are removed.
Also:
* Unify replace initialization step across implementations:
add prepare_for_replace and finalize_replace
* alloca is removed in favor of mem_root allocation. This memory is reused
across the rows.
* An rpl-related callback is added to the replace branch, meaning that an extra
check is made per row replace even for the common case. It can be avoided with
templates if considered a problem.
MDEV-6247 added PROCESSLIST states for when a Replication
SQL thread processes Row events, including a WSRep variant
that dynamically includes the Galera Sequence Number.
MDEV-7409 further expanded on it by adding the table name to the states.
However, PROCESSLIST __cannot__ support generated states.
Because it loads the state texts asynchronously,
only permanently static strings are safe.
Even thread-local memory can become invalid when the thread terminates,
which can happen in the middle of generating a PROCESSLIST.
To prioritize memory safety, this commit reverts both variants to
static strings as the non-WSRep variant was before MDEV-7409.
* __Fully__ revert MDEV-7409 (d9898c9a71)
* Remove the WSRep override from MDEV-6247
* Remove `THD::wsrep_info` and its compiler
flag `WSREP_PROC_INFO` as they are now unused
This commit also includes small optimizations
from MDEV-36839’s previous draft, #4133.
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
Assertion fails because table is opened by admin_recreate_table():
71 result_code= (thd->open_temporary_tables(table_list) ||
72 mysql_recreate_table(thd, table_list, recreate_info, false));
And that is called because t2 is failed with HA_ADMIN_NOT_IMPLEMENTED:
1093 if (result_code == HA_ADMIN_NOT_IMPLEMENTED && need_repair_or_alter)
1094 {
1095 /*
1096 repair was not implemented and we need to upgrade the table
1097 to a new version so we recreate the table with ALTER TABLE
1098 */
1099 result_code= admin_recreate_table(thd, table, &recreate_info);
1100 }
Actually 'table' is t2 but open_temporary_tables() opens whole list,
i.e. t2 and everything what follows it before first_not_own_table().
Therefore t3 is also opened for t2 processing what is wrong.
The fix opens exactly one specific table for HA_ADMIN_NOT_IMPLEMENTED.
Problem:
Empty queries are incremented if no rows are sent to the client in the
EXECUTE phase of select query. With cursor protocol, rows are not sent
during EXECUTE phase; they are sent later in FETCH phase. Hence,
queries executed with cursor protocol are always falsely treated as
empty in EXECUTE phase.
Fix:
For cursor protocol, empty queries are now counted during the FETCH
phase. This ensures counter correctly reflects whether any rows were
actually sent to the client.
Tests included in `mysql-test/main/show.test`.
Valgrind is single threaded and only changes threads as part of
system calls or waits.
Some busy loops were identified and fixed where the server assumes
that some other thread will change the state, which will not happen
with valgrind.
Based on patch by Monty. Original patch introduced VALGRIND_YIELD,
which emits pthread_yield() only in valgrind builds. However it was
agreed that it is a good idea to emit yield() unconditionally, such
that other affected schedulers (like SCHED_FIFO) benefit from this
change. Also avoid pthread_yield() in favour of standard
std::this_thread::yield().
This is needed to make it easy for users to automatically ignore long
char and varchars when using ANALYZE TABLE PERSISTENT.
These fields can cause problems as they will consume
'CHARACTERS * MAX_CHARACTER_LENGTH * 2 * number_of_rows' space on disk
during analyze, which can easily be much bigger than the analyzed table.
This commit adds a new user variable, analyze_max_length, default value 4G.
Any field that is bigger than this in bytes, will be ignored by
ANALYZE TABLE PERSISTENT unless it is specified in FOR COLUMNS().
While doing this patch, I noticed that we do not skip GEOMETRY columns from
ANALYZE TABLE, like we do with BLOB. This should be fixed when merging
to the 'main' branch. At the same time we should add a resonable default
value for analyze_max_length, probably 1024, like we have for
max_sort_length.
wait_for_prior_commit() can be called multiple times per event group,
only do my_error() the first time the call fails.
Remove redundant set_overwrite_status() calls.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Reviewed-by: Monty <monty@mariadb.org>
mysql_prepare_create_table: Extract a Key initialization part that
relates to length calculation and long unique index designation.
append_system_key_parts call also moves there.
Move this initialization before the duplicate elimination.
Extract WITHOUT OVERPLAPS check into a separate function. It had to be moved
earlier in the code to preserve the order of the error checks, as in the tests.
MDEV-28127 did is_equal() which compared vcol expressions
literally. But another table vcol expression is not equal because of
different table name.
We implement another comparison method is_identical() which respects
different table name in vcol comparison. If any field item points to
table_A and compared field item points to table_B, such items are
treated as equal in (table_A, table_B) comparison. This is done by
cloning table_B expression and renaming any table_B entries to table_A
in it.
The problems were that:
1) resources was freed "asimetric" normal execution in send_eof,
in case of error in destructor.
2) destructor was not called in case of SP for result objects.
(so if the last SP execution ended with error resorces was not
freeded on reinit before execution (cleanup() called before next
execution) and destructor also was not called due to lack of
delete call for the object)
Result cleanup() renamed to reset_for_next_ps_execution() to better
reflect function().
All result method revised and freeing resources made "symetric".
Destructor of result object called for SP.
Added skipped invalidation in case of error in insert.
Removed misleading naming of reset(thd) (could be mixed with
with reset()).
This commit updates default memory allocations size used with MEM_ROOT
objects to minimize the number of calls to malloc().
Changes:
- Updated MEM_ROOT block sizes in sql_const.h
- Updated MALLOC_OVERHEAD to also take into account the extra memory
allocated by my_malloc()
- Updated init_alloc_root() to only take MALLOC_OVERHEAD into account as
buffer size, not MALLOC_OVERHEAD + sizeof(USED_MEM).
- Reset mem_root->first_block_usage if and only if first block was used.
- Increase MEM_ROOT buffers sized used by my_load_defaults, plugin_init,
Create_tmp_table, allocate_table_share, TABLE and TABLE_SHARE.
This decreases number of malloc calls during queries.
- Use a small buffer for THD->main_mem_root in THD::THD. This avoids
multiple malloc() call for new connections.
I tried the above changes on a complex select query with 12 tables.
The following shows the number of extra allocations that where used
to increase the size of the MEM_ROOT buffers.
Original code:
- Connection to MariaDB: 9 allocations
- First query run: 146 allocations
- Second query run: 24 allocations
Max memory allocated for thd when using with heap table: 61,262,408
Max memory allocated for thd when using Aria tmp table: 419,464
After changes:
Connection to MariaDB: 0 allocations
- First run: 25 allocations
- Second run: 7 allocations
Max memory allocated for thd when using with heap table: 61,347,424
Max memory allocated for thd when using Aria table: 529,168
The new code uses slightly more memory, but avoids memory fragmentation
and is slightly faster thanks to much fewer calls to malloc().
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
Added Query_time (total time spent running queries) to status_variables.
Other things:
- Added SHOW_MICROSECOND_STATUS type that shows an ulonglong variable
in microseconds converted to a double (in seconds).
- Changed Busy_time and Cpu_time to use SHOW_MICROSECOND_STATUS, which
simplified the code and avoids some double divisions for each query.
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
Ignore snapshot isolation conflict during fragment removal, before
streaming transaction commits. This happens when a streaming
transaction creates a read view that precedes the INSERTion of
fragments into the streaming_log table. Fragments are INSERTed
using a different transaction. These fragment are then removed
as part of COMMIT of the streaming transaction. This fragment
removal operation could fail when the fragments were not part
the transaction's read view, thus violating snapshot isolation.
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
The functions queue_compare, qsort2_cmp, and qsort_cmp2
all had similar interfaces, and were used interchangable
and unsafely cast to one another.
This patch consolidates the functions all into the
qsort_cmp2 interface.
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
Post-fix for MDEV-35144.
Cannot allocate options values on the statement arena, because
HA_CREATE_INFO is shallow-copied for every execution, so if the
option_list was initially empty, it will be reset for every execution
and any values allocated on the statement arena will be lost.
Cannot allocate option values on the execution arena, because
HA_CREATE_INFO is shallow-copied for every execution, so if the
option_list was initially NOT empty, any values appended to the
end will be preserved and if they're on the execution arena their
content will be destroyed.
Let's use thd->change_item_tree() to save and restore necessary pointers
for every execution.
followup for 3da565c41d
Replication of non-transactional engines is experimental and
uses TOI. This naturally means that if there is open transaction
with transactional engine it's changes will be rolled back.
Fixed by adding error message if non-transactional engine
is part of multi-engine transaction with warning.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
