1
0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Commit Graph

5039 Commits

Author SHA1 Message Date
Marko Mäkelä
bff2c4e547 Merge 13.1 to main 2026-10-13 09:56:55 +03:00
Marko Mäkelä
0b24391c0b Merge 12.3 to 13.1 2026-10-13 08:39:01 +03:00
Monty
c37016b0e5 Fixed internal temporary buffer sizes to use tmp_memory_table_size
tmp_memory_table_size is limiting the size of internal temporary memory
tables. max_heap_table_size is there to limiting the size of explictely
created memory tables. max_heap_table_size can be much larger than
tmp_memory_table_size as the memory used by temporary tables is in the
control of the user.

This commit changes the usage of max_heap_table_size for internal buffers
to min(max_heap_table_size, tmp_memory_table_size), like we do for
internal temporary tables.

This changes the in memory buffer allocations for:
- GROUP_CONCAT()
- Calculating the cost for scanning memory tables (the original code was
  wrong here as it used the wrong size for memory tables).
- ANALYZE TABLE buffer sizes for calculating distinct column values

Other things:
- Add THD::ram_limitation() to provide consistent memory limitations
  in all code that used variables.tmp_memory_table_size as buffers.
  If tmp_memory_table_size == 0, then 8192 is used.
  This replaces Item_sum::ram_limitation which used 1024 as min buffer,
  which is way to little for any practical case.
- Added security guard in heap_prepare_hp_create_info to ensure that
  max_table_size is calculated same way as in MariaDB server.
- Fixed initial memory allocations for Item_func_group::concat which
  allocated 'max allowed memory' at start. Now it allocates only 1/16
  of that memory at start.
2026-10-11 18:42:17 +03:00
Arcadiy Ivanov
f1a72e2cca MDEV-40376 Protocol::end_statement assertion when window function fills HEAP tmp table
`save_window_function_values()` did not take into account that one can
get `HA_ERR_RECORD_FILE_FULL` on `ha_update_row()`. With blob support in
heap, it can now happen more easily (expressions wider than 512
characters are promoted to TEXT in tmp tables, and each update of a blob
result column allocates a continuation record). However, it could also
happen with Aria tables, which is apparently not tested. The error was
returned as a plain `true` with an empty diagnostics area: debug builds
hit `Assertion '0'` in `Protocol::end_statement()`, release builds send
a bare OK packet after the result set metadata, which the client
mis-parses (appears as a hang / lost connection).

Errors of the computation are now exposed:
`save_window_function_values()` calls `handler::print_error()` for any
failure it does not recover from, the previously ignored `ha_rnd_pos()`
return values are checked and reported, and `compute_window_func()`
distinguishes a read error from EOF and stops the row scan as soon as
the per-row loop fails.

The overflow itself is recovered from by converting the tmp table to
the disk-based tmp engine in place: the rows keep their identity and
their contents, only their positions change, and those are translated
in the rowid sequence that the computation is built on.

The filesort result of a window sort is a sequence of row positions
that covers every row of the table exactly once (the sort is set up
without a limit) and is the only place where the positions are kept:
the row scan and all frame cursors read the rows through it. The
conversion (`Window_rowid_remapper`) therefore copies the rows in the
order of that sequence, which makes the new position of a row known as
soon as the row has been written, and stores it back into the very slot
the old position was read from.

For the new position to always fit into the slot, a window sort stores
the row positions in slots of a fixed 8 bytes
(`Filesort::min_ref_length`, set to the new `TMP_TABLE_MAX_REF_LENGTH`),
which holds the position of any engine an internal tmp table can use: a
pointer into memory or a data file offset. `make_sortkey()` zero-pads a
shorter position to its slot, and `SORT_INFO::ref_length` carries the
slot width to the readers, so `init_read_record()` no longer re-derives
it from `handler::ref_length`, which changes when the table is
converted. Sorts that do not set a minimum, and engines whose positions
are wider than 8 bytes, are unchanged byte for byte. A sequence held in
memory is thereby rewritten in place, keeping its layout, so the frame
cursors' places in it stay valid.

A sequence held in a temporary file is written and read through an
`IO_CACHE`, which encrypts the file when tmp file encryption is
enabled, so the file can not be rewritten in place: the old sequence is
streamed out of its cache and the translated sequence into a new cached
temporary file, which then replaces the old one under the cache. The
frame cursors' slave caches stay linked to the master through
`next_file_user` across the replacement, and are re-created from the
new master afterwards (`Frame_cursor::rowids_rewritten()`). Should
re-creating one fail, the cursor forgets its already-released cache, so
that it is not released a second time when the cursor is destroyed, and
reports the failure.

The row whose update did not fit is written from `record[0]`, which
holds its new image, instead of being read from the table, so the
conversion applies the update that failed. Blob values of `record[0]`
that were read from the HEAP table can point into the handler's shared
blob reassembly buffer (`hp_read_blobs()`, blob values that span
multiple allocation blocks), which reading the other rows overwrites,
so they are first given memory of their own.

When the window sort was set up with a deferred filter (a HAVING clause
deferred to the final ORDER BY sort), the sequence omits the rows the
filter rejected and the conversion drops them: every later reader of
the table applies the same filter, either directly or by reading the
table through a sort that does, so such rows can never reach the
result.

`create_internal_tmp_table_from_heap()` gains a `Tmp_table_row_copier`
hook for the copy from the HEAP table to the Aria or MyISAM table that
replaces it. The plain copy loop and the append of the pending
`record[0]` become `Tmp_table_default_copier`, used when the caller
passes no copier, so both cases of the conversion run through the same
code, defined together with `Window_rowid_remapper` just before the
function. `ha_end_bulk_insert()` is called also when the copy fails, so
the new handler does not keep bulk insert state when its table is
dropped; the pending row is thereby written while the bulk insert is
still active, which does not delay its duplicate detection, as bulk
insert does not cache unique keys (`maria_init_bulk_insert()` skips
`HA_NOSAME` keys). An ignored duplicate of the pending row is passed to
the caller in `Tmp_table_row_copier::duplicate_row_error`, which sets
`*is_duplicate`.

Carrying the running computation over the conversion, instead of
re-running it, has two visible consequences:

 - Compound expressions containing window functions (`items_to_copy`)
   are evaluated exactly once per row, as they are when the table does
   not overflow, so the conversion does not have to be refused when
   such an expression is non-deterministic (`RAND_TABLE_BIT`: a user
   variable assignment, a non-deterministic stored function); those
   statements produce their result.
 - The statement sorts once, so sort related aggregate warnings, e.g.
   the `max_sort_length` truncation counts, are reported for one pass.

The rows are copied in the order of the sequence, so the converted
table does not hold them in the order they were inserted in. This is
safe even when that order is what the table was built for
(`TABLE::keep_row_order`, set for `ROWNUM()` with GROUP BY or ORDER
BY): before HEAP supported blobs, such a tmp table was created in the
disk engine from the start and these statements simply ran, so refusing
here would be a regression against that behavior.

1. `ROWNUM()` values are materialized into the tmp table rows during
   the fill, before the window computation begins; the copy moves them
   verbatim, so their pairing with the rows can not change.
2. The rowid sequence is translated in place, so the running window
   computation continues over exactly the same row order, and
   tie-sensitive window function values (`ROW_NUMBER`, frames over tied
   keys) keep the values the interrupted pass had already produced.
3. Consumers that scan the converted table afterwards see the rows in
   the copy order. Among rows with equal sort keys that order differs
   from the insertion order, but such tie order is unspecified and
   already differs between the memory and disk tmp engines.

The converted table keeps `keep_row_order= true` so the copy order is
also the order the disk engine preserves from then on.

Tests, in `heap.blob_window_overflow` unless noted: a single window
over all rows and a partitioned window overflow mid-computation and
must transparently convert; the side-effecting window expression test
verifies that the user variable is assigned exactly once per row,
against the same query that does not overflow; a fourth test covers the
sequence being an array in memory instead of a merge file; a fifth test
covers the `keep_row_order` conversion, verifying that every `ROWNUM()`
value keeps its insertion-order pairing with its row across the
conversion; a sixth test covers blob values larger than a HEAP
allocation block surviving the conversion of the pending row;
`heap.blob_window_overflow_encrypt` runs the conversion with encrypted
temporary files; `heap.blob_window_overflow_debug` covers the failure
to re-create a slave cache of the sequence
(`simulate_window_seq_slave_oom`), which previously hung the server in
the cursor destructor.
2026-10-11 18:42:16 +03:00
Monty
d0b279f185 MDEV-25292 Atomic CREATE OR REPLACE TABLE
Atomic CREATE OR REPLACE allows to keep an old table intact if the
command fails or during the crash. That is done by renaming the
original table to temporary name, as a backup and restoring it if the
CREATE fails. When the command is complete and logged the backup
table is deleted.

Atomic replace algorithm

 Two DDL chains are used for CREATE OR REPLACE:
 ddl_log_state_create (C) and ddl_log_state_rm (D).

  1. (C) Log rename of ORIG to TMP table (Rename TMP to original).
  2. Rename orignal to TMP.
  3. (C) Log CREATE_TABLE_ACTION of ORIG (drops ORIG);
  4. Do everything with ORIG (like insert data)
  5. (D) Log drop of TMP
  6. Write query to binlog (this marks (C) to be closed in
     case of failure)
  7. Execute drop of TMP through (D)
  8. Close (C) and (D)

  If there is a failure before 6) we revert the changes in (C)
  Chain (D) is only executed if 6) succeded (C is closed on
  crash recovery).

Foreign key errors will be found at the 1) stage.

Additional notes

- CREATE TABLE without REPLACE and temporary tables is not affected
  by this commit.
  set @@drop_before_create_or_replace=1 can be used to
  get old behaviour where existing tables are dropped
  in CREATE OR REPLACE.

- CREATE TABLE is reverted if binlogging the query fails.

- Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
  this commit. Conflicting tables marked with this flag will be
  deleted with CREATE OR REPLACE.

- Replication execution is not affected by this commit.
  - Replication will first drop the conflicting table and then
    creating the new one.

- CREATE TABLE .. SELECT XID usage is fixed and now there is no need
  to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
  do_postlock()). XID is now correctly updated so it disables
  DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
  final stage when the table is ready. So if we have XID in the
  binary log we don't need to drop the table.

- Three variations of CREATE OR REPLACE handled:

  1. CREATE OR REPLACE TABLE t1 (..);
  2. CREATE OR REPLACE TABLE t1 LIKE t2;
  3. CREATE OR REPLACE TABLE t1 SELECT ..;

- Test case uses 6 combinations for engines (aria, aria_notrans,
  myisam, ib, lock_tables, expensive_rename) and 2 combinations for
  binlog types (row, stmt). Combinations help to check differences
  between the results. Error failures are tested for the above three
  variations.

- expensive_rename tests CREATE OR REPLACE without atomic
  replace. The effect should be the same as with the old behaviour
  before this commit.

- Triggers mechanism is unaffected by this change. This is tested in
  create_replace.test.

- LOCK TABLES is affected. Lock restoration must be done after new
  table is created or TMP is renamed back to ORIG

- Moved ddl_log_complete() from send_eof() to finalize_ddl(). This
  checkpoint was not executed before for normal CREATE TABLE but is
  executed now.

- CREATE TABLE will now rollback also if writing to the binary
  logging failed. See rpl_gtid_strict.test

backup ddl log changes

- In case of a successfull CREATE OR REPLACE we only log
  the CREATE event, not the DROP TABLE event of the old table.

ddl_log.cc changes

  ddl_log_execute_action() now properly return error conditions.
  ddl_log_disable_entry() added to allow one to disable one entry.
  The entry on disk is still reserved until ddl_log_complete() is
  executed.

On XID usage

  Like with all other atomic DDL operations XID is used to avoid
  inconsistency between master and slave in the case of a crash after
  binary log is written and before ddl_log_state_create is closed. On
  recovery XIDs are taken from binary log and corresponding DDL log
  events get disabled.  That is done by
  ddl_log_close_binlogged_events().

On linking two chains together

  Chains are executed in the ascending order of entry_pos of execute
  entries. But entry_pos assignment order is undefined: it may assign
  bigger number for the first chain and then smaller number for the
  second chain. So the execution order in that case will be reverse:
  second chain will be executed first.

  To avoid that we link one chain to another. While the base chain
  (ddl_log_state_create) is active the secondary chain
  (ddl_log_state_rm) is not executed. That is: only one chain can be
  executed in two linked chains.

  The interface ddl_log_link_chains() was defined in "MDEV-22166
  ddl_log_write_execute_entry() extension".

Atomic info parameters in HA_CREATE_INFO

  Many functions in CREATE TABLE pass the same parameters. These
  parameters are part of table creation info and should be in
  HA_CREATE_INFO (or whatever). Passing parameters via single
  structure is much easier for adding new data and
  refactoring.

Aria changes:
- Fixed issue in Aria engine with CREATE + locked tables
  that data was not properly commited in some cases in
  case of crashes.

InnoDB related changes (by Marko Mäkelä):
- table_name_t::is_create_or_replace(): A new predicate to check for
  CREATE OR REPLACE TABLE will rename an old table to
  and eventually drop after creating the replacement.
 - dict_table_t::parse_name(): Do acquire MDL on #sql-create- names
  for partitioned tables.
- dict_table_rename_in_cache(): On CREATE OR REPLACE TABLE ... SELECT,
  forget the original dict_table_t::mdl_name so that purge will
  acquire MDL on the #sql-create- name instead. In this way, the
  MDL_EXCLUSIVE that the CREATE OR REPLACE TABLE holds on the
  user-visible name will not unnecessarily block any purge of old history
  until the very end when the #sql-create- table will be dropped.
- ha_innobase::delete_table(): Do not check FOREIGN KEY consistency
  when dropping an #sql-create- table.
- row_rename_table_for_mysql(): Update SYS_FOREIGN.ID also
  when renaming to #sql-create- in order to avoid any
  duplicate key error when CREATE OR REPLACE TABLE is
  creating some FOREIGN KEY constraints by names
  that existed in the old table.

Other changes:
- Removed some auto variables in log.cc for better code readability.
- Fixed old bug that CREATE ... SELECT would not be able to auto repair
  a table that is part of the SELECT.
- Marked MyISAM that it does not support ROLLBACK (not required but
  done for better consistency with other engines).
- maria_create_trn_for_mysql() does not register a new transaction
  handler for commits. This was needed to ensure create or replace
  will not end with an active transaction.
- We do not get anymore warnings about "Engine not supporting atomic
  create" when doing a legal CREATE OR REPLACE on a table with
  foreign key constraints.
- Updated VIDEX engine flags to disable CREATE SEQUENCE.
- Removed mysql_mutex_unlock(&LOCK_gdl) / mysql_mutex_lock(&LOCK_gdl)
  around calls to binlog as these are unsafe. The binlog code uses
  global variables that needs protection from other caller.
- EITS data is preserved if create or replace fails if
  drop_before_create_or_replace=OFF. If ON, then create or replace
  will drop EITS before the drop of the original table (as before).
- Using CREATE OR REPLACE on a encrypted table that the user cannot
  decrypt will fail instead of replacing the encrypted table.
  The encrypted table will unchanged.

Known issues:
- One cannot use create or replace on an InnoDB tables that has foreign
  key references point to it
- CREATE OR REPLACE TEMPORARY table is not full atomic. Any conflicting
   table will always be dropped before creating a new one. (Old behaviour).

Bug fixes related to this MDEV:
MDEV-36435 Assertion failure in finalize_locked_tables()
MDEV-36439 Assertion `thd_arg->lex->sql_command != SQLCOM_CREATE_SEQUENCE...
MDEV-36498 Failed CoR in non-atomic mode no longer generates DROP in RBR...
MDEV-36508 Temporary files #sql-create-....frm occasionally stay after
           crash recovery
MDEV-38479 Crash in CREATE OR REPLACE SEQUENCE when new sequence cannot
           be created
MDEV-36497 Assertion failure after atomic CoR with Aria under lock in
           transactional context
MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
           table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge
MDEV-39367 MSAN/valgrind errors in temp_file_size_cb_func,
           main.tmp_space_usage fails
MDEV-39446 Atomic CREATE OR REPLACE fails if a table cannot be decrypted
MDEV-40776 Atomic CREATE OR REPLACE silently breaks the foreign key
MDEV-40765 Assertion `"unexpected references" == 0' failed upon failing
           CREATE OR REPLACE
MDEV-40994 Atomic CoR: Transaction not registered for MariaDB 2PC, but
           transaction is active

Reverted commits:
- MDEV-36685 "CREATE-SELECT may lose in binlog side-effects of
  stored-routine" as it did not take into account that it safe to
  clear binlogs if the created table is non transactional and there
  are no other non transactional tables used.
  This was done because it caused extra logging when it is not
  needed (not using any non transactional tables) and it also did
  not solve side effects when using statement based loggging.
2026-10-09 15:46:32 +03:00
Sergei Golubchik
b2bc6c2f70 Merge branch '11.8' into 12.3 2026-10-07 22:10:46 +02:00
Dave Gosselin
db8ec28147 MDEV-40573: Crash on multi-table DELETE with an impossible WHERE
A DELETE containing a single table, an index hint, an impossible WHERE
condition, and a window function will take the multi-delete code path
but never initialize tables for deletion, leading to a crash.  Such
a statement would never delete rows from the target table.  Record in the
multi_delete whether it was ever initialized for execution, and don't
attempt to delete anything if it wasn't initialized.

The index hint forces the single table DELETE to take the multi-table
codepath.  Since this case has an impossible WHERE condition, we set
subq_exit_fl which later causes JOIN::optimize_stage2 to skip the
multi-delete table initialization.  It's not safe to attempt
initialization when trying to find a "tableless" subquery plan, so
defend against this case with the new multi-delete flag added by
this commit.
2026-09-25 10:49:28 -04:00
Marko Mäkelä
84a39c73e3 Merge 13.1 into main 2026-09-10 09:07:23 +03:00
Alexander Barkov
bfabe0e530 MDEV-39518 Allow prepared statements in stored functions in assignment right hand
Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.

Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:

  CREATE PROCEDURE p1()
  BEGIN
    -- case 1: DEFAULT clause
    DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK

    -- case 2: SP variable assignment statement
    DECLARE spvar2 INT;
    SET spvar2= f1_with_ps(); -- OK
  END;

- Only assignments to SP variables works for now:
  * SET spvar= func_with_ps(); -- OK
  * SET @uvar= func_with_ps(); -- Error

- Only bare function calls are supported for now. Using a function in
  an expression does not make it PS-safe yet:
    SET v= f1()+0;

- The parser now does not reject PS statements in stored functions.
  PS applicability in stored functions is now detected at run time.
  Note, PS statements in triggers are still prohibited by the parser.

- Functions with PS do not acquire MDL locks on tables, and no MDL is
  taken on the routines themselves either. They work like procedures in
  terms of table opening and routine locking: a concurrent DROP FUNCTION
  can complete while such a function is executing.

- Functions with PS are not replicated as a single `SELECT f1()` call.
  They are replicated per-statement, like procedures.

Helper changes:
- Changing the return result for LEX::sp_variable_declarations_init()
  from void to bool to catch errors in the caller properly.

Misc:
- This patch incorporates fixes for the following bugs found during debugging:
  MDEV-40224,MDEV-40225,MDEV-40226,MDEV-40227,MDEV-40240,
  MDEV-40285,MDEV-40288,MDEV-40315,MDEV-40318,MDEV-40890,MDEV-40900,
  MDEV-40901,MDEV-40913,MDEV-40914,MDEV-41013,MDEV-41015,MDEV-41019

Assisted-by: Claude - reviews and minor clean-ups
2026-09-08 08:12:46 +04:00
Oleksandr Byelkin
b140f801e6 Merge branch '13.0' into 13.1 2026-09-04 12:21:02 +02:00
Oleksandr Byelkin
71dd635f68 Merge tag '12.3' into 13.0 2026-09-04 12:14:15 +02:00
Oleksandr Byelkin
f860f2a937 Merge branch '11.8' into 12.3 2026-08-20 08:27:14 +02:00
Oleksandr Byelkin
3b8db45403 Merge branch '11.4' into 11.8 2026-08-19 11:28:01 +02:00
Oleksandr Byelkin
170b1d7073 Merge branch '10.11' into 11.4 2026-08-19 08:40:05 +02:00
Sergei Golubchik
a1b3f980dd MDEV-33463 Add an option to truncate excessively long queries in the slow log 2026-08-14 17:30:08 +02:00
bsrikanth-mariadb
719a1985d7 MDEV-38803: Rename 2 files
change opt_store_replay_context.h and .cc files to
opt_context_store_replay.h and .cc respectively
2026-08-12 14:31:53 +03:00
bsrikanth-mariadb
85595ce69c MDEV-36523: Load basic stats from trace into optimizer
This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is controlled
by optimizer_replay_context, and points to an user defined variable name,
that has the context information.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index, cost of reading an index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records, along with their costs are also
loaded from the trace.

The trace context which is in JSON format is firstly set into the
user defined session variable, and this variable name is set to
optimizer_replay_context system variable.
Later, when a user issues a query, the contents of the json context is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.

The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.

Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
2026-08-12 14:31:52 +03:00
bsrikanth-mariadb
9dd9ea3928 MDEV-36511: Dump basic stats of a table to trace
This feature stores the basic stats of the base tables that are used in
a query, to the optimizer trace. This feature is also controlled by
optimizer_record_context, and is not enabled by default. The stats
such as num_of_records present in the table, indexes if present
then their names, along with the average number of records_per_key
with in each index are dumped to the trace. Additionally, stats from
range analysis of the queries are also dumped into the trace

The approach taken here is to extend the existing function
store_tables_context_in_trace() and add new dump_table_stats_to_trace()
to opt_trace_ddl_info.cc. For storing the range analysis stats into the
trace, they are first recorded in the hash defined in
Optimizer_Stats_Context_Recorder, and is used later while dumping the
stats into the trace from dump_table_stats_to_trace().
Several new tests are also added to opt_trace_store_stats.test
2026-08-12 14:31:52 +03:00
Sergei Golubchik
1a572838e2 MDEV-40554 KILL checks user (not priv_user) and doesn't verify hostname
The "is it my own thread" check compared the login user name
(Security_context::user) and ignored the host, so u1@localhost could see
and kill threads of u1@'127.0.0.1'.

Compare the authenticated account instead:

* all comparisons are done in sctx->is_priv_user() now
* change user_matches() to priv_user_matches(), which uses is_priv_user()
* use it in KILL, KILL USER, SHOW PROCESSLIST, I_S.PROCESSLIST,
  COM_PROCESS_INFO and SHOW EXPLAIN/ANALYZE FOR.
* all the remaining places use is_priv_user() directly instead
  of doing strcmp: SHOW GRANTS, SHOW CREATE PROCEDURE, I_S.VIEWS,
  the DEFINER clause, optimizer trace and change_security_context().

Assisted-by: Claude:claude-5-opus
2026-08-01 19:44:19 +02:00
Sergei Golubchik
20d2217afa MDEV-40337 store user vars in a dedicated memroot
user_var_entry objects and their names have a connection lifetime,
they exist until the connection ends (or is reset), and then they're
all deleted at once. This is exactly the use case for MEM_ROOT,
let's store them there.

Additionally, let's set MY_ROOT_USE_VMEM flag to keep this memroot
off the general heap where user_var_entry values are stored
and where heap buffer overflows can happen.

The latter makes memory allocations for the MEM_ROOT about 10x more
expensive, so let's always start with an empty memroot (= zero overhead
if no user variables are used) and on THD cleanup let's retain one
memroot block (= zero overhead if the next connection takes THD from the
cache and uses user variables up to one block size).

Assisted-By: Claude:claude-4.8-opus
2026-07-29 13:05:09 +02:00
Vladislav Vaintroub
934108aab5 MDEV-14443 DENY statement
Implements DENY/REVOKE DENY and associated tasks.
2026-07-20 13:23:00 +03:00
Sergei Golubchik
0350ad8cb0 MDEV-30041 don't set utf8_is_utf8mb3 by default in the old-mode 2026-07-10 14:43:05 +02:00
forkfun
867d16cea7 Merge branch '11.4' into '11.8' 2026-06-23 12:52:39 +02:00
forkfun
bbd558c386 Merge branch '10.11' into '11.4' 2026-06-22 12:26:09 +02:00
Oleksandr Byelkin
0d59c434fa Merge branch '10.6' into 10.11 2026-06-21 15:32:20 +02:00
Kristian Nielsen
039def0efd Merge 13.0 to main
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-16 15:03:05 +02:00
Kristian Nielsen
37964c65fa Merge 12.3 to 13.0
This patch to mysqldump is only merged partially:

  commit 666ff1db36
  Author: Monty <monty@mariadb.org>
  Date:   Fri May 15 16:39:35 2026 +0300

      MDEV-39519 Change mariadb-dump to use SHOW REPLICA STATUS instead of SHOW SLAVE STATUS

The functionality for --dump-slave cannot be merged as the underlying
functionality was removed in this commit:

  commit 21c940d599
  Author: ParadoxV5 <paradox.ver5@gmail.com>
  Date:   Thu Dec 4 21:22:20 2025 -0700

      MDEV-37146 mariadb-dump: Replace SHOW SLAVE STATUS usage with information_schema.SLAVE_STATUS

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-16 12:24:32 +02:00
Kristian Nielsen
fcc5fc8916 Merge 11.8 to 12.3
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 16:53:03 +02:00
Kristian Nielsen
420e28ce28 Merge 11.4 to 11.8
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 15:15:10 +02:00
Kristian Nielsen
85ec122c15 Merge 10.11 to 11.4
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 12:50:45 +02:00
Kristian Nielsen
40d6228811 Fix inconsistent terminology
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 10:43:23 +02:00
Dave Gosselin
0cd55af869 MDEV-39870: macOSB Build Warning on Typecast
On THD::reset, typecast the result of query_start_sec_part() to
tv_usec in a more portable way.  Previously, we assumed that the
left hand side was always long-compatible but that isn't the case
on mac.
2026-06-08 09:02:32 -04:00
Bill Jin
f40ea8f4e6 MDEV-39548 Further cleanup of MDL_request boilerplate
- Change thd->backup_commit_lock from MDL_request* to MDL_ticket*,
  and convert related call sites in handler.cc, log.cc, sql_class.cc,
  and xa.cc to use MDL_ACQUIRE_LOCK.

- Convert reload_acl_and_cache() in sql_reload.cc, purge_master_logs()
  and reset_master() in sql_repl.cc to MDL_ACQUIRE_LOCK, holding the
  ticket in a local MDL_ticket* (these functions used a local
  MDL_request originally and should not touch thd->backup_commit_lock).

- Convert acquire_lock() in partition_info.cc to MDL_ACQUIRE_LOCK.

- Add MDL_REQUEST_LIST_ADD() helper for enqueuing lock requests into
  MDL_request_list, and convert call sites in sql_base.cc and sp.cc.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2026-06-05 13:15:49 +04:00
Oleksandr Byelkin
9f98f82b14 Merge branch '11.8' into 12.3 2026-05-26 21:15:58 +02:00
Sergei Golubchik
6b35f1748d Merge branch '11.4' into 11.8 2026-05-23 21:43:50 +02:00
Sergei Golubchik
b4a80422bf Merge branch '10.11' into 11.4 2026-05-23 16:59:15 +02:00
Sergei Golubchik
197f92bee0 Merge branch '10.6' into 10.11 2026-05-23 16:55:50 +02:00
Oleksandr Byelkin
f572addcc1 Merge branch '12.3' into 13.0 2026-05-22 09:52:05 +02:00
Oleksandr Byelkin
3f388a1cce Merge branch '11.8' into 12.3 2026-05-22 09:27:18 +02:00
Sergei Golubchik
a343e60d2f Merge branch '11.4' into 11.8 2026-05-21 20:17:03 +02:00
Sergei Golubchik
5000962deb Merge branch '10.11' into 11.4 2026-05-21 16:54:28 +02:00
Sergei Golubchik
4b2289fec6 Merge branch '10.6' into 10.11 2026-05-21 16:19:19 +02:00
Sergei Golubchik
fe3039eef7 MDEV-39673 group_concat ignores max_allowed_packet
GROUP_CONCAT is limited by group_concat_max_len,
but also, as a string function it must respect max_allowed_packet.

Let's introduce THD::gconcat_max_len() helper to simplify checks.

Also:
* make max group_concat_max_len value the same as max max_allowed_packet
* use the same MY_MIN((ulonglong) ..., UINT_MAX32) in
  Item_func_json_objectagg as in Item_func_group_concat
* use overflow-safe type for lengths in Item_func_quote
  (MAX_MAX_ALLOWED_PACKET is 1G so uint cannot overflow yet, but it's
  a fragile assumption)
2026-05-20 22:49:21 +02:00
Daniel Black
509e40e706 MDEV-30518: JSON functions cannot be killed on big-endian
The json_engine killed_ptr was of type uchar however the enum
in the server is dependant on the architecture. On big-endian
architectures like IBM Z, retreiving a uchar* retreived the
unmodifed part of the THD->kill enum location and was always 0.

As C++11 allows enums to inherit a type, used uint32_t as the base
class of killed_state type in the server and used uint32_t in the
json library.

reload_acl_and_cache required an expression change to avoid
the compile error:
sql/sql_reload.cc:472:24: error: enumerated and non-enumerated
type in conditional expression [-Werror=enum-conversion]
  472 |  return result || (thd ? thd->killed : 0);

Added the kill_ptr assignment the following functions can be
killed:
* Item_func_json_depth::val_int
* Item_func_json_type::val_str
* Item_func_json_length::val_int

Item_func_json_array_append::val_str, check_killed() only applied
to json_error: label and not return_null.

Item_func_json_overlaps::val_bool ensure check_killed() is called.

Item_func_json_format::val_str(), corresponding to the SQL,
json_compact, json_detailed, json_loose - add debug instrumentation
for the func_json_notembedded test.

MDEV-26726 (fcd345de48) added a json_pause_execution sync point
however the test case used debug_max_statement_time, which doesn't
exist. Fix the sync point name in the test.
2026-05-14 08:44:01 +10:00
Michael Widenius
7828fb475b MDEV-32745 Add a simple MySQL to MariaDB upgrade helper
The tool is named mariadb-migrate-config-file.
The main purpose of the tool is to change MySQL option
files to work both for MySQL and MariaDB.
There are options to do the changes in the options file inline,
or at-end-of-file. One can also remove or comment unknown options.

The list of supported options is generated compile time from
mariadbd --help. All server options, including compiled plugins, are
supported.

The bulk of the code comes from Väinö.
Monty has updated it with a lot of extra options.
Wlad helped with cmake integration

Other things:
- Fixed a memory leak in sql_plugin.cc
- plugin-load will now in case of errors try to load all given plugins
  before aborted
- If silent-startup is used, plugin-load will not give errors for
  plugins it cannot load or warnings about plugin marturity level.
- my_rm_tree() will now delete symlinks, not the actual file, if
  MY_NOSYMLINK flag is used.
- my_stat() will now give data for symlink if MY_NOSYMLINKS is used.
- Added 'number of lines' option to mysqltest --cat_file

@Authors: Väinö Mäkelä <vaino.makela@iki.fi>,monty@mariadb.com
2026-05-01 15:14:06 +03:00
Sergei Golubchik
35f65007b7 cleanup: change sql_command_flags from uint to cf_flags_t
introduce a dedicated enum type for sql_command_flags
to simplify debugging:

(gdb) p sql_command_flags[SQLCOM_SELECT]
$2 = (CF_REEXECUTION_FRAGILE | CF_CAN_GENERATE_ROW_EVENTS | CF_PREOPEN_TMP_TABLES | CF_CAN_BE_EXPLAINED | CF_OPTIMIZER_TRACE)
2026-04-30 21:55:42 +02:00
Rucha Deodhar
6a7a44aba0 MDEV-5092: Implement UPDATE with result set (UPDATE ... RETURNING)
The patch introduces the OLD_VALUE() expression to reference the value
of a column before it was updated. The parser is extended to support
RETURNING and OLD_VALUE(), and RETURNING expressions are stored in a
separate returning_list in SELECT_LEX with independent wildcard tracking.
RETURNING is rejected for multi-table UPDATE.

During setup of RETURNING fields, THD::is_setting_returning is used
when resolving fields, particularly for updates through views.
When resolving view fields, Item_direct_view_ref may point to the
view's item_list, losing the information about whether the value
should be old or new. The original item in returning_list still
contains the correct is_old_value_reference flag, which is copied
back to the resolved item.

OLD_VALUE() is implemented by extending Item_field with a new
Item_old_field class. Item_field::set_field() initializes
Field::ptr_old to the corresponding location in record[1],
which stores the old row during UPDATE execution.

When sending result rows, Item_old_field::send() temporarily switches
the field pointer from record[0] (current row) to ptr_old so
OLD_VALUE() returns the value before the update.

The UPDATE execution path is modified to send a result set when
RETURNING is present instead of an OK packet.
2026-04-30 09:41:26 -04:00
Sergey Vojtovich
685e0f8170 Remove redundant THD::get_thd() method
THD::get_thd() is never used after MDEV-34848 cleanup.
2026-03-21 21:00:57 +04:00
kevdn
5677d05ca0 MDEV-34848 Remove MDL_context_owner abstraction entirely
MDL_context_owner was introduced in MySQL for gtest unit testing of MDL,
which MariaDB doesn't use. Its virtual methods (enter_cond, exit_cond,
is_killed, get_thd, notify_shared_lock) are not MDL-specific - they are
part of THD and were only encapsulated in MDL_context_owner as a
misleading abstraction.

This is a cleanup that:
- Removes the MDL_context_owner class entirely
- Removes THD's inheritance from MDL_context_owner
- Changes MDL to use THD* directly instead of MDL_context_owner*
- Removes the redundant MDL_context::get_owner() method
- Moves ENTER_COND/EXIT_COND macros to sql_class.h near THD_ENTER_COND
2026-03-21 21:00:57 +04:00
Sergei Golubchik
3e97105975 13.0 deprecations
removed:

* DES encryption
* --secure-auth
* --old
* spider table options: bfz, btt, cmd, ctp, cwg, isa, ilm, ios, smd, stc, stl

extended under old-mode:

* YEAR(2), still available when old-mode=2_DIGIT_YEAR

un-deprecated:

* keep_files_on_create, originally (MDEV-23570) the idea was to make it
TRUE and deprecate. It cannot be removed when FALSE, but TRUE breaks
mariabackup.aria_backup where a table is altered from Aria to InnoDB
during a backup, so both t.MAD/t.MAI and t.ibd gets into a backup.
2026-02-16 21:55:18 +01:00