1
0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Commit Graph

1054 Commits

Author SHA1 Message Date
Arcadiy Ivanov
03c04bbd6c MDEV-40781 Duplicate row despite DISTINCT when tmp table converts
`create_internal_tmp_table_from_heap()` writes the pending `record[0]`,
the row whose write filled the in-memory table, into the new table.
Since **MDEV-40376** (`636f154bb49`) that write happens *before*
`ha_end_bulk_insert()` rather than after it.

`ha_maria::start_bulk_insert()` disables **all** indexes of an internal
temporary table that is about to receive at least
`MARIA_MIN_ROWS_TO_DISABLE_INDEXES` (100) rows:

```c
if (file->open_flags & HA_OPEN_INTERNAL_TABLE)
{
  /* Internal table; If we get a duplicate something is very wrong */
  file->update|= HA_STATE_CHANGED;
  index_disabled= share->base.keys > 0;
  maria_clear_all_keys_active(file->s->state.key_map);
}
```

`maria_write()` then skips `_ma_check_unique()` entirely, so the unique
constraint that implements `DISTINCT` for a key too wide to be an index
is not enforced. The rows copied out of the in-memory table are already
distinct and need no checking against each other, but the pending row is
exactly the row whose duplicate status is unknown, and it was written
inside that window. A `SELECT DISTINCT` over wide columns could
therefore return a duplicate row.

Note that the justification given in `636f154bb49` is not the mechanism
at work here. It refers to the bulk insert key *tree*, a different
branch of `ha_maria::start_bulk_insert()`; setting
`bulk_insert_buffer_size=0` does not avoid the problem.

The fix splits the copy in two:

1. `Tmp_table_row_copier` gains a second virtual,
   `write_pending_row()`, defaulting to a no-op.
2. `copy_rows()` now only copies the rows the in-memory table holds.
3. `create_internal_tmp_table_from_heap()` calls
   `ha_end_bulk_insert()` and then `write_pending_row()`, so the
   pending row is written with the indexes of the new table back in
   place and a duplicate of an already copied row is detected.

`Window_rowid_remapper` keeps writing its pending row within
`copy_rows()` and inherits the no-op default. Its new position is only
known once the rows before it have been written, and nothing is lost by
writing it with the indexes still disabled: it replaces a row that is
already in the table rather than adding one, and an update of a window
function value cannot collide with another row, as a deduplicating key
is not built on the columns it changes.

The new test covers `SELECT DISTINCT`, `SELECT DISTINCT ... ORDER BY`,
`GROUP BY`, `UNION` and `INSERT ... SELECT DISTINCT`, and asserts that
the conversion actually happened so that a future sizing change cannot
silently void the coverage.
2026-10-11 18:42:17 +03:00
Arcadiy Ivanov
f1a72e2cca MDEV-40376 Protocol::end_statement assertion when window function fills HEAP tmp table
`save_window_function_values()` did not take into account that one can
get `HA_ERR_RECORD_FILE_FULL` on `ha_update_row()`. With blob support in
heap, it can now happen more easily (expressions wider than 512
characters are promoted to TEXT in tmp tables, and each update of a blob
result column allocates a continuation record). However, it could also
happen with Aria tables, which is apparently not tested. The error was
returned as a plain `true` with an empty diagnostics area: debug builds
hit `Assertion '0'` in `Protocol::end_statement()`, release builds send
a bare OK packet after the result set metadata, which the client
mis-parses (appears as a hang / lost connection).

Errors of the computation are now exposed:
`save_window_function_values()` calls `handler::print_error()` for any
failure it does not recover from, the previously ignored `ha_rnd_pos()`
return values are checked and reported, and `compute_window_func()`
distinguishes a read error from EOF and stops the row scan as soon as
the per-row loop fails.

The overflow itself is recovered from by converting the tmp table to
the disk-based tmp engine in place: the rows keep their identity and
their contents, only their positions change, and those are translated
in the rowid sequence that the computation is built on.

The filesort result of a window sort is a sequence of row positions
that covers every row of the table exactly once (the sort is set up
without a limit) and is the only place where the positions are kept:
the row scan and all frame cursors read the rows through it. The
conversion (`Window_rowid_remapper`) therefore copies the rows in the
order of that sequence, which makes the new position of a row known as
soon as the row has been written, and stores it back into the very slot
the old position was read from.

For the new position to always fit into the slot, a window sort stores
the row positions in slots of a fixed 8 bytes
(`Filesort::min_ref_length`, set to the new `TMP_TABLE_MAX_REF_LENGTH`),
which holds the position of any engine an internal tmp table can use: a
pointer into memory or a data file offset. `make_sortkey()` zero-pads a
shorter position to its slot, and `SORT_INFO::ref_length` carries the
slot width to the readers, so `init_read_record()` no longer re-derives
it from `handler::ref_length`, which changes when the table is
converted. Sorts that do not set a minimum, and engines whose positions
are wider than 8 bytes, are unchanged byte for byte. A sequence held in
memory is thereby rewritten in place, keeping its layout, so the frame
cursors' places in it stay valid.

A sequence held in a temporary file is written and read through an
`IO_CACHE`, which encrypts the file when tmp file encryption is
enabled, so the file can not be rewritten in place: the old sequence is
streamed out of its cache and the translated sequence into a new cached
temporary file, which then replaces the old one under the cache. The
frame cursors' slave caches stay linked to the master through
`next_file_user` across the replacement, and are re-created from the
new master afterwards (`Frame_cursor::rowids_rewritten()`). Should
re-creating one fail, the cursor forgets its already-released cache, so
that it is not released a second time when the cursor is destroyed, and
reports the failure.

The row whose update did not fit is written from `record[0]`, which
holds its new image, instead of being read from the table, so the
conversion applies the update that failed. Blob values of `record[0]`
that were read from the HEAP table can point into the handler's shared
blob reassembly buffer (`hp_read_blobs()`, blob values that span
multiple allocation blocks), which reading the other rows overwrites,
so they are first given memory of their own.

When the window sort was set up with a deferred filter (a HAVING clause
deferred to the final ORDER BY sort), the sequence omits the rows the
filter rejected and the conversion drops them: every later reader of
the table applies the same filter, either directly or by reading the
table through a sort that does, so such rows can never reach the
result.

`create_internal_tmp_table_from_heap()` gains a `Tmp_table_row_copier`
hook for the copy from the HEAP table to the Aria or MyISAM table that
replaces it. The plain copy loop and the append of the pending
`record[0]` become `Tmp_table_default_copier`, used when the caller
passes no copier, so both cases of the conversion run through the same
code, defined together with `Window_rowid_remapper` just before the
function. `ha_end_bulk_insert()` is called also when the copy fails, so
the new handler does not keep bulk insert state when its table is
dropped; the pending row is thereby written while the bulk insert is
still active, which does not delay its duplicate detection, as bulk
insert does not cache unique keys (`maria_init_bulk_insert()` skips
`HA_NOSAME` keys). An ignored duplicate of the pending row is passed to
the caller in `Tmp_table_row_copier::duplicate_row_error`, which sets
`*is_duplicate`.

Carrying the running computation over the conversion, instead of
re-running it, has two visible consequences:

 - Compound expressions containing window functions (`items_to_copy`)
   are evaluated exactly once per row, as they are when the table does
   not overflow, so the conversion does not have to be refused when
   such an expression is non-deterministic (`RAND_TABLE_BIT`: a user
   variable assignment, a non-deterministic stored function); those
   statements produce their result.
 - The statement sorts once, so sort related aggregate warnings, e.g.
   the `max_sort_length` truncation counts, are reported for one pass.

The rows are copied in the order of the sequence, so the converted
table does not hold them in the order they were inserted in. This is
safe even when that order is what the table was built for
(`TABLE::keep_row_order`, set for `ROWNUM()` with GROUP BY or ORDER
BY): before HEAP supported blobs, such a tmp table was created in the
disk engine from the start and these statements simply ran, so refusing
here would be a regression against that behavior.

1. `ROWNUM()` values are materialized into the tmp table rows during
   the fill, before the window computation begins; the copy moves them
   verbatim, so their pairing with the rows can not change.
2. The rowid sequence is translated in place, so the running window
   computation continues over exactly the same row order, and
   tie-sensitive window function values (`ROW_NUMBER`, frames over tied
   keys) keep the values the interrupted pass had already produced.
3. Consumers that scan the converted table afterwards see the rows in
   the copy order. Among rows with equal sort keys that order differs
   from the insertion order, but such tie order is unspecified and
   already differs between the memory and disk tmp engines.

The converted table keeps `keep_row_order= true` so the copy order is
also the order the disk engine preserves from then on.

Tests, in `heap.blob_window_overflow` unless noted: a single window
over all rows and a partitioned window overflow mid-computation and
must transparently convert; the side-effecting window expression test
verifies that the user variable is assigned exactly once per row,
against the same query that does not overflow; a fourth test covers the
sequence being an array in memory instead of a merge file; a fifth test
covers the `keep_row_order` conversion, verifying that every `ROWNUM()`
value keeps its insertion-order pairing with its row across the
conversion; a sixth test covers blob values larger than a HEAP
allocation block surviving the conversion of the pending row;
`heap.blob_window_overflow_encrypt` runs the conversion with encrypted
temporary files; `heap.blob_window_overflow_debug` covers the failure
to re-create a slave cache of the sequence
(`simulate_window_seq_slave_oom`), which previously hung the server in
the cursor destructor.
2026-10-11 18:42:16 +03:00
Dave Gosselin
531d2e2114 MDEV-39014: Factor JOIN_TAB_RANGE allocation into static create()
Adds a static JOIN_TAB_RANGE::create(thd, count) helper that allocates
a JOIN_TAB array and wraps it in a JOIN_TAB_RANGE, replacing the
inline allocation in JOIN::get_best_combination().  Also gives
JOIN_TAB_RANGE::start and ::end default initializers and documents
their half-open range convention.
2026-09-16 10:20:51 -04:00
Dave Gosselin
2bd0af7349 MDEV-39602: Document the join_type enumeration
Replace the one line comment on join_type in sql_select.h with a
comment on each value.  Each comment explains what the access method
is, the conditions under which the optimizer chooses it, and what
EXPLAIN will show for it.
2026-06-22 09:21:14 -04:00
Kristian Nielsen
37964c65fa Merge 12.3 to 13.0
This patch to mysqldump is only merged partially:

  commit 666ff1db36
  Author: Monty <monty@mariadb.org>
  Date:   Fri May 15 16:39:35 2026 +0300

      MDEV-39519 Change mariadb-dump to use SHOW REPLICA STATUS instead of SHOW SLAVE STATUS

The functionality for --dump-slave cannot be merged as the underlying
functionality was removed in this commit:

  commit 21c940d599
  Author: ParadoxV5 <paradox.ver5@gmail.com>
  Date:   Thu Dec 4 21:22:20 2025 -0700

      MDEV-37146 mariadb-dump: Replace SHOW SLAVE STATUS usage with information_schema.SLAVE_STATUS

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-16 12:24:32 +02:00
Kristian Nielsen
fcc5fc8916 Merge 11.8 to 12.3
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 16:53:03 +02:00
Kristian Nielsen
420e28ce28 Merge 11.4 to 11.8
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 15:15:10 +02:00
drrtuy
2be1c9681e Expose static symbols leveraged in DuckDB to stringify WHERE conditions for other MariaDB engines. 2026-06-08 11:50:55 +02:00
Alexander Barkov
90f8178f33 MDEV-10152 Add support for TYPE .. IS REF CURSOR
Adding support for the strict cursor data types:

Example 1a:
  TYPE rec0_t IS RECORD (a INT, VARCHAR(10));
  TYPE cur0_t IS REF CURSOR RETURN rec0_t;

Example 1b:
  TYPE rec0_t IS RECORD (a t1.a%TYPE, b t1.b%TYPE);
  TYPE cur0_t IS REF CURSOR RETURN rec0_t;

Example 1c:
  TYPE rec0_t IS RECORD (a INT, VARCHAR(10));
  r0 rec0_t;
  TYPE cur0_t IS REF CURSOR RETURN r0%TYPE;

Example 1d:
  TYPE rec0_t IS RECORD (a t1.a%TYPE, b t1.b%TYPE);
  r0 rec0_t;
  TYPE cur0_t IS REF CURSOR RETURN r0%TYPE;

Example2a:
  TYPE cur0_t IS REF CURSOR RETURN t1%ROWTYPE; -- t1 is a table

Example 2b:
  r0 t1%ROWTYPE;
  TYPE cur0_t IS REF CURSOR RETURN r0%TYPE;

Example3a:
  CURSOR cursor_sample IS SELECT a,b FROM t1;
  TYPE cur0_t IS REF CURSOR RETURN cursor_sample%ROWTYPE;

Example3b:
  CURSOR cursor_sample IS SELECT a,b FROM t1;
  r0 cursor_sample%ROWTYPE;
  TYPE cur0_t IS REF CURSOR RETURN r0%TYPE;

If a cursor variable is declared with a RETURN clause then:
1. At OPEN type the data type of the SELECT list row is compared
  for compatibility with the cursor RETURN data type.
  The SELECT list row must be assignable to the RETURN type row.
  If case if assignability is not meet, an error is raised
  Assignability means:
  - The arity of the SELECT list must be equal to the arity
    of the RETURN clause
  - Every n-th field of the SELECT list must be assignable to the
    n-th field of the RETURN Clause

2. At FETCH time, the data is fetched in two steps:
  a. On the first step the data is fetched into a virtual table
     with the row type described in the RETURN clause
  b. On the second step the data is copied from the virtual table
     to the target fetch list. Data type conversion can happen
     on this step.

Change details:

Adding new methods:
- sp_cursor::check_assignability_to
- Virtual_tmp_table::check_assignability_from
- Virtual_tmp_table::sp_set_from_select_list
- Virtual_tmp_table::sp_save_in_vtable
- Virtual_tmp_table::sp_save_in_target_list
- LEX::check_ref_cursor_components
- LEX::make_sp_instr_copy_struct_for_last_context_variables
- LEX::declare_type_ref_cursor
- sp_cursor::Select_fetch_into_spvars::send_data_with_return_type

Adding new members:
- sp_instr_copen_by_ref::m_cursor_name
- Select_fetch_into_spvars::m_return_type
- Select_materialize::m_cursor_name
- Select_materialize::m_return_type

Adding new virtual methods:
- Item::resolve_spvar_cursor_rowtype
- Type_handler::Spvar_definition_resolve_type_refs
- Server_side_cursor::check_assignability_to
- Overriding Select_materialize::prepare to raise an error when the cursor
  returned data type is not compatible with the RETURN clause

Making these methods virtual:
- Field::check_assignability_from

Adding new classes:
- sp_type_def_ref
- RowTypeBuffer

Adding new constructors to:
- Spvar_definition

Adding new helper methods (e.g. to reuse the code)
- Field::store_field_maybe_null
- ChanBuffer::append_ulonglong
- sp_pcontext::set_type_for_last_context_variables

Minor changes:
- Making TABLE::export_structure const
- Overriding Item_splocal::type_extra_attributes. It was forgotten in earlier changes.

Adding new error messages
- ER_CANNOT_CAST_ON_IDENT1_ASSIGNMENT_FOR_OPERATION
- ER_CANNOT_CAST_ON_IDENT2_ASSIGNMENT_FOR_OPERATION
2026-04-06 15:16:38 +04:00
OmarGamal10
62270ffc5c MDEV-38877: Unnecessary filesort on derived table materialization
Fixes unnecessary filesort on derived tables when ordered/grouped by a
field in the key. The data is inherently sorted, wrapping the result set
in a filesort is redundant.
2026-03-12 09:16:49 +13:00
Sergei Golubchik
4456eb1a5e Merge branch '12.2' into 12.3 2026-02-07 22:12:32 +01:00
Sergei Golubchik
d653fcb564 Merge branch '11.8' into 12.2 2026-02-03 14:53:15 +01:00
Sergei Golubchik
9bfea48ce1 Merge branch '11.4' into 11.8 2026-01-31 14:08:54 +01:00
Sergei Golubchik
40f7084661 Merge branch '10.11' into 11.4 2026-01-28 21:52:18 +01:00
Monty
be237b3b0e MDEV-36290: Improved support of replication between tables of different structure
One can have data loss in multi-master setups when 1) both masters
update the same table, 2) ALTER TABLE is run on one master which
re-arranges the column ordering, and 3) transactions are binlogged
in ROW binlog_format.

This is because the slave assumes that all columns are in the same
order on the master and slave and all columns on the master also
exists on the slave. This happens even if binlog_row_metadata=FULL is
used.  If this is not the case, this will lead to silent data loss.

A new option for slave_type_conversions bit field,
ERROR_IF_MISSING_FIELD, has been added, along with a new error,
ER_SLAVE_INCOMPATIBLE_TABLE_DEF. This allows the user to define if
the slave should abort replication if it is missing some field that
existed on the master. The option is off by default to keep things
compatible with earlier versions.
If a field is missing on the slave and log_warnings >= 1, a warning
will be logged to the error log.

This patch fixes this, when binlog_row_metadata=FULL is used on the
master, by mapping fields with identical names on the master and slave.
If slave has fields that does not exist in the row event, these will
be set to their default value.

The main idea is that we added two conversion tables:
m_tabledef.master_to_slave_map[master_column_index] -> slave_column_index
and m_tabledef.master_to_slave_error[master_column_index] which contains
an error number if the master_column does not exist on the slave or
it is not possible to convert the master data to the slave column.
master_to_slave_error[#] contains 0 if the column exists and is compatible.

General code changes:
- Instead of looping over row fields in the order of slave table
  we are instead looping over fields in the order of the binary log.
- We are using table->write_set to know which fields should be updated
  on the slave. This is reflected in unpack_row
- We are calling TABLE::mark_columns_per_binlog_row_image() to ensure
  that rpl_write_set is properly set. This is needed if the slave also
  is doing binary logging.
- Before replication aborted if the master and slave tables were too
  different.  Now replication is only aborted if the row actually uses
  columns that does not exists on the slave (and ERROR_IF_MISSING_FIELD
  is used) or uses columns that cannot be converted.
  - Instead of giving errors in compatible_with(), used when table is
    accessed by first the row event, we are instead giving errors
    when we examine a row event and notice that it is accessing
    a not existing or not compatible field.

Other code changes:
- Removed conv_table argument from compatible_with() and store it
  directly in RPL_TABLE_LIST->m_conv_table
- table_def::compatible_with() returns now 1 on error (not 0).
- Remove m_width and skip arguments from prepare_record() as we are
  now using table->write_set() to check which elements need a default
  value.
- Moved DBUG_ENTER() to it's proper place (after variable
  declarations) in a few functions.
- Some changes in unpack_row():
  - Replaced null_mask and null_ptr with an indexed bit check for
    simplicity.
  - Removed check of rgi == null and table_found which never worked.
  - Updated comments to reflect current code.
  - Indentation changes as the code now uses 'continue' instead of
    'if-else' in the main loop.
  - The code to throw away 'extra master fields' is not needed as we
    are now looping over fields in binary log, not over fields in
    slave table.
- Simplified get_table_data(TABLE *table_arg) by returning found
  table_list.
- Errors for row events are now initialized in compatible_with(),
  checked in check_wrong_column_usage() and reported in
  give_compatibility_error().

Co-authored-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
2026-01-27 12:59:30 -07:00
bsrikanth-mariadb
dc28140c89 MDEV-31255: Crash with fulltext search subquery in explain delete/update
ft_handler isn't getting initialized for subqueries inside explain
delete/update queries. However, ft_handler is accessed inside ha_ft_read(),
and is the reason for NULL pointer exception.
This is not the case with non-explain delete/update queries, as
well as explain/non-explain select queries.

Follow the approach the SELECT statements are using in
JOIN::optimize_constant_subqueries(): remove SELECT_DESCRIBE
flag when invoking optimization of constant subqueries.

Single-table UPDATE/DELETEs have SELECT_LEX but don't have JOIN.
So, we make optimize_constant_subqueries() not to be a member
of JOIN class, and instead move it to SELECT_LEX, and then
invoke it from single-table UPDATE/DELETE as well as for SELECT queries.
2026-01-27 16:42:41 +05:30
Monty
797f50d107 Removed not used members in class POSITION
Removed members:
- records_after_filter
- prev_record_reads
2025-11-25 10:25:01 +02:00
Oleksandr Byelkin
fd15fd2765 Merge branch 'bb-12.1-release' into bb-12.2-release 2025-11-06 07:29:30 +01:00
Oleksandr Byelkin
ba00960fda Merge branch 'bb-11.8-release' into bb-12.1-release 2025-11-05 08:58:12 +01:00
Sergei Golubchik
f230c0ff6e Merge branch '10.11' into 11.4 2025-11-04 13:44:16 +01:00
Sergei Golubchik
5ce9a03602 Merge branch '11.4' into 11.8 2025-11-04 12:39:27 +01:00
Sergei Golubchik
1093a2f3b8 Merge branch '10.11' into 11.4 2025-11-03 14:23:51 +01:00
Sergei Petrunia
68432a0bc3 MDEV-29638 Crash when considering Split-Materialized plan
Variant 2: "Don't call optimize_stage2 too early"

The affected query had Split-Materialized derived table in another
derived table:

  select *   -- select#1
  from (
    select *  -- select#2
    from t1,
         (select * from t2 ... group by t2.group_col) DT -- select#3
    where t1.col=t2.group_col
  ) TBL;

The optimization went as follows:
JOIN::optimize() calls select_lex->handle_derived(DT_OPTIMIZE) which
calls JOIN::optimize() for all (direct and indirect) children SELECTs.

select#1->handle_derived() calls JOIN::optimize() for select#2 and #3;
select#2->handle_derived() calls JOIN::optimize() for select#3 the second
time. That call would call JOIN::optimize_stage2(), assuming the query
plan choice has been finalized for select#3.

But after that, JOIN::optimize() for select#2 would continue and consider
Split-Materialized for select#3. This would attempt to pick another join
order and cause a crash.

The fix is to have JOIN::optimize() not call optimize_stage2() ever.
Finalizing the query plan choice is now done by a separate call:
  select_lex->handle_derived(thd->lex, DT_OPTIMIZE_STAGE2)

which invokes JOIN::optimize_stage2() and saves the query plan.
2025-10-28 10:04:40 +02:00
Rex Johnston
8d08350dd3 MDEV-36321 keys generated on derived tables produce wrong out_rows estimates
record per key statistics on keys generated for derived tables are missing,
causing a fallback to the usage of MATCHING_ROWS_IN_OTHER_TABLE in
best_access_path().
This can grossly overestimate the number of rows expected during a join
and cause a bad join order.  This patch tries to infer a few simple
things about the derived table, like when the key being created matches
the group by list inside (if any) inside the derived table.

Approved by Sergei Petrunia (sergey@mariadb.com)
Tested by Lena Startseva (lena.startseva@mariadb.com)
2025-10-16 09:31:56 +12:00
Oleksandr Byelkin
c976b527db Merge branch '11.8' into bb-12.1-release 2025-10-08 09:05:38 +02:00
Aleksey Midenkov
ff33f49d9a Merge 11.4 into 11.8 2025-09-29 18:25:09 +03:00
Oleksandr Byelkin
15b1426c3a Merge branch '10.11' into bb-11.4-release 2025-09-15 16:17:33 +02:00
Monty
882f6fa3aa Fixed typos
- Removed duplicate words, like "the the" and "to to"
- Removed duplicate lines (one double sort line found in mysql.cc)
- Fixed some typos found while searching for duplicate words.

Command used to find duplicate words:
egrep -rI "\s([a-zA-Z]+)\s+\1\s" | grep -v param

Thanks to Artjoms Rimdjonoks for the command and pointing out the
spelling errors.
2025-09-04 18:08:39 +03:00
Oleg Smirnov
349f5bf2da MDEV-34870: implement join order hints
This commit implements optimizer hints allowing to affect the order
of joining tables:

 - JOIN_FIXED_ORDER similar to existing STRAIGHT_JOIN hint;
 - JOIN_ORDER to apply the specified table order;
 - JOIN_PREFIX to hint what tables should be first in the join;
 - JOIN_SUFFIX to hint what tables should be last in the join.
2025-05-05 12:02:47 +07:00
Oleg Smirnov
2c8f6058c1 MDEV-34888 Implement SEMIJOIN() and SUBQUERY() hints 2025-05-05 12:02:47 +07:00
Oleg Smirnov
af14196b8a MDEV-33281 Make BNL() hint enable hashed join buffers
Since BNL() hint does not specify which whether hashed or non-hashed
join cache should be employed, allow usage of hashed ones
where possible
2025-05-05 12:02:47 +07:00
Vasilii Lakhin
40c5b62531 Fix remaining typos 2025-04-29 11:18:00 +10:00
Alexander Barkov
f11504af51 MDEV-20034 Add support for the pre-defined weak SYS_REFCURSOR
This patch adds support for SYS_REFCURSOR (a weakly typed cursor)
for both sql_mode=ORACLE and sql_mode=DEFAULT.

Works as a regular stored routine variable, parameter and return value:

- can be passed as an IN parameter to stored functions and procedures
- can be passed as an INOUT and OUT parameter to stored procedures
- can be returned from a stored function

Note, strongly typed REF CURSOR will be added separately.

Note, to maintain dependencies easier, some parts of sql_class.h
and item.h were moved to new header files:

- select_results.h:
  class select_result_sink
  class select_result
  class select_result_interceptor

- sp_cursor.h:
  class sp_cursor_statistics
  class sp_cursor

- sp_rcontext_handler.h
  class Sp_rcontext_handler and its descendants

The implementation consists of the following parts:
- A new class sp_cursor_array deriving from Dynamic_array

- A new class Statement_rcontext which contains data shared
  between sub-statements of a compound statement.
  It has a member m_statement_cursors of the sp_cursor_array data type,
  as well as open cursor counter. THD inherits from Statement_rcontext.

- A new data type handler Type_handler_sys_refcursor in plugins/type_cursor/
  It is designed to store uint16 references -
  positions of the cursor in THD::m_statement_cursors.

- Type_handler_sys_refcursor suppresses some derived numeric features.
  When a SYS_REFCURSOR variable is used as an integer an error is raised.

- A new abstract class sp_instr_fetch_cursor. It's needed to share
  the common code between "OPEN cur" (for static cursors) and
  "OPER cur FOR stmt" (for SYS_REFCURSORs).

- New sp_instr classes:
  * sp_instr_copen_by_ref      - OPEN sys_ref_curor FOR stmt;
  * sp_instr_cfetch_by_ref     - FETCH sys_ref_cursor INTO targets;
  * sp_instr_cclose_by_ref     - CLOSE sys_ref_cursor;
  * sp_instr_destruct_variable - to destruct SYS_REFCURSOR variables when
                                 the execution goes out of the BEGIN..END block
                                 where SYS_REFCURSOR variables are declared.
- New methods in LEX:
  * sp_open_cursor_for_stmt   - handles "OPEN sys_ref_cursor FOR stmt".
  * sp_add_instr_fetch_cursor - "FETCH cur INTO targets" for both
                                static cursors and SYS_REFCURSORs.
  * sp_close - handles "CLOSE cur" both for static cursors and SYS_REFCURSORs.

- Changes in cursor functions to handle both static cursors and SYS_REFCURSORs:
  * Item_func_cursor_isopen
  * Item_func_cursor_found
  * Item_func_cursor_notfound
  * Item_func_cursor_rowcount

- A new system variable @@max_open_cursors - to limit the number
  of cursors (static and SYS_REFCURSORs) opened at the same time.
  Its allowed range is [0-65536], with 50 by default.

- A new virtual method Type_handler::can_return_bool() telling
  if calling item->val_bool() is allowed for Items of this data type,
  or if otherwise the "Illegal parameter for operation" error should be raised
  at fix_fields() time.

- New methods in Sp_rcontext_handler:
  * get_cursor()
  * get_cursor_by_ref()

- A new class Sp_rcontext_handler_statement to handle top level statement
  wide cursors which are shared by all substatements.

- A new virtual method expr_event_handler() in classes Item and Field.
  It's needed to close (and make available for a new OPEN)
  unused THD::m_statement_cursors elements which do not have any references
  any more. It can happen in various moments in time, e.g.
  * after evaluation parameters of an SQL routine
  * after assigning a cursor expression into a SYS_REFCURSOR variable
  * when leaving a BEGIN..END block with SYS_REFCURSOR variables
  * after setting OUT/INOUT routine actual parameters from formal
    parameters.
2025-04-19 10:59:58 +04:00
Vasilii Lakhin
717c12de0e Fix typos in C comments inside sql/ 2025-03-14 12:08:56 +04:00
Marko Mäkelä
bb9f010432 Merge 11.4 into 11.8 2025-03-05 20:39:47 +02:00
Sergei Golubchik
9ee09a33bb Merge branch '11.7' into 11.8 2025-02-11 20:29:43 +01:00
Sergei Petrunia
ef966af801 MDEV-30877: Output cardinality for derived table ignores GROUP BY
(Variant 3) (commit in 11.4)
When a derived table has a GROUP BY clause:

  SELECT ...
    FROM  (SELECT ... GROUP BY col1, col2) AS tbl

The optimizer would use inner join's output cardinality as an estimate
of derived table size, ignoring the fact that GROUP BY operation would
produce much fewer groups.

Add code to produce tighter bounds:
- The GROUP BY list is split into per-table lists. If GROUP BY list has
  expressions that refer to multiple tables, we fall back to join output
  cardinality.
- For each table, the first cardinality estimate is join_tab->read_records.
- Then, we try to get a tighter bound by using index statistics.
- If indexes do not cover all GROUP BY columns, we try to use per-column
  EITS statistics.
2025-02-10 22:06:49 +02:00
Sergei Petrunia
1c2a83179d MDEV-35616: Add basic optimizer support for virtual column
(Review input addressed)

After this patch, the optimizer can handle virtual column expressions
in WHERE/ON clauses. If the table has an indexed virtual column:

  ALTER TABLE t1
    ADD COLUMN vcol INT AS (col1+1),
    ADD INDEX idx1(vcol);

and the query uses the exact virtual column expression:

  SELECT * FROM t1 WHERE col1+1 <= 100

then the optimizer will be able use index idx1 for it.

This is achieved by walking the WHERE/ON clauses and replacing instances
of virtual column expression (like "col1+1" above) with virtual column's
Item_field (like "vcol"). The latter can be processed by the optimizer.

Replacement is considered (and done) only in items that are potentially
usable to the range optimizer.
2025-01-25 10:50:52 +02:00
Oleksandr Byelkin
195dcfec6f MDEV-35793: Server crashes in Item_func_vec_distance_common::get_const_arg
The problem was caused by this scenario: The query had both SELECT DISTINCT
and ORDER BY. DISTINCT was converted into GROUP BY. Then, vector index was
used to resolve the GROUP BY.
When join_read_first() initialized vector index scan, it used the ORDER BY
clause instead of GROUP BY, which caused a crash.

Fixed by making test_if_skip_sort_order() remember which ordering the scan
produces in JOIN_TAB::full_index_scan_order, and join_read_first() using that.
2025-01-21 12:54:43 +01:00
Marko Mäkelä
15700f54c2 Merge 11.4 into 11.7 2025-01-09 09:41:38 +02:00
Marko Mäkelä
17f01186f5 Merge 10.11 into 11.4 2025-01-09 07:58:08 +02:00
Marko Mäkelä
3f914afd3a Merge 10.6 into 10.11 2025-01-02 12:39:56 +02:00
Yuchen Pei
671f80c738 Merge branch '10.5' into 10.6 2024-12-17 11:06:09 +11:00
Oleg Smirnov
e640373389 Revert "MDEV-26427 MariaDB Server SEGV on INSERT .. SELECT"
This reverts commit 49e14000ee
as it introduces regression MDEV-29935 and has to be reconsidered
in general
2024-12-14 13:08:17 +07:00
Marko Mäkelä
33907f9ec6 Merge 11.4 into 11.7 2024-12-02 17:51:17 +02:00
Marko Mäkelä
2719cc4925 Merge 10.11 into 11.4 2024-12-02 11:35:34 +02:00
Marko Mäkelä
3d23adb766 Merge 10.6 into 10.11 2024-11-29 13:43:17 +02:00
Marko Mäkelä
7d4077cc11 Merge 10.5 into 10.6 2024-11-29 12:37:46 +02:00
Brandon Nesterenko
dbfee9fc2b MDEV-34348: Consolidate cmp function declarations
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict

The functions queue_compare, qsort2_cmp, and qsort_cmp2
all had similar interfaces, and were used interchangable
and unsafely cast to one another.

This patch consolidates the functions all into the
qsort_cmp2 interface.

Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
2024-11-23 08:14:22 -07:00
Oleksandr Byelkin
b12ff287ec Merge branch '11.6' into 11.7 2024-11-10 19:22:21 +01:00