MDEV-39519 added MySQL 8 compatibility to `mariadb-dump --dump-slave`
by attempting `SHOW REPLICA STATUS` first and then `SHOW SLAVE STATUS`.
This conflicted with MDEV-37146, where `mariadb-dump --dump-slave`
queries either `SELECT … FROM information_schema.SLAVE_STATUS`
or `SHOW ALL SLAVES STATUS` depending on the server version.
This commit merges MDEV-37146 and MDEV-39519:
* Use MDEV-39519’s strategy based on syntax error handling.
* Use MDEV-37146’s preference order:
1. `SELECT … FROM information_schema.SLAVE_STATUS`
2. `SHOW ALL SLAVES STATUS`
3. `SHOW REPLICA STATUS` (for MySQL compatibility _only_)
* Send `STOP`/`START REPLICA SQL_THREAD FOR CHANNEL '…'`
commands for both MariaDB 10.7+ and MySQL.
* Refactor column indices to variables set when a query succeeds.
* Partially revert MDEV-37146’s removal of `--dump-slave`’s support for
pre-GTID & pre-multi-source, but tailored for MySQL compatibility;
coverage for MariaDB pre-10.0 is not fully restored.
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
log_t::set_archive(archive=false): Ensure that both the latest checkpoint
and the latest log record (which has possibly not been written out yet)
will carry the log_sys.get_sequence_bit(lsn)==1, to guarantee a successful
recovery after the switch to the innodb_log_archive=OFF format.
Tested by: Matthias Leich
Reviewed by: Thirunarayanan Balathandayuthapani
* keep `vec_len >= subdist_part * 2` logic in one place only
* keep "distance-greater-than" mode logic in one place only
* simplify VECTOR_DIMENSIONS (no need to have a special ctx->vec_len
path if the other one always works)
* new plugin = maturity beta
* remove redundant casts, etc
* moved vector_indexes_fields_enum to the global scope to use it
for setting schema->idx_field1/schema->idx_field2
* open the hlindex graph table, if needed, otherwise most values
are unknown unless a user did vector search before
* added TABLE_CATALOG column
* remove CACHE_OVERFLOWS column, doesn't work as implemented,
the fix is complex and isn't worth it
* add privilege checks (MDEV-40793)
in the test:
* prefer query_vertical for readability
* select all columns at least once
* select INDEX_SIZE even if engine-dependent, use rdiff files
* test how get_all_tables only open one specific table, and
even only .frm file, if possible
Create a GitHub action into the MariaDB server tree to build the Markdown
documentation using the cmake target provided by MDEV-39718.
This workflow generates API documentation for the plugin using a
Docker container packed with all the extra doxygen/moxygen tools.
Fixed some more doxygen problems.
Moved the repo to ghcr.io/mariadb/mariadb-doc-gen:latest.
DENY ... TO PUBLIC denies everyone, including whoever tries to revoke
it, via the "deny wins" merge at every scope (global, db, table, column,
routine). Allow REVOKE DENY ... FROM PUBLIC when the revoker can UPDATE
mysql.global_priv (same as hand-editing).
Assisted-by: Claude:claude-5-sonnet
Move the std::function callbacks into the members instead of copying
them, and pass the captureless FTS comparators as plain functions
instead of lambdas. This is to avoid GCC -Wmaybe-uninitialized false
positive in RecordCallback
deferred_spaces.deferred_dblwr(): Skip newly created tablespaces
to avoid a bogus invocation of fil_space_free().
fil_name_process(): Simplify the logic. If no matching tablespace is
found but file_name_t::create_lsn had been set in response to parsing
a FILE_CREATE record, try to apply FILE_RENAME to deferred_spaces.
log_parse_file(): Parse each FILE_ record only once. In multi-batch
recovery, there may be redundant calls that would break the logic of
fil_name_process().
fil_delete_apply(): Wrappers for fil_space_free(). When recovering
a log in innodb_log_archive=ON format, we must apply FILE_DELETE
records in order to avoid a future clash with FILE_CREATE or FILE_RENAME.
log_t::write_checkpoint(): Extend the the correct file.
log_t::archived_mmap_switch_prepare(): Tolerate a near-concurrent
buf_flush_archive_create() from the buf_flush_page_cleaner() thread.
log_t::set_archive(): When a checkpoint needs to be forced, set a
minimum target that guarantees progress, instead of flushing
the entire buffer pool. Remove some duplicated reads of
last_checkpoint_lsn.
When switching innodb_log_archive from OFF to ON,
accurately remember whether any log records may have been
written with get_sequence_bit() == 0, to ensure that a
checkpoint will be waited for on a subsequent switch
from ON to OFF.
log_t::set_recovered(): If innodb_log_archive=ON, only make a
future set_archive(false) trigger a checkpoint if we may have
recovered a server that was killed between set_archive(true)
and write_checkpoint().
MDEV-32570 added fragmentation of large row events. When binlog
compression is enabled (log_bin_compress=ON), the compression is
bypassed, and these binary log events remain as regular row events, when
they should be compressed row events.
This is because the compression happens during the Log_event::write()
function, whereas row data fragmentation happens before
Log_event::write() is called. The Rows_log_event super-class stores the
row data buffers to be written to disk. The regular Rows_log_event
sub-classes's implementations of Log_event::write() write this data
as-is. The compressed sub-classes's implementation of ::write()
over-write these buffers with the compressed rows data before writing to
disk.
To fragment a large row event, the server fragments the Rows_log_event's
row data buffers, to be written by multiple Partial_rows_log_event's,
and each Partial_rows_log_event::write() call writes its portion of the
row data buffer to disk. However, this happens before compression ever
has a chance to take place, and thereby, an event that should be
compressed, never is.
MDEV-39762 added event structure validation for compressed events, and
discovered that these events carry the compressed event type, but are
not actually compressed. Event validation thereby fails.
This patch adds a workaround to override the event type of the
fragmented row events to be regular row events, to be consistent with
the actual on-disk content. The underlying problem still needs to be
addressed though, and is tracked by MDEV-40851.
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
- In ha_partition.cc:check_parallel_search(), remove check if
item_field->field is null. This is not needed as the function is run
after fix_field() which guarnatees that the field is always set.
- Added DBUG_ASSERT(new_field) to Item_field::fix_fields() to check if a
select-list item, found by name or alias when resolving ORDER BY/GROUP
BY/HAVING, can have field == 0. This error path is not covered by any
mtr test.
This allows
create table t1 (v vector(64) not null);
insert into t1 select vec_fromtext(concat('[',group_concat(1),']')) from seq_1_to_64;
which was banned in the previous fix
bb0ac43701, though this also introduces
the inconsistency(?) where
create table t1 as select vec_fromtext(concat('[',group_concat(1),']')) from seq_1_to_64;
still fails ER_TRUNCATED_WRONG_VALUE
see updated tests
TODO:
- Changes in vector_utf16.result does not look right
- the following tests crashes:
CREATE TABLE t1 (v VECTOR(2));
--error ER_TOO_BIG_FIELDLENGTH
INSERT INTO t1 VALUES (VEC_FROMTEXT(CONCAT('[1.', REPEAT('0',70000), ',2]')));
DROP TABLE t1;
SELECT VEC_FROMTEXT('😀😀😀');
Align ro_after_init using MAXPAGESIZE instead of COMMONPAGESIZE.
COMMONPAGESIZE may be smaller than the actual page size supported by
the target ABI. This can leave ro_after_init sharing an OS page with
adjacent sections, causing mprotect() to change permissions on data
outside ro_after_init.
Use MAXPAGESIZE so the section boundaries are aligned to the maximum
page size required by the target linker/ABI.
This is particularly important on architectures such as ppc64le and
aarch64, where the runtime page size can differ from COMMONPAGESIZE.
Before:
.data 0x...1b80000
ro_after_init 0x...1c70000
.bss 0x...1c72000
After:
ro_after_init starts and ends on MAXPAGESIZE boundaries, ensuring
mprotect() only affects pages belonging to ro_after_init.
Co-authored-by: ChatGPT GPT-5.6 Luna <noreply@openai.com>
The server crashed on DBUG_ASSERT on a SELECT into:
- a `ROW TYPE OF table1` field variable
- a `ROW TYPE OF cursor1` field variable
Fix:
- Adding a class my_var_sp_row_field_by_name
- Adding a method sp_rcontext::set_variable_row_field_by_name()
- Fixing the DBUG_ASSERT
The cleanup of the old connection goes on in the background.
It can take longer on a busy server and this triggers the active sessions
warning in DROP user.
Stablizied the test by disabling the warnings.
Handle corrupted row events by reporting whether the field length could
not be determined from metadata or whether the field extends past the
row buffer instead of falling out through debug-only assertion paths
or ad hoc exits during flashback row conversion. Propagate flashback
conversion failures through the normal mysqlbinlog error path so debug
builds produce the intended diagnostics cleanly.
Add replication tests covering corrupted BLOB metadata and corrupted
BLOB length prefixes using server-side debug injection to write broken
binlog contents and mysqltest-friendly mysqlbinlog invocation patterns.
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
Reviewed-by: Georgi Kodinov <joro@mariadb.org>
Generated the plugin API headers using a shell script.
Fixed some doxygen comment mistakes in the headers.
Added a cmake conveninence target to generate the docs into $BUILD_DIR/docs
Added a main page for the API docs.
Included all of the existing group .md files into the CMake target
Leveraged moxygen 2.1.11's fixes to produce the full API docs in a single go
Removed the list of output .md files from the CMake target and switched to a
stamp file to avoid unnecessary rebuilds of the docs when the list of .md
files changes.
Addressed various review comments.
remove dead code, handling of MariaDB/MySQL servers that didn't
have SQL_QUOTE_SHOW_CREATE.
SQL_QUOTE_SHOW_CREATE was added in 2000
(commit 5762523b78, MySQL 3.23.25-beta)
Problem:
If:
- A routine A() opened a SYS_REFCURSOR with a function B() in the SELECT list
- The function B() also opened some SYS_REFCURSORs
Then reallocation of the cursor array THD::m_statement_cursors could happen
during the execution of B(), so all sp_cursor_array_element pointers inside
sp_instr_copen_by_ref::exec_core() of routine A() became invalid.
Fix:
Chaging the data type of sp_cursor_array:
- from Dynamic_array<sp_cursor_array_element>
- to Dynamic_array<sp_cursor_array_element*>
So now only reallocations of the array of cursor pointers happen,
while sp_cursor_array_element instances always stay on their originally
allocated memory positions.
Note:
sp_cursor_array_element instances are allocated using the standart C++ "new"
and deleted using the standard C++ "delete". Using a MEM_ROOT does not
seem to be relevant here.
Selecting from the information_schema.plugins causes the loading
of all plugins. Because rockdb leaks, and duckdb triggers an
address sanitizer warning on shutdown avoid this table.
Use the information_schema.ENGINES to validate that InnoDB is
disabled per the original request in the review of MDEV-32205.
SET sql_mode=ORACLE;
DELIMITER $$
CREATE OR REPLACE PACKAGE pkg AS
-- Declare a package public data type
TYPE varchar_array IS TABLE OF VARCHAR(2000) INDEX BY INTEGER;
END;
$$
DELIMITER ;
DELIMITER $$
CREATE OR REPLACE PROCEDURE p1 AS
v pkg.varchar_array; -- Use the package public data type
BEGIN
v(0):='test';
SELECT v(0);
END;
$$
DELIMITER ;
Note, the change is done only for sql_mode=ORACLE, because the TYPE
declaration is not available for the default mode.
Where package-wide types are available
--------------------------------------
- Variabe list type:
DECLARE var pkg1.type1;
- RETURN type for a package routine:
CREATE FUNCTION .. RETURN pkg1.type1 ...
- Parameter type for a package routine:
PROCEDURE p1(param1 pkg1.type1);
- Assoc array element type:
TYPE assoc1_t IS TABLE OF pkg1.type1 ...
- REF CURSOR RETURN type:
TYPE cur1_t IS REF CURSOR RETURN pkg1.type1;
Change details
--------------
- Adding a member Lex_length_and_dec_st::m_foreign_module_type
It's set to true when the data type was initialized from a TYPE
in foreign routine (e.g. in PACKAGE spec).
It's needed to prevent use of qualified identifiers in public contexts,
i.e. in schema public routine parameter types and schema publuc function
RETURN types.
Adding a helper method sp_head::check_applicability() which prevents
use of qualified types in public context.
- Adding a helper method sp_head::raise_unknown_data_type().
- Adding methods LEX::set_field_type_typedef_package_spec() for
2-step and 3-step qualified indentifiers.
It's used in field_type_all_with_typedefs which covers cases:
- Variabe list type : DECLARE var pkg1.type1;
- RETURN type : CREATE FUNCTION .. RETURN pkg1.type1 ...
- Parameter type : PROCEDURE p1(param1 pkg1.type1);
- Assoc array element type : TYPE assoc1_t IS TABLE OF pkg1.type1 ...
- Adding a method LEX::declare_type_ref_cursor_return_typedef().
It handles cases when a new TYPE REF CURSOR RETURN is declared,
for both for qualified RETURN types and non-qualified RETURN types:
- TYPE cur0_t IS REF CURSOR RETURN rec1_t;
- TYPE cur0_t IS REF CURSOR RETURN pkg1.rec1_t;
- TYPE cur0_t IS REF CURSOR RETURN db1.pkg1.rec1_t;
The code was moved from LEX::declare_type_ref_cursor() into
LEX::declare_type_ref_cursor_return_typedef() and extended
to cover qualified RETURN types.
- Adding a method Sql_path::find_package_spec_type().
It iterates through all schemas specified in @@path and searches
for the given type in the given package.
- Adding a helper method sp_pcontext::type_defs_add_ref_cursor()
to reuse the code.
- Adding a new method sp_package::get_typedef() to search
for TYPE definitions in PACKAGE specifications.
- Adding a new method sp_head::get_typedef_package_spec()
to search for TYPE definitions used by a PROCEDURE or FUNCTION.
- Adding a helper method
Sp_handler::sp_cache_routine_reentrant_suppress_errors
Adding a method Sp_handler::find_package_spec().
The test used "restart_abort:" in the expect file, which MTR never
recognized. It fell through to the else branch, deleted restart_opts,
and started the server with defaults (heartbeat_period=60 instead of 0).
Replace restart_abort with direct --exec $MYSQLD calls.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
The query plan was not stable.
The test used to create table t1, and then do this in a loop:
INSERT INTO t1 SELECT ...
SELECT ... FROM t1;
DELETE FROM t1;
Apparently deleted records could be purged faster or slower (there is
no way to wait for purge to complete) and this caused variation in
records_in_range() return value.
Changed the test to use TRUNCATE TABLE t1 instead of DELETE.
The changes of MDEV-39558 2b6529426a
added length check assertion in Field_varstring constructors, and
length check in type inference for SELECT set operations, to emit
errors before reaching the assertions.
That change caused an error to turn into an assertion failure in a
separate path, when the length limit violation is not detected before
tripping the assertion. So in this patch we fix it by adding an
earlier length check in that path.
The reason that we place this check inside
Item_func_vec_fromtext::fix_length_and_dec rather than say
`create_field_for_create_select is for consistency:
If
create table t1 as select
vec_fromtext(concat('[',group_concat(1),']')) as c1 from seq_1_to_64;
fails due to length limit violation, then so should
create table t1 (v vector(64) not null);
insert into t1 select vec_fromtext(concat('[',group_concat(1),']'))
from seq_1_to_64;
Also use max_char_length() instead of max_length. This is a more
accurate length of characters. And add handling of empty string edge
case. Added testcases accordingly.
The change that uses max_char_length() causes side effects where
creating a table using a VEC_FROMTEXT(CHAR(1)) would result in a
0-dimensional vector field. This is accurate but 0-dim vector table
fields should not be allowed. So we make cases like this result in
a one-dimensional field.
Also fixed the underflow in (args[0]->max_length - 1) * 2 when the
arg's max length is 0. Previously this underflow would cause
create table t1 select vec_fromtext(NULL)
to fail with ER_TOO_BIG_FIELDLENGTH. Now it will be a VECTOR(1) field
As the test result is dependent of SHOW PROCESSLIST output,
adjust the wait condition to ensure the state is in sleeping
rather than "init" or another state.
