Atomic CREATE OR REPLACE allows to keep an old table intact if the
command fails or during the crash. That is done by renaming the
original table to temporary name, as a backup and restoring it if the
CREATE fails. When the command is complete and logged the backup
table is deleted.
Atomic replace algorithm
Two DDL chains are used for CREATE OR REPLACE:
ddl_log_state_create (C) and ddl_log_state_rm (D).
1. (C) Log rename of ORIG to TMP table (Rename TMP to original).
2. Rename orignal to TMP.
3. (C) Log CREATE_TABLE_ACTION of ORIG (drops ORIG);
4. Do everything with ORIG (like insert data)
5. (D) Log drop of TMP
6. Write query to binlog (this marks (C) to be closed in
case of failure)
7. Execute drop of TMP through (D)
8. Close (C) and (D)
If there is a failure before 6) we revert the changes in (C)
Chain (D) is only executed if 6) succeded (C is closed on
crash recovery).
Foreign key errors will be found at the 1) stage.
Additional notes
- CREATE TABLE without REPLACE and temporary tables is not affected
by this commit.
set @@drop_before_create_or_replace=1 can be used to
get old behaviour where existing tables are dropped
in CREATE OR REPLACE.
- CREATE TABLE is reverted if binlogging the query fails.
- Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
this commit. Conflicting tables marked with this flag will be
deleted with CREATE OR REPLACE.
- Replication execution is not affected by this commit.
- Replication will first drop the conflicting table and then
creating the new one.
- CREATE TABLE .. SELECT XID usage is fixed and now there is no need
to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
do_postlock()). XID is now correctly updated so it disables
DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
final stage when the table is ready. So if we have XID in the
binary log we don't need to drop the table.
- Three variations of CREATE OR REPLACE handled:
1. CREATE OR REPLACE TABLE t1 (..);
2. CREATE OR REPLACE TABLE t1 LIKE t2;
3. CREATE OR REPLACE TABLE t1 SELECT ..;
- Test case uses 6 combinations for engines (aria, aria_notrans,
myisam, ib, lock_tables, expensive_rename) and 2 combinations for
binlog types (row, stmt). Combinations help to check differences
between the results. Error failures are tested for the above three
variations.
- expensive_rename tests CREATE OR REPLACE without atomic
replace. The effect should be the same as with the old behaviour
before this commit.
- Triggers mechanism is unaffected by this change. This is tested in
create_replace.test.
- LOCK TABLES is affected. Lock restoration must be done after new
table is created or TMP is renamed back to ORIG
- Moved ddl_log_complete() from send_eof() to finalize_ddl(). This
checkpoint was not executed before for normal CREATE TABLE but is
executed now.
- CREATE TABLE will now rollback also if writing to the binary
logging failed. See rpl_gtid_strict.test
backup ddl log changes
- In case of a successfull CREATE OR REPLACE we only log
the CREATE event, not the DROP TABLE event of the old table.
ddl_log.cc changes
ddl_log_execute_action() now properly return error conditions.
ddl_log_disable_entry() added to allow one to disable one entry.
The entry on disk is still reserved until ddl_log_complete() is
executed.
On XID usage
Like with all other atomic DDL operations XID is used to avoid
inconsistency between master and slave in the case of a crash after
binary log is written and before ddl_log_state_create is closed. On
recovery XIDs are taken from binary log and corresponding DDL log
events get disabled. That is done by
ddl_log_close_binlogged_events().
On linking two chains together
Chains are executed in the ascending order of entry_pos of execute
entries. But entry_pos assignment order is undefined: it may assign
bigger number for the first chain and then smaller number for the
second chain. So the execution order in that case will be reverse:
second chain will be executed first.
To avoid that we link one chain to another. While the base chain
(ddl_log_state_create) is active the secondary chain
(ddl_log_state_rm) is not executed. That is: only one chain can be
executed in two linked chains.
The interface ddl_log_link_chains() was defined in "MDEV-22166
ddl_log_write_execute_entry() extension".
Atomic info parameters in HA_CREATE_INFO
Many functions in CREATE TABLE pass the same parameters. These
parameters are part of table creation info and should be in
HA_CREATE_INFO (or whatever). Passing parameters via single
structure is much easier for adding new data and
refactoring.
Aria changes:
- Fixed issue in Aria engine with CREATE + locked tables
that data was not properly commited in some cases in
case of crashes.
InnoDB related changes (by Marko Mäkelä):
- table_name_t::is_create_or_replace(): A new predicate to check for
CREATE OR REPLACE TABLE will rename an old table to
and eventually drop after creating the replacement.
- dict_table_t::parse_name(): Do acquire MDL on #sql-create- names
for partitioned tables.
- dict_table_rename_in_cache(): On CREATE OR REPLACE TABLE ... SELECT,
forget the original dict_table_t::mdl_name so that purge will
acquire MDL on the #sql-create- name instead. In this way, the
MDL_EXCLUSIVE that the CREATE OR REPLACE TABLE holds on the
user-visible name will not unnecessarily block any purge of old history
until the very end when the #sql-create- table will be dropped.
- ha_innobase::delete_table(): Do not check FOREIGN KEY consistency
when dropping an #sql-create- table.
- row_rename_table_for_mysql(): Update SYS_FOREIGN.ID also
when renaming to #sql-create- in order to avoid any
duplicate key error when CREATE OR REPLACE TABLE is
creating some FOREIGN KEY constraints by names
that existed in the old table.
Other changes:
- Removed some auto variables in log.cc for better code readability.
- Fixed old bug that CREATE ... SELECT would not be able to auto repair
a table that is part of the SELECT.
- Marked MyISAM that it does not support ROLLBACK (not required but
done for better consistency with other engines).
- maria_create_trn_for_mysql() does not register a new transaction
handler for commits. This was needed to ensure create or replace
will not end with an active transaction.
- We do not get anymore warnings about "Engine not supporting atomic
create" when doing a legal CREATE OR REPLACE on a table with
foreign key constraints.
- Updated VIDEX engine flags to disable CREATE SEQUENCE.
- Removed mysql_mutex_unlock(&LOCK_gdl) / mysql_mutex_lock(&LOCK_gdl)
around calls to binlog as these are unsafe. The binlog code uses
global variables that needs protection from other caller.
- EITS data is preserved if create or replace fails if
drop_before_create_or_replace=OFF. If ON, then create or replace
will drop EITS before the drop of the original table (as before).
- Using CREATE OR REPLACE on a encrypted table that the user cannot
decrypt will fail instead of replacing the encrypted table.
The encrypted table will unchanged.
Known issues:
- One cannot use create or replace on an InnoDB tables that has foreign
key references point to it
- CREATE OR REPLACE TEMPORARY table is not full atomic. Any conflicting
table will always be dropped before creating a new one. (Old behaviour).
Bug fixes related to this MDEV:
MDEV-36435 Assertion failure in finalize_locked_tables()
MDEV-36439 Assertion `thd_arg->lex->sql_command != SQLCOM_CREATE_SEQUENCE...
MDEV-36498 Failed CoR in non-atomic mode no longer generates DROP in RBR...
MDEV-36508 Temporary files #sql-create-....frm occasionally stay after
crash recovery
MDEV-38479 Crash in CREATE OR REPLACE SEQUENCE when new sequence cannot
be created
MDEV-36497 Assertion failure after atomic CoR with Aria under lock in
transactional context
MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge
MDEV-39367 MSAN/valgrind errors in temp_file_size_cb_func,
main.tmp_space_usage fails
MDEV-39446 Atomic CREATE OR REPLACE fails if a table cannot be decrypted
MDEV-40776 Atomic CREATE OR REPLACE silently breaks the foreign key
MDEV-40765 Assertion `"unexpected references" == 0' failed upon failing
CREATE OR REPLACE
MDEV-40994 Atomic CoR: Transaction not registered for MariaDB 2PC, but
transaction is active
Reverted commits:
- MDEV-36685 "CREATE-SELECT may lose in binlog side-effects of
stored-routine" as it did not take into account that it safe to
clear binlogs if the created table is non transactional and there
are no other non transactional tables used.
This was done because it caused extra logging when it is not
needed (not using any non transactional tables) and it also did
not solve side effects when using statement based loggging.
This is part of MDEV-25292 Atomic CREATE OR REPLACE TABLE.
Removed default values for arguments, added flags argument to specify
filename flags (FN_TO_IS_TMP, FN_FROM_IS_TMP) and forward the flag to
build_table_name().
Original patch from: Aleksey Midenkov <midenok@gmail.com>
Json_writer had two separate code paths: add_unquoted_str() for
numbers/bool/null, and add_escaped_str() which added the surrounding
quotes itself for strings. Single_line_formatting_helper, which
buffers consecutive array/object elements to decide if they fit on
one line, assumed only strings could ever be buffered and always
wrapped the flushed values in quotes. As a result, arrays of numbers
(e.g. "depends_on_map_bits", "rec_per_key") were incorrectly rendered
with their elements quoted as strings.
Unify both paths into add_escaped_quoted_str(): the caller now hands
over bytes that are already in their final on-the-wire form. String
escaping (json_escape_to_string) writes its own surrounding quotes,
while numbers/bool/null are passed through unquoted, so the one-line
helper just concatenates the buffered payloads on flush instead of
adding quotes itself. A DBUG_ASSERT in add_escaped_quoted_str() now
checks that every payload is already a quoted string or a bare
number/bool/null token, so a caller that violates the contract trips
an assertion in debug builds instead of silently producing invalid
JSON.
Also:
- Fix Json_writer_array::add(ulonglong)/(size_t), which went through
add_ll() with a cast to longlong and corrupted large unsigned
values (e.g. ULLONG_MAX); route them through add_ull() instead.
- Fix mysql-test/include/opt_context_schema.inc: "subquery_runs" was
nested inside the preceding object instead of being a sibling
member, and "rec_per_key" items are now declared as "number" to
match the corrected output.
- Update recorded .result files for opt_trace, opt_context_*, and
subselect_mat_analyze_json to reflect numbers/booleans no longer
being quoted inside JSON arrays.
- Extend unittest/sql/my_json_writer-t.cc with coverage for arrays
of primitives: plain integers, mixed types, sizes, multi-line
arrays, values flushed before a nested object, strings that were
already escaped by the one-line buffer, and invalid utf8mb4 input
through add_str().
- Rewrite the "multi-line array of integers" test to actually
overflow the one-line buffer (9 seven-digit numbers instead of 7),
since the previous version fit on one line and never exercised the
element-per-line flush path it claimed to test.
- Drop the now-redundant single-argument add_escaped_quoted_str()
overload; all call sites already know their length.
- Update json_escape_to_string()'s doc comments (my_json_writer.h,
sql_json_lib.h) to state that it quotes its output, not just
escapes it.
Extend the RISC-V CRC-32C path with a Zvbc vector implementation:
- mysys/CMakeLists.txt: detect Zvbc + RVV intrinsics under
-march=rv64gc_zbb_zvbc; when available rebuild the new
crc32c_riscv_zvbc.cc with that march (optional, Zbc unaffected);
- mysys/crc32/crc32c_riscv_zvbc.cc (new): K-lane vector core --
4 lanes x 128-bit folding, vlseg2e64 de-interleaved 64B loads,
vclmul_vx broadcast constants, single-element vector CLMUL for
the Barrett merge (no scalar Zbc instruction anywhere), bit-exact
with the Zbc core (same fold constants k1..k4 and Barrett math);
- adaptive VLEN: one e64m1 vector pair at VLEN>=256, two pairs at
VLEN=128, so 128-bit cores execute the vector path at full width;
- mysys/crc32/crc32c_riscv.cc + crc32c.cc: runtime dispatch via
riscv_hwprobe -- Zbc is preferred when present (existing
deployments keep the optimized scalar core); the Zvbc vector
path accelerates cores that implement Zvbc but not scalar Zbc.
Correctness: RFC 3720 + random/chained/boundary inputs bit-identical
to slicing-by-4; official unittest/mysys/crc32-t.c 36/36.
Performance on Spacemit X100 (k3, VLEN=256, gcc 14.3.0), vs inherited
slicing-by-4 crc32c_slow (official slow path, same my_crc32c entry):
len slow (MB/s) PR-B2 Zvbc (MB/s) vs slow
128 B 289 1147 4.0x
256 B 296 2105 7.1x
1 KiB 297 5466 18.5x
4 KiB 297 9116 30.7x
64 KiB 296 11398 38.7x
Dispatch prefers the scalar Zbc path when Zbc is available, so
existing Zbc deployments see no behavior change or regression;
the vector path is selected on cores without scalar Zbc.
No behavior change on non-riscv64 builds or toolchains without
Zvbc (optional compile check).
Review revisions requested on PR #5746 (dr-m):
- mysys/crc32/crc32c_riscv.cc: assemble the ZBC/ZVBC extension bits
without a branch at both sites, dropping the redundant intermediate
variable; replace `(void) hwprobe;` with `std::ignore = hwprobe;` and
include <tuple>. <tuple> introduces no static constructor, so the
ifunc resolver's load-time constraint is unaffected.
- mysys/crc32/crc32c.cc: expand the comment above the Zbc preference
with the structural reason -- the vector core deliberately reuses the
scalar fold to stay bit-exact, so its vl is capped at 4 and a wider
VLEN buys nothing; scalar Zbc is preferred because the two are level
on large input and the scalar path is clearly ahead on small input.
Comment-only.
Assisted-by: YuanSheng:DeepSeek-V4-Flash
Co-authored-by: Xiaofei Gong <gongxiaofei24@iscas.ac.cn>
Signed-off-by: Jiakai Xu <xujiakai2025@iscas.ac.cn>
Copy/paste friendly output was only reachable by starting the client with
--silent --skip-column-names, which cannot be done from a running
interactive session.
Add \S, a statement terminator which prints the result of one statement in
the tab separated format without column names.
com_silent() sets output_plain, opt_silent and column_names around
com_go(), then restores them, the same way com_ego() handles vertical.
output_plain selects print_tab_data() ahead of the vertical and table
branches, so \S gives the same output whether the session was started
plainly or with --table, --vertical or --silent. --html and --xml still
win, matching \G.
The server_audit_timestamp_format %f specifier always rendered zero
microseconds. The server downcast the precise time to seconds before
passing it to audit plugins, and the plugin then re-fetched the time
itself at write time.
Pass the server's high-resolution time through the audit API instead:
- extend mysql_event_general with general_time_microseconds (added in
MYSQL_AUDIT_INTERFACE_VERSION 0x0304), keeping general_time in
seconds for backward compatibility
- the server_audit plugin uses event->general_time_microseconds for
query log entries instead of re-fetching the time at write time
Connection and table events carry no timestamp in the audit API, so the
plugin keeps taking the time at event time for those entries.
log_t::set_recovered(): Do not unnecessarily set the
circular_recovery_from_sequence_bit_0 flag for innodb_log_archive=ON
format files.
The purpose of the flag is to ensure that an extra checkpoint will be
written when converting the log to innodb_log_archive=OFF format. The
scenario that we want to prevent is that the log originally was in
innodb_log_archive=OFF format and had wrapped around an odd number of
times since the file creation, that is, the sequence bit at the end
of the mini-transactions since the latest checkpoint is 0. After a
conversion to innodb_log_archive=ON format, old records would carry
the sequence bit 0 and new ones the bit 1. This is fine, because the
recovery will ignore the sequence bit; innodb_log_archive=ON files
never wrap around. However, when the log is converted back to
innodb_log_archive=OFF format, we must guarantee that all sequence
bits since the latest checkpoint were written as 1.
Add an accelerated CRC32C path for RISC-V 64-bit using the carry-less
multiply (clmul/clmulh) extension of the Zbc extension. The same
instructions also accelerate the ISO 3309 CRC-32 (Ethernet, zlib) that
my_checksum() uses, so both polynomials are served by a single core,
the way crc32_avx512() already serves both on x86:
- mysys/CMakeLists.txt: detect the extension by compiling the same
inline assembler that the implementation uses, add
-DHAVE_RISCV_ZBC and build crc32/crc32c_riscv_zbc.cc with
-march=rv64gc_zbc_zbb. Only that compilation unit is built for the
extension; the detection and dispatch in crc32/crc32c_riscv.cc use
the normal settings, so that no code that may run on a CPU without
the extension can be compiled for it (MDEV-24745).
- mysys/crc32/crc32c.cc: on RISC-V, my_crc32c is an indirect function
(ifunc) whose resolver selects crc32c_riscv_zbc() or the existing
crc32c_slow() at load time. The other architectures keep the
existing *_available() dispatch unchanged.
- mysys/crc32/crc32c_riscv_zbc.cc (new): 128-bit 4-way parallel
folding + Barrett reduction using clmul/clmulh inline asm, with a
bitwise fallback for short inputs. The polynomial-dependent
constants (fold factors, Barrett constants, reflected polynomial)
live in a constexpr rv_crc32_tab passed to the one core function,
so the same code serves both polynomials:
* crc32c_riscv_zbc(): CRC-32C, Castagnoli 0x1EDC6F41
* crc32_riscv_zbc(): ISO 3309, 0x04C11DB7
- mysys/crc32/crc32c_riscv.cc (new): the runtime Zbc probe,
rv_zbc_supported(). It uses the riscv_hwprobe system call and
nothing else; a /proc/cpuinfo probe cannot be used, because the
function is also called from the ifunc resolver and that runs before
the C library is initialised.
- mysys/crc32ieee.cc: likewise, my_checksum is dispatched by an ifunc
on RISC-V, its resolver selecting crc32_riscv_zbc() or the existing
my_crc32_zlib(), instead of the runtime init_crc32() used elsewhere.
On RISC-V the dynamic linker calls a resolver as
(hwcap, __riscv_hwprobe, reserved). rv_zbc_supported() uses the
__riscv_hwprobe pointer passed there when it is available; a C library
older than glibc 2.40 passes NULL in that position, and the system call
is then made directly. A resolver cannot report "extension absent" by
returning NULL, as that crashes at the first call, so it returns a real
fallback; crc32c_slow() and my_crc32_zlib() are both linked
unconditionally and so are the natural choice.
Correctness: RFC 3720 + official unittest/mysys/crc32-t.c (36/36; 18
of its cases exercise my_checksum(), i.e. ISO 3309). my_checksum() was
also cross-checked against zlib crc32() over an exhaustive sweep of
lengths 0..4096 x 16 input alignments plus non-zero seeds: 0
mismatches.
Performance on Spacemit X100 (k3), vs inherited slicing-by-4
crc32c_slow (official slow path, same my_crc32c entry):
len slow (MB/s) Zbc (MB/s) speedup
128 B 56 477 8.58x
256 B 55 920 16.59x
1 KiB 55 3125 56.50x
4 KiB 55 7216 130.21x
64 KiB 55 11893 214.68x
ISO 3309 (my_checksum, 64 MiB buffer on the same machine): 0.65 GB/s
on the generic zlib path before, about 8 GB/s with the Zbc core after
(~12x), on par with my_crc32c.
No behavior change on non-riscv64 or Zbc-less RISC-V builds: the code
is compiled only when the extension is available, and each entry point
selects the accelerated implementation only when the probe reports Zbc.
Assisted-by: YuanSheng:DeepSeek-V4-Flash
Co-authored-by: Xiaofei Gong <gongxiaofei24@iscas.ac.cn>
Signed-off-by: Jiakai Xu <xujiakai2025@iscas.ac.cn>
Adds a static JOIN_TAB_RANGE::create(thd, count) helper that allocates
a JOIN_TAB array and wraps it in a JOIN_TAB_RANGE, replacing the
inline allocation in JOIN::get_best_combination(). Also gives
JOIN_TAB_RANGE::start and ::end default initializers and documents
their half-open range convention.
Currently Optional_metadata_fields has many members that use classes
from the C++ standard library, most notably the use of std::vector and
std::string. This is inconsistent with the coding standards, as MariaDB
defines its own types for arrays/lists/strings. This patch updates
these variables to use existing MariaDB types.
Additionally, the structure of the fields is changed. Instead of having
a separate list to track each metadata field; the structure now exists
at the column level (Column_metadata), and it has individual member
variables to describe its metadata. This allows for more
straightforward memory management, as everything will now be allocated
at once.
There is also a bug fix in this patch. Prior to this refactor, YEAR
columns would not consume their "unsigned" bit flag, resulting in
numeric columns after a YEAR type to have incorrect signage. This is
now fixed.
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>.
Aggregates lacked the SQL-standard FILTER clause,
forcing CASE-based workarounds that reduced readability across (sum, avg, count, …).
This update introduces the ability to specify a FILTER clause for aggregate functions,
allowing for more granular control over which rows are included in the aggregation.
Also, improves standards compliance and makes queries clearer and more readable.
The FILTER(WHERE ...) condition may contain any expression allowed in regular WHERE clauses,
except window functions, and outer references.
WolfSSL code path already operates without fixed DH parameters. OpenSSL code path still sets fixed static precomputed DH params, which is now prohibited by IETF.
Also OPENSSL_init_ssl is not required since OpenSSL 1.1.0, for over 10 years now. Also cleaned up at the same time.
https://www.rfc-editor.org/rfc/rfc10015.html#section-2:
> Clients MUST NOT offer and servers MUST NOT select non-ephemeral FFDH cipher suites in (D)TLS 1.2 connections.
https://www.rfc-editor.org/rfc/rfc10015.html#section-3:
> Clients MUST NOT offer and servers MUST NOT select FFDHE cipher suites in (D)TLS 1.2 connections.
And the depreciated tables include all ciphersuites that can use SSL_CTX_set_tmp_dh as part of the connection.
Also for a very long time OpenSSL was handling these automatically anyway, back when DHE was still recommended.
Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.
Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:
CREATE PROCEDURE p1()
BEGIN
-- case 1: DEFAULT clause
DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK
-- case 2: SP variable assignment statement
DECLARE spvar2 INT;
SET spvar2= f1_with_ps(); -- OK
END;
- Only assignments to SP variables works for now:
* SET spvar= func_with_ps(); -- OK
* SET @uvar= func_with_ps(); -- Error
- Only bare function calls are supported for now. Using a function in
an expression does not make it PS-safe yet:
SET v= f1()+0;
- The parser now does not reject PS statements in stored functions.
PS applicability in stored functions is now detected at run time.
Note, PS statements in triggers are still prohibited by the parser.
- Functions with PS do not acquire MDL locks on tables, and no MDL is
taken on the routines themselves either. They work like procedures in
terms of table opening and routine locking: a concurrent DROP FUNCTION
can complete while such a function is executing.
- Functions with PS are not replicated as a single `SELECT f1()` call.
They are replicated per-statement, like procedures.
Helper changes:
- Changing the return result for LEX::sp_variable_declarations_init()
from void to bool to catch errors in the caller properly.
Misc:
- This patch incorporates fixes for the following bugs found during debugging:
MDEV-40224,MDEV-40225,MDEV-40226,MDEV-40227,MDEV-40240,
MDEV-40285,MDEV-40288,MDEV-40315,MDEV-40318,MDEV-40890,MDEV-40900,
MDEV-40901,MDEV-40913,MDEV-40914,MDEV-41013,MDEV-41015,MDEV-41019
Assisted-by: Claude - reviews and minor clean-ups
MDEV-39519 added MySQL 8 compatibility to `mariadb-dump --dump-slave`
by attempting `SHOW REPLICA STATUS` first and then `SHOW SLAVE STATUS`.
This conflicted with MDEV-37146, where `mariadb-dump --dump-slave`
queries either `SELECT … FROM information_schema.SLAVE_STATUS`
or `SHOW ALL SLAVES STATUS` depending on the server version.
This commit merges MDEV-37146 and MDEV-39519:
* Use MDEV-39519’s strategy based on syntax error handling.
* Use MDEV-37146’s preference order:
1. `SELECT … FROM information_schema.SLAVE_STATUS`
2. `SHOW ALL SLAVES STATUS`
3. `SHOW REPLICA STATUS` (for MySQL compatibility _only_)
* Send `STOP`/`START REPLICA SQL_THREAD FOR CHANNEL '…'`
commands for both MariaDB 10.7+ and MySQL.
* Refactor column indices to variables set when a query succeeds.
* Partially revert MDEV-37146’s removal of `--dump-slave`’s support for
pre-GTID & pre-multi-source, but tailored for MySQL compatibility;
coverage for MariaDB pre-10.0 is not fully restored.
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
log_t::set_archive(archive=false): Ensure that both the latest checkpoint
and the latest log record (which has possibly not been written out yet)
will carry the log_sys.get_sequence_bit(lsn)==1, to guarantee a successful
recovery after the switch to the innodb_log_archive=OFF format.
Tested by: Matthias Leich
Reviewed by: Thirunarayanan Balathandayuthapani
* keep `vec_len >= subdist_part * 2` logic in one place only
* keep "distance-greater-than" mode logic in one place only
* simplify VECTOR_DIMENSIONS (no need to have a special ctx->vec_len
path if the other one always works)
* new plugin = maturity beta
* remove redundant casts, etc
* moved vector_indexes_fields_enum to the global scope to use it
for setting schema->idx_field1/schema->idx_field2
* open the hlindex graph table, if needed, otherwise most values
are unknown unless a user did vector search before
* added TABLE_CATALOG column
* remove CACHE_OVERFLOWS column, doesn't work as implemented,
the fix is complex and isn't worth it
* add privilege checks (MDEV-40793)
in the test:
* prefer query_vertical for readability
* select all columns at least once
* select INDEX_SIZE even if engine-dependent, use rdiff files
* test how get_all_tables only open one specific table, and
even only .frm file, if possible
Create a GitHub action into the MariaDB server tree to build the Markdown
documentation using the cmake target provided by MDEV-39718.
This workflow generates API documentation for the plugin using a
Docker container packed with all the extra doxygen/moxygen tools.
Fixed some more doxygen problems.
Moved the repo to ghcr.io/mariadb/mariadb-doc-gen:latest.
DENY ... TO PUBLIC denies everyone, including whoever tries to revoke
it, via the "deny wins" merge at every scope (global, db, table, column,
routine). Allow REVOKE DENY ... FROM PUBLIC when the revoker can UPDATE
mysql.global_priv (same as hand-editing).
Assisted-by: Claude:claude-5-sonnet
Move the std::function callbacks into the members instead of copying
them, and pass the captureless FTS comparators as plain functions
instead of lambdas. This is to avoid GCC -Wmaybe-uninitialized false
positive in RecordCallback
deferred_spaces.deferred_dblwr(): Skip newly created tablespaces
to avoid a bogus invocation of fil_space_free().
fil_name_process(): Simplify the logic. If no matching tablespace is
found but file_name_t::create_lsn had been set in response to parsing
a FILE_CREATE record, try to apply FILE_RENAME to deferred_spaces.
log_parse_file(): Parse each FILE_ record only once. In multi-batch
recovery, there may be redundant calls that would break the logic of
fil_name_process().
fil_delete_apply(): Wrappers for fil_space_free(). When recovering
a log in innodb_log_archive=ON format, we must apply FILE_DELETE
records in order to avoid a future clash with FILE_CREATE or FILE_RENAME.
log_t::write_checkpoint(): Extend the the correct file.
log_t::archived_mmap_switch_prepare(): Tolerate a near-concurrent
buf_flush_archive_create() from the buf_flush_page_cleaner() thread.
log_t::set_archive(): When a checkpoint needs to be forced, set a
minimum target that guarantees progress, instead of flushing
the entire buffer pool. Remove some duplicated reads of
last_checkpoint_lsn.
When switching innodb_log_archive from OFF to ON,
accurately remember whether any log records may have been
written with get_sequence_bit() == 0, to ensure that a
checkpoint will be waited for on a subsequent switch
from ON to OFF.
log_t::set_recovered(): If innodb_log_archive=ON, only make a
future set_archive(false) trigger a checkpoint if we may have
recovered a server that was killed between set_archive(true)
and write_checkpoint().
MDEV-32570 added fragmentation of large row events. When binlog
compression is enabled (log_bin_compress=ON), the compression is
bypassed, and these binary log events remain as regular row events, when
they should be compressed row events.
This is because the compression happens during the Log_event::write()
function, whereas row data fragmentation happens before
Log_event::write() is called. The Rows_log_event super-class stores the
row data buffers to be written to disk. The regular Rows_log_event
sub-classes's implementations of Log_event::write() write this data
as-is. The compressed sub-classes's implementation of ::write()
over-write these buffers with the compressed rows data before writing to
disk.
To fragment a large row event, the server fragments the Rows_log_event's
row data buffers, to be written by multiple Partial_rows_log_event's,
and each Partial_rows_log_event::write() call writes its portion of the
row data buffer to disk. However, this happens before compression ever
has a chance to take place, and thereby, an event that should be
compressed, never is.
MDEV-39762 added event structure validation for compressed events, and
discovered that these events carry the compressed event type, but are
not actually compressed. Event validation thereby fails.
This patch adds a workaround to override the event type of the
fragmented row events to be regular row events, to be consistent with
the actual on-disk content. The underlying problem still needs to be
addressed though, and is tracked by MDEV-40851.
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
