This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 13.1.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 12.3.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This commit rewrites the rpl_master_has_bug() mechanism to solve a problem
with invalid memory access. The rpl_master_has_bug() mechanism detects
certain bugs depending on the master version, and uses that to enable
specific work-arounds on the slave. The problem was that
rpl_master_has_bug() accessed Relay_log_info::description_event_for_exec
that is not valid to access from concurrent parallel replication worker
threads, only from the SQL driver thread. Thus it could use the wrong event
or access invalid/freed memory.
This patch instead computes a bitmask of detected bugs when the SQL driver
thread processes the format description event, and reads that bitmask with
an atomic load from the worker threads. The bitmask of bugs can only change
when the master restarts with a new version, and we do not replicate events
concurrently across a format description event from a master restart. Thus,
the bitmask is safe to read concurrently from the Relay_log_info object
without locking.
This also avoids an expensive match of each entry in the bug list against
the master server version done for every single call to
rpl_master_has_bug(), which could be quite expensive when done eg. per field
in row events as in Field_string::compatible_field_size().
Also remove redundant conditional in table_def::compatible_with().
Thanks to Andrei Elkin for the idea to safely read the bitmask
concurrently from the Relay_log_info.
Reviewed-by: Andrei Elkin <andrei.elkin@pp.inet.fi>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Problem was that check for partitioned tables was missing
because then partition implementing handlerton should be used in
condition instead.
Thanks to Roel Van de Paar <roel.vandepaar@mariadb.com> for
providing test case and fix candidate.
In MDEV-25644 vers_check_update() sets vers_write to false in case it
returns false. It is ok for UPDATE but is not correct for ODKU is bulk
insert requires vers_write on next tuple.
The fix return vers_write value back when vers_check_update() and
related vers_insert_history_row() are done in ODKU.
Problem:
REPLACE DELAYED into a system-versioned table read an uninitialized
lex->vers_conditions.delete_history in TABLE::delete_row().
Delayed_insert has its own THD, and hence its own LEX, of which the
constructor copies only a few fields from the parser LEX.
vers_conditions was not copied, so it stayed uninitialized. It is read
when a REPLACE resolves a duplicate on a versioned table.
Fix:
Copy vers_conditions from the parser LEX in the Delayed_insert
constructor.
Add new acquire_lock() overloads on MDL_context that combine MDL_request
initialization and lock acquisition into a single call, returning
MDL_ticket* directly. Convert applicable call sites across the codebase
to use the new ACQUIRE_LOCK macro.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
The patch introduces the OLD_VALUE() expression to reference the value
of a column before it was updated. The parser is extended to support
RETURNING and OLD_VALUE(), and RETURNING expressions are stored in a
separate returning_list in SELECT_LEX with independent wildcard tracking.
RETURNING is rejected for multi-table UPDATE.
During setup of RETURNING fields, THD::is_setting_returning is used
when resolving fields, particularly for updates through views.
When resolving view fields, Item_direct_view_ref may point to the
view's item_list, losing the information about whether the value
should be old or new. The original item in returning_list still
contains the correct is_old_value_reference flag, which is copied
back to the resolved item.
OLD_VALUE() is implemented by extending Item_field with a new
Item_old_field class. Item_field::set_field() initializes
Field::ptr_old to the corresponding location in record[1],
which stores the old row during UPDATE execution.
When sending result rows, Item_old_field::send() temporarily switches
the field pointer from record[0] (current row) to ptr_old so
OLD_VALUE() returns the value before the update.
The UPDATE execution path is modified to send a result set when
RETURNING is present instead of an OK packet.
Remove the unscoped #pragma GCC diagnostic ignored
"-Wdeprecated-declarations" from include/violite.h and replace all
sprintf/vsprintf calls with snprintf/vsnprintf across the codebase.
Where possible, pass actual buffer sizes as function parameters
instead of using hardcoded constants or recomputing strlen().
Notable API changes:
- get_date(): added to_len parameter for buffer size
- nice_time(), end_timer() in client/mysql.cc: added buff_size
- generate_new_name(): added name_size parameter
- calc_md5(): added buffer_size parameter (fixes off-by-one)
- MD5_HASH_TO_STRING macro: added _size parameter
- Added MYSQL_UDF_MAX_RESULT_LENGTH constant in mysql_com.h
Also fixed a pre-existing bug in sql/rpl_mi.cc where the ellipsis
"..." was written to the wrong buffer (dbuff instead of buff).
Vendored code is excluded: extra/readline, extra/wolfssl,
wsrep-lib, libmariadb, storage/mroonga/vendor, zlib.
Prior to this change, AFTER UPDATE did not fire, if data was unchanged.
Now this is fixed. Also added a test to verify that trigger behavior for
on-duplicate-update is the same as normal update, in case data does not
change.
When prepare_for_replace() fails, the code returned early via
DBUG_RETURN(1), bypassing the abort label which resets
thd->abort_on_warning to 0. This left the THD in a dirty state,
triggering the dispatch_command assertion at end of command dispatch.
Signed-off-by: Amodh Dhakal <amodhakal@gmail.com>
Problem:
=========
An assertion failure occurs in InnoDB during consecutive
ALTER TABLE operations using the COPY algorithm. The crash
happens during the table rename phase because the
source table (dict_table_t::n_ref_count) is non-zero, despite
the thread holding an exclusive Metadata Lock (MDL).
Reason:
========
When ALTER IGNORE TABLE is executed via the COPY algorithm,
it generates undo logs for every row inserted into the
intermediate table (e.g., #sql-alter-...). The background Purge
Thread, responsible for cleaning up these undo logs, attempts
to take an MDL on the table to prevent the table from
being dropped while in use.
Race condition:
==================
First ALTER: Creates #sql-alter-, copies data, and renames it to t1.
Purge Activation: The Purge thread picks up the undo logs from step 1.
It takes an MDL on the temporary name (#sql-alter-) and increments
the table's n_ref_count.
Identity Shift: InnoDB renames the physical table object to t1, but
the Purge thread still holds a reference to this object.
Second ALTER: Starts a new copy process. When it attempts to rename
the "new" t1 to a backup name, it checks if n_ref_count == 0.
Because the Purge thread is still "pinning" the object to
clean up logs from the first ALTER, the count is > 0,
triggering the assertion failure.
Solution:
========
ALTER IGNORE TABLE needs row-level undo logging to easily
roll back the last inserted row in case of duplicate key errors.
By discarding the last undo log record after inserting each row,
purge will not process any log records generated by
ALTER IGNORE TABLE, preventing unexpected access from the purge
subsystem during subsequent DDL operations.
Make skip_alter_undo (1-bit) to (2-bit enum)
to support different ALTER operation modes:
- NO_UNDO (0): Normal mode with standard undo logging
- SKIP_UNDO (1): ALTER mode that skips undo logging
- IGNORE_UNDO (2): ALTER IGNORE mode that rewrites undo blocks
trx_undo_report_row_operation(): Add ALTER IGNORE undo
rewriting logic. Store old undo record info before writing
new records for IGNORE_UNDO mode. Reset undo top_offset
to maintain only latest insert undo
row_ins_clust_index_entry_low(): Prevent ALTER IGNORE from
using bulk insert optimization. since it requires to write
undo log for each row operation that's incompatible with
existing bulk operation.
A new assertion added by MDEV-36290 is correct in trying to identify
default_fields that weren't correctly accounted in parse_vcol_defs.
It caught a case for INSERT DELAYED on tables using TIMESTAMP and
DATETIME types with ON UPDATE default values. That is, the insert
delayed logic opens the table in one thread, and then copies the
field info to another table in the user thread. This copy loses most
flag metadata for these fields though. In this case, the flag
ON_UPDATE_NOW_FLAG was dropped, and the parse_col_defs() logic was
unable to properly account for that default field, and the assertion
fired.
This patch fixes this by restoring the ON_UPDATE_NOW_FLAG during the
field copy. Also, a comment is added to Field::make_new_field() to
explain the state of the returned field.
Reviewed-by: Monty <monty@mariadb.com>
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
Reason:
======
During InnoDB DDL, statistics updation fails due to lock wait
timeout and calls push_warning_printf() to generate warnings
but then returns success, causing the SQL layer
to attempt calling set_ok_status() when the diagnostics area
is already set.
Solution:
=========
By temporarily setting abort_on_warning to false around operations
that prevents warning to error escalation and restore the original
setting after calling HA_EXTRA_END_ALTER_COPY for alter operation.
don't copy field default values and check constraints in
CREATE ... SELECT.
CREATE ... SELECT means a table is created from a *result set*
not from some other table.
For backward compatibility, though, let's keep copying constant
default values and the "compressed" attribute.
Caused by optimization done in 2e2b2a0469.
Cannot use lookup_handler in default branch of locate_dup_record() as
InnoDB update depends on positioned record and update is done in table
main handler.
The patch reverts some non-pure changes done by 2e2b2a0469 to
original logic from 72429cad. There was no long_unique_table condition
to init search on table->file, so we get into default branch with long
unique and table->file search uninitialized.
ha_rnd_init_with_error() on demand for HA_DUPLICATE_POS branch was
original logic as well.
More info: 2e2b2a0469 reverts 5e345281e3, but it seems to be OK as
MDEV-3888 test case passes. mysql-5.6.13 has the original code with
HA_WHOLE_KEY as well.
Timestamp-versioned row deletion was exposed to a collisional problem: if
current timestamp wasn't changed, then a sequence of row delete+insert could
get a duplication error. A row delete would find another conflicting history row
and return an error.
This is true both for REPLACE and DELETE statements, however in REPLACE, the
"optimized" path is usually taken, especially in the tests. There, delete+insert
is substituted for a single versioned row update. In the end, both paths end up
as ha_update_row + ha_write_row.
The solution is to handle a history collision somehow.
From the design perspective, the user shouldn't experience history rows loss,
unless there's a technical limitation.
To the contrary, trxid-based changes should never generate history for the same
transaction, see MDEV-15427.
If two operations on the same row happened too quickly, so that they happen at
the same timestamp, the history row shouldn't be lost. We can still write a
history row, though it'll have row_start == row_end.
We cannot store more than one such historical row, as this will violate the
unique constraint on row_end. So we will have to phisically delete the row if
the history row is already available.
In this commit:
1. Improve TABLE::delete_row to handle the history collision: if an update
results with a duplicate error, delete a row for real.
2. use TABLE::delete_row in a non-optimistic path of REPLACE, where the
system-versioned case now belongs entirely.
