Analysis:
When json is being parsed, the step decreases without a out-of-bound check
resulting in failure.
Fix:
Before decreasing the step, check if it will result into out of bound.
json_get_path_start() set p->last_step to p->steps - 1, creating a
pointer before the beginning of the steps[] array. This is undefined
behavior flagged by UBSAN as "index -1 out of bounds for type
json_path_step_t[32]".
Use NULL as the sentinel value instead, and check for NULL in
json_get_path_next() rather than comparing against p->steps.
Backport to 10.6 of commit 4f546897b8.
Additional test case from Rucha Deodhar
Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
Signed-off-by: Varun Deep Saini <deepsainivarun@gmail.com>
* use max_length=640
* also fix mroonga_highlight_html, mroonga_normalize, mroonga_snippet_html
* remove disable_cursor_protocol from all mroonga tests
* remove redundant checks for `len` in `get_param_length()`
* validate return value `length` in `get_param_length()`
* fix all `Item_param::set_param_XXX()` methods to never read past `len`
note that all `Item_param::set_param_XXX()` methods correctly handle
zero-length values.
Stop parsing index flags after reporting an invalid flag in
mrn_parse_grn_index_column_flags().
Otherwise, an invalid index comment such as flags "COMPRESS_ZSTD" can
make CREATE TABLE loop forever while emitting warnings. Add a regression
test for this case and check that the invalid flag is reported once and
the default index flags are used.
Problem:
========
Purge threads computing virtual columns could crash due to:
1. Stale TABLE* pointers when tables are flushed/rebuilt during purge
2. open_purge_table() called close_thread_tables() on failure, making
MDL tickets invalid before purge could release them
3. Purge coordinator opened TABLE* but workers accessed it with wrong
TABLE->in_use
4. No retry mechanism when open_purge_table() failed due to concurrent
FLUSH TABLES, BACKUP STAGE, or ALTER TABLE operations
Solution:
========
1. Removed close_thread_tables() from open_purge_table(). Purge
coordinator thread should close explicitly in close_and_reopen()
2. Added retry logic: when open_purge_table() returns NULL due to
table flush/rebuild, set must_wait() flag and retry in
close_and_reopen()
3. Update close_and_reopen() with purge_table parameter to
close the failed table. Pass it to trx_purge_close_tables()
for proper cleanup
4. Properly set and reset TABLE::in_use during purge operations:
- Set to coordinator_thd in row_purge_parse_undo_rec() when opening
- Reset in trx_purge_close_table() when closing
5. The auto_increment initialization now happens unconditionally for
purge threads , ensuring the auto_increment counter is always
properly initialized when purge opens tables with virtual columns
Problem:
========
A single InnoDB purge worker thread can process undo logs from different
tables within the same batch. But get_purge_table(), open_purge_table()
incorrectly assumes that a 1:1 relationship between a purge worker thread
and a table within a single batch. Based on this wrong assumtion,
InnoDB attempts to reuse TABLE objects cached in thd->open_tables for
virtual column computation.
1) Purge worker opens Table A and caches the TABLE pointer in thd->open_tables.
2) Same purge worker moves to Table B in the same batch, get_purge_table()
retrieves the cached pointer for Table A instead of opening Table B.
3) Because innobase::open() is ignored for Table B, the virtual column
template is never initialized.
4) virtual column computation for Table B aborts the server
Solution:
========
- Introduced purge_table class which has the following
purge_table: Stores either TABLE* (for tables with indexed virtual
columns) or MDL_ticket* (for tables without) in a single union
using LSB as a flag.
For tables with indexed virtual columns: opens TABLE*, accesses
MDL_ticket* via TABLE->mdl_ticket
For tables without indexed virtual columns: stores only MDL_ticket*.
trx_purge_attach_undo_recs(): Coordinator opens both dict_table_t*
and TABLE* with proper MDL protection. Workers access cached
table pointers from purge_node_t->tables without opening
their own handles
purge_sys.coordinator_thd: Distinguish coordinator from workers
in cleanup logic. Skip innobase_reset_background_thd() for
coordinator thread to prevent premature table closure during
batch processing. Workers still call cleanup to release their
thread-local resources
trx_purge_close_tables():
Rewrite for purge coordinator thread
1) Close all dict_table_t* objects first
2) Call close_thread_tables() once for all TABLE* objects
3) Release MDL tickets last, after tables are closed
Added table->lock_mutex protection when reading (or) writing
vc_templ->mysql_table and mysql_table_query_id. Clear cached
TABLE* pointers before closing tables to prevent stale pointer
access
Declared open_purge_table() and close_thread_tables() in trx0purge.cc
Declared reset_thd() in row0purge.cc and dict0stats_bg.cc.
Removed innobase_reset_background_thd()
Problem:
When a prepared statement uses `DEFAULT` with sequence, on its
second or subsequent execution, server tries to allocate TABLE_LIST
object (used for adding sequence table to `internal_tables` list) on
statement's memory root. This fails with an assert because statement's
memory root is marked read-only after the first execution.
CREATE SEQUENCE s;
CREATE TABLE t (a INT DEFAULT(NEXTVAL(s)));
PREPARE stmt FROM "UPDATE t SET a = ?";
EXECUTE stmt USING 3; -- first execution, marks mem_root read-only
EXECUTE stmt USING DEFAULT; -- tries to allocate on second execution
Fix:
Temporarily clear the read-only flag on the statement memory root to
allow the allocation. Also adds a debug hook to verify allocation
happens not more than once per prepared statement and per internal
table added to the list.
introduce safe() wrapper for parameters that can later be interpolated
into command line.
Use as
- var="foo$BAR"
+ var="foo$(safe BAR)"
A parameter is safe, if it contains no spaces, single quotes or backticks
Problem:
InnoDB index corruption occurs when an index is renamed to a name that
differs only in case (e.g., 'b' to 'B'). The SQL layer uses
case-insensitive comparison and fails to recognize the change.
Fix:
Use case-sensitive comparison when matching index names during
ALTER TABLE to correctly identify and handle case changes.
Since ATM wsrep does not replicate any cascaded changes (because they are
not binlogged in the upstream code) we have to rely on foreign checks on
replicas to reproduce the cascade. Thus setting wsrep_slave_fk_checks=OFF
will result in inconsistency in any setup where there are cascading relations.
Mark this option as deprecated and make it a NOOP.
lock_clust_rec_read_check_and_lock(): Refine the check whether
the transaction that last modified the record is still active.
The only thing that should matter is whether we are allowed to see
the record. If the implicit lock holder transaction was active and
we succeeded in acquiring the lock, this means that the transaction
had been committed. We must return DB_RECORD_CHANGED (ER_CHECKREAD)
in that case. We want to avoid returning it before the lock wait,
in case the other transaction will be rolled back.
Thanks to Vadim Tkachenko of Percona for reporting this bug,
as well as Kyle Kingsbury for the broader testing that led to this
finding. Vadim's test case was simplified by me and the root cause
analyzed with https://rr-project.org and an additional patch that
added std::this_thread::yield() at the start of
trx_t::commit_persist(). An even better spot should have been
right after the call to trx_t::commit_state().
The addition to the test innodb.lock_isolation is based on the work
by Teemu Ollakka, which was essential for refining this fix.
Reviewed by: Thirunarayanan Balathandayuthapani
Remove the complexity of check_for_crashed_tables
by removing it all together. When check_for_crashed_tables
was written there was a desire to recover MyISAM tables.
With Aria being default from 10.4 for system tables,
and all non-MyISAM engines being able to do crash
recovery there isn't the need to autocheck as part
of a system service.
With check_for_crashed_tables removed there is no need
for a mailx package recommendation.
But doesn't package it. The devel-6 branch of columnstore
is in a low maintaince state. We can't readd aarch64.
To prevent our CI resources building aarch64 columnstore
on the 10.6 and 10.11 branches we adjust our autobake-deb.sh
to keep the disable there.
The version check to 10 is so that when this is merged to
11.4 it becomes no longer impacting.
In all supported Debian versions, which are all 1.69+,
this package only provided a library stub. The header
files where moved into libboost-dev when upstream removed
the need for a library.
cmake generator expressions are expanded *during the build step*.
If the file is created *during the configuration step* its name
cannot use generator expressions.
Analysis:
=========
1) Starts the CREATE TABLE ... SELECT and acquires an Intent Shared lock on
mysql.innodb_table_stats to read the rows. (Transaction: T1)
2) The table t1 is populated. As part of the post-copy phase
(introduced by MDEV-35163), InnoDB triggers the statistics update.
3) InnoDB opens a new internal transaction to update the statistics.
This transaction(T2) attempts to acquire an LOCK_X on the same
table in mysql.innodb_table_stats. T2 (x-lock) must wait for T1 to
commit or rollback because X-locks are incompatible with
IS locks from other trasnaction.
Solution:
========
Currently, Intermediate tables don't trigger dict_stats_update_if_needed(),
so they require explicit stats update via alter_stats_rebuild().
For CREATE...SELECT stats are updated automatically through the normal DML
path via dict_stats_update_if_needed()
Skip calling dict_stats_update() in alter_stats_rebuild() for
non-intermediate tables when using copy algorithm, since their statistics will
be updated automatically via dict_stats_update_if_needed().
