The problem is the same as in MDEV-18166: columns in virtual field
expression are not marked for read, while the field itself does.
field->register_field_in_read_map() should be called for read-marking all
fields.
The test is reproduced only in 10.4+, however the fix is applicable to
10.2+.
cherry-pick commit: 1fff2398ef
MDEV-22530 post push fixes from 10.6.
Followup. If the KILL happens - report it as a failure,
don't eat it up silently. Note that this has to be done after `table_name`
is populated, so that the error message could show it.
Problem:
========
Aborting OPTIMIZE TABLE still logs in binary logs and replicates to the
Slave server. "Optimize table" command under execution, is killed by using
"Ctrl-C" as shown below.
MariaDB [test]> optimize table t2;
^CCtrl-C -- query killed. Continuing normally.
In spite of query execution being interrupted the query gets written to
binary log.
Analysis:
========
Admin command execution logic is not handling KILL command, hence it
ignores the KILL command and completes its execution.
Fix:
===
Check for thread killed notification, during admin command execution and
handle it. If thread kill occurs prior to any table modification the query
will not be written to binary log. If kill happens after at least one table
is modified then the query will be written to binary log. Ex: command in
execution is 'OPTIMIZE TABLE t1,t2' and the thread kill happens after t1
table is modified then 'OPTIMIZE TABLE t1,t2' will be written to binary log
as admin commands will not make the slave to diverge from master.
Problem:
=======
In slave_parallel_mode=optimistic configuration, when admin commands and
DML operation on the same table are scheduled simultaneously for execution,
it results in lock conflict and slave server either hangs due to
deadlock or goes down with an assert.
Analysis:
========
Admin commands OPTIMIZE, REPAIR and ANALYZE are written to binary log as
ordinary transactions. When 'slave_parallel_mode' is 'optimistic' DMLs are
allowed to run in parallel. But these locks are not detected by parallel
replication deadlock detection-and-handling mechanism. At times they result
in deadlock or assertion.
Fix:
===
Flag admin commands as DDL in Gtid_log_event at the time of writing to
binary log. Add a new bit EXECUTED_TABLE_ADMIN_CMD to
'm_unsafe_rollback_flags'. During 'mysql_admin_table' command execution it
accepts a list of tables to be processed and executes them in a loop. Upon
successful execution enable 'EXECUTED_TABLE_ADMIN_CMD' bit in
thd->transaction.stmt_unsafe_rollback_flags. Gtid_log_event constructor
will notice this flag and mark the current transaction with 'FL_DDL' flag.
Gtid_log_events marked as FL_DDL will not be scheduled parallel execution,
on the slave. They will execute in isolation to prevent deadlocks.
Note: Removed the call to 'trans_commit_implicit' from 'mysql_admin_table'
function as 'mysql_execute_command' will take care of invoking
'trans_commit_implicit'.
The reason for the failure is that
thd->mdl_context.release_transactional_locks()
was called after commit & rollback even in cases where the current
transaction is still active.
For 10.2, 10.3 and 10.4 the fix is simple:
- Replace all calls to thd->mdl_context.release_transactional_locks() with
thd->release_transactional_locks(). The thd function will only call
the mdl_context function if there are no active transactional locks.
In 10.6 we will better fix where we will change the return value for
some trans_xxx() functions to indicate if transaction did close the
transaction or not. This will avoid the need of the indirect call.
Other things:
- trans_xa_commit() and trans_xa_rollback() will automatically
call release_transactional_locks() if the transaction is closed.
- We can't do that for the other functions as the caller of many of these
are doing additional work (like close_thread_tables) before calling
release_transactional_locks().
- Added missing abort_result_set() and missing DBUG_RETURN in
select_create::send_eof()
- Fixed wrong indentation in injector::transaction::commit()
Do not collect EITS statistics for this statement:
ALTER TABLE t ANALYZE PARTITION p
EITS stats are currently global, not per-partition.
Collecting global stats when we are asked to process just one partition
causes issues for DBAs.
MDEV-21398 Deadlock (server hang) or assertion failure in
Diagnostics_area::set_error_status upon ALTER under lock
This failure could only happen if one locked the same table
multiple times and then did an ALTER TABLE on the table.
Major change is to change all instances of
table->m_needs_reopen= true;
to
table->mark_table_for_reopen();
The main fix for the problem was to ensure that we mark all
instances of the table in the locked_table_list and when we
reopen the tables, we first close all tables before reopening
and locking them.
Other things:
- Don't call thd->locked_tables_list.reopen_tables if there
are no tables marked for reopen. (performance)
Fixing a race condition while collecting the engine independent statistics.
Thread1>
1) start running "ANALYZE TABLE t PERISTENT FOR COLUMNS (..) INDEXES ($list)"
2) Walk through $list and save it in TABLE::keys_in_use_for_query
3) Close/re-open tables
Thread2>
1) Make some use of table t. This involves taking table t from
the table cache, and putting it back (with TABLE::keys_in_use_for_query reset to 0)
Thread1>
continue collecting EITS stats. Since TABLE::keys_in_use_for_query is set to 0 we
will not collect statistics for indexes in $list.
There was two separate problems:
- Aria pagecache didn't properly handle re-reading of blocks
that have given errors before (this triggered an assert)
- temporary tables that where opened several times where
not properly closed in ALTER, REPAIR or OPTIMIZE table
Other things
- Added a couple of asserts that will make it easier to
find problems like this in the future.
now we can afford it. Fix -Werror errors. Note:
* old gcc is bad at detecting uninit variables, disable it.
* time_t is int or long, cast it for printf's
MDEV-10130 Assertion `share->in_trans == 0' failed in storage/maria/ma_close.c
MDEV-10378 Assertion `trn' failed in virtual int ha_maria::start_stmt
The problem was that maria_handler->trn was not properly reset
at commit/rollback and ha_maria::exernal_lock() could get confused
because.
There was some old code in ha_maria::implicit_commit() that tried
to take care of this, but it was not bullet proof.
Fixed by adding list of all tables that is part of the maria transaction to
TRN.
A nice side effect was of the fix is that loops in
ha_maria::implict_commit() got to be much simpler.
Other things:
- Fixed a bug in mysql_admin_table() where argument open_for_modify
was wrongly reset for the next table in the chain
- rollback admin command also in case of fatal error.
- Split _ma_set_trn_for_table() to three version to simplify code
and debugging.
- Several new asserts to detect the original problem (that file was
not properly removed from trn before calling ma_close())
ALTER TABLE ... ADD PARTITION modifies the open TABLE structure,
and sets table->need_reopen=1 to reset these modifications
in case of an error.
But under LOCK TABLES the table isn't get reopened, despite need_reopen.
Fixed by reopening need_reopen tables under LOCK TABLE.
PREBUILT->TABLE->N_MYSQL_HANDLES_OPENED == 1
ANALYSIS:
=========
Adding unique index to a InnoDB table which is locked as
mutliple instances may trigger an InnoDB assert.
When we add a primary key or an unique index, we need to
drop the original table and rebuild all indexes. InnoDB
expects that only the instance of the table that is being
rebuilt, is open during the process. In the current
scenario we have opened multiple instances of the table.
This triggers an assert during table rebuild.
'Locked_tables_list' encapsulates a list of all
instances of tables locked by LOCK TABLES statement.
FIX:
===
We are now temporarily closing all the instances of the
table except the one which is being altered and later
reopen them via Locked_tables_list::reopen_tables().
remove HA_EXTRA_PREPARE_FOR_RENAME - neither OPTIMIZE nor REPAIR need it
(was introduced in b58e79566c when replacing remove_table_from_cache()
with wait_while_table_is_used() even though remove_table_from_cache()
did not have it).
- Fix win64 pointer truncation warnings
(usually coming from misusing 0x%lx and long cast in DBUG)
- Also fix printf-format warnings
Make the above mentioned warnings fatal.
- fix pthread_join on Windows to set return value.
Reexecution of prepared "ANALYZE TABLE merge_table, table" may miss to
reinitialize "table" for subsequent execution and trigger assertion failure.
This happens because MERGE engine may adjust table->next_global chain, which
gets cleared by close_thread_tables()/ha_myisammrg::detach_children() later.
Since reinitilization iterates next_global chain, it won't see tables following
merge table.
Fixed by appending saved next_global chain after merge children.
