When checkpoint age goes beyond the sync flush threshold and
buf_flush_sync_lsn is set, page cleaner enters into "furious flush"
stage to aggressively flush dirty pages from flush list and pull
checkpoint LSN above safe margin. In this stage, page cleaner skips
doing LRU flush and eviction.
In 10.6, all other threads entirely rely on page cleaner to generate
free pages. If free pages get over while page cleaner is busy in
"furious flush" stage, a session thread could wait for free page in the
middle of a min-transaction(mtr) while holding latches on other pages.
It, in turn, can prevent page cleaner to flush such pages preventing
checkpoint LSN to move forward creating a deadlock situation. Even
otherwise, it could create a stall and hang like situation for large BP
with plenty of dirty pages to flush before the stage could finish.
Fix: During furious flush, check and evict LRU pages after each flush
iteration.
- Added a counter innodb_num_bulk_insert_operation in
INFORMATION_SCHEMA.GLOBAL_STATUS. This counter is incremented
whenever a InnoDB undergoes bulk insert operation.
- Change the innodb_instant_alter_column to atomic variable.
Problem:
========
- Partition update operation enables the bulk insert for the
transaction while moving the row between partitions. This leads
to debug assert failure while removing the row from one
of the partition.
Solution:
========
- Disallow the bulk insert operation for non-insert operation
of partition table.
In case of partition insert, InnoDB fails to end the bulk insert
for one of the partition. It leads to bulk insert operation for
the consecutive delete statement.
trx_t::bulk_insert_apply_for_table(): Irrespective of bulk insert
value, InnoDB should end the bulk insert for the table.
bulk_insert_apply_for_table(dict_table_t*)
This issue is caused by
commit 188c5da72a0057e4572b1d7e4efcd0c39332a839 (MDEV-32453).
trx_t::bulk_insert_apply_for_table(): Remove the assert
check_unique_secondary and check_foreigns. InnoDB can
apply the bulk insert operation even after disabling
the check_foreigns and check_unique_secondary variable.
- This issue is caused by commit 188c5da72a0057e4572b1d7e4efcd0c39332a839 (MDEV-32453).
InnoDB fails to end the bulk insert for the table after
applying the bulk insert operation. This leads to assertion
during commit process.
Reason:
=======
- InnoDB fails to apply the buffered insert operation if the
after insert trigger does change the same table. This behaviour
leads to empty table for the subsequent insert operation
and server abort.
Solution:
========
- InnoDB should apply buffered insert operation if
"after insert" trigger changes the same table.
- This commit is different from 10.6 commit c438284863db2ccba8a04437c941a5c8a2d9225b.
Due to Commit 045757af4c301757ba449269351cc27b1691a7d6 (MDEV-24621),
InnoDB does buffer and pre-sort the records for each index, and build
the indexes one page at a time.
Multiple large insert ignore statment aborts the server during bulk
insert operation. Problem is that InnoDB merge record exceeds
the page size. To avoid this scenario, InnoDB should catch
too big record while buffering the insert operation itself.
row_merge_buf_encode(): returns length of the encoded index record
row_merge_buf_write(): Catches the DB_TOO_BIG_RECORD earlier and
returns error
- InnoDB bulk insert operation aborts the server for redundant
table. InnoDB miscalculates the record size in temporary file
for the redundant table. CHAR in redundant row format table
always fixed length, but in temporary file, it is variable-length
for variable-length character sets.
In MariaDB, we have a confusing problem where:
* The transaction_isolation option can be set in a configuration file, but it cannot be set dynamically.
* The tx_isolation system variable can be set dynamically, but it cannot be set in a configuration file.
Therefore, we have two different names for the same thing in different contexts. This is needlessly confusing, and it complicates the documentation. The same thing applys for transaction_read_only.
MySQL 5.7 solved this problem by making them into system variables. https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-20.html
This commit takes a similar approach by adding new system variables and marking the original ones as deprecated. This commit also resolves some legacy problems related to SET STATEMENT and transaction_isolation.
Problem:
========
- InnoDB replace statement returns can't find record as result during
bulk insert operation. InnoDB returns DB_END_OF_INDEX blindly when
bulk transaction is visible to current transaction even though
the search tuple is inserted as a part of current replace statement.
Solution:
=========
row_search_mvcc(): InnoDB should allow the transaction to read
all the rows when innodb intends to do any locking on the
record even though bulk insert transaction changes are
visible to the current transaction
- InnoDB does rollback the whole transaction and discards the
savepoint when there is a failure happens during bulk
insert operation. When server request to release the savepoint,
InnoDB should return DB_SUCCESS when it deals with bulk
insert operation
This bug was previously fixed in 10.6.11 by:
MDEV-28327 InnoDB persistent statistics fail to update after bulk insert
Adding MTR tests only.
Also, fixing the old test for MDEV-28327 to make "mtr" reliably pass
with/without --mysqld=--innodb-stats-persistent=0, and with different page sizes,
as suggested by Marko.
- Background statistics thread should keep the table in the
statistics queue itself when the table under bulk insert operation
dict_stats_analyze_index(): Set the maximum value for index_stats_t
if the table is in bulk operation
dict_stats_update(), dict_stats_update_transient_for_index(),
dict_stats_update_transient(): Returns DB_SUCCESS_LOCKED_REC
if the table under bulk insert operation
dict_stats_process_entry_from_recalc_pool(): Add the table
back to recalc pool if the table under bulk insert operation
- During bulk insert, server detects the error and does rollback
operation. At that time, InnoDB fails to clean up the bulk insert
buffer
trx_t::rollback_finish(): Removed the clean up of modified tables
from transaction
trx_t::commit_cleanup(): Free the bulk buffer for bulk insert operation
trx_t::commit(): Check whether modified table wasn't dropped only
for non-dictionary transaction
- Change partition does undo logging of all rows unnecessarily and
it invokes bulk insert during DDL. Better to avoid the logging of undo
records during copy of the parititon.
If transaction does bulk insert and disables the foreign_key_check
then InnoDB fails with the assert failure. InnoDB has strict
assertion that check_foreigns and unique_secondary_check
should be enabled if the transaction does bulk insert
in innodb_prepare_commit_versioned().
- Trigger statement initiate update statement after bulk insert
operation and leads to disable of bulk operation. During commit,
InnoDB expects transaction to be in bulk insert mode before
applying the bulk insert operation. InnoDB transaction should
apply all bulk insert operation before disabling bulk insert
operation.
- InnoDB bulk insert operation fails to rollback when it detect
DB_DUPLICATE_KEY error. It leads to orphaned records in primary
indexes. Consecutive update/delete operation assumes that record
should exist in secondary index and it leads to failure.
- After MDEV-24621, InnoDB does buffer the insert bulk operation
for all indexes expect spatial one. But it leads to search the
primary key lookup and it leads to failure. So InnoDB should avoid
bulk insert when table has spatial index involved.
- InnoDB should check whether bulk transaction id set to its own
transaction id before start bulk insert operation.
- When bulk insert failure happens, InnoDB should set the error info
of the transaction.
InnoDB fails to add the tuple size which is greater than
innodb_sort_buffer_size. InnoDB should write the field
which are greater than 2000 bytes into the temporary file
and place the offset, length and make it as a new tuple.
InnoDB should buffer the newly created tuple without any
problem during bulk index
This issue is caused by MDEV-24621
(commit 045757af4c301757ba449269351cc27b1691a7d6).
InnoDB tries to insert the number of rows into an empty spatial
index table, but it fails to apply the buffered insert.
InnoDB should insert into the spatial index directly instead of
buffering the insert operation.
InnoDB fails to apply buffered insert operation for
'mysql/transaction_registry' table during system versioning DDL.
To avoid this, DDL calls extra(HA_EXTRA_IGNORE_INSERT) to
inform the InnoDB for applying the buffered insert operation.
InnoDB fails to rollback the bulk insert buffered operation when
trx_mark_sql_stat_end() encounters the DB_DUPLICATE_KEY error.
In this case, check table fails with secondary index row count
mismatch. InnoDB gives the error "ER_ERROR_DURING_COMMIT"
while encountering the DB_DUPLICATE_KEY in trx_mark_sql_stat_end()
row_get_prebuilt_insert_row(): Remove some fallback code that had been
added in commit 8ea923f55b7666a359ac2c54f6c10e8609d16846 (MDEV-24818).
It seems that after all, statement boundaries are being reliably
indicated by ha_innobase::start_stmt() or
(for partitioned tables) ha_innobase::external_lock().
row_ins_clust_index_entry_low(): Do not enable bulk insert if
any record locks exist on the table. Bulk insert is assumed to
be covered only by an exclusive table lock, with no row-level
locking or undo logging.
In commit 8ea923f55b7666a359ac2c54f6c10e8609d16846 (MDEV-24818)
when we optimized multi-statement INSERT transactions into empty tables,
we would roll back the entire transaction on any error. But, we would
fail to invalidate any SAVEPOINT that had been requested in the past.
trx_t::savepoints_discard(): Renamed from trx_roll_savepoints_free().
row_mysql_handle_errors(): If we were in bulk insert, invoke
trx_t::savepoints_discard(). In this way, a future attempt of
ROLLBACK TO SAVEPOINT will return an error.
