- This commit is different from 10.6 commit c438284863.
Due to Commit 045757af4c (MDEV-24621),
InnoDB does buffer and pre-sort the records for each index, and build
the indexes one page at a time.
Multiple large insert ignore statment aborts the server during bulk
insert operation. Problem is that InnoDB merge record exceeds
the page size. To avoid this scenario, InnoDB should catch
too big record while buffering the insert operation itself.
row_merge_buf_encode(): returns length of the encoded index record
row_merge_buf_write(): Catches the DB_TOO_BIG_RECORD earlier and
returns error
- InnoDB bulk insert operation aborts the server for redundant
table. InnoDB miscalculates the record size in temporary file
for the redundant table. CHAR in redundant row format table
always fixed length, but in temporary file, it is variable-length
for variable-length character sets.
In MariaDB, we have a confusing problem where:
* The transaction_isolation option can be set in a configuration file, but it cannot be set dynamically.
* The tx_isolation system variable can be set dynamically, but it cannot be set in a configuration file.
Therefore, we have two different names for the same thing in different contexts. This is needlessly confusing, and it complicates the documentation. The same thing applys for transaction_read_only.
MySQL 5.7 solved this problem by making them into system variables. https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-20.html
This commit takes a similar approach by adding new system variables and marking the original ones as deprecated. This commit also resolves some legacy problems related to SET STATEMENT and transaction_isolation.
Problem:
========
- InnoDB replace statement returns can't find record as result during
bulk insert operation. InnoDB returns DB_END_OF_INDEX blindly when
bulk transaction is visible to current transaction even though
the search tuple is inserted as a part of current replace statement.
Solution:
=========
row_search_mvcc(): InnoDB should allow the transaction to read
all the rows when innodb intends to do any locking on the
record even though bulk insert transaction changes are
visible to the current transaction
- InnoDB does rollback the whole transaction and discards the
savepoint when there is a failure happens during bulk
insert operation. When server request to release the savepoint,
InnoDB should return DB_SUCCESS when it deals with bulk
insert operation
This bug was previously fixed in 10.6.11 by:
MDEV-28327 InnoDB persistent statistics fail to update after bulk insert
Adding MTR tests only.
Also, fixing the old test for MDEV-28327 to make "mtr" reliably pass
with/without --mysqld=--innodb-stats-persistent=0, and with different page sizes,
as suggested by Marko.
- Background statistics thread should keep the table in the
statistics queue itself when the table under bulk insert operation
dict_stats_analyze_index(): Set the maximum value for index_stats_t
if the table is in bulk operation
dict_stats_update(), dict_stats_update_transient_for_index(),
dict_stats_update_transient(): Returns DB_SUCCESS_LOCKED_REC
if the table under bulk insert operation
dict_stats_process_entry_from_recalc_pool(): Add the table
back to recalc pool if the table under bulk insert operation
- During bulk insert, server detects the error and does rollback
operation. At that time, InnoDB fails to clean up the bulk insert
buffer
trx_t::rollback_finish(): Removed the clean up of modified tables
from transaction
trx_t::commit_cleanup(): Free the bulk buffer for bulk insert operation
trx_t::commit(): Check whether modified table wasn't dropped only
for non-dictionary transaction
- Change partition does undo logging of all rows unnecessarily and
it invokes bulk insert during DDL. Better to avoid the logging of undo
records during copy of the parititon.
If transaction does bulk insert and disables the foreign_key_check
then InnoDB fails with the assert failure. InnoDB has strict
assertion that check_foreigns and unique_secondary_check
should be enabled if the transaction does bulk insert
in innodb_prepare_commit_versioned().
- Trigger statement initiate update statement after bulk insert
operation and leads to disable of bulk operation. During commit,
InnoDB expects transaction to be in bulk insert mode before
applying the bulk insert operation. InnoDB transaction should
apply all bulk insert operation before disabling bulk insert
operation.
- InnoDB bulk insert operation fails to rollback when it detect
DB_DUPLICATE_KEY error. It leads to orphaned records in primary
indexes. Consecutive update/delete operation assumes that record
should exist in secondary index and it leads to failure.
- After MDEV-24621, InnoDB does buffer the insert bulk operation
for all indexes expect spatial one. But it leads to search the
primary key lookup and it leads to failure. So InnoDB should avoid
bulk insert when table has spatial index involved.
- InnoDB should check whether bulk transaction id set to its own
transaction id before start bulk insert operation.
- When bulk insert failure happens, InnoDB should set the error info
of the transaction.
InnoDB fails to add the tuple size which is greater than
innodb_sort_buffer_size. InnoDB should write the field
which are greater than 2000 bytes into the temporary file
and place the offset, length and make it as a new tuple.
InnoDB should buffer the newly created tuple without any
problem during bulk index
This issue is caused by MDEV-24621
(commit 045757af4c).
InnoDB tries to insert the number of rows into an empty spatial
index table, but it fails to apply the buffered insert.
InnoDB should insert into the spatial index directly instead of
buffering the insert operation.
InnoDB fails to apply buffered insert operation for
'mysql/transaction_registry' table during system versioning DDL.
To avoid this, DDL calls extra(HA_EXTRA_IGNORE_INSERT) to
inform the InnoDB for applying the buffered insert operation.
InnoDB fails to rollback the bulk insert buffered operation when
trx_mark_sql_stat_end() encounters the DB_DUPLICATE_KEY error.
In this case, check table fails with secondary index row count
mismatch. InnoDB gives the error "ER_ERROR_DURING_COMMIT"
while encountering the DB_DUPLICATE_KEY in trx_mark_sql_stat_end()
row_get_prebuilt_insert_row(): Remove some fallback code that had been
added in commit 8ea923f55b (MDEV-24818).
It seems that after all, statement boundaries are being reliably
indicated by ha_innobase::start_stmt() or
(for partitioned tables) ha_innobase::external_lock().
row_ins_clust_index_entry_low(): Do not enable bulk insert if
any record locks exist on the table. Bulk insert is assumed to
be covered only by an exclusive table lock, with no row-level
locking or undo logging.
In commit 8ea923f55b (MDEV-24818)
when we optimized multi-statement INSERT transactions into empty tables,
we would roll back the entire transaction on any error. But, we would
fail to invalidate any SAVEPOINT that had been requested in the past.
trx_t::savepoints_discard(): Renamed from trx_roll_savepoints_free().
row_mysql_handle_errors(): If we were in bulk insert, invoke
trx_t::savepoints_discard(). In this way, a future attempt of
ROLLBACK TO SAVEPOINT will return an error.
In commit 8ea923f55b (MDEV-24818)
when we optimized multi-statement INSERT into an empty table,
we would sometimes wrongly enable bulk insert into a table that
is actually already using row-level locking and undo logging.
trx_has_lock_x(): New predicate, to check if the transaction of
the current thread is holding an exclusive lock on a table.
trx_undo_report_row_operation(): Only invoke
trx_mod_table_time_t::start_bulk_insert() if
trx_has_lock_x() holds.
If the user "opts in" (as in the parent
commit 92b2a911e5),
we can optimize multiple INSERT statements to use table-level locking
and undo logging.
There will be a change of behavior:
CREATE TABLE t(a PRIMARY KEY) ENGINE=InnoDB;
SET foreign_key_checks=0, unique_checks=0;
BEGIN; INSERT INTO t SET a=1; INSERT INTO t SET a=1; COMMIT;
will end up with an empty table, because in case of an error,
the entire transaction will be rolled back, instead of rolling
back the failing statement. Previously, the second INSERT statement
would have been logged row by row, and only that second statement
would have been rolled back, leaving the first INSERT intact.
lock_table_x_unlock(), trx_mod_table_time_t::WAS_BULK: Remove.
Because we cannot really support statement rollback in this
optimized mode, we will not optimize the locking. The exclusive
table lock will be held until the end of the transaction.
In MDEV-515, we enabled an optimization where an insert into an
empty table will use table-level locking and undo logging.
This may break applications that expect row-level locking.
The SQL statements created by the mysqldump utility will include the
following:
SET unique_checks=0, foreign_key_checks=0;
We will use these flags to enable the table-level locked and logged
insert. Unless the parameters are set, INSERT will be executed in
the old way, with row-level undo logging and implicit record locks.
trx_t::commit_in_memory(): Invoke mod_tables.clear().
trx_free_at_shutdown(): Invoke mod_tables.clear() for transactions
that are discarded on shutdown.
Everywhere else, assert mod_tables.empty() on freed transaction objects.
This failure is caused by commit 43ca6059ca
(MDEV-24720). InnoDB fails to remove the ahi entries
during rollback of bulk insert operation. InnoDB should
remove the AHI entries of root page before reinitialising it.
Reviewed-by: Marko Mäkelä
InnoDB fails to remove the ahi entries during rollback
of bulk insert operation. InnoDB throws the error when
validates the ahi hash tables. InnoDB should remove
the ahi entries while freeing the segment only during
bulk index rollback operation.
Reviewed-by: Marko Mäkelä
In commit 3cef4f8f0f (MDEV-515)
we inadvertently broke CREATE TABLE...REPLACE SELECT statements
by wrongly disabling row-level undo logging.
select_create::prepare(): Only invoke extra(HA_EXTRA_BEGIN_ALTER_COPY)
if no special treatment of duplicates is needed.
