In InnoDB, an INSERT will not create an explicit lock object. Instead,
the inserted record is initially implicitly locked by the transaction
that wrote its trx_t::id to the hidden system column DB_TRX_ID.
(Other transactions would check if DB_TRX_ID is referring to a
transaction that has not been committed.)
If a record was inserted in the current transaction, it would be
implicitly locked by that transaction. Only if some other transaction
is requesting access to the record, the implicit lock should be
converted to an explicit one, so that the waits-for graph can be
constructed for detecting deadlocks and lock wait timeouts.
Before this fix, InnoDB would convert implicit locks to
explicit ones, even if no conflict exists.
lock_rec_convert_impl_to_expl(): Return whether caller_trx
already holds an explicit lock that covers the record.
row_vers_impl_x_locked_low(): Avoid a lookup if the record matches
caller_trx->id.
lock_trx_has_expl_x_lock(): Renamed from lock_trx_has_rec_x_lock().
row_upd_clust_step(): In a debug assertion, check for implicit lock
before invoking lock_trx_has_expl_x_lock().
rw_trx_hash_t::find(): Make do_ref_count a mandatory parameter.
Assert that trx_id is not 0 (the caller should check it).
trx_sys_t::is_registered(): Only invoke find() if id != 0.
trx_sys_t::find(): Add the optional parameter do_ref_count.
lock_rec_queue_validate(): Avoid lookup for trx_id == 0.
For some reason, some of these suppressions would fail to suppress
when the code is compiled with clang 6.0, Debug and -DWITH_ASAN=ON.
Possibly it is related to the number of .* or the length of the
regular expression strings.
NULL values when there is no DEFAULT
- Merged the alter_non_null test case to alter_not_null test case.
Renamed the alter_non_null_debug to alter_not_null_debug test case
NULL values when there is no DEFAULT
Copy and inplace algorithm works similarly for
NULL to NOT NULL conversion for the following cases:
(1) strict sql mode - Should give error.
(2) non-strict sql mode - Should give warnings alone
(3) alter ignore table command. - Should give warnings alone.
The following conditions will decide the query cache retrieval or
storing inside innodb:
(1) There should not be any locks on the table.
(2) Some other trx shouldn't invalidated the cache before the
transaction started.
(3) Read view shouldn't exist. If exists then the view
low_limit_id should be greater than or equal to the transaction that
invalidates the cache for the particular table.
For read-only transaction: should satisfy the above (1) and (3)
For read-write transaction: should satisfy the above (1), (2), (3).
- Changed the variable from query_cache_inv_id to query_cache_inv_trx_id.
- Moved the function row_search_check_if_query_cache_permitted from
row0sel.h and made it as static function in ha_innodb.cc
fil_page_decompress(): Replaces fil_decompress_page().
Allow the caller detect errors. Remove
duplicated code. Use the "safe" instead of "fast" variants of
decompression routines.
fil_page_compress(): Replaces fil_compress_page().
The length of the input buffer always was srv_page_size (innodb_page_size).
Remove printouts, and remove the fil_space_t* parameter.
buf_tmp_buffer_t::reserved: Make private; the accessors acquire()
and release() will use atomic memory access.
buf_pool_reserve_tmp_slot(): Make static. Remove the second parameter.
Do not acquire any mutex. Remove the allocation of the buffers.
buf_tmp_reserve_crypt_buf(), buf_tmp_reserve_compression_buf():
Refactored away from buf_pool_reserve_tmp_slot().
buf_page_decrypt_after_read(): Make static, and simplify the logic.
Use the encryption buffer also for decompressing.
buf_page_io_complete(), buf_dblwr_process(): Check more failures.
fil_space_encrypt(): Simplify the debug checks.
fil_space_t::printed_compression_failure: Remove.
fil_get_compression_alg_name(): Remove.
fil_iterate(): Allocate a buffer for compression and decompression
only once, instead of allocating and freeing it for every page
that uses compression, during IMPORT TABLESPACE. Also, validate the
page checksum before decryption, and reduce the scope of some variables.
fil_page_is_index_page(), fil_page_is_lzo_compressed(): Remove (unused).
AbstractCallback::operator()(): Remove the parameter 'offset'.
The check for it in FetchIndexRootPages::operator() was basically
redundant and dead code since the previous refactoring.
fil_page_decompress(): Replaces fil_decompress_page().
Allow the caller detect errors. Remove
duplicated code. Use the "safe" instead of "fast" variants of
decompression routines.
fil_page_compress(): Replaces fil_compress_page().
The length of the input buffer always was srv_page_size (innodb_page_size).
Remove printouts, and remove the fil_space_t* parameter.
buf_tmp_buffer_t::reserved: Make private; the accessors acquire()
and release() will use atomic memory access.
buf_pool_reserve_tmp_slot(): Make static. Remove the second parameter.
Do not acquire any mutex. Remove the allocation of the buffers.
buf_tmp_reserve_crypt_buf(), buf_tmp_reserve_compression_buf():
Refactored away from buf_pool_reserve_tmp_slot().
buf_page_decrypt_after_read(): Make static, and simplify the logic.
Use the encryption buffer also for decompressing.
buf_page_io_complete(), buf_dblwr_process(): Check more failures.
fil_space_encrypt(): Simplify the debug checks.
fil_space_t::printed_compression_failure: Remove.
fil_get_compression_alg_name(): Remove.
fil_iterate(): Allocate a buffer for compression and decompression
only once, instead of allocating and freeing it for every page
that uses compression, during IMPORT TABLESPACE.
fil_node_get_space_id(), fil_page_is_index_page(),
fil_page_is_lzo_compressed(): Remove (unused code).
When attempting to rename a table to a non-existing database,
InnoDB would misleadingly report "OS error 71" when in fact the
error code is InnoDB's own (OS_FILE_NOT_FOUND), and not report
both pathnames. Errors on rename could occur due to reasons
connected to either pathname.
os_file_handle_rename_error(): New function, to report errors in
renaming files.
When attempting to rename a table to a non-existing database,
InnoDB would misleadingly report "OS error 71" when in fact the
error code is InnoDB's own (OS_FILE_NOT_FOUND), and not report
both pathnames. Errors on rename could occur due to reasons
connected to either pathname.
os_file_handle_rename_error(): New function, to report errors in
renaming files.
ha_innobase::optimize(): If both innodb_defragment and
innodb_optimize_fulltext_only are at their default settings (OFF),
fall back to ALTER TABLE. Else process one or both options.
When the transaction isolation level is SERIALIZABLE, or when
a locking read is performed in the REPEATABLE READ isolation level,
InnoDB must lock delete-marked records in order to prevent another
transaction from inserting something.
However, at READ UNCOMMITTED or READ COMMITTED isolation level or
when the parameter innodb_locks_unsafe_for_binlog is set, the
repeatability of the reads does not matter, and there is no need
to lock any records.
row_search_for_mysql(): Skip locks on delete-marked committed records
upfront, instead of invoking row_unlock_for_mysql() afterwards.
The unlocking never worked for secondary index records.
MDEV-9931 introduced a counter for keeping track of reads of the
first page of InnoDB data files, because the original implementation
of data-at-rest-encryption for InnoDB introduced new code paths for
reading the pages.
Ultimately, the extra reads of the first page were removed, and
the encryption subsystem will be initialized whenever we first read
the first page of each data file, in fil_node_open_file(). It should not
be that interesting to observe how many times an InnoDB data file was
opened for the first time.
InnoDB does not allow FOREIGN KEY constraints to exist for TEMPORARY TABLE.
InnoDB introduced a dedicated tablespace for temporary tables, and actually
stopped creating persistent metadata and data for temporary tables.
row_table_add_foreign_constraints(): Do not create a persistent
transaction.
dict_create_foreign_constraints_low(): Add the persistent transaction to
the update the foreign key relation in dictionary.
dict_create_foreign_constraints_low(): Remove a duplicated check for
partitioned tables.
Problem:
We keep pinning pages in dict_stats_analyze_index_below_cur(),
but doesn't release these pages. When we have a relative small
buffer pool size, and big innodb_stats_persistent_sample_pages,
there will be no free pages for use.
Solution:
Use a separate mtr in dict_stats_analyze_index_below_cur(),
and commit mtr before return.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
RB: 11362
In debug builds of MySQL, there is an configuration variable
that allows an InnoDB log checkpoint to be initiated:
SET GLOBAL innodb_log_checkpoint_now=ON;
Setting this variable while a table-rebuilding ALTER TABLE is executing
may result in an infinite loop.
checkpoint_now_set(): Account for log_sys->append_on_checkpoint->size().
Note that this function contains race conditions, because it is accessing
fields of log_sys without holding log_sys->mutex. We think that this is
acceptable, because this variable only exists for debugging purposes, in
debug builds of MySQL.
RB: 9947
Reviewed-by: Sunny Bains <sunny.bains@oracle.com>
Problem:
The function row_build_index_entry_low() takes a dtuple_t object ('row') and
dict_index_t object ('index') as input and returns a new dtuple_t object
('entry') as output. The dtuple_t object 'row' that is given as input might
have been constructed from a different dict_index_t object (!= index). So
when accessing the externally stored data of the given 'row' we need to make
use of the correct index object.
Solution:
Store the page size information in the associated row_ext_t object.
rb#6086 approved by Vasil and Jimmy.
Problem:
Suppose there are two tables in the database related through a foreign key
constraint - the parent table and the child table. Suppose there is an ALTER
TABLE on the child table which gets interrupted in such a way that the child
table is not available any more.
After crash recovery, an ALTER TABLE on the parent table identifies that its
foreign keys cannot be loaded. This results in an error and a debug assert.
Solution:
Remove the debug assert and change error to a warning.
rb#8658 approved by Marko.
The server crashes on a SELECT because of space id mismatch. The
mismatch happens if the server crashes during an ALTER TABLE.
There are actually two cases of inconsistency, and three fixes needed
for the InnoDB problems.
We have dictionary data (tablespace or table name) in 3 places:
(a) The *.frm file is for the old table definition.
(b) The InnoDB data dictionary is for the new table definition.
(c) The file system did not rename the tablespace files yet.
In this fix, we will not care if the *.frm file is in sync with the
InnoDB data dictionary and file system. We will concentrate on the
mismatch between (b) and (c).
Two scenarios have been mentioned in this bug report. The simpler one
first:
1. The changes to SYS_TABLES were committed, and MLOG_FILE_RENAME2
records were written in a single mini-transaction commit.
The files were not yet renamed in the file system.
2a. The server is killed, without making a log checkpoint.
3a. The server refuses to start up, because replaying MLOG_FILE_RENAME2
fails.
I failed to repeat this myself. I repeated step 3a with a saved
dataset. The problem seems to be that MLOG_FILE_RENAME2 replay is
incorrectly being skipped when there is no page-redo log or
MLOG_FILE_NAME record for the old name of the tablespace.
FIX#1: Recover the id-to-name mapping also from MLOG_FILE_RENAME2
records when scanning the redo log. It is not necessary to write
MLOG_FILE_NAME records in addition to MLOG_FILE_RENAME2 records for
renaming tablespace files.
The scenario in the original Description involves a log checkpoint:
1. The changes to SYS_TABLES were committed, and MLOG_FILE_RENAME2
records were written in a single mini-transaction commit.
2. A log checkpoint and a server kill was injected.
3. Crash recovery will see no records (other than the MLOG_CHECKPOINT).
4. dict_check_tablespaces_and_store_max_id() will emit a message about
a non-found table #sql-ib22*.
5. A mismatch is triggering the assertion failure.
In my test, at step 4 the SYS_TABLES root page (0:8) contains these 3
records right before the page supremum:
* delete-marked (committed) name=#sql-ib21* record, with space=10.
* name=#sql-ib22*, space=9.
* name=t1, space=10.
space=10 is the rebuilt table (#sql-ib21*.ibd in the file system).
space=9 is the old table (t1.ibd in the file system).
The function dict_check_tablespaces_and_store_max_id() will enter
t1.ibd with space_id=10 into the fil_system cache without noticing
that t1.ibd contains space_id=9, because it invokes
fil_open_single_table_tablespace() with validate=false.
In MySQL 5.6, the space_id from all *.ibd files are being read when
the redo log checkpoint LSN disagrees with the FIL_PAGE_FILE_FLUSH_LSN
in the system tablespace. This field is only updated during a clean
shutdown, after performing the final log checkpoint.
FIX#2: dict_check_tablespaces_and_store_max_id() should pass
validate=true to fil_open_single_table_tablespace() when a non-clean
shutdown is detected, forcing the first page of each *.ibd file to be
read. (We do not want to slow down startup after a normal shutdown.)
With FIX#2, the SELECT would fail to find the table. This would
introduce a regression, because before WL#7142, a copy of the table
was accessible after recovery.
FIX#3: Maintain a list of MLOG_FILE_RENAME2 records that have been
written to the redo log, but not performed yet in the file system.
When performing a checkpoint, re-emit these records to the redo
log. In this way, a mismatch between (b) and (c) should be impossible.
fil_name_process(): Refactored from fil_name_parse(). Adds an item to
the id-to-filename mapping.
fil_name_parse(): Parses and applies a MLOG_FILE_NAME,
MLOG_FILE_DELETE or MLOG_FILE_RENAME2 record. This implements FIX#1.
fil_name_write_rename(): A wrapper function for writing
MLOG_FILE_RENAME2 records.
fil_op_replay_rename(): Apply MLOG_FILE_RENAME2 records. Replaces
fil_op_log_parse_or_replay(), whose logic was moved to fil_name_parse().
fil_tablespace_exists_in_mem(): Return fil_space_t* instead of bool.
dict_check_tablespaces_and_store_max_id(): Add the parameter
"validate" to implement FIX#2.
log_sys->append_on_checkpoint: Extra log records to append in case of
a checkpoint. Needed for FIX#3.
log_append_on_checkpoint(): New function, to update
log_sys->append_on_checkpoint.
mtr_write_log(): New function, to append mtr_buf_t to the redo log.
fil_names_clear(): Append the data from log_sys->append_on_checkpoint
if needed.
ha_innobase::commit_inplace_alter_table(): Add any MLOG_FILE_RENAME2
records to log_sys->append_on_checkpoint(), and remove them once the
files have been renamed in the file system.
mtr_buf_copy_t: A helper functor for copying a mini-transaction log.
rb#6282 approved by Jimmy Yang
