This assertion should have been relaxed when implementing the first part of
MDEV-15563: instant removal of NOT NULL attribute for ROW_FORMAT=REDUNDANT
tables.
For ROW_FORMAT=REDUNDANT, there is no bitmap of null columns;
the null flags are encoded in the end offset of each field.
We do not really care about the number of fields that can be NULL.
Ensure that the 'auxiliary transactions' that are there for
flushing the incomplete undo log of the to-be-recovered DDL
transactions are actually making modifications.
The error message modified.
Then the TABLE_SHARE::error_table_name() implementation taken from 10.3,
to be used as a name of the table in this message.
Innochecksum was being built as if WITH_INNODB_BUG_ENDIAN_CRC32:BOOL=OFF
had been specified.
Also, clean up tests:
innodb.innochecksum: Useless; superceded by innodb_zip.innochecksum.
innodb.innodb_zip_innochecksum: Remove; duplicated innodb_zip.innochecksum.
innodb.innodb_zip_innochecksum2: Remove; duplicated innodb_zip.innochecksum_2.
innodb.innodb_zip_innochecksum3: Remove; duplicated innodb_zip.innochecksum_3.
No test case was added. I tested manually by adding debug instrumentation
to both innochecksum and buf_page_is_checksum_valid_crc32() to make
innochecksum write the buggy crc32, and to get warnings for falling back
to the buggy checksum. Automating this would require that tests be
adjusted depending on the build options.
Allow instant changes of columns in ROW_FORMAT=REDUNDANT
from NOT NULL to NULL.
Later, this may be implemented for ROW_FORMAT=COMPACT or DYNAMIC,
but in that case any indexes on the table must be rebuilt.
dict_table_t::prepare_instant(): Add some debug assertions,
and relax a debug assertion so that the number of fields is
allowed not to change.
dict_index_t::instant_add_field(): Relax a debug assertion,
allowing a column to change from NOT NULL to NULL.
dict_table_t::instant_column(): Add debug assertions.
instant_alter_column_possible(): Allow ALTER_COLUMN_NULLABLE
when applicable.
innodb_insert_sys_columns(): Add the parameter bool update=false
to run UPDATE instead of INSERT.
innobase_instant_add_col(): Remove; let the only caller invoke
innodb_insert_sys_columns() directly.
innobase_instant_try(): Update the SYS_COLUMNS record if the
column is changed. Only convert the table to the instant ALTER TABLE
format if necessary. For ALTER_COLUMN_NULLABLE in ROW_FORMAT=REDUNDANT,
there is no data format change.
With innodb_default_row_format=redundant, InnoDB would crash when
using table options that are incompatible with ROW_FORMAT=REDUNDANT.
create_table_info_t::m_default_row_format: Cache the value of
innodb_default_row_format.
create_table_info_t::check_table_options(): Validate ROW_TYPE_DEFAULT
with m_default_row_format.
create_table_info_t::innobase_table_flags(): Use the
cached m_default_row_format.
create_table_info_t: Never read m_form->s->row_type.
Use m_create_info->row_type instead.
dict_tf_set(): Never set invalid flags for ROW_FORMAT=REDUNDANT.
ha_innobase::truncate(): Set info.row_type based on the ROW_FORMAT
of the current table.
Also, apply the MDEV-17957 changes to encrypted page checksums,
and remove error message output from the checksum function,
because these messages would be useless noise when mariabackup
is retrying reads of corrupted-looking pages, and not that
useful during normal server operation either.
The error messages in fil_space_verify_crypt_checksum()
should be refactored separately.
ha_innobase::truncate(): Because CREATE TEMPORARY TABLE
allows invalid table options when innodb_file_per_table=1,
do allow them also in TRUNCATE for temporary tables.
btr_node_ptr_max_size(): Treat CHAR(0) from SQL as a special case.
The InnoDB internal SQL parser maps the type "CHAR" to DATA_VARCHAR,
but MariaDB does allow CHAR(0) with an empty value, and does enforce
the length limitation.
btr_cur_pessimistic_insert(): Convert the metadata field of the metadata
record into BLOB before inserting, just like btr_cur_optimistic_insert()
does.
ha_innobase::prepare_inplace_alter_table(): check max column length for every
index in a table, not just added in this particular ALTER TABLE with ADD INDEX ones.
We failed to reset the dict_table_t::persistent_autoinc after
instantly dropping an AUTO_INCREMENT column, causing a bogus
call to row_parse_int() on a subsequent insert.
The result is updated here because by changing the defaults for use_stat_tables
EITs statistcs are providing more accurate statistics than INNODB statistics.
row_undo_ins_remove_clust_rec(): When rolling back the insert
of a metadata record, also reset the root page state within the
same mini-transaction.
dict_table_t::rollback_instant(): Do not allow index->n_core_fields
to be larger than index->n_fields. This logic exists in the 10.3
branch but was incorrectly removed as part of MDEV-15562 in 10.4.0.
- This is a regression of commit b26e603aeb. While dropping
the incompletely created table, InnoDB shouldn't consider that operation as non-atomic one.
This is a regression due to MDEV-17816.
When creating a table fails, we must roll back the dictionary
transaction. Because the rollback may rename tables, and because
InnoDB lacks proper undo logging for CREATE operations, we must
drop the incompletely created table before rolling back the
transaction, which could include a RENAME operation.
But, we must not blindly drop the table by name; after all,
the operation could have failed because another table by the
same name already existed.
create_table_info_t::m_drop_before_rollback: A flag that is set
if the table needs to be dropped before transaction rollback.
create_table_info_t::create_table(): Remove some duplicated
error handling.
ha_innobase::create(): On error, only drop the table if it was
actually created.
If a table had a KEY_BLOCK_SIZE attribute, but no ROW_FORMAT,
it would be created as ROW_FORMAT=COMPRESSED in InnoDB.
However, TRUNCATE TABLE would lose the KEY_BLOCK_SIZE attribute
and create the table with the innodb_default_row_format (DYNAMIC).
This is a regression that was introduced by MDEV-13564.
update_create_info_from_table(): Copy also KEY_BLOCK_SIZE.
The error handling in the MDEV-13564 TRUNCATE TABLE was broken
when an error occurred during table creation.
row_create_index_for_mysql(): Do not drop the table on error.
fts_create_one_common_table(), fts_create_one_index_table():
Do drop the table on error.
create_index(), create_table_info_t::create_table():
Let the caller handle the index creation errors.
ha_innobase::create(): If create_table_info_t::create_table()
fails, drop the incomplete table, roll back the transaction,
and finally return an error to the caller.
Several race conditions between MDEV-15562 instant ALTER TABLE and purge
were observed.
The most obvious race condition resulted in a reported assertion failure in
dict_index_t::instant_add_field(): instant.n_core_fields == n_core_fields
would not hold if the table was emptied by purge after the time
dict_table_t::prepare_instant() was called.
During purge, it can turn out that the table is logically empty, only
containing a metadata record. If the metadata record is of the type
created by MDEV-11369 instant ADD COLUMN, it can be removed and
dict_index_t::clear_instant_add() can be called. This will convert the
table to the canonical non-instant format. (If the metadata record is
of the MDEV-15562 type, then it can only be deleted if the table becomes
empty as the result of rollback of an instant ALTER TABLE operation.)
row_purge_remove_clust_if_poss_low(): Add a debug check that ensures
that purge can never remove a MDEV-15562 metadata record.
ha_innobase::open(): Add a comment about the necessity of rolling
back any recovered instant ALTER TABLE transaction on the table.
instant_metadata_lock(): An auxiliary function to acquire a page latch
on the metadata record, to prevent race conditions.
dict_table_t::prepare_instant(), dict_index_t::instant_add_field(),
dict_table_t::rollback_instant(), innobase_instant_try():
Invoke instant_metadata_lock() in order to prevent race conditions.
dict_index_t::instant_add_field(): Correct debug assertions.
The == was guaranteed by code in dict_table_t::prepare_instant()
that was introduced in MDEV-15562. Due to the race condition,
we could occasionally have <=, but never >= like the code was
after MDEV-11369.
ha_innobase_inplace_ctx::instant_column(): Wrapper for
dict_table_t::instant_column(). Add debug assertions.
There was a race condition between ALTER TABLE and purge.
If a table turns out to be logically empty when instant ALTER TABLE
is executing, we will convert the table to the canonical format,
to avoid overhead during subsequent accesses, and to allow the
data file to be imported into older versions of MariaDB.
It could happen that at the time the table is logically empty,
there still exists an undo log record for updating the hidden
metadata record for an earlier instant ALTER TABLE operation.
If the table was converted to the canonical format before
purge processes this undo log record, the undo log record
could be referring to index fields that no longer exist,
causing a crash.
To prevent the race condition, we must delete the old undo log records.
We do this lazily by assigning a new table ID, so that the table lookup
for the old undo log records will fail.
dict_table_t::reassign_id(): Reassign the table_id to
effectively lazily delete old undo log records.
innobase_instant_try(): Invoke index->table->reassign_id() before
index->clear_instant_alter().
dict_index_t::clear_instant_alter(): Remove a loop that became
redundant in commit ae2004c616
and caused corruption of n_nullable when dropping the last column
which was not declared NOT NULL.
dict_index_t::clear_instant_alter(): Correctly move all fields
corresponding to instantly dropped columns to the end of the array.
This fixes a regression that was introduced in
commit 5aaee3746e.
