database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-07-30 16:24:05 +03:00
Code Activity

MDEV-33449 improving repair of tables

Browse Source 
This task is to ensure we have a clear definition and rules of how to
repair or optimize a table.

The rules are:

- REPAIR should be used with tables that are crashed and are
  unreadable (hardware issues with not readable blocks, blocks with
  'unexpected data' etc)
- OPTIMIZE table should be used to optimize the storage layout for the
  table (recover space for delete rows and optimize the index
  structure.
- ALTER TABLE table_name FORCE should be used to rebuild the .frm file
  (the table definition) and the table (with the original table row
  format). If the table is from and older MariaDB/MySQL release with a
  different storage format, it will convert the data to the new
  format. ALTER TABLE ... FORCE is used as part of mariadb-upgrade

Here follows some more background:

The 3 ways to repair a table are:
1) ALTER TABLE table_name FORCE" (not other options).
   As an alias we allow: "ALTER TABLE table_name ENGINE=original_engine"
2) "REPAIR TABLE" (without FORCE)
3) "OPTIMIZE TABLE"

All of the above commands will optimize row space usage (which means that
space will be needed to hold a temporary copy of the table) and
re-generate all indexes. They will also try to replicate the original
table definition as exact as possible.

For ALTER TABLE and "REPAIR TABLE without FORCE", the following will hold:
If the table is from an older MariaDB version and data conversion is
needed (for example for old type HASH columns, MySQL JSON type or new
TIMESTAMP format) "ALTER TABLE table_name FORCE, algorithm=COPY" will be
used.

The differences between the algorithms are
1) Will use the fastest algorithm the engine supports to do a full repair
   of the table (except if data conversions are is needed).
2) Will use the storage engine internal REPAIR facility (MyISAM, Aria).
   If the engine does not support REPAIR then
   "ALTER TABLE FORCE, ALGORITHM=COPY" will be used.
   If there was data incompatibilities (which means that FORCE was used)
   then there will be a warning after REPAIR that ALTER TABLE FORCE is
   still needed.
   The reason for this is that REPAIR may be able to go around data
   errors (wrong incompatible data, crashed or unreadable sectors) that
   ALTER TABLE cannot do.
3) Will use the storage engine internal OPTIMIZE. If engine does not
   support optimize, then "ALTER TABLE FORCE" is used.

The above will ensure that ALTER TABLE FORCE is able to
correct almost any errors in the row or index data.  In case of
corrupted blocks then REPAIR possible followed by ALTER TABLE is needed.
This is important as mariadb-upgrade executes ALTER TABLE table_name
FORCE for any table that must be re-created.

Bugs fixed with InnoDB tables when using ALTER TABLE FORCE:
- No error for INNODB_DEFAULT_ROW_FORMAT=COMPACT even if row length
  would be too wide. (Independent of innodb_strict_mode).
- Tables using symlinks will be symlinked after any of the above commands
  (Independent of the setting of --symbolic-links)

If one specifies an algorithm together with ALTER TABLE FORCE, things
will work as before (except if data conversion is required as then
the COPY algorithm is enforced).

ALTER TABLE .. OPTIMIZE ALL PARTITIONS will work as before.

Other things:
- FORCE argument added to REPAIR to allow one to first run internal
  repair to fix damaged blocks and then follow it with ALTER TABLE.
- REPAIR will not update frm_version if ha_check_for_upgrade() finds
  that table is still incompatible with current version. In this case the
  REPAIR will end with an error.
- REPAIR for storage engines that does not have native repair, like InnoDB,
  is now using ALTER TABLE FORCE.
- REPAIR csv-table USE_FRM now works.
  - It did not work before as CSV tables had extension list in wrong
    order.
- Default error messages length for %M increased from 128 to 256 to not
  cut information from REPAIR.
- Documented HA_ADMIN_XX variables related to repair.
- Added HA_ADMIN_NEEDS_DATA_CONVERSION to signal that we have to
  do data conversions when converting the table (and thus ALTER TABLE
  copy algorithm is needed).
- Fixed typo in error message (caused test changes).
This commit is contained in:
Monty
2024-01-29 11:52:44 +02:00
committed by Sergei Golubchik
parent 2464ee758a
commit c4cad8d50c
41 changed files with 410 additions and 193 deletions
Download Patch File Download Diff File Expand all files Collapse all files

70
sql/sql_table.cc
View File

@ -10214,11 +10214,12 @@ bool mysql_alter_table(THD *thd, const LEX_CSTRING *new_db,
TABLE *table, *new_table= nullptr;
DDL_LOG_STATE ddl_log_state;
Turn_errors_to_warnings_handler errors_to_warnings;
HA_CHECK_OPT check_opt;
#ifdef WITH_PARTITION_STORAGE_ENGINE
bool partition_changed= false;
bool fast_alter_partition= false;
#endif
bool require_copy_algorithm;
bool partial_alter= false;
/*
start_alter_id is the gtid seq no of the START Alter - the 1st part
@ -10898,19 +10899,56 @@ do_continue:;
DBUG_RETURN(TRUE);
#endif /* WITH_WSREP */
/*
ALTER TABLE ... ENGINE to the same engine is a common way to
request table rebuild. Set ALTER_RECREATE flag to force table
rebuild.
*/
if (new_db_type == old_db_type)
{
if (create_info->used_fields & HA_CREATE_USED_ENGINE)
alter_info->flags|= ALTER_RECREATE;
/*
Check if we are using ALTER TABLE FORCE without any other options
(except ENGINE == current_engine).
In this case we will try to recreate an identical to the original.
This code is also used with REPAIR and OPTIMIZE.
*/
if (alter_info->flags == ALTER_RECREATE &&
((create_info->used_fields & ~HA_CREATE_USED_ENGINE) == 0))
create_info->recreate_identical_table= 1;
}
/*
We can use only copy algorithm if one of the following is true:
- If the table is from an old MariaDB version and requires data
modifications. In this case we ignore --alter-algorithm as
as we cannot use any other algorithm than COPY (using other
algorithms could open up the problem that the table .frm version
is updated without data transformations and the table would be
corrupted without any way for MariaDB to notice this during
check/upgrade).
This logic ensurses that ALTER TABLE ... FORCE (no other
options) will always be be able to repair a table structure and
convert data from any old format.
- In-place is impossible for given operation.
- Changes to partitioning which were not handled by fast_alter_part_table()
needs to be handled using table copying algorithm unless the engine
supports auto-partitioning as such engines can do some changes
using in-place API.
*/
if (is_inplace_alter_impossible(table, create_info, alter_info)
|| IF_PARTITIONING((partition_changed &&
!(old_db_type->partition_flags() & HA_USE_AUTO_PARTITION)), 0))
check_opt.init();
require_copy_algorithm= (table->file->ha_check_for_upgrade(&check_opt) ==
HA_ADMIN_NEEDS_DATA_CONVERSION);
if (require_copy_algorithm ||
is_inplace_alter_impossible(table, create_info, alter_info) ||
IF_PARTITIONING((partition_changed &&
!(old_db_type->partition_flags() & HA_USE_AUTO_PARTITION)), 0))
{
if (alter_info->algorithm_is_nocopy(thd))
if (alter_info->algorithm_is_nocopy(thd) &&
!(require_copy_algorithm && alter_info->algorithm_not_specified()))
{
my_error(ER_ALTER_OPERATION_NOT_SUPPORTED, MYF(0),
alter_info->algorithm_clause(thd), "ALGORITHM=COPY");
@ -10920,15 +10958,6 @@ do_continue:;
alter_info->set_requested_algorithm(Alter_info::ALTER_TABLE_ALGORITHM_COPY);
}
/*
ALTER TABLE ... ENGINE to the same engine is a common way to
request table rebuild. Set ALTER_RECREATE flag to force table
rebuild.
*/
if (new_db_type == old_db_type &&
create_info->used_fields & HA_CREATE_USED_ENGINE)
alter_info->flags|= ALTER_RECREATE;
/*
Handling of symlinked tables:
If no rename:
@ -12470,6 +12499,7 @@ copy_data_between_tables(THD *thd, TABLE *from, TABLE *to,
mysql_recreate_table()
thd Thread handler
table_list Table to recreate
partition_admin Optimizing partitions
table_copy Recreate the table by using
ALTER TABLE COPY algorithm
@ -12478,7 +12508,8 @@ copy_data_between_tables(THD *thd, TABLE *from, TABLE *to,
*/
bool mysql_recreate_table(THD *thd, TABLE_LIST *table_list,
Recreate_info *recreate_info, bool table_copy)
Recreate_info *recreate_info,
bool table_copy)
{
Table_specification_st create_info;
Alter_info alter_info;
@ -12495,12 +12526,13 @@ bool mysql_recreate_table(THD *thd, TABLE_LIST *table_list,
create_info.init();
create_info.row_type=ROW_TYPE_NOT_USED;
create_info.alter_info= &alter_info;
create_info.recreate_identical_table= 1;
/* Force alter table to recreate table */
alter_info.flags= (ALTER_CHANGE_COLUMN | ALTER_RECREATE);
alter_info.partition_flags= thd->lex->alter_info.partition_flags;
if (table_copy)
alter_info.set_requested_algorithm(
Alter_info::ALTER_TABLE_ALGORITHM_COPY);
if (table_copy && !(alter_info.partition_flags & ALTER_PARTITION_ADMIN))
alter_info.set_requested_algorithm(Alter_info::ALTER_TABLE_ALGORITHM_COPY);
bool res= mysql_alter_table(thd, &null_clex_str, &null_clex_str, &create_info,
table_list, recreate_info, &alter_info, 0,
@ -12829,7 +12861,7 @@ bool Sql_cmd_create_table_like::execute(THD *thd)
/*
Since CREATE_INFO is not full without Alter_info, it is better to pass them
as a signle parameter. TODO: remove alter_info argument where create_info is
as a single parameter. TODO: remove alter_info argument where create_info is
passed.
*/
create_info.alter_info= &alter_info;