database/mariadb
database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-06-01 19:42:01 +03:00
Code
67,489 Commits 3,010 Branches 355 Tags
Commit Graph

11 Commits

Author SHA1 Message Date
Davi Arnaut
a5efb91dea Bug#49938: Failing assertion: inode or deadlock in fsp/fsp0fsp.c 
Bug#54678: InnoDB, TRUNCATE, ALTER, I_S SELECT, crash or deadlock

- Incompatible change: truncate no longer resorts to a row by
row delete if the storage engine does not support the truncate
method. Consequently, the count of affected rows does not, in
any case, reflect the actual number of rows.

- Incompatible change: it is no longer possible to truncate a
table that participates as a parent in a foreign key constraint,
unless it is a self-referencing constraint (both parent and child
are in the same table). To work around this incompatible change
and still be able to truncate such tables, disable foreign checks
with SET foreign_key_checks=0 before truncate. Alternatively, if
foreign key checks are necessary, please use a DELETE statement
without a WHERE condition.

Problem description:

The problem was that for storage engines that do not support
truncate table via a external drop and recreate, such as InnoDB
which implements truncate via a internal drop and recreate, the
delete_all_rows method could be invoked with a shared metadata
lock, causing problems if the engine needed exclusive access
to some internal metadata. This problem originated with the
fact that there is no truncate specific handler method, which
ended up leading to a abuse of the delete_all_rows method that
is primarily used for delete operations without a condition.

Solution:

The solution is to introduce a truncate handler method that is
invoked when the engine does not support truncation via a table
drop and recreate. This method is invoked under a exclusive
metadata lock, so that there is only a single instance of the
table when the method is invoked.

Also, the method is not invoked and a error is thrown if
the table is a parent in a non-self-referencing foreign key
relationship. This was necessary to avoid inconsistency as
some integrity checks are bypassed. This is inline with the
fact that truncate is primarily a DDL operation that was
designed to quickly remove all data from a table.

mysql-test/suite/innodb/t/innodb-truncate.test:
  Add test cases for truncate and foreign key checks.
  Also test that InnoDB resets auto-increment on truncate.
mysql-test/suite/innodb/t/innodb.test:
  FK is not necessary, test is related to auto-increment.
  
  Update error number, truncate is no longer invoked if
  table is parent in a FK relationship.
mysql-test/suite/innodb/t/innodb_mysql.test:
  Update error number, truncate is no longer invoked if
  table is parent in a FK relationship.
  
  Use delete instead of truncate, test is used to check
  the interaction of FKs, triggers and delete.
mysql-test/suite/parts/inc/partition_check.inc:
  Fix typo.
mysql-test/suite/sys_vars/t/foreign_key_checks_func.test:
  Update error number, truncate is no longer invoked if
  table is parent in a FK relationship.
mysql-test/t/mdl_sync.test:
  Modify test case to reflect and ensure that truncate takes
  a exclusive metadata lock.
mysql-test/t/trigger-trans.test:
  Update error number, truncate is no longer invoked if
  table is parent in a FK relationship.
sql/ha_partition.cc:
  Reorganize the various truncate methods. delete_all_rows is now
  passed directly to the underlying engines, so as truncate. The
  code responsible for truncating individual partitions is moved
  to ha_partition::truncate_partition, which is invoked when a
  ALTER TABLE t1 TRUNCATE PARTITION p statement is executed.
  
  Since the partition truncate no longer can be invoked via
  delete, the bitmap operations are not necessary anymore. The
  explicit reset of the auto-increment value is also removed
  as the underlying engines are now responsible for reseting
  the value.
sql/handler.cc:
  Wire up the handler truncate method.
sql/handler.h:
  Introduce and document the truncate handler method. It assumes
  certain use cases of delete_all_rows.
  
  Add method to retrieve the list of foreign keys referencing a
  table. Method is used to avoid truncating tables that are
  parent in a foreign key relationship.
sql/share/errmsg-utf8.txt:
  Add error message for truncate and FK.
sql/sql_lex.h:
  Introduce a flag so that the partition engine can detect when
  a partition is being truncated. Used to give a special error.
sql/sql_parse.cc:
  Function mysql_truncate_table no longer exists.
sql/sql_partition_admin.cc:
  Implement the TRUNCATE PARTITION statement.
sql/sql_truncate.cc:
  Change the truncate table implementation to use the new truncate
  handler method and to not rely on row-by-row delete anymore.
  
  The truncate handler method is always invoked with a exclusive
  metadata lock. Also, it is no longer possible to truncate a
  table that is parent in some non-self-referencing foreign key.
storage/archive/ha_archive.cc:
  Rename method as the description indicates that in the future
  this could be a truncate operation.
storage/blackhole/ha_blackhole.cc:
  Implement truncate as no operation for the blackhole engine in
  order to remain compatible with older releases.
storage/federated/ha_federated.cc:
  Introduce truncate method that invokes delete_all_rows.
  This is required to support partition truncate as this
  form of truncate does not implement the drop and recreate
  protocol.
storage/heap/ha_heap.cc:
  Introduce truncate method that invokes delete_all_rows.
  This is required to support partition truncate as this
  form of truncate does not implement the drop and recreate
  protocol.
storage/ibmdb2i/ha_ibmdb2i.cc:
  Introduce truncate method that invokes delete_all_rows.
  This is required to support partition truncate as this
  form of truncate does not implement the drop and recreate
  protocol.
storage/innobase/handler/ha_innodb.cc:
  Rename delete_all_rows to truncate. InnoDB now does truncate
  under a exclusive metadata lock.
  
  Introduce and reorganize methods used to retrieve the list
  of foreign keys referenced by a or referencing a table.
storage/myisammrg/ha_myisammrg.cc:
  Introduce truncate method that invokes delete_all_rows.
  This is required in order to remain compatible with earlier
  releases where truncate would resort to a row-by-row delete.
 2010-10-06 11:34:28 -03:00
Alexander Nozdrin
cc0925f5d9 Bug#27480 (Extend CREATE TEMPORARY TABLES privilege 
to allow temp table operations) -- prerequisite patch #3.

Rename open_temporary_table() to open_table_uncached().
open_temporary_table() will be introduced in following patches
to open temporary tables for a statement.
 2010-08-31 14:03:36 +04:00
Jon Olav Hauglid
eb498cce4d Manual merge from mysql-5.5-bugfixing to mysql-5.5-runtime. 2010-08-18 13:29:04 +02:00
Mattias Jonsson
4b20ccafaa Bug#49907: ALTER TABLE ... TRUNCATE PARTITION does not wait for 
locks on the table

Fixing the partitioning specifics after TRUNCATE TABLE in
bug-42643 was fixed.

Reorganize of code to decrease the size of the giant switch
in mysql_execute_command, and to prepare for future parser
reengineering. Moved code into Sql_statement objects.

Updated patch according to davi's review comments.

libmysqld/CMakeLists.txt:
  Added new files.
libmysqld/Makefile.am:
  Added new files.
mysql-test/r/not_partition.result:
  now returning error on partitioning commands
  if partitioning is not enabled.
mysql-test/r/partition_disabled.result:
  There is no partition handlerton, so it cannot
  find the specified engine in the .frm file.
mysql-test/r/partition_truncate.result:
  Updated test results.
mysql-test/suite/parts/inc/partition_mgm.inc:
  Added check that TRUNCATE PARTITION does not delete on failure.
mysql-test/suite/parts/r/partition_debug_sync_innodb.result:
  updated results.
mysql-test/suite/parts/r/partition_mgm_lc0_archive.result:
  updated results.
mysql-test/suite/parts/r/partition_mgm_lc1_archive.result:
  updated results.
mysql-test/suite/parts/r/partition_mgm_lc2_archive.result:
  updated results.
mysql-test/suite/parts/t/partition_debug_sync_innodb.test:
  Test case for this bug.
mysql-test/t/not_partition.test:
  Added check for TRUNCATE PARTITION without partitioning.
mysql-test/t/partition_truncate.test:
  Added test of TRUNCATE PARTITION on non partitioned table.
sql/CMakeLists.txt:
  Added new files.
sql/Makefile.am:
  Added new files.
sql/datadict.cc:
  Moved out the storage engine check into an own
  function, including assert for lock.
sql/datadict.h:
  added dd_frm_storage_engine.
sql/sql_alter_table.cc:
  moved the code for SQLCOM_ALTER_TABLE in mysql_execute_command
  into its own file, and using the Sql_statement object to
  prepare for future parser reengineering.
sql/sql_alter_table.h:
  Created Sql_statement object for ALTER TABLE.
sql/sql_lex.cc:
  resetting m_stmt.
sql/sql_lex.h:
  Temporary hack for forward declaration of enum_alter_table_change_level.
sql/sql_parse.cc:
  Moved out ALTER/ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE
  from the giant switch into their own Sql_statement
  objects.
sql/sql_parse.h:
  Exporting check_merge_table_access.
sql/sql_partition_admin.cc:
  created Sql_statement for
  ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR/TRUNCATE
  PARTITION. To be able to reuse the TABLE equivalents.
sql/sql_partition_admin.h:
  Added Sql_statement of partition admin statements.
sql/sql_table.cc:
  Moved table maintenance code into sql_table_maintenance.cc
sql/sql_table.h:
  Moved table maintenance code into sql_table_maintenance.h
  exporting functions used by sql_table_maintenance.
sql/sql_table_maintenance.cc:
  Moved table maintenance code from sql_table.cc
sql/sql_table_maintenance.h:
  Sql_statement objects for ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE.
  Also declaring the keycache functions.
sql/sql_truncate.cc:
  Moved code from SQLCOM_TRUNCATE in mysql_execute_command into
  Truncate_statement::execute.
  Added check for partitioned table on TRUNCATE PARTITION.
  Moved locking fix for partitioned table into
  Alter_table_truncate_partition::execute.
sql/sql_truncate.h:
  Truncate_statement declaration (sub class of Sql_statement).
sql/sql_yacc.yy:
  Using the new Sql_statment objects.
 2010-08-16 14:53:30 +02:00
Konstantin Osipov
523066987d A fix for Bug#41158 "DROP TABLE holds LOCK_open during unlink()". 
Remove acquisition of LOCK_open around file system operations,
since such operations are now protected by metadata locks.
Rework table discovery algorithm to not require LOCK_open.

No new tests added since all MDL locking operations are covered
in lock.test and mdl_sync.test, and as long as these tests
pass despite the increased concurrency, consistency must be
unaffected.

mysql-test/t/disabled.def:
  Disable NDB tests due to Bug#55799.
sql/datadict.cc:
  No longer necessary to protect ha_create_table() with
  LOCK_open. Serial execution is now ensured by metadata
  locks.
sql/ha_ndbcluster.cc:
  Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.cc:
  Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.h:
  Update function signature.
sql/handler.cc:
  Implement ha_check_if_table_exists().
  @todo: some engines provide ha_table_exists_in_engine()
  handlerton call, for those we perhaps shouldn't
  call ha_discover(), to be more efficient.
  Since currently it's only NDB, postpone till
  integration with NDB.
sql/handler.h:
  Declare ha_check_if_table_exists() function.
sql/mdl.cc:
  Remove an obsolete comment.
sql/sql_base.cc:
  Update to a new signature of close_cached_tables():
  from now on we always call it without LOCK_open.
  Update comments.
  Remove get_table_share_with_create(), we should
  not attempt to create a table under LOCK_open.
  Introduce get_table_share_with_discover() instead,
  which would request a back off action if the table
  exists in engine.
  Remove acquisition of LOCK_open for 
  data dictionary operations, such as check_if_table_exists().
  Do not use get_table_share_with_create/discover for views,
  where it's not needed.
  Make tdc_remove_table() optionally acquire LOCK_open
  to simplify usage of this function.
  Use the right mutex in the partitioning code when
  manipulating with thd->open_tables.
sql/sql_base.h:
  Update signatures of changes functions.
sql/sql_insert.cc:
  Do not wrap quick_rm_table() with LOCK_open acquisition, 
  this is unnecessary.
sql/sql_parse.cc:
  Update to the new calling convention of tdc_remove_table().
  Update to the new signature of close_cached_tables().
  Update comments.
sql/sql_rename.cc:
  Update to the new calling convention of tdc_remove_table().
  Remove acquisition of LOCK_open around filesystem
  operations.
sql/sql_show.cc:
  Remove get_trigger_table_impl().
  Do not acquire LOCK_open for a dirty read of the trigger
  file.
sql/sql_table.cc:
  Do not acquire LOCK_open for filesystem operations.
sql/sql_trigger.cc:
  Do not require LOCK_open for trigger file I/O.
sql/sql_truncate.cc:
  Update to the new signature of tdc_remove_table().
sql/sql_view.cc:
  Do not require LOCK_open for view I/O.
  Use tdc_remove_table() to expel view share.
  Update comments.
sql/sys_vars.cc:
  Update to the new signature of close_cached_tables().
 2010-08-09 22:33:47 +04:00
Jon Olav Hauglid
f45523188e merge from mysql-trunk-bugfixing 2010-07-13 10:39:24 +02:00
Davi Arnaut
f56dd32bf7 Bug#34043: Server loops excessively in _checkchunk() when safemalloc is enabled 
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.

Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.

Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost. 

The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.

Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.

client/mysqldump.c:
  Pass my_free directly as its signature is compatible with the
  callback type -- which wasn't the case for free_table_ent.
 2010-07-08 18:20:08 -03:00
Jon Olav Hauglid
9ff272fbbd A 5.5 version of the fix for Bug #54360 "Deadlock DROP/ALTER/CREATE 
DATABASE with open HANDLER"

Remove LOCK_create_db, database name locks, and use metadata locks instead.
This exposes CREATE/DROP/ALTER DATABASE statements to the graph-based
deadlock detector in MDL, and paves the way for a safe, deadlock-free
implementation of RENAME DATABASE.

Database DDL statements will now take exclusive metadata locks on
the database name, while table/view/routine DDL statements take
intention exclusive locks on the database name. This prevents race
conditions between database DDL and table/view/routine DDL.
(e.g. DROP DATABASE with concurrent CREATE/ALTER/DROP TABLE)

By adding database name locks, this patch implements
WL#4450 "DDL locking: CREATE/DROP DATABASE must use database locks" and
WL#4985 "DDL locking: namespace/hierarchical locks".

The patch also changes code to use init_one_table() where appropriate.
The new lock_table_names() function requires TABLE_LIST::db_length to
be set correctly, and this is taken care of by init_one_table().

This patch also adds a simple template to help work with 
the mysys HASH data structure.

Most of the patch was written by Konstantin Osipov.
 2010-07-01 15:53:46 +02:00
Davi Arnaut
ad08138798 Bug#42643: InnoDB does not support replication of TRUNCATE TABLE 
Post-merge fix: Pass the right parameter type to open_and_lock_tables.
Passing FALSE ensures that derived table handling is disabled, truncate
only operates on base tables.

sql/sql_truncate.cc:
  Pass FALSE to disable derived table handling. This was probably
  reminiscent of the use of open_n_lock_single_table, which
  can't be used as in some cases (e.g. merge) more than one
  table will be opened.
sql/sql_truncate.h:
  TABLE_LIST is a struct.
 2010-05-27 18:11:55 -03:00
Davi Arnaut
fcbc77b064 Bug#42643: InnoDB does not support replication of TRUNCATE TABLE 
Post-merge fix: Retrieve handler statistics to workaround quirks
of the archive storage engine.

mysql-test/t/archive.test:
  Add test case.
sql/sql_truncate.cc:
  Call info method with has side-effects related to the archive
  storage engine. It will cause rows to become visible, allowing
  a error to be thrown once we attempt to delete the rows.
 2010-05-27 17:40:54 -03:00
Davi Arnaut
4f18083b08 Bug#42643: InnoDB does not support replication of TRUNCATE TABLE 
The problem was that TRUNCATE TABLE didn't take a exclusive
lock on a table if it resorted to truncating via delete of
all rows in the table. Specifically for InnoDB tables, this
could break proper isolation as InnoDB ends up aborting some
granted locks when truncating a table.

The solution is to take a exclusive metadata lock before
TRUNCATE TABLE can proceed. This guarantees that no other
transaction is using the table.

Incompatible change: Truncate via delete no longer fails
if sql_safe_updates is activated (this was a undocumented
side effect).

libmysqld/CMakeLists.txt:
  Add new files to the build list.
libmysqld/Makefile.am:
  Add new files to the build list.
mysql-test/extra/binlog_tests/binlog_truncate.test:
  Add test case for Bug#42643
mysql-test/include/mix1.inc:
  Update test case as TRUNCATE TABLE now grabs a exclusive lock.
  Ensure that TRUNCATE waits for granted locks on the table.
mysql-test/suite/binlog/t/binlog_truncate_innodb.test:
  As with other data modifying statements, TRUNCATE is still not
  possible in a transaction with isolation level READ COMMITTED
  or READ UNCOMMITED. It would be possible to implement so, but
  it is not worth the effort.
mysql-test/suite/binlog/t/binlog_truncate_myisam.test:
  Test under different binlog formats.
mysql-test/suite/binlog/t/disabled.def:
  Re-enable test case.
mysql-test/t/innodb_bug38231.test:
  Truncate no longer works with row-level locks.
mysql-test/t/mdl_sync.test:
  Ensure that a acquired lock is not given up due to a conflict.
mysql-test/t/partition_innodb_semi_consistent.test:
  End transaction as to release metadata locks.
mysql-test/t/truncate.test:
  A metadata lock is now taken before the object is verified.
sql/CMakeLists.txt:
  Add new files to the build list.
sql/Makefile.am:
  Add new files to the build list.
sql/datadict.cc:
  Introduce a new file specific for data dictionary operations.
sql/datadict.h:
  Add header file.
sql/sql_base.cc:
  Rename data dictionary function.
sql/sql_bitmap.h:
  Include dependency.
sql/sql_delete.cc:
  Move away from relying on mysql_delete() to delete all rows of
  a table. Thus, move any bits related to truncate to sql_truncate.cc
sql/sql_delete.h:
  Remove parameter.
sql/sql_parse.cc:
  Add protection against the global read lock -- a intention
  exclusive lock can be acquired in the truncate path.
sql/sql_show.cc:
  Add sync point for testing scenarios where a pending flush
  is ignored.
sql/sql_truncate.cc:
  Acquire a shared metadata lock before accessing table metadata.
  Upgrade the lock to a exclusive one if the table can be re-created.
  Rework binlog rules to better reflect the requirements.
sql/sql_yacc.yy:
  Set appropriate lock types for table to be truncated.
sql/table.h:
  Move to data dictionary header.
 2010-05-25 17:01:38 -03:00