-------------------------------------------------------------------
revno: 2630.6.6
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-3726
timestamp: Tue 2008-05-27 16:15:44 +0400
message:
Implement code review fixes for WL#3726 "DDL locking for all
metadata objects": cleanup the code from share->mutex
acquisitions, which are now obsolete.
sql/ha_partition.cc:
Rename share->mutex to share->LOCK_ha_data. The mutex never
protected the entire share. The right way to protect a share
is to acquire an MDL lock.
sql/ha_partition.h:
Rename share->mutex to share->LOCK_ha_data.
sql/sql_base.cc:
Remove LOCK_table_share. Do not acquire share->mutex when
deleting a table share or incrementing its ref_count.
All these operations are and must continue to be protected by LOCK_open
and respective MDL locks.
sql/sql_view.cc:
Remove acquisition of share->mutex when incrementing share->ref_count.
sql/table.cc:
Simplify release_table_shar() by removing dead code related
to share->mutex from it.
sql/table.h:
Rename TABLE_SHARE::mutex to TABLE_SHARE::LOCK_ha_data
to better reflect its purpose.
storage/myisam/ha_myisam.cc:
Rename share->mutex to share->LOCK_ha_data.
Conflicts
=========
Text conflict in .bzr-mysql/default.conf
Text conflict in libmysqld/CMakeLists.txt
Text conflict in libmysqld/Makefile.am
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/extra/rpl_tests/rpl_row_sp006.test
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_create_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_sp006_InnoDB.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_sp006.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/Makefile.am
Text conflict in sql/log_event_old.cc
Text conflict in sql/rpl_rli.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_binlog.cc
Text conflict in sql/sql_lex.h
21 conflicts encountered.
NOTE
====
mysql-5.1-rpl-merge has been made a mirror of mysql-next-mr:
- "mysql-5.1-rpl-merge$ bzr pull ../mysql-next-mr"
This is the first cset (merge/...) committed after pulling
from mysql-next-mr.
get_table_share, drop_open_table
In the partition handler code, LOCK_open and share->LOCK_ha_data
are acquired in the wrong order in certain cases. When doing a
multi-row INSERT (i.e a INSERT..SELECT) in a table with auto-
increment column(s). the increments must be in a monotonically
continuous increasing sequence (i.e it can't have "holes"). To
achieve this, a lock is held for the duration of the operation.
share->LOCK_ha_data was used for this purpose.
Whenever there was a need to open a view _during_ the operation
(views are not currently pre-opened the way tables are), and
LOCK_open was grabbed, a deadlock could occur. share->LOCK_ha_data
is other places used _while_ holding LOCK_open.
A new mutex was introduced in the HA_DATA_PARTITION structure,
for exclusive use of the autoincrement data fields, so we don't
need to overload the use of LOCK_ha_data here.
A module test case has not been supplied, since the problem occurs
as a result of a race condition, and testing for this condition
is thus not deterministic. Testing for it could be done by
setting up a test case as described in the bug report.
"insert into.. select * from"
When inserting into a partitioned table using 'insert into
<target> select * from <src>', read_buffer_size bytes of memory
are allocated for each partition in the target table.
This resulted in large memory consumption when the number of
partitions are high.
This patch introduces a new method which tries to estimate the
buffer size required for each partition and limits the maximum
buffer size used to maximum of 10 * read_buffer_size,
11 * read_buffer_size in case of monotonic partition functions.
sql/ha_partition.cc:
Introduced a method ha_partition::estimate_read_buffer_size
to estimate buffer size required for each partition.
Method ha_partition::start_part_bulk_insert updated
to update the read_buffer_size before calling bulk upload
in storage engines.
Added thd in ha_partition::start_part_bulk_insert method signature.
sql/ha_partition.h:
Introduced a method ha_partition::estimate_read_buffer_size.
Added thd in ha_partition::start_part_bulk_insert method signature.
(Backport)
Problem is that when insert (ha_start_bulk_insert) in i partitioned table,
it will call ha_start_bulk_insert for every partition, used or not.
Solution is to delay the call to the partitions ha_start_bulk_insert until
the first row is to be inserted into that partition
sql/ha_partition.cc:
Bug#35845: unneccesary call to ha_start_bulk_insert for not used partitions
Using a bitmap for keeping record of which partitions for which
ha_start_bulk_insert has been called, and check against that if one
should call it before continue with the insert/update, or if it has already
been called.
This way it will only call ha_start_bulk_insert for the used partitions.
There is also a little prediction on how many rows that will be inserted into
the current partition, it will guess on equal distribution of the records
across all partitions, accept for the first used partition, which will guess
at 50% of the given estimate, if it is a monotonic partitioning function.
sql/ha_partition.h:
Bug#35845: unneccesary call to ha_start_bulk_insert for not used partitions
Added help variables and function for delaying ha_bulk_insert until it has
to be called.
Fixed a comment.
Inserting a negative value in the autoincrement column of a
partitioned innodb table was causing the value of the auto
increment counter to wrap around into a very large positive
value. The consequences are the same as if a very large positive
value was inserted into a column, e.g. reduced autoincrement
range, failure to read autoincrement counter.
The current patch ensures that before calculating the next
auto increment value, the current value is within the positive
maximum allowed limit.
mysql-test/suite/parts/inc/partition_auto_increment.inc:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Adds tests for performing insert,update and delete on a partition
table with negative auto_increment values.
mysql-test/suite/parts/r/partition_auto_increment_innodb.result:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Result file for the innodb engine.
mysql-test/suite/parts/r/partition_auto_increment_memory.result:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Result file for the memory engine.
mysql-test/suite/parts/r/partition_auto_increment_myisam.result:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Result file for the myisam engine.
mysql-test/suite/parts/r/partition_auto_increment_ndb.result:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Result file for the ndb engine.
mysql-test/suite/parts/t/partition_auto_increment_archive.test:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Adds a variable that allows the Archive engine to skip tests
that involve insertion of negative auto increment values.
mysql-test/suite/parts/t/partition_auto_increment_blackhole.test:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Adds a variable that allows the Blackhole engine to skip tests
that involve insertion of negative auto increment values.
sql/ha_partition.cc:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Ensures that the current value is lesser than the upper limit
for the type of the field before setting the next auto increment
value to be calculated.
sql/ha_partition.h:
Bug#45823 Assertion failure in file row/row0mysql.c line 1386
Modifies the set_auto_increment_if_higher function, to take
into account negative auto increment values when doing a
comparison.
Addition of hander function was_semi_consistent_read
mysql-test/r/partition_innodb_semi_consistent.result:
post push fix for bug#40595
Addition of hander function was_semi_consistent_read
Added test result
mysql-test/t/partition_innodb_semi_consistent-master.opt:
post push fix for bug#40595
Addition of hander function was_semi_consistent_read
Added test opt file
mysql-test/t/partition_innodb_semi_consistent.test:
post push fix for bug#40595
Addition of hander function was_semi_consistent_read
Added test case
sql/ha_partition.cc:
post push fix for bug#40595
Addition of hander function was_semi_consistent_read
The lack of was_semi_consistent_read opened a regression
for bug-31310 when useing a partitioned InnoDB table
on tables with partitions
Problem was that the handler function try_semi_consistent_read
was not propagated to the innodb handler.
Solution was to implement that function in the partitioning
handler.
mysql-test/r/partition_innodb.result:
Bug#40595: Non-matching rows not released with READ-COMMITTED
on tables with partitions
Updated test result.
mysql-test/t/partition_innodb.test:
Bug#40595: Non-matching rows not released with READ-COMMITTED
on tables with partitions
Added test case for bug#40595.
Note: the replace_regex is taking long time. I have not found
any better regex (it takes time using 'sed' too).
sql/ha_partition.cc:
Bug#40595: Non-matching rows not released with READ-COMMITTED
on tables with partitions
Added function to the partitioning handler for handling
semi consistent reads (unlock_row was already implemented,
and this is needed for unlock_row to work properly in innodb).
It uses pruning for optimizing the call.
sql/ha_partition.h:
Bug#40595: Non-matching rows not released with READ-COMMITTED
on tables with partitions
Added function to the partitioning handler for handling
semi consistent reads (unlock_row was already implemented,
and this is needed for unlock_row to work properly in innodb).
Problem was that partitioning cached the table flags.
These flags could change due to TRANSACTION LEVEL changes.
Solution was to remove the cache and always return the table flags
from the first partition (if the handler was initialized).
mysql-test/r/partition_innodb_stmt.result:
Bug#39084: Getting intermittent errors with statement-based binary logging
New test result file.
mysql-test/t/partition_innodb_stmt.test:
Bug#39084: Getting intermittent errors with statement-based binary logging
New test file.
sql/ha_partition.cc:
Bug#39084: Getting intermittent errors with statement-based binary logging
Removed m_table_flags, and added m_handler_status.
Added checks that all partitions have the same
table flags.
Moved some variable initializations.
Updated some comments.
Fixed typo initialise -> initialize
Changed HA_EXTTA_NO_READCHECK to do nothing, since it
is only used in ha_open, which is called for every
partition in ::open anyway.
sql/ha_partition.h:
Bug#39084: Getting intermittent errors with statement-based binary logging
Removed m_table_flags, and added m_handler_status.
Always return the first partitions table flags, instead of using
cached table flags.
Added define of enabled/disabled partitioning table flags
Fixed type initialise -> initialize
Updated some comments.
sql/handler.cc:
Bug#39084: Getting intermittent errors with statement-based binary logging
Fixed type initialise -> initialize.
sql/handler.h:
Bug#39084: Getting intermittent errors with statement-based binary logging
Added comment to understand where the cached value is set.
breaks auto increment
The auto_increment value was not initialized if
the first statement after opening a table was
an 'UPDATE'.
solution was to check initialize if it was not,
before trying to increase it in update.
mysql-test/suite/parts/inc/partition_auto_increment.inc:
Bug#40176: update as first partitioning statement
breaks auto increment
Added tests for verifying the bug and show some more
auto_increment flaws
mysql-test/suite/parts/r/partition_auto_increment_archive.result:
Bug#40176: update as first partitioning statement
breaks auto increment
Updated test results, due to added tests
mysql-test/suite/parts/r/partition_auto_increment_blackhole.result:
Bug#40176: update as first partitioning statement
breaks auto increment
Updated test results, due to added tests
mysql-test/suite/parts/r/partition_auto_increment_innodb.result:
Bug#40176: update as first partitioning statement
breaks auto increment
Updated test results, due to added tests
mysql-test/suite/parts/r/partition_auto_increment_memory.result:
Bug#40176: update as first partitioning statement
breaks auto increment
Updated test results, due to added tests
mysql-test/suite/parts/r/partition_auto_increment_myisam.result:
Bug#40176: update as first partitioning statement
breaks auto increment
Updated test results, due to added tests
mysql-test/suite/parts/r/partition_auto_increment_ndb.result:
Bug#40176: update as first partitioning statement
breaks auto increment
Updated test results, due to added tests
sql/ha_partition.cc:
Bug#40176: update as first partitioning statement
breaks auto increment
make sure that the auto_increment value is initialized
before updating it.
(missed initializing in mysql_update_row).
sql/ha_partition.h:
Bug#40176: update as first partitioning statement
breaks auto increment
Assert that it is initialized, before updating
the auto_increment value
on non-partitioned table
Problem was that partitioning specific commands was accepted
for non partitioned tables and treated like
ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE, after bug-20129 was fixed,
which changed the code path from mysql_alter_table to
mysql_admin_table.
Solution was to check if the table was partitioned before
trying to execute the admin command
mysql-test/r/partition_mgm_err.result:
Bug#39434: ALTER TABLE CHECK/OPTIMIZE/ANALYZE PARTITION work
on non-partitioned table
Updated test result
mysql-test/t/partition_mgm_err.test:
Bug#39434: ALTER TABLE CHECK/OPTIMIZE/ANALYZE PARTITION work
on non-partitioned table
Updated test case
sql/ha_partition.cc:
Bug#39434: ALTER TABLE CHECK/OPTIMIZE/ANALYZE PARTITION work
on non-partitioned table
Simplified the code by using ALTER_ADMIN_PARTITION for all
commands that go through mysql_admin_tables and is set
for partitioning specific commands that.
sql/ha_partition.h:
Bug#39434: ALTER TABLE CHECK/OPTIMIZE/ANALYZE PARTITION work
on non-partitioned table
Simplified the code by using ALTER_ADMIN_PARTITION for all
commands that go through mysql_admin_tables and is set
for partitioning specific commands that.
sql/sql_lex.h:
Bug#39434: ALTER TABLE CHECK/OPTIMIZE/ANALYZE PARTITION work
on non-partitioned table
Simplified the code by using ALTER_ADMIN_PARTITION for all
commands that go through mysql_admin_tables and is set
for partitioning specific commands that.
Removed ALTER_ANALYZE/CHECK/OPTIMIZE/REPAIR_PARTITION and
added ALTER_ADMIN_PARTITION instead.
sql/sql_partition.cc:
Bug#39434: ALTER TABLE CHECK/OPTIMIZE/ANALYZE PARTITION work
on non-partitioned table
Simplified the code by using ALTER_ADMIN_PARTITION for all
commands that go through mysql_admin_tables and is set
for partitioning specific commands that.
Removed ALTER_ANALYZE/CHECK/OPTIMIZE/REPAIR_PARTITION and
added ALTER_ADMIN_PARTITION instead.
sql/sql_table.cc:
Bug#39434: ALTER TABLE CHECK/OPTIMIZE/ANALYZE PARTITION work
on non-partitioned table
Give error and return if trying partitioning admin command
on non partitioned table.
Simplified the code by using ALTER_ADMIN_PARTITION for all
commands that go through mysql_admin_tables and is set
for partitioning specific commands that.
Removed ALTER_ANALYZE/CHECK/OPTIMIZE/REPAIR_PARTITION and
added ALTER_ADMIN_PARTITION instead.
sql/sql_yacc.yy:
Bug#39434: ALTER TABLE CHECK/OPTIMIZE/ANALYZE PARTITION work
on non-partitioned table
Simplified the code by using ALTER_ADMIN_PARTITION for all
commands that go through mysql_admin_tables and is set
for partitioning specific commands that.
Removed ALTER_ANALYZE/CHECK/OPTIMIZE/REPAIR_PARTITION and
added ALTER_ADMIN_PARTITION instead.
index column
There was actually two problems
1) when clustered pk, order by non pk index should also
compare with pk as last resort to differ keys from each
other
2) bug in the index search handling in ha_partition (was
found when extending the test case
Solution to 1 was to include the pk in key compare if
clustered pk and search on other index.
Solution for 2 was to remove the optimization from
ordered scan to unordered scan if clustered pk.
mysql-test/r/partition_innodb.result:
Bug#37721: ORDER BY when WHERE contains non-partitioned
index column
updated test result.
mysql-test/t/partition_innodb.test:
Bug#37721: ORDER BY when WHERE contains non-partitioned
index column
Added test case for bug verification.
sql/ha_partition.cc:
Bug#37721: ORDER BY when WHERE contains non-partitioned
index column
using m_curr_key_info with both given index and PK
if clustered PK.
Also including PK in read_set.
Added debug prints for easier verification.
sql/ha_partition.h:
Bug#37721: ORDER BY when WHERE contains non-partitioned
index column
Changed m_curr_key_info to a null terminated array
with max 2 keys and a terminating null.
For use with key_rec_cmp with both given index and PK.
sql/key.cc:
Bug#37721: ORDER BY when WHERE contains non-partitioned
index column
added handling of a null terminated array of keys for
use in compare.
InnoDB Plugin locks table
The fast/on-line add/drop index handler calls was not implemented
whithin the partitioning.
This implements it in the partitioning handler.
Since this is only used by the not included InnoDB plugin, there
is no test case. (Have tested it manually with the plugin, and
it does not allow unique indexes not including partitioning
function, or removal of pk, which in innodb generates a new pk,
which is not in the partitioning function.)
NOTE: This introduces a new handler method, and because of that
changes the storage engine api. (One cannot use a handlerton to
see the capabilities of a table's handler if it is partitioned.
So I added a wrapper function in the handler that defaults to
the handlerton function, which the partitioning handler overrides.
sql/ha_partition.cc:
Bug#37453: Dropping/creating index on partitioned table with
InnoDB Plugin locks table
Added support for fast/on-line add/drop index.
Implemented alter_table_flags as bit-or of the partitioned
hton and the first partitions alter_table_flags.
It is only to forward the calls for the other functions:
check_if_incompatible_data
add_index
prepare_drop_index
final_drop_index
to all parts handler
sql/ha_partition.h:
Bug#37453: Dropping/creating index on partitioned table with
InnoDB Plugin locks table
Added support for fast/on-line add/drop index.
sql/handler.h:
Bug#37453: Dropping/creating index on partitioned table with
InnoDB Plugin locks table
Added the function on handler level, defaulting to use
the handlerton function, but a handler can override it.
Needed for partitioned tables.
NOTE: Change of storage engine api.
sql/sql_partition.cc:
Bug#37453: Dropping/creating index on partitioned table with
InnoDB Plugin locks table
Using the new handler function, instead of the handlerton
function. This works better with the partitioning handler.
sql/sql_table.cc:
Bug#37453: Dropping/creating index on partitioned table with
InnoDB Plugin locks table
Using the new handler function, instead of the handlerton
function. This works better with the partitioning handler.
Also using new process info for 'manage keys' (kind of fix
for bug-37550).
and
Bug#33555: Group By Query does not correctly aggregate partitions
Backport of bug-33257 which is the same bug.
read_range_*() calls was not passed to the partition handlers,
but was translated to index_read/next family calls.
Resulting in duplicates rows and wrong aggregations.
mysql-test/r/partition_range.result:
Bug#30573: Ordered range scan over partitioned tables returns some rows twice
Updated result file
mysql-test/t/partition_range.test:
Bug#30573: Ordered range scan over partitioned tables returns some rows twice
Re-enabled the test
sql/ha_partition.cc:
Bug#30573: Ordered range scan over partitioned tables returns some rows twice
backport of bug-33257, correct handling of read_range_* calls,
without converting them to index_read/next calls
sql/ha_partition.h:
Bug#30573: Ordered range scan over partitioned tables returns some rows twice
backport of bug-33257, correct handling of read_range_* calls,
without converting them to index_read/next calls
Problem was a mutex added in bug n 27405 for solving a problem
with auto_increment in partitioned innodb tables.
(in ha_partition::write_row over partitions file->ha_write_row)
Solution is to use the patch for bug#33479, which refines the
usage of mutexes for auto_increment.
Backport of bug-33479 from 6.0:
Bug-33479: auto_increment failures in partitioning
Several problems with auto_increment in partitioning
(with MyISAM, InnoDB. Locking issues, not handling
multi-row INSERTs properly etc.)
Changed the auto_increment handling for partitioning:
Added a ha_data variable in table_share for storage engine specific data
such as auto_increment value handling in partitioning, also see WL 4305
and using the ha_data->mutex to lock around read + update.
The idea is this:
Store the table's reserved auto_increment value in
the TABLE_SHARE and use a mutex to, lock it for reading and updating it
and unlocking it, in one block. Only accessing all partitions
when it is not initialized.
Also allow reservations of ranges, and if no one has done a reservation
afterwards, lower the reservation to what was actually used after
the statement is done (via release_auto_increment from WL 3146).
The lock is kept from the first reservation if it is statement based
replication and a multi-row INSERT statement where the number of
candidate rows to insert is not known in advance (like INSERT SELECT,
LOAD DATA, unlike INSERT VALUES (row1), (row2),,(rowN)).
This should also lead to better concurrancy (no need to have a mutex
protection around write_row in all cases)
and work with any local storage engine.
mysql-test/suite/parts/inc/partition_auto_increment.inc:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
Test source file for testing auto_increment
mysql-test/suite/parts/r/partition_auto_increment_archive.result:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
result file for testing auto_increment
mysql-test/suite/parts/r/partition_auto_increment_blackhole.result:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
result file for testing auto_increment
mysql-test/suite/parts/r/partition_auto_increment_innodb.result:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
result file for testing auto_increment
mysql-test/suite/parts/r/partition_auto_increment_memory.result:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
result file for testing auto_increment
mysql-test/suite/parts/r/partition_auto_increment_myisam.result:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
result file for testing auto_increment
mysql-test/suite/parts/r/partition_auto_increment_ndb.result:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
result file for testing auto_increment
mysql-test/suite/parts/t/partition_auto_increment_archive.test:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
test file for testing auto_increment
mysql-test/suite/parts/t/partition_auto_increment_blackhole.test:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
test file for testing auto_increment
mysql-test/suite/parts/t/partition_auto_increment_innodb.test:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
test file for testing auto_increment
mysql-test/suite/parts/t/partition_auto_increment_memory.test:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
test file for testing auto_increment
mysql-test/suite/parts/t/partition_auto_increment_myisam.test:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
test file for testing auto_increment
mysql-test/suite/parts/t/partition_auto_increment_ndb.test:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
test file for testing auto_increment
sql/ha_partition.cc:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: Failures using auto_increment and partitioning
Changed ha_partition::get_auto_increment from file->get_auto_increment
to file->info(HA_AUTO_STATUS), since it is works better with InnoDB
(InnoDB can have issues with partitioning and auto_increment,
where get_auto_increment sometimes can return a non updated value.)
Using the new table_share->ha_data for keeping the auto_increment
value, shared by all instances of the same table.
It is read+updated when holding a auto_increment specific mutex.
Also added release_auto_increment to decrease gaps if possible.
And a lock for multi-row INSERT statements where the number of candidate
rows to insert is not known in advance (like INSERT SELECT, LOAD DATA;
Unlike INSERT INTO (row1),(row2),,(rowN)).
Fixed a small bug, copied++ to (*copied)++ and the same for deleted.
Changed from current_thd, to ha_thd()
sql/ha_partition.h:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: Failures using auto_increment and partitioning
Added a new struct HA_DATA_PARTITION to be used in table_share->ha_data
Added a private function to set auto_increment values if needed
Removed the restore_auto_increment (the hander version is better)
Added lock/unlock functions for auto_increment handling.
Changed copied/deleted to const.
sql/handler.h:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: auto_increment failures in partitioning
Added const for changed_partitions
Added comments about SQLCOM_TRUNCATE for delete_all_rows
sql/table.h:
Bug#38804: Query deadlock causes all tables to be inaccessible.
Backporting from 6.0 of:
Bug-33479: Failures using auto_increment and partitioning
Added a variable in table_share: ha_data for storage of storage engine
specific data (such as auto_increment handling in partitioning).
partition is corrupt
The main problem was that ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR
PARTITION took another code path (over mysql_alter_table instead of
mysql_admin_table) which differs in two ways:
1) alter table opens the tables in a different way than admin tables do
resulting in returning with error before it tried the command
2) alter table does not start to send any diagnostic rows to the client
which the lower admin functions continue to use -> resulting in
assertion crash
The fix:
Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION to use
the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE t.
Adding check in mysql_admin_table to setup the partition list for
which partitions that should be used.
Partitioned tables will still not work with
REPAIR TABLE/PARTITION USE_FRM, since that requires moving partitions
to tables, REPAIR TABLE t USE_FRM, and check that the data still
fulfills the partitioning function and then move the table back to
being a partition.
NOTE: I have removed the following functions from the handler
interface:
analyze_partitions, check_partitions, optimize_partitions,
repair_partitions
Since they are not longer needed.
THIS ALTERS THE STORAGE ENGINE API
mysql-test/r/handler_innodb.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/r/innodb.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/r/innodb_mysql.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/r/partition.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/r/trigger-trans.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/suite/ndb/r/ndb_partition_key.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/ndb/t/ndb_partition_key.test:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/parts/inc/partition_alter4.inc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/parts/r/partition_alter4_innodb.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/parts/r/partition_alter4_myisam.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/rpl/r/rpl_failed_optimize.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/t/partition.test:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
sql/ha_partition.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a function for returning admin commands result rows
Updated handle_opt_partitions to handle admin commands result rows,
and some error filtering (as mysql_admin_table do).
Removed the functions analyze/check/optimize/repair_partitions
since they have no longer any use.
sql/ha_partition.h:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Removed analyze/check/optimize/repair_partitions since they
are no longer are needed.
sql/handler.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Removed analyze/check/optimize/repair_partitions since they
are no longer are needed.
sql/handler.h:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Removed analyze/check/optimize/repair_partitions since they
are no longer are needed.
sql/mysql_priv.h:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added set_part_state for reuse of code in mysql_admin_table.
(Originally fond in sql/sql_partition.cc:prep_alter_part_table)
sql/protocol.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added one assert and a debug print.
sql/sql_partition.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Refactored code for setting up partition state, set_part_state,
now used in both prep_alter_part_table and
sql_table.cc:mysql_admin_table.
Removed code for handling ANALYZE/CHECK/OPTIMIZE/REPAIR partitions,
since it is now handled by mysql_admin_table.
sql/sql_table.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added functionality in mysql_admin_table to work with partitioned
tables.
Fixed a possible assertion bug for HA_ADMIN_TRY_ALTER
(If analyze would output a row, it fails since the row was already
started).
sql/sql_yacc.yy:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION
to use the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE
instead of taking the ALTER TABLE path.
Added reset of alter_info for ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE
since it is now used by partitioned tables.
storage/myisam/mi_check.c:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Changed warning message from "Found X parts Should be: Y parts"
to "Found X key parts. Should be Y", since it could be confusing
with partitioned tables.
problem was that ha_partition::records was not implemented, thus
using the default handler::records, which is not correct if the engine
does not support HA_STATS_RECORDS_IS_EXACT.
Solution was to implement ha_partition::records as a wrapper around
the underlying partitions records.
The rows column in explain partitions will now include the total
number of records in the partitioned table.
(recommit after removing out-commented code)
Problem was that auto_repair, is_crashed and check_and_repair was not
implemented in ha_partition.
Solution, implemented them as loop over all partitions for is_crashed and
check_and_repair, and using the first partition for auto_repair.
(Recommit after fixing review comments)
mysql-test/lib/mtr_report.pl:
Bug#35161: --myisam-recover does not work for partitioned MyISAM tables
Added filter for crashed tables, when testing auto repair
mysql-test/std_data/corrupt_t1#P#p1.MYI:
Bug#35161: --myisam-recover does not work for partitioned MyISAM tables
Corrupt MYI file for testing auto repair
mysql-test/std_data/corrupt_t1.MYI:
Bug#35161: --myisam-recover does not work for partitioned MyISAM tables
Corrupt MYI file for testing auto repair
mysql-test/suite/parts/r/partition_repair_myisam.result:
Bug#35161: --myisam-recover does not work for partitioned MyISAM tables
Result file for testing auto repair of crashed myisam partitions
mysql-test/suite/parts/t/partition_repair_myisam-master.opt:
Bug#35161: --myisam-recover does not work for partitioned MyISAM tables
opt file for testing auto repair of crashed myisam partitions
mysql-test/suite/parts/t/partition_repair_myisam.test:
Bug#35161: --myisam-recover does not work for partitioned MyISAM tables
Test file for testing auto repair of crashed myisam partitions
sql/ha_partition.cc:
Bug#35161: --myisam-recover does not work for partitioned MyISAM tables
Added auto_repair as returning the first partitions auto_repair
Added is_crashed and check_and_repair as loop over all partitions
sql/ha_partition.h:
Bug#35161: --myisam-recover does not work for partitioned MyISAM tables
Activating check_and_repair, auto_repair and is_crashed
- Reserver namespace and place in frm for TABLE_CHECKSUM and PAGE_CHECKSUM create options
- Added syncing of directory when creating .frm files
- Portability fixes
- Added missing cast that could cause bugs
- Code cleanups
- Made some bit functions inline
- Moved things out of myisam.h to my_handler.h to make them more accessable
- Renamed some myisam variables and defines to make them more globaly usable (as they are used outside of MyISAM)
- Fixed bugs in error conditions
- Use compiler time asserts instead of run time
- Fixed indentation
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP as the old name was wrong
(Added a define for old value to ensure we don't break any old code)
Added HA_EXTRA_PREPARE_FOR_RENAME as a signal for rename (before we used a DROP signal which is wrong)
- Initialize error messages early to get better errors when mysqld or an engine fails to start
- Fix windows bug that query_performance_frequency was not initialized if registry code failed
- thread_stack -> my_thread_stack_size
BitKeeper/etc/ignore:
added libmysqld/scheduler.cc libmysqld/sql_connect.cc libmysqld/sql_tablespace.cc
include/Makefile.am:
Added my_bit.h
include/m_string.h:
Added bzero_if_purify() to simplify code
include/my_base.h:
Reserve options for the future
Added HA_OPTION_NULL_FIELDS, HA_OPTION_PAGE_CHECKSUM, HA_CREATE_PAGE_CHECKSUM
Added new error message HA_ERR_NEW_FILE
Added optional new row type BLOCK_RECORD
Renamed HA_EXTRA_PREPARE_FOR_DELETE to HA_EXTRA_PREPARE_FOR_DROP
Added HA_EXTRA_PREARE_FOR_RENAME to inform handler we will do a rename
(Added define to make things compatible until 6.0)
Moved invalidator_by_filename form myisam.h
include/my_dbug.h:
Poirtablity fix
include/my_global.h:
Added helper macros STATIC_INLINE and MY_ERRPTR
Added NEED_EXPLICIT_SYNC_DIR
include/my_handler.h:
Added missing casts
Moved some constants and macros out from myisam.h to make these generally available
Renamed mi_compare_text() to ha_compare_text() as this function is not myisam specific
Renamed mi_portable_sizeof_char_ptr to portable_sizeof_char_ptr
Added registering of handler messages for better error reporting during startup
include/my_sys.h:
Added my_sync_dir() and my_sync_dir_by_file()
More comments
Some indentation fixes
Moved bit functions to my_bit.h
Added prototype for crc32()
include/myisam.h:
Moved things from here to my_handler.h to make them more accessable
libmysql/Makefile.shared:
Added my_sync
mysys/array.c:
Fixed indentation and spelling errors
Split set_dynamic() to two functions
Added allocate_dynamic() as a new visiable function
(no new code, only refactoring)
mysys/mf_iocache.c:
More DBUG
mysys/mf_keycache.c:
More explicite ASSERT
Removed some casts
Fixed indentation
mysys/mf_tempfile.c:
Fixed bug with possible dangling file descriptor
mysys/my_atomic.c:
Use compile time asserts instead of run time
mysys/my_bit.c:
Make most bit functions inline
mysys/my_bitmap.c:
Added my_bit.h
mysys/my_compress.c:
Fixed indentation
mysys/my_create.c:
Added my_sync_by_dir()
mysys/my_delete.c:
Added my_sync_by_dir()
mysys/my_error.c:
init_glob_errs() is now done in my_init()
mysys/my_handler.c:
mi_compare_text() -> ha_compare_text() as this is not MyISAM specific
Added functions to initialize handler error messages
Fixed indentation
More clear usage of include files
mysys/my_init.c:
Added my_thread_stack_size to be used by other programs
Ensure that global error messages are always initialized
Fix windows bug that query_performance_frequency was not initialized if registry code failed
mysys/my_open.c:
More comments
Removed duplicate code
mysys/my_pread.c:
Ensure that my_errno is set even if errno is 0
mysys/my_realloc.c:
Added comment
mysys/my_rename.c:
Added syncing of directories
mysys/my_symlink.c:
Added my_sync_by_dir()
mysys/my_sync.c:
Added my_sync_dir()
On recent Mac OS X, fcntl(F_FULLFSYNC) is recommended over fsync()
(see "man fsync" on Mac OS X 10.3).
my_sync_dir(): to sync a directory after a file creation/deletion/
renaming; can be called directly or via MY_SYNC_DIR in my_create/
my_delete/my_rename(). No-op except on Linux (see "man fsync" on Linux).
my_sync_dir_from_file(): same as above, just more practical when the
caller has a file name but no directory name ready.
Should the #warning even be a #error? I mean do we want to release
binaries which don't guarantee any durability?
mysys/safemalloc.c:
Added sf_malloc_report_allocated() (Debugging aid)
sql/gen_lex_hash.cc:
Remove inline for big function
sql/ha_partition.cc:
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
prepare_for_delete -> prepare_for_rename() as this is the the time this function is called
sql/ha_partition.h:
prepare_for_delete -> prepare_for_rename() as this is the the time this function is called
sql/handler.cc:
ha_init_errors() is now called at startup before plugins
This allows us to get better error messages
sql/handler.h:
Reserve enum value for Maria
Add future proof enum for page checksums
sql/item_func.cc:
Include my_bit.h
sql/lex.h:
Added future proof CREATE table options
sql/log.cc:
Added comment
sql/mysql_priv.h:
thread_stack moved to mysys
sql/mysqld.cc:
thread_stack moved to mysys
thread_stack -> my_thread_stack_size
Initialize myisam key caches before plugins starts
Initialize error to allow storage engine to give better error messages if init failes.
Fixed indentation
Group all MyISAM options together
Added new status variable 'Opened_table_definitions' to allow one to monitor if table definition cache is too small
Clarified some option help messages
sql/opt_range.cc:
Removed wrong usage of SAFE_MODE (this disabled key usage for UPDATES, which was never the intention)
Removed print if total cost in a place where it didn't have any usable value
sql/set_var.cc:
thread_stack -> my_thread_stack
sql/sql_class.cc:
Intialize transaction object properly
sql/sql_parse.cc:
thread_stack -> my_thread_stack
sql/sql_select.cc:
Include my_bit.h
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
sql/sql_show.cc:
Simplify handling of ha_choice variables
Added future safe PAGE_CHECKSUM option
Addid missing 'transactional=#' in information schema
sql/sql_table.cc:
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_FORCE_REOPEN when doing reopen
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_RENAME when doing rename
Removed not needed initialization
sql/sql_test.cc:
thread_stack -> my_thread_stack
sql/sql_yacc.yy:
Simplify handling of ha_choice variables
Added future proof create table options TABLE_CHECKSUM=# & PAGE_CHECKSUM=#
sql/table.cc:
Save page_checksum in .frm
sql/table.h:
Added variable to hold create table option PAGE_CHECKSUM
sql/unireg.cc:
Added syncing of directories
storage/myisam/ft_boolean_search.c:
mi_compare_text() -> ha_compare_text()
storage/myisam/ft_eval.c:
mi_compare_text() -> ha_compare_text()
storage/myisam/ft_nlq_search.c:
mi_compare_text() -> ha_compare_text()
storage/myisam/ft_parser.c:
mi_compare_text() -> ha_compare_text()
storage/myisam/ft_stopwords.c:
mi_compare_text() -> ha_compare_text()
storage/myisam/ft_test1.c:
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/ft_update.c:
mi_compare_text() -> ha_compare_text()
storage/myisam/ha_myisam.cc:
Include my_bit.h
storage/myisam/mi_check.c:
MI_MAX_POSSIBLE_KEY_BUFF -> HA_MAX_POSSIBLE_KEY_BUFF
mi_compare_text() -> ha_compare_text()
Added BLOCK_RECORD to avoid compiler warnings
storage/myisam/mi_checksum.c:
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/mi_create.c:
MI_MAX_POSSIBLE_KEY -> HA_MAX_POSSIBLE_KEY
MI_MAX_KEY_BLOCK_SIZE -> HA_MAX_KEY_BLOCK_SIZE
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/mi_dynrec.c:
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/mi_extra.c:
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
storage/myisam/mi_open.c:
MI_MAX_POSSIBLE_KEY -> HA_MAX_POSSIBLE_KEY
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/mi_packrec.c:
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/mi_range.c:
mi_compare_text -> ha_compare_text
storage/myisam/mi_test1.c:
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/mi_test2.c:
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/mi_unique.c:
mi_compare_text() -> ha_compare_text()
storage/myisam/mi_write.c:
mi_compare_text() -> ha_compare_text()
storage/myisam/myisamchk.c:
Include my_bit.h
storage/myisam/myisamdef.h:
Moved store_key_length_inc to handler.h
storage/myisam/myisampack.c:
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisam/sp_test.c:
mi_portable_sizeof_char_ptr -> portable_sizeof_char_ptr
storage/myisammrg/ha_myisammrg.cc:
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
include/my_bit.h:
New BitKeeper file ``include/my_bit.h''
into trift-lap.fambruehe:/MySQL/M51/antony-5.1
mysql-test/suite/funcs_1/include/ndb_tb2.inc:
Auto merged
mysql-test/suite/funcs_1/r/innodb_views.result:
Auto merged
mysql-test/suite/funcs_1/r/memory_views.result:
Auto merged
mysql-test/suite/funcs_1/r/myisam_views.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_cursors.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_func_view.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_storedproc_07.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_storedproc_08.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_trig_0102.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_trig_03.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_trig_03e.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_trig_0407.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_trig_08.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_trig_09.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_trig_1011ext.result:
Auto merged
mysql-test/suite/funcs_1/r/ndb_views.result:
Auto merged
mysql-test/suite/funcs_1/views/views_master.inc:
Auto merged
sql/ha_partition.h:
Auto merged
storage/innobase/handler/ha_innodb.cc:
Auto merged
configure.in:
Version is updated already.
sql/ha_partition.cc:
Manual merge of 5.1.22 changes into main.
In the ha_partition::position() we don't calculate the number
of the partition of the record, but use m_last_part value instead,
relying on that it's previously set by some other call like ::write_row().
Delete_rows_log_event::do_exec_row() calls find_and_fetch_row(),
where we used position() + rnd_pos() call for the InnoDB-based PARTITION-ed
table as there HA_PRIMARY_KEY_REQUIRED_FOR_POSITION enabled.
fixed by introducing new handler::rnd_pos_by_record() method to be
used for random record-based positioning
sql/ha_partition.cc:
Bug #28430 Failure in replication of innodb partitioned tables on row/mixed format.
ha_partition::rnd_pos_by_record() implemented
sql/ha_partition.h:
Bug #28430 Failure in replication of innodb partitioned tables on row/mixed format.
ha_partition::rnd_pos_by_record() declared
sql/handler.h:
Bug #28430 Failure in replication of innodb partitioned tables on row/mixed format.
handler::rnd_pos_by_record() introduced
sql/log_event.cc:
Bug #28430 Failure in replication of innodb partitioned tables on row/mixed format.
handler::rnd_pos_by_record used instead of position() + rnd_pos() call
