Problem: in 5.0 'check table for upgrade' doesn't detect
incompatible collation changes made in 5.0.48.
Fix: backport #39585 fix to 5.0
sql/handler.cc:
Fix for bug#40984: backport fix from 39585 into 5.0
- backport of #39585 fix
sql/handler.h:
Fix for bug#40984: backport fix from 39585 into 5.0
- backport of #39585 fix
Cache size is truncated via 32bit ulong in ha_init_key_cache() and
ha_resize_key_cache()
This change fixes the cast to size_t instead of ulong. This cast is safe,
because key_buffer_size parameter is limited to SIZE_T_MAX
Cache size is truncated via 32bit ulong in ha_init_key_cache() and
ha_resize_key_cache()
This change fixes the cast to size_t instead of ulong. This cast is safe,
because key_buffer_size parameter is limited to SIZE_T_MAX
Problem was that partitioning cached the table flags.
These flags could change due to TRANSACTION LEVEL changes.
Solution was to remove the cache and always return the table flags
from the first partition (if the handler was initialized).
mysql-test/r/partition_innodb_stmt.result:
Bug#39084: Getting intermittent errors with statement-based binary logging
New test result file.
mysql-test/t/partition_innodb_stmt.test:
Bug#39084: Getting intermittent errors with statement-based binary logging
New test file.
sql/ha_partition.cc:
Bug#39084: Getting intermittent errors with statement-based binary logging
Removed m_table_flags, and added m_handler_status.
Added checks that all partitions have the same
table flags.
Moved some variable initializations.
Updated some comments.
Fixed typo initialise -> initialize
Changed HA_EXTTA_NO_READCHECK to do nothing, since it
is only used in ha_open, which is called for every
partition in ::open anyway.
sql/ha_partition.h:
Bug#39084: Getting intermittent errors with statement-based binary logging
Removed m_table_flags, and added m_handler_status.
Always return the first partitions table flags, instead of using
cached table flags.
Added define of enabled/disabled partitioning table flags
Fixed type initialise -> initialize
Updated some comments.
sql/handler.cc:
Bug#39084: Getting intermittent errors with statement-based binary logging
Fixed type initialise -> initialize.
sql/handler.h:
Bug#39084: Getting intermittent errors with statement-based binary logging
Added comment to understand where the cached value is set.
Problem was that partitioning cached the table flags.
These flags could change due to TRANSACTION LEVEL changes.
Solution was to remove the cache and always return the table flags
from the first partition (if the handler was initialized).
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
mysql-test/r/xa.result:
Add test case result for Bug#28323
mysql-test/t/xa.test:
Add test case for Bug#28323
sql/handler.cc:
Reset XID only at the end of the global transaction.
sql/share/errmsg.txt:
Add new error codes.
sql/sql_class.h:
Remember the error reported by the Resource Manager.
sql/sql_parse.cc:
Rollback the transaction if the Resource Manager reported
a error and rolled back its branch of the transaction.
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
mysql-test/r/xa.result:
Add test case result for Bug#28323
mysql-test/t/xa.test:
Add test case for Bug#28323
sql/handler.cc:
Reset XID only at the end of the global transaction.
sql/share/errmsg.txt:
Add new error codes.
sql/sql_class.h:
Remember the error reported by the Resource Manager.
sql/sql_parse.cc:
Rollback the transaction if the Resource Manager reported
a error and rolled back its branch of the transaction.
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
collation change made in 5.1.24-rc
Problem: 'CHECK TABLE ... FOR UPGRADE' did not check for
incompatible collation changes made in MySQL 5.1.24-rc.
Fix: add the check.
sql/handler.cc:
- check for incompatible collation changes made in 5.1.24-rc:
bug #27877:
utf8_general_ci
ucs2_general_ci
collation change made in 5.1.24-rc
Problem: 'CHECK TABLE ... FOR UPGRADE' did not check for
incompatible collation changes made in MySQL 5.1.24-rc.
Fix: add the check.
upgrade from <=5.0.46 to >=5.0.48
Problem: 'check table .. for upgrade' doesn't detect
incompatible collation changes made in 5.0.48.
Fix: check for incompatible collation changes.
sql/handler.cc:
Fix for bug#39585: innodb and myisam corruption after binary
upgrade from <=5.0.46 to >=5.0.48
- check for incompatible collation changes made in 5.0.48:
bug #29461
latin7_general_ci
latin7_general_cs
latin7_estonian_cs
latin2_hungarian_ci
koi8u_general_ci
cp1251_ukrainian_ci
cp1250_general_ci
bug #29499, bug #27562
ascii_general_ci
upgrade from <=5.0.46 to >=5.0.48
Problem: 'check table .. for upgrade' doesn't detect
incompatible collation changes made in 5.0.48.
Fix: check for incompatible collation changes.
The failure was caused by executing a CREATE-SELECT statement that creates a
table in another database than the current one. In row-based logging, the
CREATE statement was written to the binary log without the database, hence
creating the table in the wrong database, causing the following inserts to
fail since the table didn't exist in the given database.
Fixed the bug by adding a parameter to store_create_info() that will make
the function print the database name before the table name and used that
in the calls that write the CREATE statement to the binary log. The database
name is only printed if it is different than the currently selected database.
The output of SHOW CREATE TABLE has not changed and is still printed without
the database name.
mysql-test/suite/rpl/t/rpl_row_create_table.test:
Added test to check that CREATE-SELECT into another database than the
current one replicates.
sql/sql_insert.cc:
Adding parameter to calls to store_create_info().
sql/sql_show.cc:
Adding parameter to calls to store_create_info().
Extending store_create_info() with parameter 'show_database' that will cause
the database to be written before the table name.
sql/sql_show.h:
Adding parameter to call to store_create_info() to tell if the database should be shown or not.
sql/sql_table.cc:
Adding parameter to calls to store_create_info().
The failure was caused by executing a CREATE-SELECT statement that creates a
table in another database than the current one. In row-based logging, the
CREATE statement was written to the binary log without the database, hence
creating the table in the wrong database, causing the following inserts to
fail since the table didn't exist in the given database.
Fixed the bug by adding a parameter to store_create_info() that will make
the function print the database name before the table name and used that
in the calls that write the CREATE statement to the binary log. The database
name is only printed if it is different than the currently selected database.
The output of SHOW CREATE TABLE has not changed and is still printed without
the database name.
"Trigger fired multiple times leads to gaps in auto_increment sequence".
The bug was that if a trigger fired multiple times inside a top
statement (for example top-statement is a multi-row INSERT,
and trigger is ON INSERT), and that trigger inserted into an auto_increment
column, then gaps could be observed in the auto_increment sequence,
even if there were no other users of the database (no concurrency).
It was wrong usage of THD::auto_inc_intervals_in_cur_stmt_for_binlog.
Note that the fix changes "class handler", I'll tell the Storage Engine API team.
mysql-test/r/trigger-trans.result:
result; before the bugfix, the sequence was 1,2,4,6,8,10,12...
mysql-test/t/trigger-trans.test:
test for BUG#31612
sql/handler.cc:
See revision comment of handler.h.
As THD::auto_inc_intervals_in_cur_stmt_for_binlog is cumulative
over all trigger invokations by the top statement, the
second invokation of the trigger arrived in handler::update_auto_increment()
with already one interval in
THD::auto_inc_intervals_in_cur_stmt_for_binlog. The method thus
believed it had already reserved one interval for that invokation,
thus reserved a twice larger interval (heuristic when we don't know
how large the interval should be: we grow by powers of two). InnoDB
thus increased its internal per-table auto_increment counter by 2
while only one row was to be inserted. Hence a gap in the sequence.
The fix is to use the new handler::auto_inc_intervals_count.
Note that the trigger's statement knows how many rows it is going
to insert, but provides estimation_rows_to_insert == 0 (see comments
in sql_insert.cc why triggers don't call handler::ha_start_bulk_insert()).
* removing white space at end of line
* we don't need to maintain THD::auto_inc_intervals_in_cur_stmt_for_binlog
if no binlogging or if row-based binlogging. Using auto_inc_intervals_count in
the heuristic makes the heuristic independent of binary logging, which is good.
sql/handler.h:
THD::auto_inc_intervals_in_cur_stmt_for_binlog served
- for binlogging
- as a heuristic when we have no estimation of how many records the
statement will insert.
But the first goal needs to be cumulative over all statements which
form a binlog event, while the second one needs to be attached to each
statement. THD::auto_inc_intervals_in_cur_stmt_for_binlog is cumulative,
leading to BUG#31612. So we introduce handler::auto_inc_intervals_count
for the second goal. See the revision comment of handler.cc.
A smaller issue was that, even when the binlog event was only one
statement (no triggers, no stored functions),
THD::auto_inc_intervals_in_cur_stmt.nb_elements() could be lower than
the number of reserved intervals (fooling the heuristic), because its
append() method collapses two contiguous intervals in one.
Note that as auto_inc_intervals_count is in class 'handler' and not
in class 'THD', it does not need to be handled in
THD::reset|restore_sub_statement_state().
sql/log.cc:
Comment is wrong: if auto_increment is second, in handler::update_auto_increment()
'append' is false and so auto_inc_intervals_in_cur_stmt_for_binlog
is empty, we do not come here.
sql/sql_class.h:
comment
"Trigger fired multiple times leads to gaps in auto_increment sequence".
The bug was that if a trigger fired multiple times inside a top
statement (for example top-statement is a multi-row INSERT,
and trigger is ON INSERT), and that trigger inserted into an auto_increment
column, then gaps could be observed in the auto_increment sequence,
even if there were no other users of the database (no concurrency).
It was wrong usage of THD::auto_inc_intervals_in_cur_stmt_for_binlog.
Note that the fix changes "class handler", I'll tell the Storage Engine API team.
This patch contains fixes for two problems:
1. As originally reported, the server crashed on Mac OS X when trying to access
an EXAMPLE table after the EXAMPLE plugin was installed.
It turned out that the dynamically loaded EXAMPLE plugin called the
function hash_earch() from a Mac OS X system library, instead of
hash_earch() from MySQL's mysys library. Makefile.am in storage/example
does not include libmysys. So the Mac OS X linker arranged the hash_search()
function to be linked to the system library when the shared object is
loaded.
One possible solution would be to include libmysys into the linkage of
dynamic plugins. But then we must have a libmysys.so, which must be
used by the server too. This could have a minimal performance impact,
but foremost the change seems to bee too risky at the current state of
MySQL 5.1.
The selected solution is to rename MySQL's hash_search() to my_hash_search()
like it has been done before with hash_insert() and hash_reset().
Since this is the third time, we need to rename a hash_*() function,
I did renamed all hash_*() functions to my_hash_*().
To avoid changing a zillion calls to these functions, and announcing
this to hundreds of developers, I added defines that map the old names
to the new names.
This change is in hash.h and hash.c.
2. The other problem was improper implementation of the handlerton-to-plugin
mapping. We use a fixed-size array to hold a plugin reference for each
handlerton. On every install of a handler plugin, we allocated a new slot
of the array. On uninstall we did not free it. After some uninstall/install
cycles the array overflowed. We did not check for overflow.
One fix is to check for overflow to stop the crashes.
Another fix is to free the array slot at uninstall and search for a free slot
at plugin install.
This change is in handler.cc.
include/hash.h:
Bug#37958 - test main.plugin crash on Mac OS X when selecting from EXAMPLE engine.
Renamed hash_*() functions to my_hash_*().
Added defines that map old names to new names.
mysys/hash.c:
Bug#37958 - test main.plugin crash on Mac OS X when selecting from EXAMPLE engine.
Renamed hash_*() functions to my_hash_*().
sql/handler.cc:
Bug#37958 - test main.plugin crash on Mac OS X when selecting from EXAMPLE engine.
Protect against a failing ha_initialize_handlerton() in ha_finalize_handlerton().
Free hton2plugin slot on uninstall of a handler plugin.
Reuse freed slost of the hton2plugin array.
Protect against array overrun.
This patch contains fixes for two problems:
1. As originally reported, the server crashed on Mac OS X when trying to access
an EXAMPLE table after the EXAMPLE plugin was installed.
It turned out that the dynamically loaded EXAMPLE plugin called the
function hash_earch() from a Mac OS X system library, instead of
hash_earch() from MySQL's mysys library. Makefile.am in storage/example
does not include libmysys. So the Mac OS X linker arranged the hash_search()
function to be linked to the system library when the shared object is
loaded.
One possible solution would be to include libmysys into the linkage of
dynamic plugins. But then we must have a libmysys.so, which must be
used by the server too. This could have a minimal performance impact,
but foremost the change seems to bee too risky at the current state of
MySQL 5.1.
The selected solution is to rename MySQL's hash_search() to my_hash_search()
like it has been done before with hash_insert() and hash_reset().
Since this is the third time, we need to rename a hash_*() function,
I did renamed all hash_*() functions to my_hash_*().
To avoid changing a zillion calls to these functions, and announcing
this to hundreds of developers, I added defines that map the old names
to the new names.
This change is in hash.h and hash.c.
2. The other problem was improper implementation of the handlerton-to-plugin
mapping. We use a fixed-size array to hold a plugin reference for each
handlerton. On every install of a handler plugin, we allocated a new slot
of the array. On uninstall we did not free it. After some uninstall/install
cycles the array overflowed. We did not check for overflow.
One fix is to check for overflow to stop the crashes.
Another fix is to free the array slot at uninstall and search for a free slot
at plugin install.
This change is in handler.cc.
SET col
When reporting a duplicate key error the server was making incorrect assumptions
on what the state of the value string to include in the error is.
Fixed by accessing the data in this string in a "safe" way (without relying on it
having a terminating 0).
Detected by code analysis and fixed a similar problem in reporting the foreign key
duplicate errors.
mysql-test/r/type_set.result:
Bug #38701: test case
mysql-test/t/type_set.test:
Bug #38701: test case
sql/handler.cc:
Bug #38701: don't rely on the presence of a terminating 0 in the string
SET col
When reporting a duplicate key error the server was making incorrect assumptions
on what the state of the value string to include in the error is.
Fixed by accessing the data in this string in a "safe" way (without relying on it
having a terminating 0).
Detected by code analysis and fixed a similar problem in reporting the foreign key
duplicate errors.
partition is corrupt
The main problem was that ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR
PARTITION took another code path (over mysql_alter_table instead of
mysql_admin_table) which differs in two ways:
1) alter table opens the tables in a different way than admin tables do
resulting in returning with error before it tried the command
2) alter table does not start to send any diagnostic rows to the client
which the lower admin functions continue to use -> resulting in
assertion crash
The fix:
Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION to use
the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE t.
Adding check in mysql_admin_table to setup the partition list for
which partitions that should be used.
Partitioned tables will still not work with
REPAIR TABLE/PARTITION USE_FRM, since that requires moving partitions
to tables, REPAIR TABLE t USE_FRM, and check that the data still
fulfills the partitioning function and then move the table back to
being a partition.
NOTE: I have removed the following functions from the handler
interface:
analyze_partitions, check_partitions, optimize_partitions,
repair_partitions
Since they are not longer needed.
THIS ALTERS THE STORAGE ENGINE API
mysql-test/r/handler_innodb.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/r/innodb.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/r/innodb_mysql.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/r/partition.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/r/trigger-trans.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/suite/ndb/r/ndb_partition_key.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/ndb/t/ndb_partition_key.test:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/parts/inc/partition_alter4.inc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/parts/r/partition_alter4_innodb.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/parts/r/partition_alter4_myisam.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
mysql-test/suite/rpl/r/rpl_failed_optimize.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a note result row.
mysql-test/t/partition.test:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Updated after fixing ANALYZE/CHECK/OPTIMIZE/REPAIR partitioned
tables.
sql/ha_partition.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added a function for returning admin commands result rows
Updated handle_opt_partitions to handle admin commands result rows,
and some error filtering (as mysql_admin_table do).
Removed the functions analyze/check/optimize/repair_partitions
since they have no longer any use.
sql/ha_partition.h:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Removed analyze/check/optimize/repair_partitions since they
are no longer are needed.
sql/handler.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Removed analyze/check/optimize/repair_partitions since they
are no longer are needed.
sql/handler.h:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Removed analyze/check/optimize/repair_partitions since they
are no longer are needed.
sql/mysql_priv.h:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added set_part_state for reuse of code in mysql_admin_table.
(Originally fond in sql/sql_partition.cc:prep_alter_part_table)
sql/protocol.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added one assert and a debug print.
sql/sql_partition.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Refactored code for setting up partition state, set_part_state,
now used in both prep_alter_part_table and
sql_table.cc:mysql_admin_table.
Removed code for handling ANALYZE/CHECK/OPTIMIZE/REPAIR partitions,
since it is now handled by mysql_admin_table.
sql/sql_table.cc:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Added functionality in mysql_admin_table to work with partitioned
tables.
Fixed a possible assertion bug for HA_ADMIN_TRY_ALTER
(If analyze would output a row, it fails since the row was already
started).
sql/sql_yacc.yy:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION
to use the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE
instead of taking the ALTER TABLE path.
Added reset of alter_info for ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE
since it is now used by partitioned tables.
storage/myisam/mi_check.c:
Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
partition is corrupt
Changed warning message from "Found X parts Should be: Y parts"
to "Found X key parts. Should be Y", since it could be confusing
with partitioned tables.
partition is corrupt
The main problem was that ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR
PARTITION took another code path (over mysql_alter_table instead of
mysql_admin_table) which differs in two ways:
1) alter table opens the tables in a different way than admin tables do
resulting in returning with error before it tried the command
2) alter table does not start to send any diagnostic rows to the client
which the lower admin functions continue to use -> resulting in
assertion crash
The fix:
Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION to use
the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE t.
Adding check in mysql_admin_table to setup the partition list for
which partitions that should be used.
Partitioned tables will still not work with
REPAIR TABLE/PARTITION USE_FRM, since that requires moving partitions
to tables, REPAIR TABLE t USE_FRM, and check that the data still
fulfills the partitioning function and then move the table back to
being a partition.
NOTE: I have removed the following functions from the handler
interface:
analyze_partitions, check_partitions, optimize_partitions,
repair_partitions
Since they are not longer needed.
THIS ALTERS THE STORAGE ENGINE API
