WL#3951 - MyISAM: Additional Error Logs for Data Corruption
When table corruption is detected, in addition to current error message
provide following information:
- list of threads (and queries) accessing a table;
- thread_id of a thread that detected corruption;
- source file name and line number where this corruption was detected;
- optional extra information (string).
----------------------------------------------------------------------
ChangeSet@1.2571, 2008-04-08 12:30:06+02:00, vvaintroub@wva. +122 -0
Bug#32082 : definition of VOID in my_global.h conflicts with Windows
SDK headers
VOID macro is now removed. Its usage is replaced with void cast.
In some cases, where cast does not make much sense (pthread_*, printf,
hash_delete, my_seek), cast is ommited.
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
BUG#47073 - valgrind errs, corruption,failed repair of partition,
low myisam_sort_buffer_size
Fixed race conditions discovered with the provided test case and
stabilized test case.
include/myisam.h:
Serialize submission of messages from multi-threaded REPAIR.
mysql-test/r/myisam.result:
REPAIR output highly depend on threads activity. Disabled
result log to make test case deterministic.
mysql-test/t/myisam.test:
REPAIR output highly depend on threads activity. Disabled
result log to make test case deterministic.
storage/myisam/ha_myisam.cc:
Serialize submission of messages from multi-threaded REPAIR.
storage/myisam/mi_check.c:
Serialize submission of messages from multi-threaded REPAIR.
storage/myisam/sort.c:
Only master thread is allowed to detach write cache from
the share.
BUG#47073 - valgrind errs, corruption,failed repair of partition,
low myisam_sort_buffer_size
Fixed race conditions discovered with the provided test case and
stabilized test case.
Implemented the server infrastructure for the fix:
1. Added a function LEX_STRING *thd_query_string(THD) to return
a LEX_STRING structure instead of char *.
This is the function that must be called in innodb instead of
thd_query()
2. Did some encapsulation in THD : aggregated thd_query and
thd_query_length into a LEX_STRING and made accessor and mutator
methods for easy code updating.
3. Updated the server code to use the new methods where applicable.
Implemented the server infrastructure for the fix:
1. Added a function LEX_STRING *thd_query_string(THD) to return
a LEX_STRING structure instead of char *.
This is the function that must be called in innodb instead of
thd_query()
2. Did some encapsulation in THD : aggregated thd_query and
thd_query_length into a LEX_STRING and made accessor and mutator
methods for easy code updating.
3. Updated the server code to use the new methods where applicable.
low myisam_sort_buffer_size
Repair by sort (default) or parallel repair of a MyISAM table
(doesn't matter partitioned or not) as well as bulk inserts
and enable indexes some times didn't failover to repair with
key cache.
The problem was that after unsuccessful attempt, data file was
closed. Whereas repair with key cache requires open data file.
Fixed by reopening data file.
Also fixed a valgrind warning, which may appear during repair
by sort or parallel repair with certain myisam_sort_buffer_size
number of rows and length of an index entry (very dependent).
mysql-test/r/myisam.result:
A test case for BUG#47073.
mysql-test/t/myisam.test:
A test case for BUG#47073.
storage/myisam/ha_myisam.cc:
Reverted fix for BUG25289. Not needed anymore.
storage/myisam/mi_check.c:
Reopen data file, when repair by sort or parallel repair
fails.
When repair by sort is requested to rebuild data file, data file
gets rebuilt while fixing first index. When rebuild is completed,
info->dfile is pointing to temporary data file, original data file
is closed.
It may happen that repair has successfully fixed first index and
rebuilt data file, but failed to fix second index. E.g.
myisam_sort_buffer_size was big enough to fix first shorter index,
but not enough to fix subsequent longer index.
In this case we end up with info->dfile pointing to temporary file,
which is removed and info->dfile is set to -1.
Though repair by sort failed, the upper layer may still want to
try repair with key cache. But it needs info->dfile pointing to
valid data file.
storage/myisam/sort.c:
When performing a copy of IO_CACHE structure, current_pos and
current_end must be updated separatly to point to memory we're
copying to (not to memory we're copying from).
As t_file2 is always WRITE cache, proper members are write_pos
and write_end accordingly.
low myisam_sort_buffer_size
Repair by sort (default) or parallel repair of a MyISAM table
(doesn't matter partitioned or not) as well as bulk inserts
and enable indexes some times didn't failover to repair with
key cache.
The problem was that after unsuccessful attempt, data file was
closed. Whereas repair with key cache requires open data file.
Fixed by reopening data file.
Also fixed a valgrind warning, which may appear during repair
by sort or parallel repair with certain myisam_sort_buffer_size
number of rows and length of an index entry (very dependent).
* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
bzr branch mysql-5.1-performance-version mysql-trunk # Summit
cd mysql-trunk
bzr merge mysql-5.1-innodb_plugin # which is 5.1 + Innodb plugin
bzr rm innobase # remove the builtin
Next step: build, test fixes.
bzr branch mysql-5.1-performance-version mysql-trunk # Summit
cd mysql-trunk
bzr merge mysql-5.1-innodb_plugin # which is 5.1 + Innodb plugin
bzr rm innobase # remove the builtin
Next step: build, test fixes.
the auto_increment value
This is an alternative patch that instead of allowing RECREATE TABLE
on TRUNCATE TABLE it implements reset_auto_increment that is called
after delete_all_rows.
Note: this bug was fixed by Mattias Jonsson:
Pusing this patch: http://lists.mysql.com/commits/70370
mysql-test/suite/parts/r/partition_auto_increment_memory.result:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
mysql-test/suite/parts/r/partition_auto_increment_myisam.result:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
sql/ha_partition.cc:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate.
storage/heap/ha_heap.cc:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate
storage/heap/ha_heap.h:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate
storage/myisam/ha_myisam.cc:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate.
storage/myisam/ha_myisam.h:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate.
the auto_increment value
This is an alternative patch that instead of allowing RECREATE TABLE
on TRUNCATE TABLE it implements reset_auto_increment that is called
after delete_all_rows.
Note: this bug was fixed by Mattias Jonsson:
Pusing this patch: http://lists.mysql.com/commits/70370
- Added a handler call (prepare_index_scan()) to inform storage engines that an index scan is about to take place.
- Extended the maximun key parts for an index from 16 to 32
- Extended MyISAM and Maria engines to support up to 32 parts
Added checks for return value from ha_index_init()
include/my_handler.h:
Extended number of key parts for MyISAM and Maria from 16 to 32
include/my_pthread.h:
Ensure we always have 256M of stack.
(Required to be able to handle the current number of keys and key parts in MyISAM)
mysql-test/r/create.result:
Extended to test for 32 key parts
mysql-test/r/myisam.result:
Test that we can create 32 but not 33 key parts
mysql-test/r/ps_1general.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_2myisam.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_3innodb.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_4heap.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_5merge.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/suite/maria/r/maria.result:
Max key length is now 1208 bytes
mysql-test/suite/maria/r/maria3.result:
Max key length is now 1208 bytes
mysql-test/suite/maria/r/ps_maria.result:
Max key length is now 1208 byte
mysql-test/t/create.test:
Extended to test for 32 key parts
mysql-test/t/myisam.test:
Test that we can create 32 but not 33 key parts
sql/handler.cc:
Check return value from ha_index_init()
sql/handler.h:
Added a handler call (prepare_index_scan()) to inform storage engines that an index scan is about to take place.
sql/sql_select.cc:
Checks all return values from ha_index_init()
Call prepare_index_scan()) to inform storage engines that an index scan is about to take place.
Fixed indentation
sql/table.cc:
Fixed wrong types for key_length (rest of code assumed this was 32 bit)
sql/unireg.h:
Extended the maximun key parts for an index from 16 to 32
storage/maria/ha_maria.cc:
Don't allocate HA_CHECK on the stack in functions where we call repair() as HA_CHECK is HUGE and will overflow stack
storage/myisam/ha_myisam.cc:
Don't allocate HA_CHECK on the stack in functions where we call repair() as HA_CHECK is HUGE and will overflow stack
storage/myisam/mi_check.c:
Fixed wrong check if value overflow
tests/mysql_client_test.c:
Added fflush() to fix output in case of error
Fixed wrong check of 'ref' length in EXPLAIN
This patch fixes compilation warning, "conversion from 'time_t' to 'ulong',
possible loss of data".
The fix is to typecast time_t to ulong before assigning it to ulong.
Backported this from 6.0-bugteam tree.
storage/archive/ha_archive.cc:
type casting time_t to ulong before assigning.
storage/federated/ha_federated.cc:
type casting time_t to ulong before assigning.
storage/innobase/handler/ha_innodb.cc:
type casting time_t to ulong before assigning.
storage/myisam/ha_myisam.cc:
type casting time_t to ulong before assigning.
This patch fixes compilation warning, "conversion from 'time_t' to 'ulong',
possible loss of data".
The fix is to typecast time_t to ulong before assigning it to ulong.
Backported this from 6.0-bugteam tree.
This patch adds corrections to the original patch
submitted 2009-04-08 (http://lists.mysql.com/commits/71607):
- fixed that the original patch didn't work because of an
incorrect condition;
- added a test case.
mysql-test/r/upgrade.result:
Bug#37631 Incorrect key file for table after upgrading from 5.0 to 5.1
Result file for test case
mysql-test/std_data/bug37631.MYD:
Bug#37631 Incorrect key file for table after upgrading from 5.0 to 5.1
table created in mysql 4.0
mysql-test/std_data/bug37631.MYI:
Bug#37631 Incorrect key file for table after upgrading from 5.0 to 5.1
table created in mysql 4.0
mysql-test/std_data/bug37631.frm:
Bug#37631 Incorrect key file for table after upgrading from 5.0 to 5.1
table created in mysql 4.0
mysql-test/t/upgrade.test:
Bug#37631 Incorrect key file for table after upgrading from 5.0 to 5.1
Adds test for checking that upgrade works
on a table which is created by a mysql
server version <= 4.0.
storage/myisam/ha_myisam.cc:
Bug#37631 Incorrect key file for table after upgrading from 5.0 to 5.1
Fix the conformance checker to relax checking
for the correct version of the tables (for tables
created by mysql server version <= 4.0)
This patch adds corrections to the original patch
submitted 2009-04-08 (http://lists.mysql.com/commits/71607):
- fixed that the original patch didn't work because of an
incorrect condition;
- added a test case.
Killing insert-select statement on MyISAM corrupts the table.
Killing the insert-select statement corrupts the MyISAM table only
when the destination table is empty and when it has indexes. When
we bulk insert huge data and if the destination table is empty we
disable the indexes for fast inserts, data is then inserted and
indexes are re-enabled after bulk_insert operation
Killing the query, aborts the repair table operation during enable
indexes phase leading to table corruption.
We now truncate the table when we detect that enable indexes is
killed for bulk insert query.As we have an empty table before the
operation, we can fix by truncating the table.
mysql-test/r/myisam.result:
Result file for BUG#40827
mysql-test/t/myisam.test:
Testcase for BUG#40827
storage/myisam/ha_myisam.cc:
Fixed end_bulk_insert() method to truncate the table when we detect enable
index operation is killed.
