crashes server!
The problem affects the scenario when index merge is followed by a filesort
and the sort buffer is not big enough for all the sort keys.
In this case the filesort function will read the data to the end through the
index merge quick access method (and thus closing the cursor etc),
but will leave the pointer to the quick select method in place.
It will then create a temporary file to hold the results of the filesort and
will add it as a sort output file (in sort.io_cache).
Note that filesort will copy the original 'sort' structure in an automatic
variable and restore it after it's done.
As a result at exiting filesort() we have a sort.io_cache filled in and
nothing else (as a result of close of the cursors at end of reading data
through index merge).
Now create_sort_index() will note that there is a select and will clean it up
(as it's been used already by filesort() reading the data in). While doing that
a special case in the index merge destructor will clean up the sort.io_cache,
assuming it's an output of the index merge method and is not needed anymore.
As a result the code that tries to read the data back from the filesort output
will get no data in both memory and disk and will crash.
Fixed similarly to how filesort() does it : by copying the sort.io_cache structure
to a local variable, removing the pointer to the io_cache (so that it's not freed
by QUICK_INDEX_MERGE_SELECT::~QUICK_INDEX_MERGE_SELECT) and restoring the original
structure (together with the valid pointer) after the cleanup is done.
This is a safe thing to do because all the structures are already cleaned up by
hitting the end of the index merge's read method (QUICK_INDEX_MERGE_SELECT::get_next())
and the cleanup code being written in a way that tolerates repeating cleanups.
mysql-test/r/index_merge.result:
Bug #44810: test case
mysql-test/t/index_merge.test:
Bug #44810: test case
sql/sql_select.cc:
Bug #44810: preserve the io_cache produced by filesort while cleaning up
the index merge quick access method (QUICK_INDEX_MERGE_SELECT).
- QUICK_INDEX_MERGE_SELECT deinitializes its rnd_pos() scan when it reaches EOF, but we
need to make the deinitialization in QUICK_INDEX_MERGE_SELECT destructor also. This is because
certain execution strategies can stop scanning without reaching EOF, then then try to do a full
table scan on this table. Failure to deinitialize caused the full scan to use (already empty)
table->sort and produce zero records.
mysql-test/r/index_merge.result:
BUG#40974: Incorrect query results when using clause evaluated using range check
- Testcase
mysql-test/t/index_merge.test:
BUG#40974: Incorrect query results when using clause evaluated using range check
- Testcase
When analyzing the possible index use cases the server was re-using an internal structure.
This is wrong, as this internal structure gets updated during the analysis.
Fixed by making a copy of the internal structure for every place it needs to be used.
Also stopped the generation of empty SEL_TREE structures that unnecessary
complicate the analysis.
mysql-test/r/index_merge.result:
Bug#37943: test case
mysql-test/t/index_merge.test:
Bug#37943: test case
sql/opt_range.cc:
Bug#37943:
- Make copy constructors for SEL_TREE and sub-structures and use them when OR-ing trees.
- don't generate empty SEL_TREEs. Return NULL instead.
- return HA_KEY_SCAN_NOT_ROR flag for HASH indexes;
- Fix ha_heap::cmp_ref() to work with BTREE index scans.
mysql-test/r/index_merge.result:
BUG#29740: testcase
mysql-test/t/index_merge.test:
BUG#29740: testcase
sql/ha_heap.h:
BUG#29740: Wrong query results for index_merge/union over HEAP table.
- make HEAP table engine return HA_KEY_SCAN_NOT_ROR flag for HASH
indexes,as HASH index does not guarantee any ordering for rows
within the hash bucket.
- Fix BTREE indexes: make ha_heap::cmp_ref() compare the rowids in the
same way as ha_key_cmp() does.
sql/opt_range.cc:
BUG#29740: Fix comment about ROR scans.
Only MyISAM tables locked with LOCK TABLES ... WRITE were affected.
A query that is optimized with index_merge doesn't reflect rows
inserted within LOCK TABLES.
MyISAM doesn't flush a state within LOCK TABLES. index_merge
optimization creates a copy of the handler, which thus gets
outdated MyISAM state.
New handler->clone() method is introduced to fix this problem.
For non-MyISAM storage engines it allocates a handler and opens
it with ha_open(). For MyISAM it additionally copies MyISAM state
pointer to cloned handler.
mysql-test/r/index_merge.result:
A test case for bug#20256.
mysql-test/t/index_merge.test:
A test case for bug#20256.
sql/ha_myisam.cc:
clone method added to handler class.
sql/ha_myisam.h:
clone method added to handler class.
sql/handler.cc:
clone method added to handler class.
sql/handler.h:
clone method added to handler class.
sql/opt_range.cc:
Use handler clone method.
1. Fix index access costs for MERGE tables, set block_size=myisam_block_size/#underlying_tables
instead of 0 which it was before.
2. Make index scans on MERGE table to return records in (key_tuple, merge_table_rowid) order,
instead of just (key_tuple) order. This makes an index scan on MERGE table to be truly a ROR-scan
which is a requirement for index_merge union/intersection.
myisammrg/myrg_queue.c:
BUG#17314: Make index scans on MERGE table return records ordered by (keytuple, merge_table_rowid).
mysql-test/r/index_merge.result:
Testcase for BUG#17314
mysql-test/r/merge.result:
BUG#17314: update testcase result
mysql-test/t/index_merge.test:
Testcase for BUG#17314
sql/ha_myisammrg.cc:
BUG#17314: For MERGE tables, set handler::block_size to myisam_block_size/#underlying_tables, and not to 0.
account that "FORCE INDEX" disables full table scans, and not range/index_merge scans.
(with post-review fixes)
mysql-test/r/index_merge.result:
Testcase for BUG#16166
mysql-test/t/index_merge.test:
Testcase for BUG#16166
sql/sql_select.cc:
BUG#16166: "Can't use index_merge with FORCE INDEX": adjust the heuristics check:
if (force-index-is-used && there-is-possible-ref-access &&
+ THERE IS NO POSSIBLE RANGE/INDEX_MERGE ACCESS)
{
...
myisam_max_extra_sort_file_size is depricated
Ensure that myisam_data_pointer_size is honoured when creating new MyISAM files
Changed default value of myisam_data_pointer_size from 4 to 6 to get rid of 'table-is-full' errors
include/myisam.h:
Change type of myisam_block_size and myisam_concurrent_insert to make them changeable in mysqld.cc
Removed not used varaible myisam_max_extra_temp_length
include/thr_lock.h:
Added extra parameter to get_status
myisam/mi_create.c:
Ensure that myisam_data_pointer_size is honoured
myisam/mi_dynrec.c:
If 'append_insert_at_end' is set, only write at end of myisam record file
myisam/mi_locking.c:
Add extra argument to 'mi_get_status' to allow thr_lock to signal that we want to do a concurrent insert
If this is used, we will append new insert rows at end of data file.
Change mi_check_status() to allow concurrent_inserts even if there are holes in the file when myisam_concurent_insert=2
myisam/mi_static.c:
Change behavior of myisam_concurrent_insert so that setting this to 2 allows inserts even if there is a hole in the data file.
Default value is 2 for MyISAM direct usage but will be set to 1 (old default) by mysqld.cc
myisam/mi_statrec.c:
If 'append_insert_at_end' is set, only write at end of myisam record file
myisam/mi_write.c:
If 'append_insert_at_end' is set, only write at end of myisam record file
myisam/myisamdef.h:
Support for insert-at-end even if there is holes in data file
mysql-test/r/gis-rtree.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/group_min_max.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/index_merge.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/index_merge_ror.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/merge.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/myisam.result:
Test of concurrent_insert=2
mysql-test/r/null.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/preload.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/ps_1general.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/range.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/r/variables.result:
concurrent_insert is now a integer, not a boolean
myisam_extra_sort_file_size is deleted
mysql-test/r/view.result:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/t/index_merge.test:
Test result changed because default value for myisam_data_pointer_size is changed from 4 -> 6
mysql-test/t/myisam.test:
Test of concurrent_insert=2
mysql-test/t/ps_1general.test:
Drop test table
mysql-test/t/variables.test:
concurrent_insert is now a integer, not a boolean
myisam_extra_sort_file_size is deleted
mysql-test/t/view.test:
Drop tables that may be left from previous tests
mysys/thr_lock.c:
Extra paramter to get_status to signal if concurrent_insert was used
sql/mysqld.cc:
concurrent-insert can now be set to 2 for concurrent inserts when there is holes in the data file
myisam_max_extra_sort_file_size is depricated
sql/set_var.cc:
concurrent-insert is now an integer, not a bool
myisam_max_extra_sort_file_size is deleted
* Make index merge quick selects code allow perform several scans.
* Delay additional handler objects creation till row retrieval is started.
include/queues.h:
Added trivial queue_remove_all macro
mysql-test/r/index_merge.result:
Testcase for BUG#4177
mysql-test/t/index_merge.test:
Testcase for BUG#4177
sql/opt_range.cc:
Fix for BUG#4177:
* For any quick select, row retrieval can be performed several times. Now all index_merge quick selects code handles such cases properly.
* In QUICK_INDEX_MERGE_SELECT we use one handler object for all merged scans, and it was possible that in
destructor several cleanup functions were called
* Additionally
- Removed redundant QUICK_INDEX_MERGE_SELECT members.
- Now QUICK_ROR_*_SELECTs create additional handler objects only when row retrieval is started
So if join optimizer chooses other access method, we don't create/delete handlers.
Note: One sp.test still fails (prime), and rpl_server_id2.test fails (will be fixed by guilhem ASAP).
mysql-test/r/index_merge.result:
Fixed syntax error (no ',' after last index in create table any more).
mysql-test/r/index_merge_bdb.result:
Fixed syntax error (no ',' after last index in create table any more).
mysql-test/r/index_merge_innodb.result:
Fixed syntax error (no ',' after last index in create table any more).
mysql-test/r/index_merge_innodb2.result:
Fixed syntax error (no ',' after last index in create table any more).
mysql-test/r/rpl_server_id1.result:
Update result after merge.
mysql-test/r/sp-error.result:
Update result after merge.
mysql-test/r/variables.result:
Update result after merge.
mysql-test/t/index_merge.test:
Fixed syntax error (no ',' after last index in create table any more).
mysql-test/t/index_merge_bdb.test:
Fixed syntax error (no ',' after last index in create table any more).
mysql-test/t/index_merge_innodb.test:
Fixed syntax error (no ',' after last index in create table any more).
mysql-test/t/index_merge_innodb2.test:
Fixed syntax error (no ',' after last index in create table any more).
mysql-test/t/sp-error.test:
Post-merge fix of error codes.
sql/opt_range.cc:
Manually merged by Monty.
sql/opt_range.h:
Manually merged by Monty.
sql/slave.cc:
Post-merge fixes with some help from Guilhem.
sql/slave.h:
Post-merge fixes with some help from Guilhem.
sql/sp_head.cc:
Got rid of warning (reordering initialization).
sql/sql_parse.cc:
Post-merge fix: Need to set/reset select_limit at SP CALL time as well.
tests/client_test.c:
Post-merge fix: key_len length in explain has changed.
