This commit is based on the work of Michal Schorm, rebased on the
earliest MariaDB version.
Th command line used to generate this diff was:
find ./ -type f \
-exec sed -i -e 's/Foundation, Inc., 59 Temple Place, Suite 330, Boston, /Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, /g' {} \; \
-exec sed -i -e 's/Foundation, Inc. 59 Temple Place.* Suite 330, Boston, /Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, /g' {} \; \
-exec sed -i -e 's/MA.*.....-1307.*USA/MA 02110-1335 USA/g' {} \; \
-exec sed -i -e 's/Foundation, Inc., 59 Temple/Foundation, Inc., 51 Franklin/g' {} \; \
-exec sed -i -e 's/Place, Suite 330, Boston, MA.*02111-1307.*USA/Street, Fifth Floor, Boston, MA 02110-1335 USA/g' {} \; \
-exec sed -i -e 's/MA.*.....-1307/MA 02110-1335/g' {} \;
(Backport to 5.3)
(variant #2, with fixed coding style)
- Make Mrr_ordered_index_reader::resume_read() restore index position
only if it was saved before with Mrr_ordered_index_reader::interrupt_read().
- The crash was caused because the optimizer called handler->multi_range_read_info()
on a derived temporary table. That table has been created, but not opened yet.
Because of that, handler::table was NULL, which caused crash.
Fixed by changing DS-MRR methods to use handler::table_share instead.
handler::table_share is set in handler ctor, so this should be safe.
-Added test and extra code to ensure we don't leave keyread on for a handler table.
-Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
mysql-test/include/default_mysqld.cnf:
Run test suite with smaller aria keybuffer size
mysql-test/suite/maria/maria3.result:
Run test suite with smaller aria keybuffer size
mysql-test/suite/sys_vars/r/aria_pagecache_buffer_size_basic.result:
Run test suite with smaller aria keybuffer size
sql/handler.cc:
Disable key read (extra safety if something went wrong)
sql/multi_range_read.cc:
Ensure we have don't leave keyread on for secondary_file
sql/opt_range.cc:
Simplify code with mark_columns_used_by_index_no_reset()
Ensure that read_keys_and_merge() disableds keyread if it enables it
sql/opt_subselect.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_derived.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_select.cc:
Use 'enable_keyread()' instead of calling HA_EXTRA_RESET. (Makes debugging easier)
Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
Remove not anymore used argument for create_internal_tmp_table()
More DBUG
sql/sql_select.h:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
- Mrr_ordered_index_reader::interrupt_read() and resume_read() should
save/restore not just index lookup tuple, but entire index tuple.
Key parts that are not used for index lookup can be still used in
pushed index condition. Failure to save/restore will cause the index
condition to be evaluated over the wrong values.
Split status variable Rows_read to Rows_read and Rows_tmp_read so that one can see how much real data is read.
Same was done with with Handler_update and Handler_write.
Fixed bug in MEMORY tables where some variables was counted twice.
Added new internal handler call 'ha_close()' to have one place to gather statistics.
Fixed bug where thd->open_options was set to wrong value when doing admin_recreate_table()
mysql-test/r/status.result:
Updated test results and added new tests
mysql-test/r/status_user.result:
Udated test results
mysql-test/t/status.test:
Added new test for temporary table status variables
sql/ha_partition.cc:
Changed to call ha_close() instead of close()
sql/handler.cc:
Added internal_tmp_table variable for easy checking of temporary tables.
Added new internal handler call 'ha_close()' to have one place to gather statistics.
Gather statistics for internal temporary tables.
sql/handler.h:
Added handler variables internal_tmp_table, rows_tmp_read.
Split function update_index_statistics() to two.
Added ha_update_tmp_row() for faster tmp table handling with more statistics.
sql/item_sum.cc:
ha_write_row() -> ha_write_tmp_row()
sql/multi_range_read.cc:
close() -> ha_close()
sql/mysqld.cc:
New status variables: Rows_tmp_read, Handler_tmp_update and Handler_tmp_write
sql/opt_range.cc:
close() -> ha_close()
sql/sql_base.cc:
close() -> ha_close()
sql/sql_class.cc:
Added handling of rows_tmp_read
sql/sql_class.h:
Added new satistics variables.
rows_read++ -> update_rows_read() to be able to correctly count reads to internal temp tables.
Added handler::ha_update_tmp_row()
sql/sql_connect.cc:
Added comment
sql/sql_expression_cache.cc:
ha_write_row() -> ha_write_tmp_row()
sql/sql_select.cc:
close() -> ha_close()
ha_update_row() -> ha_update_tmp_row()
sql/sql_show.cc:
ha_write_row() -> ha_write_tmp_row()
sql/sql_table.cc:
Fixed bug where thd->open_options was set to wrong value when doing admin_recreate_table()
sql/sql_union.cc:
ha_write_row() -> ha_write_tmp_row()
sql/sql_update.cc:
ha_write_row() -> ha_write_tmp_row()
sql/table.cc:
close() -> ha_close()
storage/heap/ha_heap.cc:
Removed double counting of statistic variables.
close -> ha_close() to get tmp table statistics.
storage/maria/ha_maria.cc:
close -> ha_close() to get tmp table statistics.
- "Using MRR" is no longer shown with range access.
- Instead, both range and BKA accesses will show one of the following:
= "Rowid-ordered scan"
= "Key-ordered scan"
= "Key-ordered Rowid-ordered scan"
depending on whether DS-MRR implementation will do scan keys in order, rowids in order,
or both.
- The patch also introduces a way for other storage engines/MRR implementations to
pass information to EXPLAIN output about the properties of employed MRR scans.
- The problem was that Mrr_ordered_index_reader's interrupt_read() and resume_read() would
save and restore 1) index tuple 2) the rowid (as bytes returned by handler->position()). Clustered
primary key columns were not saved/restored.
They are not explicitly present in the index tuple (i.e. table->key_info[secondary_key].key_parts
doesn't list them), but they are actually there, in particular
table->field[clustered_primary_key_member].part_of_key(secondary_key) == 1. Index condition pushdown
code [correctly] uses the latter as inidication that pushed index condition can refer to clustered PK
members.
The fix was to make interrupt_read()/resume_read() to save/restore clustered primary key members as well,
so that we get correct values for them when evaluating pushed index condition.
[3rd attempt: remove the debugging aids, fix comments in testcase]
- Make DsMrr_impl::dsmrr_init() handle the case of
1. 1st MRR scan using DS-MRR strategy (i.e. doing key sorting and rowid sorting)
2. 2nd MRR scan getting a buffer that's too small to fit one key element
and one rowid element, and so falling back to default MRR implementation
In this case, dsmrr_init() is invoked with {primary_handler, secondary_handler}
initialized for DS-MRR scan and have to reset them to be initialized for the
default MRR scan.
(attempt 2, with simplified testcase)
sql/multi_range_read.cc:
Added printing of error if something goes wrong in get_next()
(Not critical for this bug fix, but this was something that I noticed while testing and found missing)
storage/myisam/mi_rkey.c:
Fixed wrong error number in mi_yield_and_check_if_killed()
- In Maria/MyISAM: Release/re-acquire locks to give queries that wait on them a chance to make progress
- In Maria/MyISAM: Change from numeric constants to ICP_RES values.
- In Maria: Do check index condition in maria_rprev() (was lost in the merge/backport?)
- In Maria/MyISAM/XtraDB: Check if the query was killed, and return immediately if it was.
Added new storage engine error: HA_ERR_ABORTED_BY_USER, for handler to signal that it detected a kill of the query and aborted
Authors: Sergey Petrunia & Monty
include/my_base.h:
Added HA_ERR_ABORTED_BY_USER, for handler to signal that it detected a kill of the query and aborted
include/my_handler.h:
Added comment
mysql-test/r/myisam_icp.result:
Updated test
mysql-test/t/myisam_icp.test:
Drop used tables at start of test
Added test case that can help with manual testing of killing index condition pushdown query.
mysys/my_handler_errors.h:
Text for new storage engine error
sql/handler.cc:
If engine got HA_ERR_ABORTED_BY_USER, send kill message.
sql/multi_range_read.cc:
Return error code
storage/maria/ha_maria.cc:
Added ma_killed_in_mariadb() to detect kill.
Ensure that file->external_ref points to TABLE object.
storage/maria/ma_extra.c:
Dummy test-if-killed for standalone
storage/maria/ma_key.c:
If ma_check_index_cond() fails, set my_errno and info->cur_row.lastpos
storage/maria/ma_rkey.c:
Release/re-acquire locks to give queries that wait on them a chance to make progress
Check if the query was killed, and return immediately if it was
storage/maria/ma_rnext.c:
Check if the query was killed, and return immediately if it was
Added missing fast_ma_writeinfo(info)
storage/maria/ma_rnext_same.c:
Check if the query was killed, and return immediately if it was
Added missing fast_ma_writeinfo(info)
storage/maria/ma_rprev.c:
Check if the query was killed, and return immediately if it was
Added missing fast_ma_writeinfo(info) and ma_check_index_cond()
storage/maria/ma_search.c:
Give error message if we find a wrong key
storage/maria/ma_static.c:
Added pointer to test-if-killed function
storage/maria/maria_def.h:
New prototypes
storage/myisam/ha_myisam.cc:
Added mi_killed_in_mariadb()
Ensure that file->external_ref points to TABLE object.
storage/myisam/mi_extra.c:
Dummy test-if-killed for standalone
storage/myisam/mi_key.c:
If ma_check_index_cond() fails, set my_errno and info->lastpos
storage/myisam/mi_rkey.c:
Ensure that info->lastpos= HA_OFFSET_ERROR in case of error
Release/re-acquire locks to give queries that wait on them a chance to make progress
Check if the query was killed, and return immediately if it was
Reorder code to do less things in case of error.
Added missing fast_mi_writeinfo()
storage/myisam/mi_rnext.c:
Check if the query was killed, and return immediately if it was
Simplify old ICP code
Added missing fast_ma_writeinfo(info)
storage/myisam/mi_rnext_same.c:
Check if the query was killed, and return immediately if it was
Added missing fast_mi_writeinfo(info)
storage/myisam/mi_rprev.c:
Check if the query was killed, and return immediately if it was
Simplify error handling of ICP
Added missing fast_mi_writeinfo(info)
storage/myisam/mi_search.c:
Give error message if we find a wrong key
storage/myisam/mi_static.c:
Added pointer to test-if-killed function
storage/myisam/myisamdef.h:
New prototypes
storage/xtradb/handler/ha_innodb.cc:
Added DB_SEARCH_ABORTED_BY_USER and ha_innobase::is_thd_killed()
Check if the query was killed, and return immediately if it was
storage/xtradb/handler/ha_innodb.h:
Added prototype
storage/xtradb/include/db0err.h:
Added DB_SEARCH_ABORTED_BY_USER
storage/xtradb/include/row0mysql.h:
Added possible ICP errors
storage/xtradb/row/row0sel.c:
Use ICP errors instead of constants.
Detect if killed and return B_SEARCH_ABORTED_BY_USER
Switch from "Disable identical key handling optimization when
IndexConditionPushdown is used" approach
To
an approach where we save/restore index tuple and so can use index condition pushdown.
- Make Mrr_ordered_index_reader() save the rowid across scan interruptions
Also
- Fix compiler warning for setup_buffer_sizes()
- Add commented key_copy/key_restore for better handling of a similar issue
with index record being destroyed by scan interruption (which causes
incorrect evaluation of pushed index condition later on).
Changed rows_read and rows_sent status variables to be longlong (to avoid compiler warnings)
sql/item_func.cc:
Fixed wrong usage of alias
sql/item_subselect.cc:
Changed buffer size to ulonglong to be able detect buffers bigger than size_t
sql/item_subselect.h:
Changed buffer size to ulonglong to be able detect buffers bigger than size_t
sql/multi_range_read.cc:
Fixed compiler warning by using correct type for function argument
sql/mysqld.cc:
Changed rows_read and rows_sent status variables to be longlong
sql/opt_subselect.h:
Fixed compiler warning by using correct type for function argument
sql/sql_class.cc:
Changed rows_read and rows_sent status variables to be longlong
sql/sql_class.h:
Changed rows_read and rows_sent status variables to be longlong
Changed max_nulls_in_row to uint as this is number of columns in row.
This fixed some compiler warnings.
sql/sql_select.cc:
Added casts to avoid compiler warnings
storage/heap/ha_heap.cc:
Initilize different types separate
storage/oqgraph/ha_oqgraph.cc:
Fixed argument to store(longlong) to avoid compiler warnings
