- Switch from uniform graph to bipartite graph with two kinds of nodes:
"values" (tables and fields) and "modules" (t.col=func(...) equalities,
multi-equalities, unique keys, inner sides of outer joins).
- Rename functions, classes, etc.
Address review feedback:
- Change from Wave-based approach (a-la const table detection) to
building and walking functional dependency graph.
- Change from piggy-backing on ref-access code and KEYUSE structures
to using our own expression analyzer.
sql/item.cc:
MWL#17: Table elimination
- Move from C-ish Field_processor_info to C++ ish and generic Field_enumerator
sql/item.h:
MWL#17: Table elimination
- Move from C-ish Field_processor_info to C++ ish and generic Field_enumerator
sql/sql_bitmap.h:
MWL#17: Table elimination
- Backport of Table_map_iterator from 6.0
This allows us to skip and speed up some very common character converts that MySQL is doing when sending data to the client
and this gives us a nice speed increase for most queries that uses only characters in the range 0x00-0x7f.
This code is based on Alexander Barkov's code that he has done in MySQL 6.0
include/m_ctype.h:
Added MY_CS_NONASCII marker
libmysqld/lib_sql.cc:
Added function net_store_data(...) that takes to and from CHARSET_INFO * as arguments
mysys/charset.c:
Mark character sets with MY_CS_NONASCII
scripts/mysql_install_db.sh:
Fixed messages to refer to MariaDB instead of MySQL
sql/protocol.cc:
Added function net_store_data(...) that takes to and from CHARSET_INFO * as arguments
sql/protocol.h:
Added function net_store_data(...) that takes to and from CHARSET_INFO * as arguments
sql/sql_string.cc:
Quicker copy of strings with no characters above 0x7f
strings/conf_to_src.c:
Added printing of MY_CS_NONASCII
strings/ctype-extra.c:
Mark incompatible character sets with MY_CS_NONASCII
Removed duplicated character set geostd
strings/ctype-sjis.c:
Mark incompatible character sets with MY_CS_NONASCII
strings/ctype-uca.c:
Mark incompatible character sets with MY_CS_NONASCII
strings/ctype-ucs2.c:
Mark incompatible character sets with MY_CS_NONASCII
strings/ctype-utf8.c:
Mark incompatible character sets with MY_CS_NONASCII
strings/ctype.c:
Added function to check if character set is compatible with latin1 in ranges 0x00-0x7f
- Last fixes
sql/item.cc:
MWL#17: Table elimination
- Don't make multiple calls of ::walk(check_column_usage_processor),
call once and cache the value
sql/item.h:
MWL#17: Table elimination
- s/KEYUSE::usable/KEYUSE::type/, more comments
sql/opt_table_elimination.cc:
MWL#17: Table elimination
- Don't make multiple calls of ::walk(check_column_usage_processor),
call once and cache the value
sql/sql_select.cc:
MWL#17: Table elimination
- s/KEYUSE::usable/KEYUSE::type/, more comments
sql/sql_select.h:
MWL#17: Table elimination
- s/KEYUSE::usable/KEYUSE::type/, more comments
sql/table.h:
MWL#17: Table elimination
- Better comments
- Added a handler call (prepare_index_scan()) to inform storage engines that an index scan is about to take place.
- Extended the maximun key parts for an index from 16 to 32
- Extended MyISAM and Maria engines to support up to 32 parts
Added checks for return value from ha_index_init()
include/my_handler.h:
Extended number of key parts for MyISAM and Maria from 16 to 32
include/my_pthread.h:
Ensure we always have 256M of stack.
(Required to be able to handle the current number of keys and key parts in MyISAM)
mysql-test/r/create.result:
Extended to test for 32 key parts
mysql-test/r/myisam.result:
Test that we can create 32 but not 33 key parts
mysql-test/r/ps_1general.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_2myisam.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_3innodb.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_4heap.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_5merge.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/suite/maria/r/maria.result:
Max key length is now 1208 bytes
mysql-test/suite/maria/r/maria3.result:
Max key length is now 1208 bytes
mysql-test/suite/maria/r/ps_maria.result:
Max key length is now 1208 byte
mysql-test/t/create.test:
Extended to test for 32 key parts
mysql-test/t/myisam.test:
Test that we can create 32 but not 33 key parts
sql/handler.cc:
Check return value from ha_index_init()
sql/handler.h:
Added a handler call (prepare_index_scan()) to inform storage engines that an index scan is about to take place.
sql/sql_select.cc:
Checks all return values from ha_index_init()
Call prepare_index_scan()) to inform storage engines that an index scan is about to take place.
Fixed indentation
sql/table.cc:
Fixed wrong types for key_length (rest of code assumed this was 32 bit)
sql/unireg.h:
Extended the maximun key parts for an index from 16 to 32
storage/maria/ha_maria.cc:
Don't allocate HA_CHECK on the stack in functions where we call repair() as HA_CHECK is HUGE and will overflow stack
storage/myisam/ha_myisam.cc:
Don't allocate HA_CHECK on the stack in functions where we call repair() as HA_CHECK is HUGE and will overflow stack
storage/myisam/mi_check.c:
Fixed wrong check if value overflow
tests/mysql_client_test.c:
Added fflush() to fix output in case of error
Fixed wrong check of 'ref' length in EXPLAIN
- fix a typo bug in has_eqref_access_candidate()
- Adjust test to remove race condition
mysql-test/r/mysql-bug41486.result:
Adjust test to remove race condition
mysql-test/t/mysql-bug41486.test:
Adjust test to remove race condition
sql/item.cc:
MWL#17: Table elimination:
fix a typo bug in has_eqref_access_candidate()
mysql-test/r/variables_debug.result:
Test that sessions do not share the same session debug variable.
mysql-test/t/variables_debug.test:
Test that sessions do not share the same session debug variable.
sql/set_var.cc:
As soon as default setting are shared between sessions we should push dbug state before changing debug setting first time.
- More testcases
- Let add_ft_key() set keyuse->usable
mysql-test/r/table_elim.result:
MWL#17: Table elimination
- More testcases
mysql-test/t/table_elim.test:
MWL#17: Table elimination
- More testcases
sql/sql_select.cc:
MWL#17: Table elimination
- Let add_ft_key() set keyuse->usable
- Make elimination check to be able detect cases like t.primary_key_col1=othertbl.col AND t.primary_key_col2=func(t.primary_key_col1).
These are needed to handle e.g. the case of func() being a correlated subquery that selects the latest value.
- If we've removed a condition with subquery predicate, EXPLAIN [EXTENDED] won't show the subquery anymore
sql/item.cc:
MWL#17: Table elimination
- Add tem_field::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item.h:
MWL#17: Table elimination
- Add tem_field::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item_subselect.cc:
MWL#17: Table elimination
- Item_subselect got 'eliminated' attribute. It is used only to determine if the subselect should be printed by EXPLAIN.
- Item_subselect got List<Item> refers_to - a list of item in the current select that are referred to from within the subselect.
- Added Item_*::check_column_usage_processor(). it allows to check which key parts a condition depends on.
- Added a comment about possible problem in Item_subselect::walk
sql/item_subselect.h:
MWL#17: Table elimination
- Item_subselect got 'eliminated' attribute. It is used only to determine if the subselect should be printed by EXPLAIN.
- Item_subselect got List<Item> refers_to - a list of item in the current select that are referred to from within the subselect.
- Added Item_*::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item_sum.cc:
MWL#17: Table elimination
sql/sql_lex.cc:
MWL#17: Table elimination
sql/sql_lex.h:
MWL#17: Table elimination
sql/sql_select.h:
MWL#17: Table elimination
Manually merge some InnoDB changes into XtraDB.
Fix ALTER TABLE bug in XtraDB with wrong comparison of row type.
mysql-test/include/varchar.inc:
Fix in test case that which of several duplicate keys triggers an error is not
deterministic.
mysql-test/mysql-test-run.pl:
InnoDB does not bother to free resources individually during shutdown, but due to using
its own memory tracking it nevertheless can free everything at exit. But XtraDB adds an
option, on by default, to skip this extra tracking. This causes lots of Valgrind
warnings, so needs to be disabled for Valgrind testing.
mysql-test/r/innodb.result:
Fix in test case that which of several duplicate keys triggers an error is not
deterministic.
mysql-test/t/innodb-use-sys-malloc.test:
InnoDB does not bother to free resources individually during shutdown, but due to using
its own memory tracking it nevertheless can free everything at exit. But XtraDB adds an
option, on by default, to skip this extra tracking. This causes lots of Valgrind
warnings, so needs to be disabled for Valgrind testing.
sql/sql_table.cc:
Add some useful DBUG while debugging alter table.
storage/xtradb/handler/ha_innodb.cc:
Fix that check_if_incompatible_data did not realise that ROW_TYPE_DEFAULT is identical
to the default row format ROW_TYPE_COMPACT, causing excessive table copying in
ALTER TABLE
Add some useful DBUG while debugging alter table.
Manually merge into XtraDB a few small changes for InnoDB from upstream MySQL.
storage/xtradb/include/pars0pars.h:
Manually merge into XtraDB a few small changes for InnoDB from upstream MySQL.
storage/xtradb/include/univ.i:
Manually merge a MariaDB fix in InnoDB into XtraDB.
- Move eliminate_tables() to before constant table detection.
- First code for benchmark
sql-bench/test-table-elimination.sh:
MWL#17: Table elimination
- sql-bench "Benchmark", incomplete
sql/sql_select.cc:
MWL#17: Table elimination
- Move eliminate_tables() to before constant table detection, this will allow
to spare const table reads (at a cost of not being able to take advantage of
tables that are constant because they have no records, but this case is of
lesser importance)
- Fix print_join() to work both for EXPLAIN EXTENDED (after table elimination) and for
CREATE VIEW (after join->prepare() but without any optimization).
mysql-test/r/union.result:
MWL#17: Table elimination
- Adjust test results
- Fix the previous cset: take into account that select_lex may be printed when
1. There is no select_lex->join at all (in that case, assume that no tables were eliminated)
2. select_lex->join exists but there was no JOIN::optimize() call yet. handle this by initializing join->eliminated really early.
include/my_sys.h:
Move generic file parsing functions out to shared code, as they are used in several places.
mysys/mf_iocache2.c:
Move generic file parsing functions out to shared code, as they are used in several places.
sql/log_event.cc:
Fix XtraDB build with embedded server.
XtraDB needs access to replication stuff, which is missing in embedded server.
Solved by defining wrapper function for this which is compiled differently for normal and
embedded case.
sql/log_event.h:
Fix XtraDB build with embedded server.
XtraDB needs access to replication stuff, which is missing in embedded server.
Solved by defining wrapper function for this which is compiled differently for normal and
embedded case.
sql/slave.cc:
Move generic file parsing functions out to shared code, as they are used in several places.
- Make elimination work with aggregate functions. The problem was that aggregate functions
reported all table bits in used_tables(), and that prevented table elimination. Fixed by
making aggregate functions return more correct value from used_tables().
mysql-test/r/ps_11bugs.result:
MWL#17: Table elimination
- Update test results. The difference is because of Item_ref change: outer references to constants
are now recognized as constants, too.
mysql-test/r/subselect.result:
- Update test results. The difference is because of Item_ref change: outer references to constants
are now recognized as constants, too.
mysql-test/r/table_elim.result:
MWL#17: Table elimination
- Check that elimination works in presense of aggreagate functions
mysql-test/t/table_elim.test:
MWL#17: Table elimination
- Check that elimination works in presense of aggreagate functions
sql/item.h:
MWL#17: Table elimination
- Add Item_ref::const_item() which calls (*ref)->const_item(). Before this diff Item_ref used the
default implementation of const_item(){ return used_tables()==0; }. This is no longer true, as
COUNT(*) now has used_tables()==0 but const_item()==FALSE.
sql/item_sum.cc:
MWL#17: Table elimination
- Make Item_sum() and it descendants not to return all bits in used_tables(). This is needed
because otherwise table elimination can't work in presense of aggregate functions
- COUNT(*) now has used_tables()==0 and const_item()==FALSE. Had to change
Item_ref::const_item() to account for this.
sql/item_sum.h:
MWL#17: Table elimination
- Add comments
Use MY_MUTEX_INIT_FAST for pool mutex
mysql-test/mysql-test-run.pl:
Added option --staging-run
Added information about --parallell=# to help message
mysql-test/suite/federated/federated_server.test:
Slow test, don't run with --staging-run
mysql-test/suite/maria/t/maria-preload.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_optimize.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_relayrotate.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_001.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_mysqlbinlog.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_sp003.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_start_stop_slave.test:
Slow test, don't run with --staging-run
mysql-test/t/compress.test:
Slow test, don't run with --staging-run
mysql-test/t/count_distinct3.test:
Slow test, don't run with --staging-run
mysql-test/t/index_merge_innodb.test:
Slow test, don't run with --staging-run
mysql-test/t/information_schema_all_engines.test:
Slow test, don't run with --staging-run
mysql-test/t/innodb_mysql.test:
Slow test, don't run with --staging-run
mysql-test/t/pool_of_threads.test:
Slow test, don't run with --staging-run
mysql-test/t/preload.test:
Slow test, don't run with --staging-run
mysql-test/t/ssl.test:
Slow test, don't run with --staging-run
mysql-test/t/ssl_compress.test:
Slow test, don't run with --staging-run
mysql-test/valgrind.supp:
Suppress warnings from SuSE 11.1 on x86
sql/scheduler.cc:
Use MY_MUTEX_INIT_FAST for pool mutex
- First code. Elimination works for simple cases, passes the testsuite.
- Known issues:
= No elimination is done for aggregate functions.
= EXPLAIN EXTENDED shows eliminated tables (I think it better not)
= No benchmark yet
= The code needs some polishing.
mysql-test/r/table_elim.result:
MWL#17: Table elimination
- Testcases
mysql-test/t/table_elim.test:
MWL#17: Table elimination
- Testcases
sql/sql_select.cc:
MWL#17: Table elimination
sql/sql_select.h:
MWL#17: Table elimination
- Added JOIN_TAB::eliminated (is JOIN_TAB the best place to store this flag?)
sql/table.h:
MWL#17: Table elimination
- ADded NESTED_JOIN::n_tables. We need to have the number of real tables remaining in an outer join nest.
- Version number.
- Valgrind false alarms in libz.
- New variant of suppression for Valgrind warning in dlclose().
- Fix double free() in plugin init error case.
configure.in:
Fix version number. We should reset the maria variant back to `1' when the MySQL version
number increases.
include/my_sys.h:
Fix false alarms in Valgrind for zlib.
Apply same fix as for archive storage handler also to the cases of compression in the
client protocol, and to the compression SQL function.
mysql-test/valgrind.supp:
A new variant of the dlclose() suppression is needed now.
mysys/my_compress.c:
Fix false alarms in Valgrind for zlib.
Apply same fix as for archive storage handler also to the cases of compression in the
client protocol, and to the compression SQL function.
sql/handler.cc:
Fix a double free() in error case for plugin initialisation.
sql/item_strfunc.cc:
Fix false alarms in Valgrind for zlib.
Apply same fix as for archive storage handler also to the cases of compression in the
client protocol, and to the compression SQL function.
mysql-test/r/information_schema.result:
Fixed a result file.
mysql-test/r/innodb-autoinc.result:
Fixed a result file.
mysql-test/t/connect.test:
Fixed a problem with merge, needed to close
connections and use the default
Fix mysql-test-run.pl to not terminate early when warnings in error logs are detected during
server shutdown. Instead, give a nice summary report at the end of the failures.
Fix code to make 100% sure no failures will go undetected.
Revert earlier wrong change.
Fix race with port allocation semaphore file permissions.
Adjust testsuite to copy with new PBXT engine now in the tree. The PBXT engine causes an
extra table to appear in the INFORMATION_SCHEMA. This causes different output for a few
test cases.
dbug/dbug.c:
If DbugParse() is called multiple times, the stack->keywords for the
top stack frame could be overwritten without being freed, causing a
memory leak reported by Valgrind.
include/my_global.h:
Add useful macro for different values when Valgrind (HAVE_purify) and not.
mysql-test/extra/rpl_tests/rpl_auto_increment.test:
Omit pbxt variables from show variables output.
mysql-test/include/have_pbxt.inc:
Add facility to disable test if PBXT engine is not available.
mysql-test/lib/mtr_report.pm:
Give a nice summary report at the end of tests of any warnings seen in logs during
server shutdowns.
mysql-test/lib/mtr_unique.pm:
Move chmod 777 to greatly reduce the risk of leaving the port semaphore file unaccessible
bu other users.
mysql-test/mysql-test-run.pl:
Don't abort in case of warnings detected, instead give a nice summary report.
Fix code to make 100% sure no failures will go undetected.
Revert earlier wrong change when master disconnects early.
mysql-test/r/information_schema.result:
Omit PBXT INFORMATION_SCHEMA table from output.
Move part of test to information_schema_all_engines.
mysql-test/r/information_schema_all_engines.result:
New file for information_schema tests that depend on which engines are available.
mysql-test/r/information_schema_db.result:
Move part of test to information_schema_all_engines.
mysql-test/r/innodb-autoinc.result:
Omit pbxt variables from show variables output.
mysql-test/r/mysqlshow.result:
Move part of test to information_schema_all_engines.
mysql-test/suite/rpl/r/rpl_auto_increment.result:
Omit pbxt variables from show variables output.
mysql-test/t/information_schema.test:
Omit PBXT INFORMATION_SCHEMA table from output.
Move part of test to information_schema_all_engines.
mysql-test/t/information_schema_all_engines.test:
New file for information_schema tests that depend on which engines are available.
mysql-test/t/information_schema_db.test:
Move part of test to information_schema_all_engines.
mysql-test/t/innodb-autoinc.test:
Omit pbxt variables from show variables output.
mysql-test/t/mysqlshow.test:
Move part of test to information_schema_all_engines.
mysql-test/valgrind.supp:
Add variant suppression (different system library versions).
Add suppression for problem with inet_ntoa().
sql/mysqld.cc:
Fix missing DBUG_RETURN.
Fix uninitialised thd->connect_utime, likely introduced by pool_of_threads.
sql/set_var.cc:
Fix one-byte buffer overflow in several places.
Fix unsafe use of String::c_ptr() of stack-allocated String buffer.
sql/sql_select.cc:
Silence valgrind warning due to GCC bug.
sql/sql_string.h:
Document potential problem with String::c_ptr() and String() constructor with caller-supplied buffer.
storage/archive/azio.c:
Silence Valgrind false warning for libz.
