One effect of this change in the test suite is that tests with very few
rows changed to use sub queries instead of materialization. This is
correct and expected as for these the materialization overhead is too high.
A lot of tests where fixed to still use materialization by adding a
few rows to the tables (most tests has only 2-3 rows and are thus easily
affected when cost computations are changed).
Other things:
- Added more variables to TMPTABLE_COSTS for better cost calculation
- Added cost of copying rows to TMPTABLE_COSTS lookup and write
- Added THD::optimizer_cache_hit_ratio for easier cost calculations
- Added DISK_FAST_READ_SIZE to be used when calculating costs when
reading big blocks from a disk
Tests with checking metadata or that cannot be run with
the view-protocol are excluded from --view-protocol.
For tests that do not allow the use of an additional connection,
the util connection is disabled with "--disable_service_connection".
Also cases with bugs for --view-protocol are disabled.
If when extracting a range condition for an index from the WHERE condition
Range Optimizer sees that the range condition covers the whole index then
such condition should be discarded because it cannot be used in any range
scan. In some cases Range Optimizer really does it, but there remained some
conditions for which it was not done. As a result the optimizer could
produce index merge plans with the full index scan for one of the indexes
participating in the index merge.
This could be observed in one of the test cases from index_merge1.inc
where a plan with index_merge_sort_union was produced and in the test case
reported for this bug where a plan with index_merge_sort_intersect was
produced. In both cases one of two index scans participating in index merge
ran over the whole index.
The patch slightly changes the original above mentioned test case from
index_merge1.inc to be able to produce an intended plan employing
index_merge_sort_union. The original query was left to show that index
merge is not used for it anymore.
It should be noted that for the plan with index_merge_sort_intersect could
be chosen for execution only due to a defect in the InnoDB code that
returns wrong estimates for the cardinality of big ranges.
This bug led to serious problems in 10.4+ where the optimization using
Rowid filters is employed (see mdev-26446).
Approved by Sergey Petrunia <sergey@mariadb.com>
- multi_range_read_info_const now uses the new records_in_range interface
- Added handler::avg_io_cost()
- Don't calculate avg_io_cost() in get_sweep_read_cost if avg_io_cost is
not 1.0. In this case we trust the avg_io_cost() from the handler.
- Changed test_quick_select to use TIME_FOR_COMPARE instead of
TIME_FOR_COMPARE_IDX to align this with the rest of the code.
- Fixed bug when using test_if_cheaper_ordering where we didn't use
keyread if index was changed
- Fixed a bug where we didn't use index only read when using order-by-index
- Added keyread_time() to HEAP.
The default keyread_time() was optimized for blocks and not suitable for
HEAP. The effect was the HEAP prefered table scans over ranges for btree
indexes.
- Fixed get_sweep_read_cost() for HEAP tables
- Ensure that range and ref have same cost for simple ranges
Added a small cost (MULTI_RANGE_READ_SETUP_COST) to ranges to ensure
we favior ref for range for simple queries.
- Fixed that matching_candidates_in_table() uses same number of records
as the rest of the optimizer
- Added avg_io_cost() to JT_EQ_REF cost. This helps calculate the cost for
HEAP and temporary tables better. A few tests changed because of this.
- heap::read_time() and heap::keyread_time() adjusted to not add +1.
This was to ensure that handler::keyread_time() doesn't give
higher cost for heap tables than for normal tables. One effect of
this is that heap and derived tables stored in heap will prefer
key access as this is now regarded as cheap.
- Changed cost for index read in sql_select.cc to match
multi_range_read_info_const(). All index cost calculation is now
done trough one function.
- 'ref' will now use quick_cost for keys if it exists. This is done
so that for '=' ranges, 'ref' is prefered over 'range'.
- scan_time() now takes avg_io_costs() into account
- get_delayed_table_estimates() uses block_size and avg_io_cost()
- Removed default argument to test_if_order_by_key(); simplifies code
The test main.index_merge_innodb is taking very much time,
especially on later versions (10.2 and 10.3).
Some of this could be attributed to the use of INSERT...SELECT,
which is time-consumingly creating explicit record locks in InnoDB
for the locking read in the SELECT part.
In 10.3 and later, some slowness can be attributed to MDEV-12288,
which makes the InnoDB purge thread spend time to reset transaction
identifiers in the inserted records. If we prevent purge from
running before all tables are dropped, the test seems to be
10% faster on an unoptimized debug build on 10.5. (A proper fix
would be to implement MDEV-515 and stop writing row-level undo log
records for inserts into an empty table or partition.)
At the same time, it should not hurt to make main.index_merge_myisam
to use the sequence engine. Not only could it be a little faster,
but the test would be slightly more readable.
This patch contains a full implementation of the optimization
that allows to use in-memory rowid / primary filters built for range
conditions over indexes. In many cases usage of such filters reduce
the number of disk seeks spent for fetching table rows.
In this implementation the choice of what possible filter to be applied
(if any) is made purely on cost-based considerations.
This implementation re-achitectured the partial implementation of
the feature pushed by Galina Shalygina in the commit
8d5a11122c32f4d9eb87536886c6e893377bdd07.
Besides this patch contains a better implementation of the generic
handler function handler::multi_range_read_info_const() that
takes into account gaps between ranges when calculating the cost of
range index scans. It also contains some corrections of the
implementation of the handler function records_in_range() for MyISAM.
This patch supports the feature for InnoDB and MyISAM.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
Also:
Changed the value of TIME_FOR_COMPARE_ROWID to make it the same as for MWL 21.
Changed some queries in range_vs_index_merge.test to make them generate
the same plans as earlier.
Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#44797 plugins w/o command-line options have no disabling option in --help
Bug#46314 string system variables don't support expressions
Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number
Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds
Bug#49417 some complaints about mysqld --help --verbose output
Bug#49540 DEFAULT value of binlog_format isn't the default value
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
Bug#49646 mysql --show-warnings crashes when server dies
CMakeLists.txt:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
client/mysql.cc:
don't crash with --show-warnings when mysqld dies
config/ac-macros/plugins.m4:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
include/my_getopt.h:
comments
include/my_pthread.h:
fix double #define
mysql-test/mysql-test-run.pl:
run sys_vars suite by default
properly recognize envirinment variables (e.g. MTR_MAX_SAVE_CORE) set to 0
escape gdb command line arguments
mysql-test/suite/sys_vars/r/rpl_init_slave_func.result:
init_slave+utf8 bug
mysql-test/suite/sys_vars/t/rpl_init_slave_func.test:
init_slave+utf8 bug
mysys/my_getopt.c:
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
mysys/typelib.c:
support for flagset
sql/ha_ndbcluster.cc:
backport from telco tree
sql/item_func.cc:
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
sql/sql_builtin.cc.in:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
sql/sql_plugin.cc:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#32902 plugin variables don't know their names
Bug#44797 plugins w/o command-line options have no disabling option in --help
sql/sys_vars.cc:
all server variables are defined here
storage/myisam/ft_parser.c:
remove unnecessary updates of param->quot
storage/myisam/ha_myisam.cc:
myisam_* variables belong here
strings/my_vsnprintf.c:
%o and %llx
unittest/mysys/my_vsnprintf-t.c:
%o and %llx tests
vio/viosocket.c:
bugfix: fix @@wait_timeout to work with socket timeouts (vs. alarm thread)
into magare.gmz:/home/kgeorge/mysql/work/merge-5.0-5.1-opt
mysql-test/r/type_time.result:
Auto merged
sql/field.cc:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
BitKeeper/deleted/.del-index_merge.result:
Auto merged
mysql-test/include/index_merge1.inc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/opt_range.cc:
Auto merged
storage/heap/ha_heap.cc:
Auto merged
storage/heap/ha_heap.h:
Auto merged
into april.(none):/home/svoj/devel/mysql/BUG20256/mysql-5.1-engines
sql/ha_myisam.cc:
Auto merged
sql/ha_myisam.h:
Auto merged
sql/handler.cc:
Auto merged
BitKeeper/deleted/.del-index_merge.result:
Auto merged
mysql-test/include/index_merge1.inc:
Manual merge.
sql/handler.h:
Manual merge.
sql/opt_range.cc:
Manual merge.
WL#3397 Refactoring storage engine test cases (for falcon)
It contains also fixes according to code review.
Contents: Testcases which were in history dedicated to InnoDB or MyISAM only.
Modifications:
1. Shift the main testing code into include/<testing field>.inc
Introduce $variables which can be used to omit tests for features which are not supported by
certain storage engines.
2. The storage engine to be tested is assigned within the toplevel script (t/<whatever>_<engine>.test)
via variable $engine_type and the the main testing code is sourced from
include/<testing field>.inc
3. Some toplevel testscripts have to be renamed to
- avoid immediate or future namespace clashes
- show via filename which storage engine is tested
4. Minor code cleanup like remove trailing spaces, some additional comments ....
mysql-test/t/unsafe_binlog_innodb-master.opt:
Rename: mysql-test/t/innodb_unsafe_binlog-master.opt -> mysql-test/t/unsafe_binlog_innodb-master.opt
mysql-test/r/read_many_rows_innodb.result:
Rename: mysql-test/r/innodb-big.result -> mysql-test/r/read_many_rows_innodb.result
mysql-test/t/cache_innodb-master.opt:
Rename: mysql-test/t/innodb_cache-master.opt -> mysql-test/t/cache_innodb-master.opt
mysql-test/t/concurrent_innodb-master.opt:
Rename: mysql-test/t/innodb_concurrent-master.opt -> mysql-test/t/concurrent_innodb-master.opt
BitKeeper/deleted/.del-index_merge.result:
Delete: mysql-test/r/index_merge.result
BitKeeper/deleted/.del-index_merge_innodb.result:
Delete: mysql-test/r/index_merge_innodb.result
BitKeeper/deleted/.del-index_merge_innodb2.result:
Delete: mysql-test/r/index_merge_innodb2.result
BitKeeper/deleted/.del-index_merge_ror.result:
Delete: mysql-test/r/index_merge_ror.result
BitKeeper/deleted/.del-index_merge_ror_cpk.result:
Delete: mysql-test/r/index_merge_ror_cpk.result
mysql-test/r/index_merge_innodb.result:
BitKeeper file /home/matthias/Arbeit/mysql-5.1-engines/src-1/mysql-test/r/index_merge_innodb.result
mysql-test/t/index_merge_innodb.test:
BitKeeper file /home/matthias/Arbeit/mysql-5.1-engines/src-1/mysql-test/t/index_merge_innodb.test
mysql-test/t/index_merge_myisam.test:
BitKeeper file /home/matthias/Arbeit/mysql-5.1-engines/src-1/mysql-test/t/index_merge_myisam.test
mysql-test/include/concurrent.inc:
1. This file contains now the main testing code of the former t/innodb_concurrent.test.
2. It is now sourced by t/concurrent_innodb.test.
mysql-test/include/deadlock.inc:
1. This file contains now the main testing code of the former t/innodb-deadlock.test
2. It is now sourced by t/deadlock_innodb.test.
mysql-test/include/handler.inc:
1. This file contains now the main testing code of the former t/innodb_handler.test + t/handler.test.
2. It is now sourced by t/handler_myisam.test and t/handler_innodb.test.
mysql-test/include/index_merge1.inc:
1. This file contains now the main testing code of the former t/index_merge.test.
2. It is now sourced by t/index_merge_myisam.test.
mysql-test/include/index_merge2.inc:
1. This file contains now the main code of t/index_merge_innodb.test.
2. It is sourced by t/index_merge_myisam.test and t/index_merge_innodb.test.
mysql-test/include/index_merge_2sweeps.inc:
1. This file contains now the main code of the former t/index_merge_innodb2.test.
2. It is sourced by t/index_merge_myisam.test and t/index_merge_innodb.test.
mysql-test/include/index_merge_ror.inc:
1. This file contains now the main code of the former t/index_merge_ror.test.
2. It is sourced by t/index_merge_myisam.test.
mysql-test/include/index_merge_ror_cpk.inc:
1. This file contains now the main testing code of the former t/index_merge_ror_cpk.test.
2. It is now sourced by t/index_merge_myisam.test and t/index_merge_innodb.test.
mysql-test/include/mix1.inc:
1. This file contains now the main testing code of the t/innodb_mysql.test
2. The name mix1.inc was used because the test contains subtests for different fields.
3. It is sourced by t/innodb_mysql.test.
4. Fixes:
- Assign $other_engine_type instead of hardcoded MyISAM.
- improve comment
- remove redundant subtest
- analyze table t4 instead of wrong table t1
- remove not needed "eval set storage_engine = $engine_type;"
mysql-test/include/mix2.inc:
1. This file is a copy of the main testing code of the t/innodb.test
A copy has to be used, because t/innodb.test is to be maintained by INNOBASE only.
2. The name mix2.inc was used because the test contains subtests for different fields.
3. It is sourced by t/mix2_myisam.test.
4. Fixes:
- improved comment
- additional "eval SET SESSION STORAGE_ENGINE = $other_engine_type;" at beginning of tests
- assign $other_engine_type instead of hardcoded MyISAM or HEAP
- assign $other_engine_type where it is needed to preserve test logics
- correct logical bugs
- improve(extend) "checksum table" test
mysql-test/include/query_cache.inc:
1. This file contains now the main testing code of the former t/innodb_cache.test.
2. It is now sourced by t/cache_innodb.test
mysql-test/include/read_many_rows.inc:
1. This file contains now the main testing code of the former t/innodb_big.test.
2. It is now sourced by t/read_many_rows_innodb.test.
mysql-test/include/rowid_order.inc:
1. This file contains now the main testing code of t/rowid_order_innodb.test.
2. It is now sourced by t/rowid_order_innodb.test.
mysql-test/include/unsafe_binlog.inc:
1. This file contains now the main testing code of the former t/innodb_unsafe_binlog.test.
2. It is now sourced by t/unsafe_binlog_innodb.test.
mysql-test/r/cache_innodb.result:
Updated result
mysql-test/r/concurrent_innodb.result:
Updated result
mysql-test/r/deadlock_innodb.result:
Updated result
mysql-test/r/handler_innodb.result:
Updated result
mysql-test/r/handler_myisam.result:
Updated result
mysql-test/r/index_merge_myisam.result:
Updated result
mysql-test/r/innodb_mysql.result:
Updated result
mysql-test/r/mix2_myisam.result:
Updated result
mysql-test/r/rowid_order_innodb.result:
Updated result
mysql-test/r/unsafe_binlog_innodb.result:
Updated result
mysql-test/t/cache_innodb.test:
1. Renaming of t/innodb_cache.test to t/cache_innodb.test
2. Main code is now sourced from include/query_cache.inc.
mysql-test/t/concurrent_innodb.test:
1. Renaming of t/innodb_concurrent.test to t/concurrent_innodb.test
2. Main code is now sourced from include/concurrent.inc.
Attention: This test fails even in the old version. (BUG#21579).
--> added to t/disabled.def
mysql-test/t/deadlock_innodb.test:
1. Renaming of t/innodb_deadlock.test to t/deadlock_innodb.test
2. Main code is now sourced from include/deadlock.inc.
mysql-test/t/disabled.def:
Add the test concurrent_innodb because of
BUG#21579 2006-08-11 mleich innodb_concurrent random failures with varying differences
mysql-test/t/handler_innodb.test:
1. Renaming of t/innodb_handler.test to t/handler_innodb.test
2. Main code is now sourced from include/handler.inc.
include/handler.inc = united code of former t/handler.test and t/innodb_handler.test
mysql-test/t/handler_myisam.test:
1. Renaming of t/handler.test to t/handler_myisam.test
2. Main code is now sourced from include/handler.inc.
include/handler.inc = united code of former t/handler.test and t/handler_innodb.test.
mysql-test/t/innodb_mysql.test:
1. Main code is now sourced from include/mix1.inc.
2. Test was not renamed because t/innodb.test refers to it.
mysql-test/t/mix2_myisam.test:
New test: MyISAM variant of mix2 ( = t/innodb.test)
mysql-test/t/read_many_rows_innodb.test:
1. Renaming of t/innodb_big.test to t/read_many_rows_innodb.test
2. Main code is now sourced from include/read_many_rows.inc.
mysql-test/t/rowid_order_innodb.test:
Main code is now sourced from t/rowid_order.inc.
mysql-test/t/unsafe_binlog_innodb.test:
1. Renaming of t/innodb_unsafe_binlog.test to t/unsafe_binlog_innodb.test
2. Main code is now sourced from include/unsafe_binlog.inc.
