The problem was in an incorrect debug assertion. The expression
used in the failing assertion states that when finding
references matching ORDER BY expressions, there can be only one
reference to a single table. But that does not make any sense,
all test cases for this bug are valid examples with multiple
identical WHERE expressions referencing the same table which
are also present in the ORDER BY list.
Fixed by removing the failing assertion. We also have to take
care of the 'found' counter so that we count multiple
references only once. We rely on this fact later in
eq_ref_table().
mysql-test/r/join.result:
Added a test case for bug #50335.
mysql-test/t/join.test:
Added a test case for bug #50335.
sql/sql_select.cc:
Removing the assertion in eq_ref_table() as it does not make
any sense. We also have to take care of the 'found' counter so
that we count multiple references only once. We rely on this
fact later in eq_ref_table().
union...order by (select... where...)
The problem is mysql is trying to materialize and
cache the scalar sub-queries at JOIN::optimize
even for EXPLAIN where the number of columns is
totally different from what's expected.
Fixed by not executing the scalar subqueries
for EXPLAIN.
The problem is that we can not use make_cond_for_table().
This function relies on used_tables() condition
which is not set properly for subqueries.
As result subquery is not filtered out.
The fix is to use remove_eq_conds() function instead
of make_cond_for_table() func. 'remove_eq_conds()'
algorithm relies on const_item() value and it allows
to handle subqueries in right way.
mysql-test/r/having.result:
test case
mysql-test/t/having.test:
test case
sql/sql_select.cc:
The fix is to use remove_eq_conds() function instead
of make_cond_for_table() function.
- Add Item_in_subselect::get_identifier() that returns subquery's id
- Change select_describe() to produce output in new format
- Update test results (checked)
The crash is the result of an attempt made by JOIN::optimize to evaluate
the WHERE condition when no records have been actually read.
The fix is to remove erroneous 'outer_join' variable check.
mysql-test/r/join.result:
test result
mysql-test/t/join.test:
test case
sql/sql_select.cc:
removed erroneous 'outer_join' variable check.
The crash happens because greedy_serach
can not determine best plan due to
wrong inner table dependences. These
dependences affects join table sorting
which performs before greedy_search starting.
In our case table which has real 'no dependences'
should be put on top of the list but it does not
happen as inner tables have no dependences as well.
The fix is to exclude RAND_TABLE_BIT mask from
condition which checks if table dependences
should be updated.
mysql-test/r/join.result:
test result
mysql-test/t/join.test:
test case
sql/sql_select.cc:
RAND_TABLE_BIT mask should not be counted as it
prevents update of inner table dependences.
For example it might happen if RAND() function
is used in JOIN ON clause.
mysql-test/r/union.result:
Update test results (checked, it should be "DEPENDENT SUBQUERY" as it will actually run subquery's join multiple times. There is no reason to do that but executor will still do it)
The problem is that when we make conditon for
grouped result const part of condition is cut off.
It happens because some parts of 'having' condition
which refer to outer join become const after
make_join_statistics. These parts may be lost
during further having condition transformation
in JOIN::exec. The fix is adding 'having'
condition check for const tables after
make_join_statistics is performed.
mysql-test/r/having.result:
test case
mysql-test/t/having.test:
test result
sql/sql_select.cc:
added 'having' condition check for const tables
after make_join_statistics is performed.
The crash happens because of discrepancy between values of
conts_tables and join->const_table_map(make_join_statisctics).
Calculation of conts_tables used condition with
HA_STATS_RECORDS_IS_EXACT flag check. Calculation of
join->const_table_map does not use this flag check.
In case of MERGE table without union with index
the table does not become const table and
thus join_read_const_table() is not called
for the table. join->const_table_map supposes
this table is const and later in make_join_select
this table is used for making&calculation const
condition. As table record buffer is not populated
it leads to crash.
The fix is adding a check if an engine supports
HA_STATS_RECORDS_IS_EXACT flag before updating
join->const_table_map.
mysql-test/r/merge.result:
test result
mysql-test/t/merge.test:
test case
sql/sql_select.cc:
adding a check if an engine supports
HA_STATS_RECORDS_IS_EXACT flag before updating
join->const_table_map.
Fix two problems:
1. Let optimize_semijoin_nests() reset sj_nest->sjmat_info irrespectively
of value of optimizer_flag. We need this in case somebody has turned optimization
off between reexecutions of the same statement.
2. Do not pull out constant tables out of semi-join nests. The problem is that pullout
operation is not undoable, and if a table is constant because it is 1/0-row table it
may cease to be constant on the next execution. Note that tables that are constant
because of possible eq_ref(const) access will still be pulled out as they are
considered functionally-dependent.
BUG#50019: Wrong result for IN-subquery with materialization
- Fix equality substitution in presense of semi-join materialization, lookup and scan variants
(started off from fix by Evgen Potemkin, then modified it to work in all cases)
tmp_table_size can now be set to 0 (to disable in memory internal temp tables)
Improved speed for internal Maria temp tables:
- Don't use packed keys, except with long text fields.
- Don't copy key all accessed pages during key search.
Some new benchmark tests to sql-bench (for group by)
BUILD/compile-pentium64-gcov:
Update script to use same pentium_config flags as other tests
BUILD/compile-pentium64-gprof:
Update script to use same pentium_config flags as other tests
include/my_sys.h:
Added count of my_sync calls
mysql-test/r/variables.result:
tmp_table_size can now be set to 0
sql-bench/test-select.sh:
Added some new test for GROUP BY on a not key field and group by with different order by
sql/mysqld.cc:
Added count of my_sync calls
tmp_table_size can now be set to 0 (to disable in memory internal temp tables)
sql/sql_select.cc:
If tmp_table_size is 0, don't use in memory temp tables (good for benchmarking MyISAM/Maria temp tables)
Don't pack keys for Maria tables; The 8K page size makes packed keys too slow for temp tables.
storage/maria/ma_key_recover.h:
Moved definition to maria_def.h
storage/maria/ma_page.c:
Moved code used to simplify comparing of identical Maria tables to own function (page_cleanup())
Fixed that one can read a page with a read lock.
storage/maria/ma_rkey.c:
For not exact key reads, cache the page where we found key (to speed up future read-next/read-prev calls)
storage/maria/ma_search.c:
Moved code to cache last key page to separate function.
Instead of copying pages, only get a link to the page. This notable speeds up key searches on bigger tables.
storage/maria/ma_write.c:
Added comment
storage/maria/maria_def.h:
Moved page_cleanup() to separate function.
SunStudio
SunStudio compilers of late warn about methods that might hide
methods in base classes due to the use of overloading combined
with overriding. SunStudio also warns about variables defined
in local socpe or method arguments that have the same name as
a member attribute of the class.
This patch renames methods that might hide base class methods,
to make it easier both for humans and compilers to see what is
actually called. It also renames variables in local scope.
sql/field.cc:
Local scope variable or method argument same as class
attribute.
sql/item_cmpfunc.cc:
Local scope variable or method argument same as class
attribute.
sql/item_create.cc:
Renaming base class create() to create_func().
sql/item_create.h:
Renaming base class create() to create_func().
sql/protocol.cc:
Local scope variable or method argument same as class
attribute.
sql/sql_profile.cc:
Local scope variable or method argument same as class
attribute.
sql/sql_select.cc:
Local scope variable or method argument same as class
attribute.
sql/sql_yacc.yy:
Renaming base class create() to create_func().
storage/federated/ha_federated.cc:
Local scope variable or method argument same as class
attribute.
storage/myisammrg/ha_myisammrg.cc:
Local scope variable or method argument same as class
attribute.
The problem is that not all column names retrieved from a SELECT
statement can be used as view column names due to length and format
restrictions. The server failed to properly check the conformity
of those automatically generated column names before storing the
final view definition on disk.
Since columns retrieved from a SELECT statement can be anything
ranging from functions to constants values of any format and length,
the solution is to rewrite to a pre-defined format any names that
are not acceptable as a view column name.
The name is rewritten to "Name_exp_%u" where %u translates to the
position of the column. To avoid this conversion scheme, define
explict names for the view columns via the column_list clause.
Also, aliases are now only generated for top level statements.
mysql-test/include/view_alias.inc:
Add test case for Bug#40277
mysql-test/r/compare.result:
Bug#40277: SHOW CREATE VIEW returns invalid SQL
mysql-test/r/group_by.result:
Bug#40277: SHOW CREATE VIEW returns invalid SQL
mysql-test/r/ps.result:
Bug#40277: SHOW CREATE VIEW returns invalid SQL
mysql-test/r/subselect.result:
Bug#40277: SHOW CREATE VIEW returns invalid SQL
mysql-test/r/subselect3.result:
Bug#40277: SHOW CREATE VIEW returns invalid SQL
mysql-test/r/type_datetime.result:
Bug#40277: SHOW CREATE VIEW returns invalid SQL
mysql-test/r/union.result:
Bug#40277: SHOW CREATE VIEW returns invalid SQL
mysql-test/r/view.result:
Add test case result for Bug#40277
mysql-test/r/view_alias.result:
Add test case result for Bug#40277
mysql-test/t/view_alias.test:
Add test case for Bug#40277
sql/sql_view.cc:
Check if auto generated column names are conforming. Also, the
make_unique_view_field_name function is not used as it uses the
original name to construct a new one, which does not work if the
name is invalid.
- The problem was that DuplicateWeedout strategy setup code wasn't aware of the
fact that join buffering will be used and applied optimization that doesn't work
together with join buffering. Fixed by making DuplicateWeedout setup code to have
a pessimistic check about whether there is a chance that join buffering will be
used.
- Make JOIN_CACHE_BKA::init() correctly process Copy_field elements that denote saving
current rowids in the join buffer.
mysql-test/r/subselect_sj2.result:
Update test results
mysql-test/r/subselect_sj2_jcl6.result:
Update test results
mysql-test/r/subselect_sj_jcl6.result:
Testcase
mysql-test/t/subselect_sj2.test:
Update test results
mysql-test/t/subselect_sj_jcl6.test:
Testcase
sql/opt_subselect.cc:
- The problem was that DuplicateWeedout strategy setup code wasn't aware of the
fact that join buffering will be used and applied optimization that doesn't work
together with join buffering. Fixed by making DuplicateWeedout setup code to have
a pessimistic check about whether there is a chance that join buffering will be
used.
sql/sql_join_cache.cc:
Make JOIN_CACHE_BKA::init() correctly process Copy_field elements that denote saving current rowids in the join buffer.
sql/sql_select.cc:
Added a question note
Save no-records constant tables in JOIN::const_table_map before we invoke
eliminate_tables(). Failure to do so caused crash when the same table was
marked as constant two times
mysql-test/r/table_elim.result:
LPBUG#524025 Running RQG outer_join test leads to crash
- Testcase
mysql-test/t/table_elim.test:
LPBUG#524025 Running RQG outer_join test leads to crash
- Testcase
sql/sql_select.cc:
LPBUG#524025 Running RQG outer_join test leads to crash
Save no-records constant tables in JOIN::const_table_map before we invoke eliminate_tables(). Failure to do so caused crash when the same table was marked as constant two times.
consider clustered primary keys
Choosing a shortest index for the covering index scan,
the optimizer ignored the fact, that the clustered primary
key read involves whole table data.
The find_shortest_key function has been modified to
take into account that fact that a clustered PK has a
longest key of possible covering indices.
mysql-test/r/innodb_mysql.result:
Test case for bug #39653.
mysql-test/t/innodb_mysql.test:
Test case for bug #39653.
sql/sql_select.cc:
Bug #39653: find_shortest_key in sql_select.cc does not
consider clustered primary keys
The find_shortest_key function has been modified to
take into account that fact that a clustered PK has a
longest key of possible covering indices.
The problem is that cond->fix_fields(thd, 0) breaks
condition(cuts off 'having'). The reason of that is
that NULL valued Item pointer is present in the
middle of Item list and it breaks the Item processing
loop.
mysql-test/r/having.result:
test case
mysql-test/t/having.test:
test case
sql/item_cmpfunc.h:
added ASSERT to make sure that we do not add NULL valued Item pointer
sql/sql_select.cc:
skip adding an item to condition if Item pointer is NULL.
skip adding a list to condition if this list is empty.
performance degradation.
Filesort + join cache combination is preferred to full index scan because it
is usually faster. But it's not the case when the index is clustered one.
Now test_if_skip_sort_order function prefers filesort only if index isn't
clustered.
mysql-test/r/innodb_mysql.result:
Added a test case for the bug#50843.
mysql-test/t/innodb_mysql.test:
Added a test case for the bug#50843.
sql/sql_select.cc:
Bug#50843: Filesort used instead of clustered index led to
performance degradation.
Now test_if_skip_sort_order function prefers filesort only if index isn't
clustered.
The problem was in an incorrect debug assertion. The expression
used in the failing assertion states that when finding
references matching ORDER BY expressions, there can be only one
reference to a single table. But that does not make any sense,
all test cases for this bug are valid examples with multiple
identical WHERE expressions referencing the same table which
are also present in the ORDER BY list.
Fixed by removing the failing assertion. We also have to take
care of the 'found' counter so that we count multiple
references only once. We rely on this fact later in
eq_ref_table().
mysql-test/r/join.result:
Added a test case for bug #50335.
mysql-test/t/join.test:
Added a test case for bug #50335.
sql/sql_select.cc:
Removing the assertion in eq_ref_table() as it does not make
any sense. We also have to take care of the 'found' counter so
that we count multiple references only once. We rely on this
fact later in eq_ref_table().
This patch implements correct NULL semantics for materialized subquery execution.
The implementation has the following properties and main limitations:
- It passes all query result tests, but fails a number of EXPLAIN tests because of
changed plans.
- The EXPLAIN output for partial matching is not decided yet.
- It works only when all necessary indexes fit into main memory. Notice that these
are not the general B-tree/Hash indexes, but instead much more compact ones,
therefore this limitation may not be a problem in many practical cases.
- It doesn't contain specialized tests.
- In several places the implementation uses methods that are modified copies of
other similar methods. These cases need to be refactored to avoid code duplication.
- Add a test if the predicate is top-level just before deciding on partial matching.
If it is top-level, use a more efficient exec method (index lookup).
- Add sorting of indexes according to their selectivity. The code is almost there.
- Needs more comments, and to sync existing ones with the implementation.
sql/item_cmpfunc.h:
Expose the Arg_comparator of a comparison predicate. This makes it possible to
directly get the comparison result {-1,0,1}, which is not possible through the
val_XXX() methods which "fold" such results into a boolean.
sql/item_subselect.cc:
The core of the implementation of MWL#68.
sql/item_subselect.h:
The core of the implementation of MWL#68.
sql/opt_subselect.cc:
Removed the limitation for materialized subquery execution that it is applicable only
for top-level predicates.
sql/sql_class.cc:
New class select_materialize_with_stats that collects data statistics about
the data being inserted into the target table.
sql/sql_class.h:
New class select_materialize_with_stats that collects data statistics about
the data being inserted into the target table.
sql/sql_select.cc:
- more complete initialization of the TABLE object of a temp table.
- call setup_subquery_materialization at one more exit point.
The problem is that during temporary table creation uneven bits
are not taken into account for hidden fields. It leads to incorrect
calculation&allocation of null bytes size for table record. And
if grouped value is null we set wrong bit for this value(see end_update()).
Fixed by adding separate calculation of uneven bit for hidden fields.
mysql-test/r/type_bit.result:
test case
mysql-test/t/type_bit.test:
test case
sql/sql_select.cc:
added separate calculation of uneven bit for hidden fields
column is used for ORDER BY
Problem: filesort isn't meant for null length sort data
(e.g. char(0)), that leads to a server crash.
Fix: disregard sort order if sort data record length is 0 (nothing
to sort).
mysql-test/r/select.result:
Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL
column is used for ORDER BY
- test result.
mysql-test/t/select.test:
Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL
column is used for ORDER BY
- test case.
sql/filesort.cc:
Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL
column is used for ORDER BY
- assert added as filesort cannot handle null length sort data.
sql/sql_select.cc:
Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL
column is used for ORDER BY
- don't sort null length data e.g. in case of ORDER BY CHAR(0).
The problem becomes apparent only if HAVE_purify is undefined.
It related to the part of code placed in open_table_from_share() fuction
where we initialize record buffer only if HAVE_purify is enabled.
So in case of HAVE_purify=OFF record buffer is not initialized
on open table stage.
Next we read key, find NULL value and update appropriate null bit
but do not update record buffer. After that the record is stored
in the join cache(store_record_in_cache). For CHAR fields we
strip trailing spaces and in our case this procedure uses
uninitialized record buffer.
The fix is to skip stripping space procedure in case of null values
for CHAR fields(partially based on 6.0 JOIN_CACHE implementation).
mysql-test/r/join.result:
test case
mysql-test/t/join.test:
test case
sql/field.cc:
code updated according to new CACHE_FIELD struct
sql/sql_select.cc:
code updated according to new CACHE_FIELD struct
sql/sql_select.h:
CACHE_FIELD struct:
added new fields: Field *field, uint type;
removed fields: Field_blob *blob_field, bool strip;
Queries optimized with GROUP_MIN_MAX didn't cleanup KEYREAD
optimization properly. As a result subsequent queries may
return incomplete rows (fields are initialized to default
values).
mysql-test/r/group_min_max.result:
A test case for BUG#49902.
mysql-test/t/group_min_max.test:
A test case for BUG#49902.
sql/opt_range.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/opt_sum.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/sql_select.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/sql_update.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/table.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/table.h:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
Grouping by a subquery in a query with a distinct aggregate
function lead to a wrong result (wrong and unordered
grouping values).
There are two related problems:
1) The query like this:
SELECT (SELECT t1.a) aa, COUNT(DISTINCT b) c
FROM t1 GROUP BY aa
returned wrong result, because the outer reference "t1.a"
in the subquery was substituted with the Item_ref item.
The Item_ref item obtains data from the result_field object
that refreshes once after the end of each group. This data
is not applicable to filesort since filesort() doesn't care
about groups (and doesn't update result_field objects with
copy_fields() and so on). Also that data is not applicable
to group separation algorithm: end_send_group() checks every
record with test_if_group_changed() that evaluates Item_ref
items, but it refreshes those Item_ref-s only after the end
of group, that is a vicious circle and the grouped column
values in the output are shifted.
Fix: if
a) we grouping by a subquery and
b) that subquery has outer references to FROM list
of the grouping query,
then we substitute these outer references with
Item_direct_ref like references under aggregate
functions: Item_direct_ref obtains data directly
from the current record.
2) The query with a non-trivial grouping expression like:
SELECT (SELECT t1.a) aa, COUNT(DISTINCT b) c
FROM t1 GROUP BY aa+0
also returned wrong result, since JOIN::exec() substitutes
references to top-level aliases in SELECT list with Item_copy
caching items. Item_copy items have same refreshing policy
as Item_ref items, so the whole groping expression with
Item_copy inside returns wrong result in filesort() and
end_send_group().
Fix: include aliased items into GROUP BY item tree instead
of Item_ref references to them.
mysql-test/r/group_by.result:
Test case for bug #45640
mysql-test/t/group_by.test:
Test case for bug #45640
sql/item.cc:
Bug #45640: optimizer bug produces wrong results
Item_field::fix_fields() has been modified to resolve
aliases in GROUP BY item trees into aliased items instead
of Item_ref items.
sql/item.h:
Bug #45640: optimizer bug produces wrong results
- Item::find_item_processor() has been introduced.
- Item_ref::walk() has been modified to apply processors
to itself too (not only to referenced item).
sql/mysql_priv.h:
Bug #45640: optimizer bug produces wrong results
fix_inner_refs() has been modified to accept group_list
parameter.
sql/sql_lex.cc:
Bug #45640: optimizer bug produces wrong results
Initialization of st_select_lex::group_fix_field has
been added.
sql/sql_lex.h:
Bug #45640: optimizer bug produces wrong results
The st_select_lex::group_fix_field field has been introduced
to control alias resolution in Itef_fied::fix_fields.
sql/sql_select.cc:
Bug #45640: optimizer bug produces wrong results
- The fix_inner_refs function has been modified to treat
subquery outer references like outer fields under aggregate
functions, if they are included in GROUP BY item tree.
- The find_order_in_list function has been modified to
fix Item_field alias fields included in the GROUP BY item
trees in a special manner.
