Analysis:
The optimizer distinguishes two kinds of 'constant' conditions:
expensive ones, and non-expensive ones. The non-expensive conditions
are evaluated inside make_join_select(), and if false, already the
optimizer detects empty query results.
In order to avoid arbitrarily expensive optimization, the evaluation of
expensive constant conditions is delayed until execution. These conditions
are attached to JOIN::exec_const_cond and evaluated in the beginning of
JOIN::exec. The relevant execution logic is:
JOIN::exec()
{
if (! join->exec_const_cond->val_int())
{
produce an empty result;
stop execution
}
continue execution
execute the original WHERE clause (that contains exec_const_cond)
...
}
As a result, when an expensive constant condition is
TRUE, it is evaluated twice - once through
JOIN::exec_const_cond, and once through JOIN::cond.
When the expensive constant condition is a subquery,
predicate, the subquery is evaluated twice. If we have
many levels of subqueries, this logic results in a chain
of recursive subquery executions that walk a perfect
binary tree. The result is that for subquries with depth N,
JOIN::exec is executed O(2^N) times.
Solution:
Notice that the second execution of the constant conditions
happens inside do_select(), in the branch:
if (join->table_count == join->const_tables) { ... }
In this case exec_const_cond is equivalent to the whole WHERE
clause, therefore the WHERE clause has already been checked in
the beginnig of JOIN::exec, and has been found to be true.
The bug is addressed by not evaluating the WHERE clause if there
was exec_const_conds, and it was TRUE.
If the optimizer switch 'semijoin_with_cache' is set to 'off' then
join cache cannot be used to join inner tables of a semijoin.
Also fixed a bug in the function check_join_cache_usage() that led
to wrong output of the EXPLAIN commands for some test cases.
In MariaDB, when running in ONLY_FULL_GROUP_BY mode,
the server produced in incorrect error message that there
is an aggregate function without GROUP BY, for artificially
created MIN/MAX functions during subquery MIN/MAX optimization.
The fix introduces a way to distinguish between artifially
created MIN/MAX functions as a result of a rewrite, and normal
ones present in the query. The test for ONLY_FULL_GROUP_BY violation
now tests in addition if a MIN/MAX function was part of a MIN/MAX
subquery rewrite.
In order to be able to distinguish these MIN/MAX functions, the
patch introduces an additional flag in Item_in_subselect::in_strategy -
SUBS_STRATEGY_CHOSEN. This flag is set when the optimizer makes its
final choice of a subuqery strategy. In order to make the choice
consistent, access to Item_in_subselect::in_strategy is provided
via new class methods.
******
Fix MySQL BUG#12329653
In MariaDB, when running in ONLY_FULL_GROUP_BY mode,
the server produced in incorrect error message that there
is an aggregate function without GROUP BY, for artificially
created MIN/MAX functions during subquery MIN/MAX optimization.
The fix introduces a way to distinguish between artifially
created MIN/MAX functions as a result of a rewrite, and normal
ones present in the query. The test for ONLY_FULL_GROUP_BY violation
now tests in addition if a MIN/MAX function was part of a MIN/MAX
subquery rewrite.
In order to be able to distinguish these MIN/MAX functions, the
patch introduces an additional flag in Item_in_subselect::in_strategy -
SUBS_STRATEGY_CHOSEN. This flag is set when the optimizer makes its
final choice of a subuqery strategy. In order to make the choice
consistent, access to Item_in_subselect::in_strategy is provided
via new class methods.
The function add_ref_to_table_cond missed updating the value of
join_tab->pre_idx_push_select_cond after having updated the value
of join_tab->select->pre_idx_push_select_cond.
The bug happened because in some cases the function JOIN::exec
did not save the value of TABLE::pre_idx_push_select_cond in
TABLE::select->pre_idx_push_select_cond for the sort table.
Noticed and fixed a bug in the function make_cond_remainder
that builds the remainder condition after extraction of an index
pushdown condition from the where condition. The code
erroneously assumed that the function make_cond_for_table left
the value of ICP_COND_USES_INDEX_ONLY in sub-condition markers.
Adjusted many result files from the regression test suite
after this fix .
The call of the virtual function cancel_pushed_idx_cond in the code of
the function test_if_skip_sort_order was misplaced when backporting the
fix for bug 58816.
- The bug was caused by the following scenario:
= a quick select is created with get_quick_select_for_ref. The quick
select refers to temporary (derived) table. It saves table->file, which
refers to a ha_heap object.
= When temp table is populated, ha_heap reaches max. size and is converted
to a ha_myisam. However, quick->file remains pointing to where ha_heap
was.
= Attempt to use the quick select causes crash.
- Fixed by introducing QUICK_SELECT_I::replace_handler(). Note that it will
not work for index_merge quick selects. Which is fine, because these
quick selects are never created for derived tables.
- The problem was that the value of READ_RECORD::file was not updated when the underlying table
was temporary and was converted from heap to myisam. Resolved by eliminating READ_RECORD::file,
always use READ_RECORD::table->file
Honor unique/not unique when creating keys for internal tempory tables.
Added new variables to be used to limit how keys are created for internal temporary tables.
include/maria.h:
Added maria_max_key_length() and maria_max_key_segments()
include/myisam.h:
Added myisam_max_key_length() and myisam_max_key_segments()
mysql-test/r/mysql.result:
Drop all used tables
mysql-test/r/subselect4.result:
Added test case for lp:879939
mysql-test/t/mysql.test:
Drop all used tables
mysql-test/t/subselect4.test:
Added test case for lp:879939
sql/mysql_priv.h:
Added internal_tmp_table_max_key_length and internal_tmp_table_max_key_segments to be used to limit how keys for derived tables are created.
sql/mysqld.cc:
Added internal_tmp_table_max_key_length and internal_tmp_table_max_key_segments to be used to limit how keys for derived tables are created.
sql/share/errmsg.txt:
Added new error message for internal errors
sql/sql_select.cc:
Give error if we try to create a wrong key (this error should never happen)
Honor unique/not unique when creating keys for internal tempory tables.
storage/maria/ha_maria.cc:
Added change_table_ptr() to ensure that external_ref points always to the correct table.
(Not having this caused an assert in the included test)
storage/maria/ha_maria.h:
Added change_table_ptr() to ensure that external_ref points always to the correct table.
storage/maria/ma_check.c:
Fixed bug in Duplicate key error printing (now row position is printed correctly)
storage/maria/ma_create.c:
maria_max_key_length() -> _ma_max_key_length()
storage/maria/ma_info.c:
Added extern function maria_max_key_length() to calculate the max key length based on current block size.
storage/maria/ma_open.c:
maria_max_key_length() -> _ma_max_key_length()
storage/maria/maria_def.h:
maria_max_key_length() -> _ma_max_key_length()
storage/myisam/ha_myisam.cc:
Added change_table_ptr() to ensure that external_ref points always to the correct table.
(Not having this caused an assert in the included test)
storage/myisam/ha_myisam.h:
Added change_table_ptr() to ensure that external_ref points always to the correct table.
- when create_ref_for_key() is constructing a ref access for
a table that's inside a SJ-Materialization nest, it may not
use references to fields of tables that are unside the nest (because
these fields will not yet have values when ref access will be used)
The check was performed in the first of create_ref_for_key's loops (the
one which counts how many key parts are usable) but not in the second
(the one which actually fills the TABLE_REF structure).
The function JOIN::drop_unused_derived_keys could erroneously set
the value of REF::key to 0 for a joined materialized view/derived table
in the case when no REF access to the table was used by the query
execution plan. This could cause a crash of the server.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
This bug manifested itself with queries containing non-correlated
IN subqueries over materialized views/derived tables.
The bug happened because the code of the function generate_derived_keys did
not take into account that the function could be called twice when the
optimizer was deciding whether in-exist transformation should be applied.
- in make_join_select(), use the correct condition to check whether the current table is a SJM nest (the previous
condition used to be correct before, but then sj-materialization temp table creation was moved to happen before
make_join_select was called)
This bug is a consequence of the fix in the function add_ref_to_table_cond
for LP bug 826935 that turned out to be not quite correct: it tried to AND
the same generated condition with two different other conditions.
This patch creates a copy of the generated condition if the condition needs
to be ANDed with two different items.
Problematic query:
insert ignore into `t1_federated` (`c1`) select `c1` from `t1_local` a
where not exists (select 1 from `t1_federated` b where a.c1 = b.c1);
When this query is killed in another connection it could lead to crash.
The problem is follwing:
An attempt to obtain table statistics for subselect table in killed query
fails with an error. So JOIN::optimize() for subquery is failed but
it does not prevent further subquery evaluation.
At the first subquery execution JOIN::optimize() is called
(see subselect_single_select_engine::exec()) and fails with
an error. 'executed' flag is set to TRUE and it prevents
further subquery evaluation. At the second call
JOIN::optimize() does not happen as 'JOIN::optimized' is TRUE
and in case of uncacheable subquery the 'executed' flag is set
to FALSE before subquery evaluation. So we loose 'optimize stage'
error indication (see subselect_single_select_engine::exec()).
In other words 'executed' flag is used for two purposes, for
error indication at JOIN::optimize() stage and for an
indication of subquery execution. And it seems it's wrong
as the flag could be reset.
mysql-test/r/error_simulation.result:
test case
mysql-test/t/error_simulation.test:
test case
sql/item_subselect.cc:
added new flag subselect_single_select_engine::optimize_error
which is used for error detection which could happen at optimize
stage.
sql/item_subselect.h:
added new flag subselect_single_select_engine::optimize_error
sql/sql_select.cc:
test case
in the function best_access_path revealed another bug: currently
table scans on NULL keys used for NOT IN subqueries cannot work
together with employment of join caches for inner tables of these
subqueries. Otherwise the result can be wrong as it could be seen
with the result of the test case constructed for bug #37894
in the file subselect3_jcl6.result.
of the 5.3 code line after a merge with 5.2 on 2010-10-28
in order not to allow the cost to access a joined table to be equal
to 0 ever.
Expanded data sets for many test cases to get the same execution plans
as before.
- The problem was that JOIN::save/restore_query_plan() did not save/restore parts of
the query plan that are located inside SJ_MATERIALIZATION_INFO structures. This could
cause parts of one plan to be used with another, which led get_best_combination() to
constructing non-sensical join plans (and crash).
Fixed by saving/restoring SJM parts of the query plans.
- check_and_do_in_subquery_rewrites() will not set SUBS_MATERIALIZATION flag when it
records that the subquery predicate is to be converted into semi-join.
If convert_join_subqueries_to_semijoins() later decides not to convert to semi-join,
let it set SUBS_MATERIALIZATION flag, if appropriate.
- are_tables_local() failed to recognize the fact that OUTER_REF_TABLE_BIT is ok
for SJ-Materialization. This caused zero-length ref access to be constructed, which
led to an assert.
- Let join buffering code correctly take into account rowids needed
by DuplicateElimination when it is calculating minimum record sizes.
- In JOIN_CACHE::write_record_data, added asserts that prevent us from
writing beyond the end of the buffer.
For any query JOIN::optimize() should call the method
SELECT::save_leaf_tables after the last transformation
that utilizes the statement memory rather than the
execution memory.
sql/item_subselect.cc:
Added check of error condtions (safety)
sql/sql_join_cache.cc:
Added DBUG to some functions.
Added error checking for calls to check_match(); This fixed the bug.
sql/sql_select.cc:
Moved variable assignment to be close to where it's used (cleanup)
Analysis:
In the test query semi-join merges the inner-most subquery
into the outer subquery, and the optimization of the merged
subquery finds some new index access methods. Later the
IN-EXISTS transformation is applied to the unmerged subquery.
Since the optimizer is instructed to not consider
materialization, it reoptimizes the plan in-place to take into
account the new IN-EXISTS conditions. Just before reoptimization
JOIN::choose_subquery_plan resets the query plan, which also
resets the access methods found during the semi-join merge.
Then reoptimization discovers there are no new access methods,
but it leaves the query plan in its reset state. Later semi-join
crashes because it assumes these access methods are present.
Solution:
When reoptimizing in-place, reset the query plan only after new
access methods were discovered. If no new access methods were
discovered, leave the current plan as it was.
