MDEV-14957: JOIN::prepare gets unusable "conds" as argument
Do not touch merged derived (it is irreversible)
Fix first argument of in_optimizer for calls possible before fix_fields()
The assertion failure was caused by an incorrectly set read_set for
functions in the ORDER BY clause in part of a union, when we are using
a mergeable view and the order by clause can be skipped (removed).
An order by clause can be skipped if it's part of one part of the UNION as
the result set is not meaningful when multiple SELECT queries are UNIONed. The
server is aware of this optimization and tries to remove the order by
clause before JOIN::prepare. The problem is that we need to throw an
error when the ORDER BY clause contains invalid columns. To do this, we
attempt resolving the ORDER BY expressions, then subsequently drop them
if resolution succeeded. However, ORDER BY resolution had the side
effect of adding the expressions to the all_fields list, which is used
to construct temporary tables to store the result. We may be ignoring
the ORDER BY statement, but the tmp table still tried to compute the
values for the expressions, even if the columns are never used.
The assertion only shows itself if the order by clause contains members
which were not previously in the select list, and are part of a
function.
There is an additional question as to why this only manifests when using
VIEWS and not when using a regular table. The difference lies with the
"reset" of the read_set for the temporary table during
SELECT_LEX::update_used_tables() in JOIN::optimize(). The changes
introduced in fdf789a7ea cleared the
read_set when a mergeable view is encountered in the TABLE_LIST
defintion.
Upon initial order_list resolution, the table's read_set is updated
correctly. JOIN::optimize() will only reset the read_set if it
encounters a VIEW. Since we no longer have ORDER BY clause in
JOIN::optimize() we never get to correctly update the read_set again.
Other relevant commit by Timour, which first introduced the order
resolution when we "can_skip_sort_order":
883af99e7d
Solution:
Don't add the resolved ORDER BY elements to all_fields. We only resolve
them to check if an error should be returned for the query. Ignore them
completely otherwise.
TRASH was mapped to TRASH_FREE and was supposed to be used for memory
that should not be accessed anymore, while TRASH_ALLOC() is to be
used for uninitialized but to-be-used memory.
But sometimes TRASH() was used in the latter sense.
Remove TRASH() macro, always use explicit TRASH_ALLOC() or TRASH_FREE().
In this case we were using the optimization derived_with_keys but we could not create a key
because the length of the key was greater than the max allowed(MI_MAX_KEY_LENGTH).
To do the join we needed to create a hash join key instead, but in the explain output it
showed that we were still referring to derived keys which were created but not used.
In the function JOIN::shrink_join_buffers the iteration over joined
tables was organized in a wrong way. This could cause a crash if
the optimizer chose to materialize a semi-join that used join caches
for which the sizes must be adjusted.
* get_rec_bits() was always reading two bytes, even if the
bit field contained only of one byte
* In various places the code used field->pack_length() bytes
starting from field->ptr, while it should be field->pack_length_in_rec()
* Field_bit::key_cmp and Field_bit::cmp_max passed field_length as
an argument to memcmp(), but field_length is the number of bits!
optimizer_switch
For DATE and DATETIME columns defined as NOT NULL,
"date_notnull IS NULL" has to be modified to:
"date_notnull IS NULL OR date_notnull == 0"
if date_notnull is from an inner table of outer join);
"date_notnull == 0" - otherwise.
This must hold for such columns of mergeable views and derived
tables as well. So far the code did the above re-writing only
for columns of base tables and temporary tables.
with joins, SQ, ORDER BY, semijoin=on
A bug in get_sort_by_table() could mislead the function
setup_semijoin_dups_elimination(). As a result the optimizer
could produce invalid execution plans for queries with ORDER BY
and subquery predicates that could be converted to semi-joins.
An overflow of the double variable storing the estimate of the
number of rows in a partial join could trigger an assertion
failure during the optimization stage.
AND PARAM IN ORDER BY
Issue:
------
This issue can occur when the ORDER BY list refers to a
column that contains a parameter in the select list.
Solution:
---------
In JOIN::update_depend_map and get_sort_by_table, the
ORDER BY list's used_tables isn't checked for parameters.
This can result in incorrect behavior.
This is a partial backport of Roy's
For each SELECT the list sj_nests is built by the
function simplify_joins() when scanning different
join nests. This function may be called several
times for the same join nest. That's why before
adding a new member to sj_nests it is necessary
to check if it's already in the list.
The code of simplify_joins() lacked this check and
as a result it could cause memory overwright for
some queries.
This is actually a legacy bug:
SQL_SELECT::test_quick_select() was called
with SQL_SELECT::head not set.
It looks like that this problem can be
reproduced only on queries with ORDER BY
that use IN predicates converted to semi-joins.
This patch corrects the fix for bug mdev-7599.
When the min/max optimization of the function
opt_sum_query() optimizes away all tables of
a subquery it should not ever be rolled back.
If the optimizer chose an execution plan where
a semi-join nest were materialized and the
result of materialization was scanned to access
other tables by ref access it could build a key
over columns of the tables from the nest that
were actually inaccessible.
The patch performs a proper check whether a key
that uses columns of the tables from a materialized
semi-join nest can be employed to access outer tables.
When an IN subquery predicate was converted to a semi-join that were
materialized and the result of the materialization happened to be
the last in the execution plan then any conjunctive condition with RAND()
turned out to be lost.
Fixed by attaching this condition to the last top base table.
This patch corrects the fix for the bug mdev-10693.
It is critical for the function get_best_combination() not to call
create_ref_for_key() for constant tables.
This bug could manifest itself only in multi-table subqueries where
one of the tables is accessed by a constant primary key.
The code that chooses between materialization of a non-correlated
IN subquery and its transformation into an EXISTS correlated
subquery assumes that the execution plan for the outer select
has been already built. However it was not always so if subqueries
occurred in the expressions used for ref access to tables of
the outer select. A call of the function create_ref_for_key() in
get_best_combination() could trigger a premature execution of
the above mentioned code when the execution plan structures for
the outer select were not fully built. This could cause a crash
of the server.
The fix postpones the calls of create_ref_for_key() until the
structures for the execution plan is fully built.
Also fixed a wrong result for a test case for mdev-7691
(the alternative one).
The test cases for all these bug have materialized semi-joins used
inside dependent sub-queries.
The patch actually reverts the change inroduced by Monty in 2003.
It looks like this change is not valid anymore after the implementation
of semi-joins.
Adjusted output from EXPLAIN for many other test cases.
Also, implement MDEV-11027 a little differently from 5.5:
recv_sys_t::report(ib_time_t): Determine whether progress should
be reported.
recv_apply_hashed_log_recs(): Rename the parameter to last_batch.
'Not exists' optimization can be used for nested outer joins
only if IS NULL predicate from the WHERE condition is activated.
So we have to check that all guards that wrap this predicate
are in the 'open' state.
This patch supports usage of 'Not exists' optimization for any
outer join, no matter how it's nested in other outer joins.
This patch is also considered as a proper fix for bugs
#49322/#58490 and LP #817360.
This patch is actually a complement for the fix of bug mdev-6892.
The procedure create_tmp_table() now must take into account
Item_direct_refs that wrap up constant fields of derived tables/views
that are used as inner tables in outer join operations.
The fix for bug mdev-5104 did not take into account that
for any call of setup_order the size of ref_array must
be big enough. This patch fixes this problem.
When JOIN::destroy() is called for a JOIN object that has
- join->tmp_join != NULL
- also has join->table[0]->sort
then the latter was not cleaned up.
This could cause a memory leak and/or asserts in the subsequent queries.
Fixed by adding a cleanup call.
The function Item_func_isnull::update_used_tables() must
handle the case when the predicate is over not nullable
column in a special way.
This is actually a bug of MariaDB 5.3/5.5, but it's probably
hard to demonstrate that it can cause problems there.
Problem:
When build_equal_items_for_cond gets called for a big query
recursively, the specified thread_stack_size exceeds. But
optimizer does not handle this condition. As a result, server
exits.
Solution:
Check if we exceed specified stack size and if yes exit
gracefully by throwing an error.
DERIVED TABLE IN JOIN
ISSUE:
------
This problem occurs under the following conditions:
1) A parameter is used in the select-list of a derived table.
2) The derived table is part of a JOIN.
SOLUTION:
---------
When a derived table is materialized, a temporary table is
created. This temporary table creates a field each for the
items in the select-list of the derived table. This set of
fields is later used to setup the join.
Currently no field is created in the temporary table if a
parameter is used in the select-list.
Create a field for the parameter. By default Item_param's
result type in a prepared statement is set to
STRING_RESULT. This can change during the execute phase
depending on the user variable. But since the execute phase
creates its own temporary table, it will be handled
separately.
This is a backport of the fix for BUG#22392374.
Clang warns on this code because it is memsetting over a vtable contained in a
struct in the best_positions array. The diagnostic text is:
mariadb/sql/sql_select.cc:24462:10: error: destination for this 'memset' call is
a pointer to class containing a dynamic class 'Duplicate_weedout_picker'; vtable
pointer will be overwritten [-Werror,-Wdynamic-class-memaccess]
memset(best_positions, 0, sizeof(POSITION) * (table_count + 1));
~~~~~~ ^
Patch contributed by David Gow.
