The problematic queries involve unions. For unions we have an
optimization where we skip the ORDER BY clause in a query from one side
of the union if it will be performed later due to UNION.
EX:
(SELECT a from t1 ORDER BY a) ORDER BY b;
The first ordering by a is not necessary and it gets removed.
The problem is that we still need to resolve the Items before removing the
ORDER BY list from the
SELECT_LEX structure. During this final resolve step however, we forgot to
allow SET functions within the ORDER BY clause. This caused us to return
an "Invalid use of group function" error during the checking performed
by fix_fields in Item_sum::init_sum_func_check.
JOIN_CACHE's were initialized in check_join_cache_usage()
from make_join_readinfo(). After that make_join_readinfo() was looking
whether it's possible to use keyread. Later, after make_join_readinfo(),
optimizer decided whether to use filesort. And even later, at the
execution time, from join_read_first(), keyread was actually enabled.
The problem is, that if a query uses a vcol, base columns that it
depends on are automatically added to the read_set - because they're
needed to calculate the vcol. But if we're doing keyread, vcol is taken
from the index, not calculated, and base columns do not need to be
in the read set (even should not be - as they aren't getting values).
The bug was that JOIN_CACHE used read_set with base columns,
they were not read because of keyread, so it was caching garbage.
So read_set is only known after the keyread was decided. And after the
filesort was decided, as filesort doesn't use keyread. But
check_join_cache_usage() needs to be done in make_join_readinfo(),
as the code below depends on these checks,
Fix: keep JOIN_CACHE checks where they were, but move initialization
down to the very end of JOIN::optimize_inner. If keyread was enabled,
update the read_set to include only columns that are part of the index.
Copy the keyread logic from join_read_first() to happen at optimize time.
* rename to "keyread" (to avoid conflicts with tokudb),
* change from bool to uint and store the keyread index number there
* provide a bool accessor to check if keyread is enabled
move TABLE::key_read into handler. Because in index merge and DS-MRR
there can be many handlers per table, and some of them use
key read while others don't. "keyread" is really per handler,
not per TABLE property.
These are different bugs, but the fixing code is the same:
if window functions are used over implicit grouping then
now the execution should follow the general path calling
the function set in JOIN::first_select.
This patch complements the patch for bug 11138.
Without this patch some table-less queries with window functions
could cause crashes due to a memory overwrite.
Using window functions over results of implicit groupings
required special handling in JOIN::make_aggr_tables_info.
The patch made sure that the result of implicit grouping
was written into a temporary table properly.
* Remove duplicate lines from tests
* Use thd instead of current_thd
* Remove extra wsrep_binlog_format_names
* Correctly merge union patch from 5.5 wrt duplicate rows.
* Correctly merge SELinux changes into 10.1
The fix for bug mdev-5104 did not take into account that
for any call of setup_order the size of ref_array must
be big enough. This patch fixes this problem.
multi-update was setting up read_set/vcol_set in
multi_update::initialize_tables() that is invoked after
the optimizer (JOIN::optimize_inner()). But some rows - if they're from
const tables - will be read already in the optimizer, and these rows
will not have all necessary column/vcol values.
* multi_update::initialize_tables() uses results from the optimizer
and cannot be moved to be called earlier.
* multi_update::prepare() is called before the optimizer, but
it cannot set up read_set/vcol_set, because the optimizer
might reset them (see SELECT_LEX::update_used_tables()).
As a fix I've added a new method, select_result::prepare_to_read_rows(),
it's called from inside the optimizer just before make_join_statistics().
This fixes a bug where handler::read_range_first (for example)
needed to compare vcol values that were not calculated yet.
As a bonus it fixes few cases where vcols were calculated twice
When JOIN::destroy() is called for a JOIN object that has
- join->tmp_join != NULL
- also has join->table[0]->sort
then the latter was not cleaned up.
This could cause a memory leak and/or asserts in the subsequent queries.
Fixed by adding a cleanup call.
The function Item_func_isnull::update_used_tables() must
handle the case when the predicate is over not nullable
column in a special way.
This is actually a bug of MariaDB 5.3/5.5, but it's probably
hard to demonstrate that it can cause problems there.
The idea of this fix was taken from the patch by Roy Lyseng
for mysql-5.6 bug iBug#14740889: "Wrong result for aggregate
functions when executing query through cursor".
Here's Roy's comment for his patch:
"
The problem was that a grouped query did not behave properly when
executed using a cursor. On further inspection, the query used one
intermediate temporary table for the grouping.
Then, Select_materialize::send_result_set_metadata created a temporary
table for storing the query result. Notice that get_unit_column_types()
is used to retrieve column meta-data for the query. The items contained
in this list are later modified so that their result_field points to
the row buffer of the materialized temporary table for the cursor.
But prior to this, these result_field objects have been prepared for
use in the grouping operation, by JOIN::make_tmp_tables_info(), hence
the grouping operation operates on wrong column buffers.
The problem is solved by using the list JOIN::fields when copying data
to the materialized table. This list is set by JOIN::make_tmp_tables_info()
and points to the columns of the last intermediate temporary table of
the executed query. For a UNION, it points to the temporary table
that is the result of the UNION query.
Notice that we have to assign a value to ::fields early in JOIN::optimize()
in case the optimization shortcuts due to a const plan detection.
A more optimal solution might be to avoid creating the final temporary
table when the query result is already stored in a temporary table.
"
The patch does not contain a test case, but the description of the
problem corresponds exactly what could be observed in the test
case for mdev-11081.
Refactour out (into a copy for now) the logic of Item_sum_hybrid, to
allow for multiple arguments. It does not contain the comparator
members. The result is the class Item_sum_hybrid_simple.
LEAD and LAG make use of this Item to store previous rows in a chache.
It also helps in specifying the field type. Currently LEAD/LAG do not
support default values.
NTH_VALUE behaves identical to LEAD and LAG, except that the starting
position cursor is placed on the top of the frame instead of the current
row.
Fix window function expressions such as win_func() <operator> expr.
The problem was found in 2 places.
First, when we have complex expressions containing window functions, we
can only compute their final value _after_ we have computed the window
function's values. These values must be stored within the temporary
table that we are using, before sending them off.
This is done by performing an extra copy_funcs call before the final
end_send() call.
Second, such expressions need to have their inner arguments,
changed such that the references within those arguments point to fields within
the temporary table.
Ex: sum(t.a) over (order by t.b) + sum(t.a) over (order by t.b)
Before this fix, t.a pointed to the original table's a field. In order
to compute the sum function's value correctly, it needs to point to the
copy of this field inside the temp table.
This is done by calling split_sum_func for each argument in the
expression in turn.
The win.test results have also been updated as they contained wrong
values for such a use case.
The condition pushed into WHERE/HAVING of a materialized
view/derived table may differ for different executions of
the same prepared statement. That's why the should be
ANDed with the existing WHERE/HAVING conditions only after all
permanent transformations of these conditions has been
performed.
