mysql-test/r/group_by.result:
Test for MDEV-6855
mysql-test/t/group_by.test:
Test for MDEV-6855
sql/item.h:
Fixed spelling error
sql/opt_range.cc:
Added handling of cond_type == Item::CACHE_ITEM in WHERE clauses for MIN/MAX optimization.
Fixed indentation
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
- Filesort has an optmization where it reads only columns that are
needed before the sorting is done.
- When ref(_or_null) is picked by the join optimizer, it may remove parts
of WHERE clause that are guaranteed to be true.
- However, if we use quick select, we must put all of the range columns into the
read set. Not doing so will may cause us to fail to detect the end of the range.
- When range optimizer cannot the lookup value into [VAR]CHAR(n) column,
it should produce:
= "Impossible range" for equality
= "no range" for non-equalities.
"HAVING SUM(DISTINCT)": WRONG RESULTS.
ISSUE:
------
If a query uses loose index scan and it has both
AGG(DISTINCT) and MIN()/MAX()functions. Then, result values
of MIN/MAX() is set improperly.
When query has AGG(DISTINCT) then end_select is set to
end_send_group. "end_send_group" keeps doing aggregation
until it sees a record from next group. And, then it will
send out the result row of that group.
Since query also has MIN()/MAX() and loose index scan is
used, values of MIN/MAX() are set as part of loose index
scan itself. Setting MIN()/MAX() values as part of loose
index scan overwrites values computed in end_send_group.
This caused invalid result.
For such queries to work loose index scan should stop
performing MIN/MAX() aggregation. And, let end_send_group to
do the same. But according to current design loose index
scan can produce only one row per group key. If we have both
MIN() and MAX() then it has to give two records out. This is
not possible as interface has to use common buffer
record[0]! for both records at a time.
SOLUTIONS:
----------
For such queries to work we need a new interface for loose
index scan. Hence, do not choose loose_index_scan for such
cases. So a new rule SA7 is introduced to take care of the
same.
SA7: "If Q has both AGG_FUNC(DISTINCT ...) and
MIN/MAX() functions then loose index scan access
method is not used."
mysql-test/r/group_min_max.result:
Expected result.
mysql-test/t/group_min_max.test:
1. Test with various combination of AGG(DISTINCT) and
MIN(), MAX() functions.
2. Corrected the plan for old queries.
sql/opt_range.cc:
A new rule SA7 is introduced.
Problem:
If there is a predicate on a column referenced by MIN/MAX and
that predicate is not present in all the disjunctions on
keyparts earlier in the compound index, Loose Index Scan will
not return correct result.
Analysis:
When loose index scan is chosen, range optimizer currently
groups all the predicates that contain group parts separately
and minmax parts separately. It therefore applies all the
conditions on the group parts first to the fetched row.
Then in the call to next_max, it processes the conditions
which have min/max keypart.
For ex in the following query:
Select f1, max(f2) from t1 where (f1 = 10 and f2 = 13) or
(f1 = 3) group by f1;
Condition (f2 = 13) would be applied even for rows that
satisfy (f1 = 3) thereby giving wrong results.
Solution:
Do not choose loose_index_scan for such cases. So a new rule
WA2 is introduced to take care of the same.
WA2: "If there are predicates on C, these predicates must
be in conjuction to all predicates on all earlier keyparts
in I."
Todo the same, fix reuses the function get_constant_key_infix().
Since this funciton will fail for all multi-range conditions, it
is re-written to recognize that if the sub-conditions are
equivalent across the disjuncts: it will now succeed.
And to achieve this a new helper function is introduced called
all_same().
The fix also moves the test of NGA3 up to the former only
caller, get_constant_key_infix().
mysql-test/r/group_min_max_innodb.result:
Added test result change for Bug#17909656
mysql-test/t/group_min_max_innodb.test:
Added test cases for Bug#17909656
sql/opt_range.cc:
Introduced Rule WA2 because of Bug#17909656
MDEV-4556 Server crashes in SEL_ARG::rb_insert with index_merge+index_merge_sort_union, FORCE INDEX
- merge_same_index_scans() may put the same SEL_ARG tree in multiple result plans.
make it call incr_refs() on the SEL_ARG trees that it does key_or() on, because
key_or(sel_arg_tree_1, sel_arg_tree_2) call may invalidate SEL_ARG trees pointed
by sel_arg_tree_1 and sel_arg_tree_2.
For queries like
update t1 set ... where <unique key> order by ... limit ...
we need to handle the fact that unique keys allow NULL
values, and hence can return more than one row.
sql/opt_range.cc:
When the unique key has multiple key parts,
check each key_part for nullability, rather than the first key part.
(s/key->part ==/key_tree->part ==/)
Also: revert the if() test, for better readability.
In the case of loose scan used as input for order by, end_send()
didn't detect correctly that a loose scan was used, and didn't copy
the non-aggregated fields from the temp table used for ORDER BY.
The fix uses the fact that the quick select used for sorting is
attached to JOIN::pre_sort_join_tab instead of JOIN::join_tab.
- merge_same_index_scans() may put the same SEL_ARG tree in multiple result plans.
make it call incr_refs() on the SEL_ARG trees that it does key_or() on, because
key_or(sel_arg_tree_1, sel_arg_tree_2) call may invalidate SEL_ARG trees pointed
by sel_arg_tree_1 and sel_arg_tree_2.
-Added test and extra code to ensure we don't leave keyread on for a handler table.
-Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
mysql-test/include/default_mysqld.cnf:
Run test suite with smaller aria keybuffer size
mysql-test/suite/maria/maria3.result:
Run test suite with smaller aria keybuffer size
mysql-test/suite/sys_vars/r/aria_pagecache_buffer_size_basic.result:
Run test suite with smaller aria keybuffer size
sql/handler.cc:
Disable key read (extra safety if something went wrong)
sql/multi_range_read.cc:
Ensure we have don't leave keyread on for secondary_file
sql/opt_range.cc:
Simplify code with mark_columns_used_by_index_no_reset()
Ensure that read_keys_and_merge() disableds keyread if it enables it
sql/opt_subselect.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_derived.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_select.cc:
Use 'enable_keyread()' instead of calling HA_EXTRA_RESET. (Makes debugging easier)
Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
Remove not anymore used argument for create_internal_tmp_table()
More DBUG
sql/sql_select.h:
Remove not anymore used argument for create_internal_tmp_table()
get_mm_leaf function can store all sorts of spatial features in
one type of field it receives from an Item_field.
So we just allow that by setting the type of this field to GEOMETRY.
per-file comments:
mysql-test/r/gis-rtree.result
result updated
mysql-test/t/gis-rtree.test
test case added.
sql/opt_range.cc
set geom_type=GEOMETRY if we got Field_geom.
GROUP BY, MYISAM
Problem:-
In a query, where we are using loose index scan optimization and
we have MIN() causes segmentation fault(where table row length
is less then key_length).
Analysis:
While using loose index scan for MIN(), we call key_copy(), to copy
the key data from record.
This function is using temporary record buffer to store key data
from the record buffer.But in case where the key length is greater
then the buffer length, this will cause a segmentation fault.
Solution:
Give a proper buffer to store a key record.
sql/opt_range.cc:
We can't use record buffer to store key data.So, give a proper buffer to store a key record.
PARTIAL INDEX
Consider the following table definition:
CREATE TABLE t (
my_col CHAR(10),
...
INDEX my_idx (my_col(1))
)
The my_idx index is not able to distinguish between rows with
equal first-character my_col-values (e.g. "f", "foo", "fee").
Prior to this CS, the range optimizer would translate
"WHERE my_col NOT IN ('f', 'h')" into (optimizer trace syntax)
"ranges": [
"NULL < my_col < f",
"f < my_col"
]
But this was not correct because the rows with values "foo"
and "fee" would not belong to any of those ranges. However, the
predicate "my_col != 'f' AND my_col != 'h'" would translate
to
"ranges": [
"NULL < my_col"
]
because get_mm_leaf() changes from "<" to "<=" for partial
keyparts. This CS changes the range optimizer implementation
for NOT IN to behave like a conjunction of NOT EQUAL: it
replaces "<" with "<=" for all but the first range when the
keypart is partial.
currently get_mm_tree skipped the evaluation of this constant
and icorrectly proceeded. The correct behavior is to return a
NULL subtree, according to the IF branch being fixed - when it
evaluates the constant it returns a value, and doesn't continue
further.
- Let index_merge allocate table handlers on quick select's MEM_ROOT,
not on statement's MEM_ROOT.
This is crucial for big "range checked for each record" queries, where
index_merge can be created and deleted many times during query exection.
We should not make O(#rows) allocations on statement's MEM_ROOT.
The range optimizer uses 'save_in_field_no_warnings()' to verify properties of
'value <cmp> field' expressions.
If this execution yields an error, it should abort.
Analysis:
Range analysis detects that the subquery is expensive and doesn't
build a range access method. Later, the applicability test for loose
scan doesn't take that into account, and builds a loose scan method
without a range scan on the min/max column. As a result loose scan
fetches the first key in each group, rather than the first key that
satisfies the condition on the min/max column.
Solution:
Since there is no SEL_ARG tree to be used for the min/max column,
it is not possible to use loose scan if the min/max column is compared
with an expensive scalar subquery. Make the test for loose scan
applicability to be in sync with the range analysis code by testing if
the min/max argument is compared with an expensive predicate.
Analys:
The cause for the wrong result was that the optimizer
incorrectly chose min/max loose scan when it is not
applicable. The applicability test missed the case when
a condition on the MIN/MAX argument was OR-ed with a
condition on some other field. In this case, the MIN/MAX
condition cannot be used for loose scan.
Solution:
Extend the test check_group_min_max_predicates() to check
that the WHERE clause is of the form: "cond1 AND cond2"
where
cond1 - does not use min_max_column at all.
cond2 - is an AND/OR tree with leaves in form "min_max_column $CMP$ const"
or $CMP$ is one of the functions between, is [not] null
Analysis:
Range analysis discoveres that the query can be executed via loose index scan for GROUP BY.
Later, GROUP BY analysis fails to confirm that the GROUP operation can be computed via an
index because there is no logic to handle duplicate field references in the GROUP clause.
As a result the optimizer produces an inconsistent plan. It constructs a temporary table,
but on the other hand the group fields are not set to point there.
Solution:
Make loose scan analysis work in sync with order by analysis. In the case of duplicate
columns loose scan will not be applicable. This limitation will be lifted in 10.0 by
removing duplicate columns.
ON COL WITH COMPOSITE INDEX
This problem is caused by the patch for the bug#11751794.
While checking for the keypart covering non grouping attribute. we are not
checking whether the root node of the SEL_ARG* tree for the index have any
cvalue or not.
sql/opt_range.cc:
check whether the keeypart_tree has any range tree.
ON COL WITH COMPOSITE INDEX
This problem is caused by the patch for the bug#11751794.
While checking for the keypart covering non grouping attribute. we are not
checking whether the root node of the SEL_ARG* tree for the index have any
cvalue or not.
from MariaDB 10.0.
The bug in mdev-3948 was an instance of the problem fixed by Sergey's patch
in 10.0 - namely that the range optimizer could change table->[read | write]_set,
and not restore it.
revno: 3471
committer: Sergey Petrunya <psergey@askmonty.org>
branch nick: 10.0-serg-fix-imerge
timestamp: Sat 2012-11-03 12:24:36 +0400
message:
# MDEV-3817: Wrong result with index_merge+index_merge_intersection, InnoDB table, join, AND and OR conditions
Reconcile the fixes from:
#
# guilhem.bichot@oracle.com-20110805143029-ywrzuz15uzgontr0
# Fix for BUG#12698916 - "JOIN QUERY GIVES WRONG RESULT AT 2ND EXEC. OR
# AFTER FLUSH TABLES [-INT VS NULL]"
#
# guilhem.bichot@oracle.com-20111209150650-tzx3ldzxe1yfwji6
# Fix for BUG#12912171 - ASSERTION FAILED: QUICK->HEAD->READ_SET == SAVE_READ_SET
# and
#
and related fixes from: BUG#1006164, MDEV-376:
Now, ROR-merged QUICK_RANGE_SELECT objects make no assumptions about the values
of table->read_set and table->write_set.
Each QUICK_ROR_SELECT has (and had before) its own column bitmap, but now, all
QUICK_ROR_SELECT's functions that care: reset(), init_ror_merged_scan(), and
get_next() will set table->read_set when invoked and restore it back to what
it was before the call before they return.
This allows to avoid the mess when somebody else modifies table->read_set for
some reason.
