Queries involving predicates of the form "const NOT BETWEEN
not_indexed_column AND indexed_column" could return wrong data
due to incorrect handling by the range optimizer.
For "c NOT BETWEEN f1 AND f2" predicates, get_mm_tree()
produces a disjunction of the SEL_ARG trees for "f1 > c" and
"f2 < c". If one of the trees is empty (i.e. one of the
arguments is not sargable) the resulting tree should be empty
as well, since the whole expression in this case is not
sargable.
The above logic is implemented in get_mm_tree() as follows. The
initial state of the resulting tree is NULL (aka empty). We
then iterate through arguments and compute the corresponding
SEL_ARG tree (either "f1 > c" or "f2 < c"). If the resulting
tree is NULL, it is simply replaced by the generated
tree. Otherwise it is replaced by a disjunction of itself and
the generated tree. The obvious flaw in this implementation is
that if the first argument is not sargable and thus produces a
NULL tree, the resulting tree will simply be replaced by the
tree for the second argument. As a result, "c NOT BETWEEN f1
AND f2" will end up as just "f2 < c".
Fixed by adding a check so that when the first argument
produces an empty tree for the NOT BETWEEN case, the loop is
aborted with an empty tree as a result. The whole idea of using
a loop for 2 arguments does not make much sense, but it was
probably used to avoid code duplication for several BETWEEN
variants.
feature
The test for bug no 50939 was put in range.test which isn't such a good idea
since it requires partitioning. Fixed by moving the test case to
partitioning_range.test.
feature
The test for bug no 50939 was put in range.test which isn't such a good idea
since it requires partitioning. Fixed by moving the test case to
partitioning_range.test.
remember range endpoints
The Loose Index Scan optimization keeps track of a sequence
of intervals. For the current interval it maintains the
current interval's endpoints. But the maximum endpoint was
not stored in the SQL layer; rather, it relied on the
storage engine to retain this value in-between reads. By
coincidence this holds for MyISAM and InnoDB. Not for the
partitioning engine, however.
Fixed by making the key values iterator
(QUICK_RANGE_SELECT) keep track of the current maximum endpoint.
This is also more efficient as we save a call through the
handler API in case of open-ended intervals.
The code to calculate endpoints was extracted into
separate methods in QUICK_RANGE_SELECT, and it was possible to
get rid of some code duplication as part of fix.
remember range endpoints
The Loose Index Scan optimization keeps track of a sequence
of intervals. For the current interval it maintains the
current interval's endpoints. But the maximum endpoint was
not stored in the SQL layer; rather, it relied on the
storage engine to retain this value in-between reads. By
coincidence this holds for MyISAM and InnoDB. Not for the
partitioning engine, however.
Fixed by making the key values iterator
(QUICK_RANGE_SELECT) keep track of the current maximum endpoint.
This is also more efficient as we save a call through the
handler API in case of open-ended intervals.
The code to calculate endpoints was extracted into
separate methods in QUICK_RANGE_SELECT, and it was possible to
get rid of some code duplication as part of fix.
for each record'
There was an error in an internal structure in the range
optimizer (SEL_ARG). Bad design causes parts of a data
structure not to be initialized when it is in a certain
state. All client code must check that this state is not
present before trying to access the structure's data. Fixed
by
- Checking the state before trying to access data (in
several places, most of which not covered by test case.)
- Copying the keypart id when cloning SEL_ARGs
mysql-test/r/range.result:
Bug#48459: Test result.
mysql-test/t/range.test:
Bug#48459: Test case.
sql/opt_range.cc:
Bug#48459: Fix + doxygenated count_key_part_usage comment.
for each record'
There was an error in an internal structure in the range
optimizer (SEL_ARG). Bad design causes parts of a data
structure not to be initialized when it is in a certain
state. All client code must check that this state is not
present before trying to access the structure's data. Fixed
by
- Checking the state before trying to access data (in
several places, most of which not covered by test case.)
- Copying the keypart id when cloning SEL_ARGs
When merging ranges during calculation of the result of OR
to two range sets the current range may be obsoleted by the
resulting merged range.
The first overlapping range can be obsoleted as well.
Fixed by moving the pointer to the first overlapping range to the
pointer of the resulting union range.
Added few comments at key places in key_or().
When merging ranges during calculation of the result of OR
to two range sets the current range may be obsoleted by the
resulting merged range.
The first overlapping range can be obsoleted as well.
Fixed by moving the pointer to the first overlapping range to the
pointer of the resulting union range.
Added few comments at key places in key_or().
When a query was using a DATE or DATETIME value formatted
using any other separator characters beside hyphen '-', a
query with a greater-or-equal '>=' condition matching only
the greatest value in an indexed column, the result was
empty if index range scan was employed.
The range optimizer got a new feature between 5.1.38 and
5.1.39 that changes a greater-or-equal condition to a
greater-than if the value matching that in the query was not
present in the table. But the value comparison function
compared the dates as strings instead of dates.
The bug was fixed by splitting the function
get_date_from_str in two: One part that parses and does
error checking. This function is now visible outside the
module. The old get_date_from_str now calls the new
function.
mysql-test/r/range.result:
Bug#47925: Test result
mysql-test/t/range.test:
Bug#47925: Test case
sql/item.cc:
Bug#47925: Fix + some edit on the comments
sql/item.h:
Bug#47925: Changed function signature
sql/item_cmpfunc.cc:
Bug#47925: Split function in two
sql/item_cmpfunc.h:
Bug#47925: Declaration of new function
sql/opt_range.cc:
Bug#47925: Added THD to function call
sql/time.cc:
Bug#47925: Added microsecond comparison
When a query was using a DATE or DATETIME value formatted
using any other separator characters beside hyphen '-', a
query with a greater-or-equal '>=' condition matching only
the greatest value in an indexed column, the result was
empty if index range scan was employed.
The range optimizer got a new feature between 5.1.38 and
5.1.39 that changes a greater-or-equal condition to a
greater-than if the value matching that in the query was not
present in the table. But the value comparison function
compared the dates as strings instead of dates.
The bug was fixed by splitting the function
get_date_from_str in two: One part that parses and does
error checking. This function is now visible outside the
module. The old get_date_from_str now calls the new
function.
The problem was in incorrect handling of predicates involving
NULL as a constant value by the range optimizer.
For example, when creating a SEL_ARG node from a condition of
the form "field < const" (which would normally result in the
"NULL < field < const" SEL_ARG), the special case when "const"
is NULL was not taken into account, so "NULL < field < NULL"
was produced for the "field < NULL" condition.
As a result, SEL_ARG structures of this form could not be
further optimized which in turn could lead to incorrectly
constructed SEL_ARG trees. In particular, code assuming SEL_ARG
structures to always form a sequence of ordered disjoint
intervals could enter an infinite loop under some
circumstances.
Fixed by changing get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
mysql-test/r/partition_pruning.result:
Fixed a broken test case.
mysql-test/r/range.result:
Added a test case for bug #47123.
mysql-test/r/subselect.result:
Fixed a broken test cases.
mysql-test/t/range.test:
Added a test case for bug #47123.
sql/opt_range.cc:
Fixed get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
The problem was in incorrect handling of predicates involving
NULL as a constant value by the range optimizer.
For example, when creating a SEL_ARG node from a condition of
the form "field < const" (which would normally result in the
"NULL < field < const" SEL_ARG), the special case when "const"
is NULL was not taken into account, so "NULL < field < NULL"
was produced for the "field < NULL" condition.
As a result, SEL_ARG structures of this form could not be
further optimized which in turn could lead to incorrectly
constructed SEL_ARG trees. In particular, code assuming SEL_ARG
structures to always form a sequence of ordered disjoint
intervals could enter an infinite loop under some
circumstances.
Fixed by changing get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
The problem was in incorrect handling of predicates involving
NULL as a constant value by the range optimizer.
For example, when creating a SEL_ARG node from a condition of
the form "field < const" (which would normally result in the
"NULL < field < const" SEL_ARG), the special case when "const"
is NULL was not taken into account, so "NULL < field < NULL"
was produced for the "field < NULL" condition.
As a result, SEL_ARG structures of this form could not be
further optimized which in turn could lead to incorrectly
constructed SEL_ARG trees. In particular, code assuming SEL_ARG
structures to always form a sequence of ordered disjoint
intervals could enter an infinite loop under some
circumstances.
Fixed by changing get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
mysql-test/r/range.result:
Added a test case for bug #47123.
mysql-test/t/range.test:
Added a test case for bug #47123.
sql/opt_range.cc:
Fixed get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
The problem was in incorrect handling of predicates involving
NULL as a constant value by the range optimizer.
For example, when creating a SEL_ARG node from a condition of
the form "field < const" (which would normally result in the
"NULL < field < const" SEL_ARG), the special case when "const"
is NULL was not taken into account, so "NULL < field < NULL"
was produced for the "field < NULL" condition.
As a result, SEL_ARG structures of this form could not be
further optimized which in turn could lead to incorrectly
constructed SEL_ARG trees. In particular, code assuming SEL_ARG
structures to always form a sequence of ordered disjoint
intervals could enter an infinite loop under some
circumstances.
Fixed by changing get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
covering index
When two range predicates were combined under an OR
predicate, the algorithm tried to merge overlapping ranges
into one. But the case when a range overlapped several other
ranges was not handled. This lead to
1) ranges overlapping, which gave repeated results and
2) a range that overlapped several other ranges was cut off.
Fixed by
1) Making sure that a range got an upper bound equal to the
next range with a greater minimum.
2) Removing a continue statement
mysql-test/r/group_min_max.result:
Bug#42846: Changed query plans
mysql-test/r/range.result:
Bug#42846: Test result.
mysql-test/t/range.test:
Bug#42846: Test case.
sql/opt_range.cc:
Bug#42846: The fix.
Part1: Previously, both endpoints from key2 were copied,
which is not safe. Since ranges are processed in ascending
order of minimum endpoints, it is safe to copy the minimum
endpoint from key2 but not the maximum. The maximum may only
be copied if there is no other range or the other range's
minimum is greater than key2's maximum.
covering index
When two range predicates were combined under an OR
predicate, the algorithm tried to merge overlapping ranges
into one. But the case when a range overlapped several other
ranges was not handled. This lead to
1) ranges overlapping, which gave repeated results and
2) a range that overlapped several other ranges was cut off.
Fixed by
1) Making sure that a range got an upper bound equal to the
next range with a greater minimum.
2) Removing a continue statement
WHERE f1 < n ignored row if f1 was indexed integer column and
f1 = TYPE_MAX ^ n = TYPE_MAX+1. The latter value when treated
as TYPE overflowed (obviously). This was not handled, it is now.
mysql-test/r/range.result:
show that on an index int column, we no longer disregard
a field val of TYPE_MAX in SELECT ... WHERE ... < TYPE_MAX+1
mysql-test/t/range.test:
show that on an index int column, we no longer disregard
a field val of TYPE_MAX in SELECT ... WHERE ... < TYPE_MAX+1
sql/opt_range.cc:
Handle overflowing of int-types in range-optimizer.
Unfortunately requires re-indentation of entire block.
Overflow (err == 1) was handled, but only if
field->cmp_type() != value->result_type(), which it
just wasn't in our case.
WHERE f1 < n ignored row if f1 was indexed integer column and
f1 = TYPE_MAX ^ n = TYPE_MAX+1. The latter value when treated
as TYPE overflowed (obviously). This was not handled, it is now.
into kaamos.(none):/data/src/opt/mysql-5.1-opt
mysql-test/r/heap.result:
Auto merged
mysql-test/r/innodb.result:
Auto merged
mysql-test/r/myisam.result:
Auto merged
BitKeeper/deleted/.del-bdb.result:
Auto merged
mysql-test/r/strict.result:
Auto merged
mysql-test/r/type_binary.result:
Auto merged
mysql-test/r/type_set.result:
Auto merged
mysql-test/r/variables.result:
Auto merged
mysql-test/r/warnings.result:
Auto merged
mysql-test/t/range.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
mysql-test/t/warnings.test:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/mysqld.cc:
Null merge.
mysql-test/include/mix1.inc:
Manual merge.
mysql-test/r/innodb_mysql.result:
Manual merge.
mysql-test/r/range.result:
Manual merge.
mysql-test/r/sp.result:
Manual merge.
mysql-test/t/sp.test:
Manual merge.
sql/handler.cc:
Manual merge.
Two disjuncts containing equalities of the form key=const1 and key=const2 can
be merged into one if const1 is equal to const2. To check it the common
collation of the constants were used rather than the collation of the field key.
For example when the default collation of the constants was cases insensitive
while the collation of the field was case sensitive, then two or-ed equality
predicates key='b' and key='B' incorrectly were merged into one f='b'. As a
result ref access was used instead of range access and wrong result sets were
returned in many cases.
Fixed the problem by comparing constant in the or-ed predicate with collation of
the key field.
mysql-test/r/range.result:
Added a test case for bug #33833.
mysql-test/t/range.test:
Added a test case for bug #33833.
sql/item.cc:
Fixed bug #33833.
Added the method eq_by_collation that compares two items almost as
the method Item::eq, but it rather enforces a given collation for
the comparison.
sql/item.h:
Fixed bug #33833.
Added the method eq_by_collation that compares two items almost as
the method Item::eq, but it rather enforces a given collation for
the comparison.
Two disjuncts containing equalities of the form key=const1 and key=const2 can
be merged into one if const1 is equal to const2. To check it the common
collation of the constants were used rather than the collation of the field key.
For example when the default collation of the constants was cases insensitive
while the collation of the field was case sensitive, then two or-ed equality
predicates key='b' and key='B' incorrectly were merged into one f='b'. As a
result ref access was used instead of range access and wrong result sets were
returned in many cases.
Fixed the problem by comparing constant in the or-ed predicate with collation of
the key field.
into mysql.com:/home/gluh/MySQL/Merge/5.1-opt
client/mysql.cc:
Auto merged
client/mysqltest.c:
Auto merged
include/mysql_com.h:
Auto merged
libmysql/CMakeLists.txt:
Auto merged
mysql-test/r/delayed.result:
Auto merged
mysql-test/r/func_misc.result:
Auto merged
mysql-test/r/innodb.result:
Auto merged
mysql-test/r/myisam.result:
Auto merged
mysql-test/r/ps.result:
Auto merged
mysql-test/t/func_misc.test:
Auto merged
mysql-test/t/merge.test:
Auto merged
mysql-test/t/myisam.test:
Auto merged
mysql-test/t/range.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/type_datetime.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
sql/field.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/set_var.h:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
storage/myisam/mi_check.c:
Auto merged
mysql-test/include/mix1.inc:
manual merge
mysql-test/r/innodb_mysql.result:
manual merge
mysql-test/r/range.result:
manual merge
sql/item_cmpfunc.cc:
manual merge
- Make conditions like "date_col $CMP$ 'datetime-const'" range-sargable
mysql-test/r/range.result:
BUG#32198: Comparison of DATE with DATETIME still not using indexes correctly
- Testcase
mysql-test/t/range.test:
BUG#32198: Comparison of DATE with DATETIME still not using indexes correctly
- Testcase
sql/field.cc:
BUG#32198: Comparison of DATE with DATETIME still not using indexes correctly
- Added comments
into mysql.com:/home/psergey/mysql-5.1-merge2
mysql-test/r/range.result:
Auto merged
mysql-test/t/range.test:
Auto merged
sql/opt_range.cc:
Auto merged
Pushbuild fixes:
- Make MAX_SEL_ARGS smaller (even 16K records_in_range() calls is
more than it makes sense to do in typical cases)
- Don't call sel_arg->test_use_count() if we've already allocated
more than MAX_SEL_ARGs elements. The test will succeed but will take
too much time for the test suite (and not provide much value).
mysql-test/r/range.result:
BUG#26624: high mem usage (crash) in range optimizer
Pushbuild fixes: make the test go faster
mysql-test/t/range.test:
BUG#26624: high mem usage (crash) in range optimizer
Pushbuild fixes: make the test go faster
Pushbuild fixes:
- Make MAX_SEL_ARGS smaller (even 16K records_in_range() calls is
more than it makes sense to do in typical cases)
- Don't call sel_arg->test_use_count() if we've already allocated
more than MAX_SEL_ARGs elements. The test will succeed but will take
too much time for the test suite (and not provide much value).
into mysql.com:/home/psergey/mysql-5.1-merge
BitKeeper/deleted/.del-mysqld.cc.rej:
Auto merged
mysql-test/r/range.result:
Auto merged
mysql-test/t/range.test:
Auto merged
sql/item_sum.cc:
Auto merged
sql/sql_list.h:
Auto merged
mysql-test/r/subselect.result:
Manual merge
mysql-test/t/subselect.test:
Manual merge
sql/opt_range.cc:
Manual merge
- Added PARAM::alloced_sel_args where we count the # of SEL_ARGs
created by SEL_ARG tree cloning operations.
- Made the range analyzer to shortcut and not do any more cloning
if we've already created MAX_SEL_ARGS SEL_ARG objects in cloning.
- Added comments about space complexity of SEL_ARG-graph
representation.
mysql-test/r/range.result:
BUG#26624: Testcase
mysql-test/t/range.test:
BUG#26624: Testcase
- Added PARAM::alloced_sel_args where we count the # of SEL_ARGs
created by SEL_ARG tree cloning operations.
- Made the range analyzer to shortcut and not do any more cloning
if we've already created MAX_SEL_ARGS SEL_ARG objects in cloning.
- Added comments about space complexity of SEL_ARG-graph
representation.
into zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-4.1-maint
configure.in:
Auto merged
mysql-test/r/range.result:
Auto merged
mysql-test/t/range.test:
Auto merged
into mysql.com:/home/gluh/MySQL/Merge/5.1-opt
BitKeeper/etc/ignore:
auto-union
mysql-test/r/func_in.result:
Auto merged
mysql-test/r/range.result:
Auto merged
mysql-test/r/sp-code.result:
Auto merged
mysql-test/r/trigger.result:
Auto merged
mysql-test/r/udf.result:
Auto merged
mysql-test/t/func_in.test:
Auto merged
mysql-test/t/range.test:
Auto merged
mysql-test/t/trigger.test:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_prepare.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_table.cc:
Auto merged
mysql-test/r/select.result:
manual merge
mysql-test/r/view.result:
manual merge
mysql-test/t/select.test:
manual merge
mysql-test/t/view.test:
manual merge
