When executing row-ordered-retrieval index merge,
the handler was cloned, but it used the wrong
memory root, so instead of allocating memory
on the thread/query's mem_root, it used the table's
mem_root, resulting in non released memory in the
table object, and was not freed until the table was
closed.
Solution was to ensure that memory used during cloning
of a handler was allocated from the correct memory root.
This was implemented by fixing handler::clone() to also
take a name argument, so it can be used with partitioning.
And in ha_partition only allocate the ha_partition's ref, and
call the original ha_partition partitions clone() and set at cloned
partitions.
Fix of .bzrignore on Windows with VS 2010
- Fixed some issues with partitions and connection_string, which also fixed lp:716890 "Pre- and post-recovery crash in Aria"
- Fixed wrong assert in Aria
Now need to merge with latest xtradb before pushing
sql/ha_partition.cc:
Ensure that m_ordered_rec_buffer is not freed before close.
sql/mysqld.cc:
Changed to use opt_stack_trace instead of opt_pstack.
Removed references to pstack
sql/partition_element.h:
Ensure that connect_string is initialized
storage/maria/ma_key_recover.c:
Fixed wrong assert
- Removed files specific to compiling on OS/2
- Removed files specific to SCO Unix packaging
- Removed "libmysqld/copyright", text is included in documentation
- Removed LaTeX headers for NDB Doxygen documentation
- Removed obsolete NDB files
- Removed "mkisofs" binaries
- Removed the "cvs2cl.pl" script
- Changed a few GPL texts to use "program" instead of "library"
Fix post-merge failure in 5.1-merge
- Let QUICK_RANGE_INTERSECT_SELECT not make assumption that HA_EXTRA_KEYREAD
scans do not touch parts of table->record[0] that refer to fields that are
not covered by the used index.
This assumption is not true for XtraDB (e.g. grep row/row0sel.c for
"init null bytes with default values as they might be").
BUG#26447 prefer a clustered key for an index scan, as secondary index is always slower
... which was fixed to cause
BUG#35850 queries take 50% longer
... and was reverted
and
BUG#39653 prefer a secondary index for an index scan, as clustered key is always slower
... which was fixed to cause
BUG#55656 mysqldump takes six days instead of half an hour
... and was amended with a special case workaround
sql/opt_range.cc:
move get_index_only_read_time() into the handler class
sql/sql_select.cc:
use cost not an index length when choosing a cheaper index
Subselect executes twice, at JOIN::optimize stage
and at JOIN::execute stage. At optimize stage
Innodb prebuilt struct which is used for the
retrieval of column values is initialized in.
ha_innobase::index_read(), prebuilt->sql_stat_start is true.
After QUICK_ROR_INTERSECT_SELECT finished his job it
restores read_set/write_set bitmaps with initial values
and deactivates one of the handlers used by
QUICK_ROR_INTERSECT_SELECT in JOIN::cleanup
(it's the case when we reuse original handler as one of
handlers required by QUICK_ROR_INTERSECT_SELECT object).
On second subselect execution inactive handler is activated
in QUICK_RANGE_SELECT::reset, file->ha_index_init().
In ha_index_init Innodb prebuilt struct is reinitialized
with inappropriate read_set/write_set bitmaps. Further
reinitialization in ha_innobase::index_read() does not
happen as prebuilt->sql_stat_start is false.
It leads to partial retrieval of required field values
and we get a mix of field values from different records
in the record buffer.
The fix is to reset
read_set/write_set bitmaps as these values
are required for proper intialization of
internal InnoDB struct which is used for
the retrieval of column values
(see build_template(), ha_innodb.cc)
mysql-test/include/index_merge_ror_cpk.inc:
test case
mysql-test/r/index_merge_innodb.result:
test case
mysql-test/r/index_merge_myisam.result:
test case
sql/opt_range.cc:
if ROR merge scan is used we need to reset
read_set/write_set bitmaps as these values
are required for proper intialization of
internal InnoDB struct which is used for
the retrieval of column values
(see build_template(), ha_innodb.cc)
- Changed to still use bcmp() in certain cases becasue
- Faster for short unaligneed strings than memcmp()
- Bettern when using valgrind
- Changed to use my_sprintf() instead of sprintf() to get higher portability for old systems
- Changed code to use MariaDB version of select->skip_record()
- Removed -%::SCCS/s.% from Makefile.am:s to remove automake warnings
Queries involving predicates of the form "const NOT BETWEEN
not_indexed_column AND indexed_column" could return wrong data
due to incorrect handling by the range optimizer.
For "c NOT BETWEEN f1 AND f2" predicates, get_mm_tree()
produces a disjunction of the SEL_ARG trees for "f1 > c" and
"f2 < c". If one of the trees is empty (i.e. one of the
arguments is not sargable) the resulting tree should be empty
as well, since the whole expression in this case is not
sargable.
The above logic is implemented in get_mm_tree() as follows. The
initial state of the resulting tree is NULL (aka empty). We
then iterate through arguments and compute the corresponding
SEL_ARG tree (either "f1 > c" or "f2 < c"). If the resulting
tree is NULL, it is simply replaced by the generated
tree. Otherwise it is replaced by a disjunction of itself and
the generated tree. The obvious flaw in this implementation is
that if the first argument is not sargable and thus produces a
NULL tree, the resulting tree will simply be replaced by the
tree for the second argument. As a result, "c NOT BETWEEN f1
AND f2" will end up as just "f2 < c".
Fixed by adding a check so that when the first argument
produces an empty tree for the NOT BETWEEN case, the loop is
aborted with an empty tree as a result. The whole idea of using
a loop for 2 arguments does not make much sense, but it was
probably used to avoid code duplication for several BETWEEN
variants.
Calculating the estimated number of records for a range scan
may take a significant time, and it was impossible for a user
to interrupt that process by killing the connection or the
query.
Fixed by checking the thread's 'killed' status in
check_quick_keys() and interrupting the calculation process if
it is set to a non-zero value.
require O(#scans) memory
When an index merge operation was restarted, it would
re-allocate the Unique object controlling the duplicate row
ID elimination. Fixed by making the Unique object a member
of QUICK_INDEX_MERGE_SELECT and thus reusing it throughout
the lifetime of this object.
In process of record search it is not taken into account
that inital quick->file->ref value could be inapplicable
to range interval. After proper row is found this value is
stored into the record buffer and later the record is
filtered out at condition evaluation stage.
The fix is store a refernce of found row to the handler ref field.
mysql-test/r/innodb_mysql.result:
test case
mysql-test/std_data/intersect-bug50389.tsv:
test case
mysql-test/t/innodb_mysql.test:
test case
sql/opt_range.cc:
store a refernce of found row to the handler ref field.
- INSERT with RAND() doesn't require row based logging again
- Some bugs fixed in opt_range() where we table->key_read was wrongly used
.bzrignore:
Ignore new xtstat binary
mysql-test/r/index_merge_myisam.result:
Update results (old result was wrong)
mysql-test/suite/binlog/r/binlog_stm_binlog.result:
Added drop table first
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
Added test for when RAND() requires row based logging
mysql-test/suite/binlog/t/binlog_stm_binlog.test:
Added drop table first
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
Added test for when RAND() requires row based logging
scripts/make_binary_distribution.sh:
Removed type from last commit
sql/item_create.cc:
Don't require row based logging when using RAND() with INSERT
sql/opt_range.cc:
Revert wrong patch from Oracle:
- As QUICK_RANGE_SELECT uses it's own 'file' handler to the tables, one can't use 'table->key_read' as a flag to detect if index only read (keyread) is used or not
- Don't set keyread if keyread is already enabled
- Don't disable key read, if we didn't enable it ourselves
- Simplify code (and ensure that we do proper cleanup of index only read)
sql/opt_range.h:
Added flags to detect if the range optimizer enabled index only read (key read) or not
sql/opt_sum.cc:
Use our more optimized macros
sql/sql_lex.h:
Added 'readable' function to check if we are in a sub query function or not (not normal query or sub query in FROM clause)
sql/sql_select.cc:
Use our more optimized keyread macros
Added ASSERTS early
Simplify code on eliminate_item_equal()
Fixed that substitute_for_best_equal_field() doesn't core dump in case of out of memory conditions.
Removed not needed test for 'field->maybe_null()'
Replaced master_unit()->item with is_subquery_function() (More readable)
sql/sql_update.cc:
Use our more optimized keyread macros
sql/table.cc:
Use our more optimized keyread macros
sql/table.h:
Use separate functions to enable/disable Index only reads
- Safer, more readable, better logging and faster.
remember range endpoints
The Loose Index Scan optimization keeps track of a sequence
of intervals. For the current interval it maintains the
current interval's endpoints. But the maximum endpoint was
not stored in the SQL layer; rather, it relied on the
storage engine to retain this value in-between reads. By
coincidence this holds for MyISAM and InnoDB. Not for the
partitioning engine, however.
Fixed by making the key values iterator
(QUICK_RANGE_SELECT) keep track of the current maximum endpoint.
This is also more efficient as we save a call through the
handler API in case of open-ended intervals.
The code to calculate endpoints was extracted into
separate methods in QUICK_RANGE_SELECT, and it was possible to
get rid of some code duplication as part of fix.
Queries optimized with GROUP_MIN_MAX didn't cleanup KEYREAD
optimization properly. As a result subsequent queries may
return incomplete rows (fields are initialized to default
values).
mysql-test/r/group_min_max.result:
A test case for BUG#49902.
mysql-test/t/group_min_max.test:
A test case for BUG#49902.
sql/opt_range.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/opt_sum.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/sql_select.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/sql_update.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/table.cc:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
sql/table.h:
Refactor of KEYREAD optimization switch so that KEYREAD
handler state is in sync with st_table::key_read flag.
All SQL code is supposed to switch KEYREAD optimization
via st_table::set_keyread().
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
for each record'
There was an error in an internal structure in the range
optimizer (SEL_ARG). Bad design causes parts of a data
structure not to be initialized when it is in a certain
state. All client code must check that this state is not
present before trying to access the structure's data. Fixed
by
- Checking the state before trying to access data (in
several places, most of which not covered by test case.)
- Copying the keypart id when cloning SEL_ARGs
mysql-test/r/range.result:
Bug#48459: Test result.
mysql-test/t/range.test:
Bug#48459: Test case.
sql/opt_range.cc:
Bug#48459: Fix + doxygenated count_key_part_usage comment.
When merging ranges during calculation of the result of OR
to two range sets the current range may be obsoleted by the
resulting merged range.
The first overlapping range can be obsoleted as well.
Fixed by moving the pointer to the first overlapping range to the
pointer of the resulting union range.
Added few comments at key places in key_or().
WHERE conditions
check_group_min_max() checks if the loose index scan
optimization is applicable for a given WHERE condition, that is
if the MIN/MAX attribute participates only in range predicates
comparing the corresponding field with constants.
The problem was that it considered the whole predicate suitable
for the loose index scan optimization as soon as it encountered
a constant as a predicate argument. This is obviously wrong for
cases when a constant is the first argument of a predicate
which does not satisfy the above condition.
Fixed check_group_min_max() so that all arguments of the input
predicate are considered to decide if it passes the test, even
though a constant has already been encountered.
mysql-test/r/group_min_max.result:
Added a test case for bug #48472.
mysql-test/t/group_min_max.test:
Added a test case for bug #48472.
sql/opt_range.cc:
Fixed check_group_min_max() so that all arguments of the input
predicate are considered to decide if it passes the test, even
though a constant has already been encountered.
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
When a query was using a DATE or DATETIME value formatted
using any other separator characters beside hyphen '-', a
query with a greater-or-equal '>=' condition matching only
the greatest value in an indexed column, the result was
empty if index range scan was employed.
The range optimizer got a new feature between 5.1.38 and
5.1.39 that changes a greater-or-equal condition to a
greater-than if the value matching that in the query was not
present in the table. But the value comparison function
compared the dates as strings instead of dates.
The bug was fixed by splitting the function
get_date_from_str in two: One part that parses and does
error checking. This function is now visible outside the
module. The old get_date_from_str now calls the new
function.
mysql-test/r/range.result:
Bug#47925: Test result
mysql-test/t/range.test:
Bug#47925: Test case
sql/item.cc:
Bug#47925: Fix + some edit on the comments
sql/item.h:
Bug#47925: Changed function signature
sql/item_cmpfunc.cc:
Bug#47925: Split function in two
sql/item_cmpfunc.h:
Bug#47925: Declaration of new function
sql/opt_range.cc:
Bug#47925: Added THD to function call
sql/time.cc:
Bug#47925: Added microsecond comparison
Problem: involving a spatial index for "non-spatial" queries
(that don't containt MBRXXX() functions) may lead to failed assert.
Fix: don't use spatial indexes in such cases.
mysql-test/r/gis-rtree.result:
Fix for bug#48258: Assertion failed when using a spatial index
- test result.
mysql-test/t/gis-rtree.test:
Fix for bug#48258: Assertion failed when using a spatial index
- test case.
sql/opt_range.cc:
Fix for bug#48258: Assertion failed when using a spatial index
- allow only spatial functions (MBRXXX) for itMBR keyparts.
The problem was in incorrect handling of predicates involving
NULL as a constant value by the range optimizer.
For example, when creating a SEL_ARG node from a condition of
the form "field < const" (which would normally result in the
"NULL < field < const" SEL_ARG), the special case when "const"
is NULL was not taken into account, so "NULL < field < NULL"
was produced for the "field < NULL" condition.
As a result, SEL_ARG structures of this form could not be
further optimized which in turn could lead to incorrectly
constructed SEL_ARG trees. In particular, code assuming SEL_ARG
structures to always form a sequence of ordered disjoint
intervals could enter an infinite loop under some
circumstances.
Fixed by changing get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
mysql-test/r/partition_pruning.result:
Fixed a broken test case.
mysql-test/r/range.result:
Added a test case for bug #47123.
mysql-test/r/subselect.result:
Fixed a broken test cases.
mysql-test/t/range.test:
Added a test case for bug #47123.
sql/opt_range.cc:
Fixed get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
The problem was in incorrect handling of predicates involving
NULL as a constant value by the range optimizer.
For example, when creating a SEL_ARG node from a condition of
the form "field < const" (which would normally result in the
"NULL < field < const" SEL_ARG), the special case when "const"
is NULL was not taken into account, so "NULL < field < NULL"
was produced for the "field < NULL" condition.
As a result, SEL_ARG structures of this form could not be
further optimized which in turn could lead to incorrectly
constructed SEL_ARG trees. In particular, code assuming SEL_ARG
structures to always form a sequence of ordered disjoint
intervals could enter an infinite loop under some
circumstances.
Fixed by changing get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
mysql-test/r/range.result:
Added a test case for bug #47123.
mysql-test/t/range.test:
Added a test case for bug #47123.
sql/opt_range.cc:
Fixed get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
covering index
When two range predicates were combined under an OR
predicate, the algorithm tried to merge overlapping ranges
into one. But the case when a range overlapped several other
ranges was not handled. This lead to
1) ranges overlapping, which gave repeated results and
2) a range that overlapped several other ranges was cut off.
Fixed by
1) Making sure that a range got an upper bound equal to the
next range with a greater minimum.
2) Removing a continue statement
mysql-test/r/group_min_max.result:
Bug#42846: Changed query plans
mysql-test/r/range.result:
Bug#42846: Test result.
mysql-test/t/range.test:
Bug#42846: Test case.
sql/opt_range.cc:
Bug#42846: The fix.
Part1: Previously, both endpoints from key2 were copied,
which is not safe. Since ranges are processed in ascending
order of minimum endpoints, it is safe to copy the minimum
endpoint from key2 but not the maximum. The maximum may only
be copied if there is no other range or the other range's
minimum is greater than key2's maximum.
can lead to bad memory access
Problem: Field_bit is the only field which returns INT_RESULT
and doesn't have unsigned flag. As it's not a descendant of the
Field_num, so using ((Field_num *) field_bit)->unsigned_flag may lead
to unpredictable results.
Fix: check the field type before casting.
mysql-test/r/type_bit.result:
Fix for bug #42803: Field_bit does not have unsigned_flag field,
can lead to bad memory access
- test result.
mysql-test/t/type_bit.test:
Fix for bug #42803: Field_bit does not have unsigned_flag field,
can lead to bad memory access
- test case.
sql/opt_range.cc:
Fix for bug #42803: Field_bit does not have unsigned_flag field,
can lead to bad memory access
- don't cast to (Field_num *) Field_bit, as it's not a Field_num
descendant and is always unsigned by nature.
* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
results in server crash
check_group_min_max_predicates() assumed the input condition
item to be one of COND_ITEM, SUBSELECT_ITEM, or FUNC_ITEM.
Since a condition of the form "field" is also a valid condition
equivalent to "field <> 0", using such a condition in a query
where the loose index scan was chosen resulted in a debug
assertion failure.
Fixed by handling conditions of the FIELD_ITEM type in
check_group_min_max_predicates().
mysql-test/r/group_min_max.result:
Added a test case for bug #46607.
mysql-test/t/group_min_max.test:
Added a test case for bug #46607.
sql/opt_range.cc:
Handle conditions of the FUNC_ITEM type in
check_group_mix_max_predicates().
with gcc 4.3.2
This patch fixes a number of GCC warnings about variables used
before initialized. A new macro UNINIT_VAR() is introduced for
use in the variable declaration, and LINT_INIT() usage will be
gradually deprecated. (A workaround is used for g++, pending a
patch for a g++ bug.)
GCC warnings for unused results (attribute warn_unused_result)
for a number of system calls (present at least in later
Ubuntus, where the usual void cast trick doesn't work) are
also fixed.
client/mysqlmanager-pwgen.c:
A fix for warn_unused_result, adding fallback to use of
srand()/rand() if /dev/random cannot be used. Also actually
adds calls to rand() in the second branch so that it actually
creates a random password.
sql/opt_range.cc:
Removed duplicate code (if statement must have been duplicated during earlier merge).
sql/sql_partition.cc:
After mergeing bug#46362 and bug#20577, the NULL partition was also searched
when col = const, fixed by checking if = or range.
