Analysis (BUG#719198):
The assert failed because the execution code for
partial matching is designed with the assumption that
NULLs on the left side are detected as early as possible,
and a NULL result is returned before any lookups are
performed at all.
However, in the case of an Item_cache object on the left
side, null was not detected properly, because detection
was done via Item::is_null(), which is not implemented at
all for Item_cache, and resolved to the default Item::is_null()
which always returns FALSE.
Solution:
Imlpement Item::is_null().
******
Analysis (BUG#730604):
The method Item_field::is_null() determines if an item is NULL from its
Item_field::field object. However, for Item_fields that represent internal
temporary tables, Item_field::field represents the field of the original
table that was the source for the temporary table (in this case t1.f3).
Both in the committed test case, and in the original bug report the current
value of t1.f3 is not NULL. This results in an incorrect count of NULLs
for this column. As a consequence, all related Ordered_key buffers are
allocated with incorrect sizes. Depending on the exact query and data,
these incorrect sizes result in various crashes or failed asserts.
Solution:
The correct value of the current field of the internal temp table is
in Item_field::result_field. This value is determined by
Item::is_null_result().
Analysis:
The assert failed because the execution code for
partial matching is designed with the assumption that
NULLs on the left side are detected as early as possible,
and a NULL result is returned before any lookups are
performed at all.
However, in the case of an Item_cache object on the left
side, null was not detected properly, because detection
was done via Item::is_null(), which is not implemented at
all for Item_cache, and resolved to the default Item::is_null()
which always returns FALSE.
Solution:
Use the property Item::null_value instead of is_null(), which
is properly updated for Item_cache objects as well.
If join condition is of the form <t2.key>=<t1.no_key> then the server
performs no index look-ups when looking for matching rows of t2 for
the rows from t1 with t1.no_key=NULL. It happens because the function
add_not_null_conds() injects an additional condition of the form
IS NOT NULL(<t1.no_key>) into the WHERE condition.
However if the join condition was of the form <t.key>=<outer_ref> no
additional null rejecting predicate was generated. This could lead
to extra records in the result set if the value of <outer_ref> happened
to be NULL.
The new code injects null rejecting predicates of the form
IS NOT NULL(<outer_ref>) and evaluates them before the first row
the subquery is constructed.
- put the code that sets HA_NULL_PART bit back
- Fix test_if_ref/part_of_refkey() so that
= NULL-ability of lookup columns does not prevent the equality
from being removed (we now have early/late NULLs filtering which
will filter out NULL values)
= equality is not removed if it is ref_or_null access, and the value
of the lookup column can alternate between the lookup value and NULL.
The bug was a result of the fix for bug 668644 that turned out to be
not quite correct. A problem appeared with HAVING conditions containing
more than one predicate. If a query with an ORDER BY clause uses
such HAVING condition and the required order can be obtained with
a range/index scan then the HAVING condition has to be pushed into
two different formulas (items). To be able to do it we have to create
a copy of the ANDOR structure of the pushed condition.
- The problem was that Mrr_ordered_index_reader's interrupt_read() and resume_read() would
save and restore 1) index tuple 2) the rowid (as bytes returned by handler->position()). Clustered
primary key columns were not saved/restored.
They are not explicitly present in the index tuple (i.e. table->key_info[secondary_key].key_parts
doesn't list them), but they are actually there, in particular
table->field[clustered_primary_key_member].part_of_key(secondary_key) == 1. Index condition pushdown
code [correctly] uses the latter as inidication that pushed index condition can refer to clustered PK
members.
The fix was to make interrupt_read()/resume_read() to save/restore clustered primary key members as well,
so that we get correct values for them when evaluating pushed index condition.
[3rd attempt: remove the debugging aids, fix comments in testcase]
Analysis:
The reason for the crash was that the inner subquery was executed
via a scan on a final temporary table applied after all other
operations. This final operation is implemented by changing the
contents of the JOIN object of the subquery to represent a table
scan over the temp table. At the same time query optimization of
the outer subquery required evaluation of the inner subquery, which
happened before the actual EXPLAIN. The evaluation left the JOIN
object of the inner subquery in the changed state, where it represented
a table scan over a temp table, and EXPLAIN crashed because the temp
table is not associated with any table reference (TABLE_LIST object).
The reason the JOIN was not restored was because its saving/restoration
was controlled by the join->select_lex->uncacheable flag, which was
not set in the case of materialization.
Solution:
In the methods Item_in_subselect::[single | row]_value_transformer() set:
select_lex->uncacheable|= UNCACHEABLE_EXPLAIN;
In addition, for symmetry, change:
master_unit->uncacheable|= UNCACHEABLE_EXPLAIN;
instead of UNCACHEABLE_DEPENDENT because if a subquery was not
dependent initially, the changed methods do not change this
fact. The subquery may later become correlated if it is transformed
to an EXISTS query, but it may stay uncorrelated if executed via
materialization.
- Make DsMrr_impl::dsmrr_init() handle the case of
1. 1st MRR scan using DS-MRR strategy (i.e. doing key sorting and rowid sorting)
2. 2nd MRR scan getting a buffer that's too small to fit one key element
and one rowid element, and so falling back to default MRR implementation
In this case, dsmrr_init() is invoked with {primary_handler, secondary_handler}
initialized for DS-MRR scan and have to reset them to be initialized for the
default MRR scan.
(attempt 2, with simplified testcase)
- Added ORDER BY to get consistent results to federated_server
- Sort slow tests first
mysql-test/lib/My/ConfigFactory.pm:
Remove usage of port as the test suite is not using that anymore and it causes some probelms in buildbot
mysql-test/lib/mtr_cases.pm:
Sort slow tests first
If a test is marked as 'big_test' also mark it as 'long_test'
mysql-test/suite/federated/federated_server.result:
Added ORDER BY to get consistent results
mysql-test/suite/federated/federated_server.test:
Added ORDER BY to get consistent results
- Make equality-substitution-for-ref-access code in JOIN::optimize() treat join_tab->ref.key_copy correctly
(in the way create_ref_for_key() has filled it).
mysql-test/suite/parts/t/partition_alter4_innodb.test:
Removed duplicated big_test marker
sql-common/my_time.c:
Get rid of compiler warning about uninitialized members
Taged a couple of tests with --big-test
configure.in:
Upgrade version number
mysql-test/suite/parts/t/partition_alter1_1_2_innodb.test:
Removed duplicate big_test tag
mysql-test/suite/parts/t/partition_alter1_2_innodb.test:
Removed duplicate big_test tag
mysql-test/suite/parts/t/partition_decimal_innodb.test:
Added big_test tag as this takes forever with valgrind
mysql-test/suite/parts/t/partition_decimal_myisam.test:
Removed duplicate big_test tag
The bug in the function print_keyuse() caused crashes if
hash join could be used. It happened because the function
ignored the fact that KEYUSE structures could be created
for hash joins as well.
When compiled with SAFEMALLOC or with Windows
Debug CRT, it allocates and initializes 5GB of memory.
The effect is 20 minutes of paging and swapping on
a 4GB VM.
Still allow the test to run with optimized binaries.
Memory is not initialized in this case, malloc()
of 5GB size will not bring the whole buffer into
physical memory.
- Fix signed/unsigned warning (error -Werror) in readline
- change regex_replace pattern to account for forward or backward slashes
in partition_recover_myisam ( fixes Windows/embedded)
- Make get_constant_key_infix() take into account that there may be SEL_TREEs with
type=SEL_ARG::MAYBE_KEY, which it cannot process, because they are not real ranges
but rather indications that we might have been able to construct a range if we had
values for some other tables' fields.
(check_quick_select() already has such check)
client/readline.cc:
Initialize not used variable (to kill wrong compiler warning)
mysql-test/suite/handler/aria.result:
Updated test result
mysql-test/suite/handler/handler.inc:
Changed index to ensure rows are in a fixed order
mysql-test/suite/handler/heap.result:
Updated test result
mysql-test/suite/handler/innodb.result:
Updated test result
mysql-test/suite/handler/myisam.result:
Updated test result
plugin/handler_socket/handlersocket/Makefile.am:
Use CXX flags to compile
sql/filesort.cc:
Initialize variable that may be used
sql/log.cc:
Initialize not used variable (to kill wrong compiler warning)
sql/opt_range_mrr.cc:
Fixed cast to avoid compiler warning
storage/xtradb/fil/fil0fil.c:
Added cast to avoid compiler warning
- Fixed mysql-test-run failures on window
- Fixed compiler warnings from my last push (sorry about that)
- Fixed that maria_chk --repair --extended works again
- Fixed compiler warnings about using not unitialized data
mysql-test/mysql-test-run.pl:
Better output
mysql-test/suite/parts/inc/partition_check_drop.inc:
Use remove_files_wildcard instead of rm
mysys/safemalloc.c:
Fixed argument to printf
storage/maria/ma_cache.c:
Don't give errors when running maria_chk
storage/maria/ma_dynrec.c:
Don't give errors when running maria_chk
storage/maria/ma_rt_test.c:
Added option --datadir for where to put logs and test data
storage/maria/ma_test1.c:
Added option --datadir for where to put logs and test data
storage/maria/ma_test2.c:
Added option --datadir for where to put logs and test data
storage/maria/maria_chk.c:
If --datadir is used but --logdir is not, set --logdir from --datadir
(this reflects how --help said how things should work)
storage/maria/maria_read_log.c:
Changed short option for 'maria-log-dir-path' from -l to -h to be same as mysqld, maria_chk, ma_test1 etc..
storage/maria/unittest/ma_test_all-t:
Allow one to specify --tmpdir for where to store logs and data
storage/xtradb/buf/buf0buf.c:
Fixed compiler warnings about using not unitialized data
storage/xtradb/row/row0upd.c:
Fixed compiler warnings about using not unitialized data
storage/xtradb/srv/srv0srv.c:
Fixed compiler warnings about using not unitialized data
even in the cases when there existed range/index-merge scans that
were cheaper than the full table scan.
This was a defect/bug of the implementation of mwl #128.
Now hash join can work not only with full table scan of the joined
table, but also with full index scan, range and index-merge scans.
Accordingly, in the cases when hash join is used the column 'type'
in the EXPLAINs can contain now 'hash_ALL', 'hash_index', 'hash_range'
and 'hash_index_merge'. If hash join is coupled with a range/index_merge
scan then the columns 'key' and 'key_len' contain info not only on
the used hash index, but also on the indexes used for the scan.
- Reduced memory usage from safe_mutex.
- Fixed problem with failing tests that could not restart mysqld becasue the port was reserved
- More DBUG information
- Fixed bug where bitmap_set_prefix() wrote over buffer area.
- Initialize n_pages_flushed in xtradb which was used uninitialized.
client/mysqltest.cc:
Reduced memory usage (400K -> 80 for simple test; 400M -> 70M for some other tests)
- Smaller dynamic arrays at start
- Made 'st_connection' significantly smaller by allocation 'mysql' on demand in mysql_init() and storing require_file in a mem_root.
- Fixed that when one does --debug we get information from safemalloc in the trace
(Most of changes are changing &connect->mysql to connect->mysql
libmysql/libmysql.c:
Don't call mysql_thread_end() if my_init() was called outside of mysql_server_init()
This is needed to get information from my_end() into the DBUG log
mysql-test/README:
Fixed wrong comment
mysql-test/mysql-test-run.pl:
Reserv 20 ports / mysql-test-run thread.
(Needed as some tests uses 9 mysqld servers)
mysys/hash.c:
More DBUG information
mysys/my_bitmap.c:
Fixed bug where bitmap_set_prefix() wrote over buffer area.
mysys/safemalloc.c:
More DBUG information
mysys/thr_mutex.c:
Initialize smaller arrays be default.
sql-common/client.c:
More DBUG_PRINT
storage/xtradb/srv/srv0srv.c:
Initialize n_pages_flushed which was used uninitialized.
- Linking now with g++ instead of gcc with 'compile-dist' to solve problems with handlersocket/client
- Fixed bug in heap tables when doing handler read next-prev over last row
BUILD/compile-dist:
- Linking now with g++ instead of gcc with 'compile-dist' to solve problems with handlersocket/client
cmd-line-utils/libedit/vi.c:
Fixed compiler warning about not checking return value for write
mysql-test/r/index_intersect.result:
Updated results (missed this file in my last push)
mysql-test/suite/handler/aria.result:
Updated test results
mysql-test/suite/handler/handler.inc:
Changed test to use read next/read prev on key where there are duplicates that can come in different order depending on system
Added testing of read next-prev over last row and read prev-next around first row
mysql-test/suite/handler/heap.result:
Updated test results
mysql-test/suite/handler/init.inc:
More rows to test
mysql-test/suite/handler/innodb.result:
Updated test results
mysql-test/suite/handler/interface.result:
Updated test results
mysql-test/suite/handler/myisam.result:
Updated test results
mysql-test/t/variables-big.test:
Fixed test to not fail on windows
mysql-test/valgrind.supp:
Removed not matching fun: to get rid of valgrind warning
storage/heap/hp_rfirst.c:
Added state so that we know if we have an active position in the index.
storage/heap/hp_rkey.c:
Added state so that we know if we have an active position in the index.
storage/heap/hp_rnext.c:
Handle reading several next after finding the last row (this caused a crash before)
storage/heap/hp_rprev.c:
Handle reading several prev after finding the first row (this caused a crash before)
storage/xtradb/buf/buf0buf.c:
Fixed compiler warning about uninitialized value
- Removed references to deleted files
- If we link staticly, check for static zlib
- This should fix the problem with 'no -lz found' link error
- Fixed build failure on window (Patch from Wlad)
- Fixed build problem with federatedx when using -Werror
BUILD/Makefile.am:
Remove removed file
config/ac-macros/zlib.m4:
If we compile with --all-static, test that we have a static libz
libmysqld/CMakeLists.txt:
Fix for build error on windows
mysql-test/suite/pbxt/r/key_cache.result:
Updated result
mysql-test/suite/pbxt/t/key_cache.test:
Fixed not updated test case
sql/CMakeLists.txt:
Fix for build error on windows
storage/federatedx/Makefile.am:
Don't use CFLAGS to compile C++ programs
storage/pbxt/src/lock_xt.cc:
Fixed compiler warning about using uninitialized b2
storage/xtradb/buf/buf0buf.c:
Fixed wrong printf
storage/xtradb/srv/srv0srv.c:
Fixed assignment of different width and test with different sign/unsigned
dbug/dbug.c:
Fixed compiler warnings on windows
mysql-test/r/index_intersect_innodb.result:
Don't print number of rows as this varies
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
Update to use new State
mysql-test/suite/handler/heap.result:
Update results
mysql-test/suite/handler/heap.test:
Changed key used in test as the key 'b' will not guarantee order of the two last elements (as the key value is the same)
mysql-test/suite/pbxt/r/lock_multi.result:
Update to use new State
mysql-test/t/index_intersect.test:
Don't print number of rows as this varies
mysql-test/valgrind.supp:
Added suppression for linux
mysys/hash.c:
Fixed compiler warnings on windows
plugin/handler_socket/handlersocket/database.cpp:
Fixed compiler warnings
sql-common/client_plugin.c:
Changed variable to define to avoid compiler warnings when variable is not used
sql-common/my_time.c:
Initialize all my_time elements to avoid compiler warnings
storage/oqgraph/ha_oqgraph.cc:
Use right function (to avoid compiler warning)
storage/pbxt/src/table_xt.cc:
Initialize variables (to avoid compiler warnings)
