The pushdown condition for the sorted table in a query can be complemented
by the conditions from HAVING. This transformation is done in JOIN::exec
pretty late after the original pushdown condition have been saved in the
field pre_idx_push_select_cond for the sorted table. So this field must
be updated after the inclusion of the condition from HAVING.
Avoid doing reallocs
Prealloc some strings / provide extension allocation size to some strings
This gave a 25 % speedup in some mysql-test-run tests.
mysys/safemalloc.c:
More DBUG_PRINT
sql/net_serv.cc:
Make all mallocs() look the similar. (just-for-safety fix)
sql/protocol.cc:
Ensure that communication packet buffer is allocated.
(It's freed by stored precedures and some DLL statements)
sql/sp.cc:
Fixed valgrind warning
sql/sql_select.cc:
Set extent allocation for buffer that has a lot of append() calls.
sql/sql_show.cc:
Fixed wrong usage of string buffer. Old code worked in test suite 'just-by-chance'
sql/sql_string.cc:
Call realloc_with_extra_if_needed() in append() functions.
sql/sql_string.h:
Added 'extra_alloc' member, to specify chunck size for realloc().
extra_alloc is addaptive to catch cases where preallocation of buffers is not done properly.
Simplified free() to allow compiler to optimize things better (and to keep things consistent).
Fixed shrink() to take into account the extra memory added to the Alloced_length in realloc(). This saves us a realloc() per query.
sql/sql_test.cc:
Set extent allocation for buffer that has a lot of append() calls.
sql/table.cc:
Set extent allocation for buffer that has a lot of append() calls.
try to find a match in the join buffer. It makes sense to check for a match
only those records satisfying WHERE/ON conditions that can be pushed to
the scanned table. It allows us to discard early some join candidates.
Such pushdown conditions were built when BNL join algorithm was employed,
but for they were not built when BNLH algorithm was used.
The patch removes this shortcoming.
Merge 5.3-mwl89 into 5.3 main.
There is one remaining test failure in this merge:
innodb_mysql_lock2. All other tests have been checked to
deliver the same results/explains as 5.3-mwl89, including
the few remaining wrong results.
plans or wrong results due to the fact that JOIN_CACHE functions
ignored the possibility of interleaving materialized semijoin
tables with tables whose records were stored in join buffers.
This fixes would become mostly unnecessary if the new code of
mwl 90 was merged into 5.3 right now.
Yet the fix the code of optimize_wo_join_buffering was needed
in any case.
- Make optimize_wo_join_buffering() handle cases where position->records_read=0 (this
happens for outer joins that have constant tables inside them). The number of
0 is not correct (should be 1 because outer join will produce at least a NULL-complemented
record) but for now we just make it work with incorrect number.
The fixes for #643424 was part of the fix for #652727, that's why both
fixes are pushed together.
- The cause for #643424 was the improper use of get_partial_join_cost(),
which assumed that the 'n_tables' parameter was the upper bound for
query plan node indexes.
Fixed by generalizing get_partial_join_cost() as a method that computes
the cost of any partial join.
- The cause of #652727 was that JOIN::choose_subquery_plan() incorrectly
deleted the contents of the old keyuse array in the cases when an injected
plan would not provide more key accesses, and reoptimization was not actually
performed.
by a function and column
The bugreport reveals two different bugs about grouping
on a function:
1) grouping by the TIME_TO_SEC function result caused
a server crash or wrong results and
2) grouping by the function returning a blob caused
an unexpected "Duplicate entry" error and wrong
result.
Details for the 1st bug:
TIME_TO_SEC() returns NULL if its argument is invalid (empty
string for example). Thus its nullability depends not only
on the nullability of its arguments but also on their values.
Fixed by (overoptimistically) setting TIME_TO_SEC() to be
nullable despite the nullability of its arguments.
Details for the 2nd bug:
The server is unable to create indices on blobs without
explicit blob key part length. However, this fact was
ignored for blob function result fields of GROUP BY
intermediate tables.
Fixed by disabling GROUP BY index creation for blob
function result fields like regular blob fields.
mysql-test/r/func_time.result:
Test case for bug #52160.
mysql-test/r/type_blob.result:
Test case for bug #52160.
mysql-test/t/func_time.test:
Test case for bug #52160.
mysql-test/t/type_blob.test:
Test case for bug #52160.
sql/item_timefunc.h:
Bug #52160: crash and inconsistent results when grouping
by a function and column
TIME_TO_SEC() returns NULL if its argument is invalid (empty
string for example). Thus its nullability depends not only
Fixed by (overoptimistically) setting TIME_TO_SEC() to be
nullable despite the nullability of its arguments.
sql/sql_select.cc:
Bug #52160: crash and inconsistent results when grouping
by a function and column
The server is unable to create indices on blobs without
explicit blob key part length. However, this fact was
ignored for blob function result fields of GROUP BY
intermediate tables.
Fixed by disabling GROUP BY index creation for blob
function result fields like regular blob fields.
Prohibited to use hash join algorithm BNLH if join attributes
need non-binary collations. It has to be done because BNLH does
not support join for such attributes yet.
Later this limitations will be lifted.
Changed default collations for the schemes of some test cases
to preserve the old execution plans.
Lines below which were added in the patch for Bug#56814 cause this crash:
+ if (table->table)
+ table->table->maybe_null= FALSE;
Consider following test case:
--
CREATE TABLE t1(f1 INT NOT NULL);
INSERT INTO t1 VALUES (16777214),(0);
SELECT COUNT(*) FROM t1 LEFT JOIN t1 t2
ON 1 WHERE t2.f1 > 1 GROUP BY t2.f1;
DROP TABLE t1;
--
We set TABLE::maybe_null to FALSE for t2 table
and in create_tmp_field() we create appropriate tmp table field
using create_tmp_field_from_item() function instead of
create_tmp_field_from_field. As a result we have
LONGLONG field. As we have GROUP BY clause we calculate
group buffer length, see calc_group_buffer().
Item from group list which is used for calculation
refer to the field from real tables and have LONG type.
So group buffer length become insufficient for storing of
LONGLONG value. It leads to overwriting of wrong memory
area in do_field_int() function which is called from
end_update().
After some investigation I found out that
create_tmp_field_from_item() is used only for OLAP
grouping and can not be used for common grouping
as it could be an incompatibility between tmp
table fields and group buffer length.
We can not remove create_tmp_field_from_item() call from
create_tmp_field as OLAP needs it and we can not use this
function for common grouping. So we should remove setting
TABLE::maybe_null to FALSE from simplify_joins().
In this case we'll get wrong behaviour of
list_contains_unique_index() back. To fix it we
could use Field::real_maybe_null() check instead of
Field::maybe_null() and add addition check of
TABLE_LIST::outer_join.
mysql-test/r/group_by.result:
test case
mysql-test/r/join_outer.result:
test case
mysql-test/t/group_by.test:
test case
mysql-test/t/join_outer.test:
test case
sql/sql_select.cc:
--remove wrong code
--use Field::real_maybe_null() check instead of
Field::maybe_null() and add addition check of
TABLE_LIST::outer_join
When join buffers are employed no index scan for the first
table with grouping columns can be used.
mysql-test/r/join_cache.result:
Added a test case for bug #664508.
Sorted results for some other test cases.
mysql-test/t/join_cache.test:
Added a test case for bug #664508.
Sorted results for some other test cases.
- A prerequisite cleanup patch for making KILL reliable.
The test case main.kill did not work reliably.
The following problems have been identified:
1. A kill signal could go lost if it came in, short before a
thread went reading on the client connection.
2. A kill signal could go lost if it came in, short before a
thread went waiting on a condition variable.
These problems have been solved as follows. Please see also
added code comments for more details.
1. There is no safe way to detect, when a thread enters the
blocking state of a read(2) or recv(2) system call, where it
can be interrupted by a signal. Hence it is not possible to
wait for the right moment to send a kill signal. It has been
decided, not to fix it in the code. Instead, the test case
repeats the KILL statement until the connection terminates.
2. Before waiting on a condition variable, we register it
together with a synchronizating mutex in THD::mysys_var. After
this, we need to test THD::killed again. At some places we did
only test it in a loop condition before the registration. When
THD::killed had been set between this test and the registration,
we entered waiting without noticing the killed flag. Additional
checks ahve been introduced where required.
In addition to the above, a re-write of the main.kill test
case has been done. All sleeps have been replaced by Debug
Sync Facility synchronization. A couple of sync points have
been added to the server code.
To avoid further problems, if the test case fails in spite of
the fixes, the test case has been added to the "experimental"
list for now.
- Most of the work on this patch is authored by Ingo Struewing
mysql-test/t/kill.test:
Re-wrote test case to use Debug Sync points instead of sleeps
sql/event_queue.cc:
Fixed kill detection in Event_queue::cond_wait() by adding a check
after enter_cond().
sql/lock.cc:
Moved Debug Sync points behind enter_cond().
Fixed comments.
sql/slave.cc:
Fixed kill detection in start_slave_thread() by adding a check
after enter_cond().
sql/sql_class.cc:
Swapped order of kill and close in THD::awake().
Added comments.
sql/sql_class.h:
Added a comment to THD::killed.
sql/sql_parse.cc:
Added a sync point in do_command().
sql/sql_select.cc:
Added a sync point in JOIN::optimize().
- Added more tests to the MWL#89 specific test, and made the test more modular.
- Updated test files.
- Fixed a memory leak.
- More comments.
mysql-test/r/subselect_mat.result:
- Updated the test file to reflect the new optimizer switches related to
materialized subquery execution.
- Added one extra test to test all cases that expose BUG#40037 (this is an old bug from 5.x).
- Updated the test result with correct results that expose BUG#40037.
mysql-test/t/subselect_mat.test:
- Updated the test file to reflect the new optimizer switches related to
materialized subquery execution.
- Added one extra test to test all cases that expose BUG#40037 (this is an old bug from 5.x).
- Updated the test result with correct results that expose BUG#40037.
sql/sql_select.cc:
Fixed a memory leak reported by Valgrind.
BUG#26447 prefer a clustered key for an index scan, as secondary index is always slower
... which was fixed to cause
BUG#35850 queries take 50% longer
... and was reverted
and
BUG#39653 prefer a secondary index for an index scan, as clustered key is always slower
... which was fixed to cause
BUG#55656 mysqldump takes six days instead of half an hour
... and was amended with a special case workaround
sql/opt_range.cc:
move get_index_only_read_time() into the handler class
sql/sql_select.cc:
use cost not an index length when choosing a cheaper index
create_sort_index() function overwrites original JOIN_TAB::type field.
At re-execution of subquery overwritten JOIN_TAB::type(JT_ALL) is
used instead of JT_FT. It misleads test_if_skip_sort_order() and
the function tries to find suitable key for the order that should
not be allowed for FULLTEXT(JT_FT) table.
The fix is to restore JOIN_TAB strucures for subselect on re-execution
for EXPLAIN.
Additional fix:
Update TABLE::maybe_null field which
affects list_contains_unique_index() behaviour as it
could have the value(maybe_null==TRUE) based on the
assumption that this join is outer
(see setup_table_map() func).
mysql-test/r/explain.result:
test case
mysql-test/t/explain.test:
test case
sql/item_subselect.cc:
Make subquery uncacheable in case of EXPLAIN. It allows to keep
original JOIN_TAB::type(see JOIN::save_join_tab) and restore it
on re-execution.
sql/sql_select.cc:
-restore JOIN_TAB strucures for subselect on re-execution for EXPLAIN
-Update TABLE::maybe_null field as it could have
the value(maybe_null==TRUE) based on the assumption
that this join is outer(see setup_table_map() func).
This change is not related to the crash problem but
affects EXPLAIN results in the test case.
Subquery executes twice, at top level JOIN::optimize and ::execute stages.
At first execution create_sort_index() function is called and
FT_SELECT object is created and destroyed. HANDLER::ft_handler is cleaned up
in the object destructor and at second execution FT_SELECT::get_next() method
returns error.
The fix is to reinit HANDLER::ft_handler field before re-execution of subquery.
mysql-test/r/fulltext.result:
test case
mysql-test/t/fulltext.test:
test case
sql/item_func.cc:
reinit ft_handler before re-execution of subquery
sql/item_func.h:
Fixed method name
sql/sql_select.cc:
reinit ft_handler before re-execution of subquery
- When building multiple-equalities for HAVING, don't set JOIN::cond_equal, set
join_having_equal instead. Setting JOIN::cond_equal based on HAVING makes
equality propagation data self-inconsistent
- Changed the default optimizer switches to provide 5.1/5.2 compatible behavior
- Added a regression test file to test consistently all cases covered by MWL#89
- Added/corrected/improved comments.
Employed the same kind of optimization as in the fix for the cases
when join buffer is used.
The optimization performs early evaluation of the conditions from
on expression with table references to only outer tables of
an outer join.
Improved handling of EXPLAIN statements for subqueries.
This patch specifically solves the problem when EXPLAIN reports:
"const row not found"
instead of
"no matching row in const table".
Added a possibility not to factor out the condition pushed to
the access index out of the condition pushed to a joined table.
This is useful for the condition pushed to the index when a hashed
join buffer for BKA is employed. In this case the index condition
may be false for some, but for all records with the same key.
So the condition must be checked not only after index lookup,
but after fetching row data as well, and it makes sense not to
factor out the condition from the condition checked after reading
row data,
The bug happened because the condition pushed to an index always
was factor out from the condition pushed to the accessed table.
Phase 3: Implementation of re-optimization of subqueries with injected predicates
and cost comparison between Materialization and IN->EXISTS strategies.
The commit contains the following known problems:
- The implementation of EXPLAIN has not been re-engineered to reflect the
changes in subquery optimization. EXPLAIN for subqueries is called during
the execute phase, which results in different code paths during JOIN::optimize
and thus in differing EXPLAIN messages for constant/system tables.
- There are some valgrind warnings that need investigation
- Several EXPLAINs with minor differences need to be reconsidered after fixing
the EXPLAIN problem above.
This patch also adds one extra optimizer_switch: 'in_to_exists' for complete
manual control of the subquery execution strategies.
