when semijoin=on
When setting the aggregate function as having no rows to report
the function no_rows_in_result() was calling Item_sum::reset().
However this function in addition to cleaning up the aggregate
value by calling aggregator_clear() was also adding the current
value to the aggregate value by calling aggregator_add().
Fixed by making no_rows_in_result() to call aggregator_clear()
directly.
Renamed Item_sum::reset to Item_sum::reset_and_add() to
and added a comment to avoid misinterpretation of what the
function does.
when semijoin=on
When setting the aggregate function as having no rows to report
the function no_rows_in_result() was calling Item_sum::reset().
However this function in addition to cleaning up the aggregate
value by calling aggregator_clear() was also adding the current
value to the aggregate value by calling aggregator_add().
Fixed by making no_rows_in_result() to call aggregator_clear()
directly.
Renamed Item_sum::reset to Item_sum::reset_and_add() to
and added a comment to avoid misinterpretation of what the
function does.
- Fixed problem with oqgraph and 'make dist'
Note that after this merge we have a problem show in join_outer where we examine too many rows in one specific case (related to BUG#57024).
This will be fixed when mwl#128 is merged into 5.3.
Analysis:
The fix for LP BUG#680846 avoids evaluation of constant expressions
with subqueries in the GROUP/ORDER clauses in the procedure
remove_const(). The purpose of remove_const is to remove constant
expressions in the GROUP/ORDER clauses.
In order delay until execution the evaluation of such subqueries,
they were not removed in the GROUP/ORDER clause. As a result temp
table creation during execution attempted to create a column in the
temp table for each constant GROUP/ORDER expression. However, the
logic in create_tmp_table is to not create temp table columns for
constant items. The crash was due to a group Item without a
corresponding column in the temp table for GROUP BY.
Solution:
The patch adds back removal of constant expressions with subqueries.
In order for such expressions to be evaluated, so that the server can
ensure that such subquries return 1 row, the evaluation of these
expressions is delayed until execution.
Changed rows_read and rows_sent status variables to be longlong (to avoid compiler warnings)
sql/item_func.cc:
Fixed wrong usage of alias
sql/item_subselect.cc:
Changed buffer size to ulonglong to be able detect buffers bigger than size_t
sql/item_subselect.h:
Changed buffer size to ulonglong to be able detect buffers bigger than size_t
sql/multi_range_read.cc:
Fixed compiler warning by using correct type for function argument
sql/mysqld.cc:
Changed rows_read and rows_sent status variables to be longlong
sql/opt_subselect.h:
Fixed compiler warning by using correct type for function argument
sql/sql_class.cc:
Changed rows_read and rows_sent status variables to be longlong
sql/sql_class.h:
Changed rows_read and rows_sent status variables to be longlong
Changed max_nulls_in_row to uint as this is number of columns in row.
This fixed some compiler warnings.
sql/sql_select.cc:
Added casts to avoid compiler warnings
storage/heap/ha_heap.cc:
Initilize different types separate
storage/oqgraph/ha_oqgraph.cc:
Fixed argument to store(longlong) to avoid compiler warnings
- Changed Cached_item_field not copy data for fields with NULL value
- In key_copy() and key_restore() don't copy data for fields with NULL value
Fixed code to avoid valgrind warnings
- Use c_ptr_safe instead of c_ptr()
Removed "QQ" from comments (QQ was ment to be used for internal comments that should be removed before pushing)
Fixed wrong alias used (from previous patch)
sql/event_db_repository.cc:
Update testing if event table is valid (to avoid valgrind errors)
sql/ha_partition.cc:
m_ordered_scan_ongoing was not initialized
Reset null bits in record to avoid valgrind errors
sql/handler.h:
Added flag if storage engine will write row verbatim and the row contains varchar or null fields
(in which case we must clear the row to avoid valgrind warnings)
sql/item_buff.cc:
Changed Cached_item_field not copy data for fields with NULL value
(Optimization and avoids valgrind warnings)
sql/item_func.cc:
c_ptr() -> c_ptr_safe()
sql/key.cc:
In key_copy() and key_restore() don't copy data for fields with NULL value
sql/opt_range.cc:
c_ptr() -> c_ptr_safe()
sql/sql_base.cc:
Added TRASH() to table->record[0] to find out if we access not initialzed data.
Initialize null_bytes to:
- Get consistent tests
- Ensure we don't get valgrind warnings for null fields (as we may only update a couple of bits in a byte)
sql/sql_class.cc:
Removed "QQ" from comments
sql/sql_insert.cc:
Initialize row to default values if we are using valgrind and row will be copied verbatim to disk in storage engine.
sql/sql_load.cc:
QQ -> TODO
sql/sql_parse.cc:
Removed old not used code marked QQ and withing "#ifdef REMOVED"
sql/sql_select.cc:
QQ -> TODO
Initialize some variables that was used uninitialized
Added DBUG_ASSERT() to find out if thd was not properly initialized for sub queries
sql/sql_test.cc:
Fixed format for printing to DBUG file
Fixed wrong alias used (from previous patch)
sql/sql_trigger.h:
QQ -> TODO
sql/table.cc:
QQ -> TODO
storage/maria/ha_maria.cc:
Mark table with HA_RECORD_MUST_BE_CLEAN_ON_WRITE, if row is written verbatim to disk and contains varchar or null fields.
storage/maria/ma_open.c:
Added flags if table has varchar or null fields
storage/maria/maria_def.h:
Added flags if table has varchar or null fields
storage/myisam/ha_myisam.cc:
Mark table with HA_RECORD_MUST_BE_CLEAN_ON_WRITE, if row is written verbatim to disk and contains varchar or null fields.
storage/myisam/mi_open.c:
Fixed memory overrun bug when using fulltext keys
storage/xtradb/row/row0sel.c:
Removed initialization of null bits. (not needed anymore)
Analysis:
JOIN::optimize performs constant optimization of GROUP by clauses
by calling remove_const():
group_list= remove_const(this, (old_group_list= group_list), conds,
rollup.state == ROLLUP::STATE_NONE,
&simple_group);
If it turns out that a GROUP clause references a field that is
computed by a single-row subquery, then the said optimization
performs premature execution of the subquery referenced by the
group clause.
Solution:
Block the evaluation of subqueries similarly to the approach
for the WHERE and JOIN..ON clauses.
Fix MySQL BUG#52344 - Subquery materialization: Assertion if subquery in on-clause of outer join
Original fix and comments from Oysten, adjusted for the different
subquery optimization in MariaDB.
"
Problem: If tables of an outer join are constant tables,
the associated on-clause will be evaluated in the optimization
phase. If the on-clause contains a query that is to be
executed with subquery materialization, this will not work
since the infrastructure for such execution is not yet set up.
Solution: Do not evaluate on-clause in optimization phase if
is_expensive() returns true for this clause. This is how the
problem is currently avoided for where-clauses. This works
because, Item_in_subselect::is_expensive_processor returns true
if query is to be executed with subquery materialization.
"
In addition, after MWL#89, in MariaDB if the IN-EXISTS strategy
is chosen, the in-to-exists predicates are insterted after
join_read_const_table() is called, resulting in evaluation of
the subquery without the in-to-exists predicates.
Open issues:
- A better fix for #57688; Igor is working on this
- Test failure in index_merge_innodb.test ; Igor promised to look at this
- Some Innodb tests fails (need to merge with latest xtradb) ; Kristian promised to look at this.
- Failing tests: innodb_plugin.innodb_bug56143 innodb_plugin.innodb_bug56632 innodb_plugin.innodb_bug56680 innodb_plugin.innodb_bug57255
- Werror is disabled; Should be enabled after merge with xtradb.
- Changed TABLE->alias to String to get fewer reallocs when alias are used.
- Preallocate some buffers
Changed some String->c_ptr() -> String->ptr() when \0 is not needed.
Fixed wrong usage of String->ptr() when we need a \0 terminated string.
Use my_strtod() instead of my_atof() to avoid having to add \0 to string.
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
zr
sql/event_db_repository.cc:
Update usage of TABLE->alias
sql/event_scheduler.cc:
c_ptr() -> c_ptr_safe()
sql/events.cc:
c_ptr() -> ptr() as \0 was not needed
sql/field.cc:
Update usage of TABLE->alias
sql/field.h:
Update usage of TABLE->alias
sql/ha_partition.cc:
Update usage of TABLE->alias
sql/handler.cc:
Update usage of TABLE->alias
Fixed wrong usage of str.ptr()
sql/item.cc:
Fixed error where code wrongly assumed string was \0 terminated.
sql/item_func.cc:
c_ptr() -> c_ptr_safe()
Update usage of TABLE->alias
sql/item_sum.h:
Use my_strtod() instead of my_atof() to avoid having to add \0 to string
sql/lock.cc:
Update usage of TABLE->alias
sql/log.cc:
c_ptr() -> ptr() as \0 was not needed
sql/log_event.cc:
c_ptr_quick() -> ptr() as \0 was not needed
sql/opt_range.cc:
ptr() -> c_ptr() as \0 is needed
sql/opt_subselect.cc:
Update usage of TABLE->alias
sql/opt_table_elimination.cc:
Update usage of TABLE->alias
sql/set_var.cc:
ptr() -> c_ptr() as \0 is needed
c_ptr() -> c_ptr_safe()
sql/sp.cc:
c_ptr() -> ptr() as \0 was not needed
sql/sp_rcontext.cc:
Update usage of TABLE->alias
sql/sql_base.cc:
Preallocate buffers
Update usage of TABLE->alias
sql/sql_class.cc:
Fix arguments to sprintf() to work even if string is not \0 terminated
sql/sql_insert.cc:
Update usage of TABLE->alias
c_ptr() -> ptr() as \0 was not needed
sql/sql_load.cc:
Preallocate buffers
Trivial optimizations
sql/sql_parse.cc:
Trivial optimization
sql/sql_plugin.cc:
c_ptr() -> ptr() as \0 was not needed
sql/sql_select.cc:
Update usage of TABLE->alias
sql/sql_show.cc:
Update usage of TABLE->alias
sql/sql_string.h:
Added move() function to move allocated memory from one object to another.
sql/sql_table.cc:
Update usage of TABLE->alias
c_ptr() -> c_ptr_safe()
sql/sql_test.cc:
ptr() -> c_ptr_safe()
sql/sql_trigger.cc:
Update usage of TABLE->alias
c_ptr() -> c_ptr_safe()
sql/sql_update.cc:
Update usage of TABLE->alias
sql/sql_view.cc:
ptr() -> c_ptr_safe()
sql/sql_yacc.yy:
ptr() -> c_ptr()
sql/table.cc:
Update usage of TABLE->alias
sql/table.h:
Changed TABLE->alias to String to get fewer reallocs when alias are used.
storage/federatedx/ha_federatedx.cc:
Use c_ptr_safe() to ensure strings are \0 terminated.
storage/maria/ha_maria.cc:
Update usage of TABLE->alias
storage/myisam/ha_myisam.cc:
Update usage of TABLE->alias
storage/xtradb/row/row0sel.c:
Ensure that null bits in record are properly reset.
(Old code didn't work as row_search_for_mysql() can be called twice while reading fields from one row.
> revision-id: alexey.kopytov@sun.com-20100824103548-ikm79qlfrvggyj9h
> parent: sunny.bains@oracle.com-20100816001222-xqc447tr6jwh8c53
> committer: Alexey Kopytov <Alexey.Kopytov@Sun.com>
> branch nick: 5.1-security
> timestamp: Tue 2010-08-24 14:35:48 +0400
> message:
> Bug #55568: user variable assignments crash server when used
> within query
>
> The server could crash after materializing a derived table
> which requires a temporary table for grouping.
>
> When destroying the temporary table used to execute a query for
> a derived table, JOIN::destroy() did not clean up Item_fields
> pointing to fields in the temporary table. This led to
> dereferencing a dangling pointer when printing out the items
> tree later in the outer SELECT.
>
> The solution is an addendum to the patch for bug37362: in
> addition to cleaning up items in tmp_all_fields3, do the same
> for items in tmp_all_fields1, since now we have an example
> where this is necessary.
sql/field.cc:
Make sure field->table_name is not set to NULL in
Field::make_field() to avoid assertion failure in
Item_field::make_field() after cleaning up items
(the assertion fired in udf.test when running
the test suite with the patch applied).
sql/sql_select.cc:
In addition to cleaning up items in tmp_all_fields3, do the
same for items in tmp_all_fields1.
Introduce a new helper function to avoid code duplication.
sql/sql_select.h:
Introduce a new helper function to avoid code duplication in
JOIN::destroy().
> revision-id: alexey.kopytov@sun.com-20100824103548-ikm79qlfrvggyj9h
> parent: sunny.bains@oracle.com-20100816001222-xqc447tr6jwh8c53
> committer: Alexey Kopytov <Alexey.Kopytov@Sun.com>
> branch nick: 5.1-security
> timestamp: Tue 2010-08-24 14:35:48 +0400
> message:
> Bug #55568: user variable assignments crash server when used
> within query
>
> The server could crash after materializing a derived table
> which requires a temporary table for grouping.
>
> When destroying the temporary table used to execute a query for
> a derived table, JOIN::destroy() did not clean up Item_fields
> pointing to fields in the temporary table. This led to
> dereferencing a dangling pointer when printing out the items
> tree later in the outer SELECT.
>
> The solution is an addendum to the patch for bug37362: in
> addition to cleaning up items in tmp_all_fields3, do the same
> for items in tmp_all_fields1, since now we have an example
> where this is necessary.
Analysis:
This another instance of the problem fixed in LP BUG#675981 -
evaluation of subqueries during EXPLAIN when the subquery plan
is incomplete because JOIN::optimize() generally doesn't create
complete execution plans for EXPLAIN statements.
In this case the call path is:
mysql_explain_union -> outer_join.exec -> outer_join.init_execution ->
create_sort_index -> filesort -> find_all_keys ->
SQL_SELECT::skip_record -> outer_where_clause.val_int -> ...
-> subselect_join.exec -> ... -> sub_select_cache
When calling sub_select_cache JOIN_TAB::cache is NULL because the cache
objects are not created for EXPLAIN statements.
Solution:
Delay the call to init_execution() after all EXPLAIN related processing
is completed. Thus init_execution() is not called at all during EXPLAIN.
Cause:
The optimize() phase for the subquery selected to use join buffering via setting
JOIN_TAB::next_select= sub_select_cache in make_join_readinfo, however, the call
to check_join_cache_usage() from make_join_readinfo didn't create the corresponding
JOIN_CACHE_BNL object because of the condition:
if ((options & SELECT_DESCRIBE) ||
(((tab->cache= new JOIN_CACHE_BNL(join, tab, prev_cache))) &&
!tab->cache->init()))
Since EXPLAIN for subqueries runs regular execution, the constant predicates that
were delayed to be evaluated at the exec() phase, were evaluated during EXPLAIN.
As a result the outer JOIN::exec called JOIN::exec for the subquery, while the
subquery execution plan was no properly created, which resulted in an failed ASSERT.
Fix:
The patch blocks evaluation of constant expensive conditions during EXPLAIN. Notice
that these conditions are "constant" with respect to the outer query, thus in
general they could be arbitrarily expensive, which may result in very slow EXPLAINs.
and related small fixes.
mysql-test/t/user_var.test:
test for bug
sql/field_conv.cc:
From the C standard, memcpy() has undefined behaviour if to->ptr==from->ptr
sql/item_func.cc:
In the case of BUG#56138, entry->value==ptr in which case memcpy()
has undefined results per the C standard.
sql/sql_select.cc:
Work around a bug in old gcc
The condition that was supposed to check whether a join table
is an inner table of a nested outer join or semi-join was not
quite correct in the code of the function check_join_cache_usage.
That's why some queries with nested outer joins triggered
an assertion failure.
Encapsulated this condition in the new method called
JOIN_TAB::is_nested_inner and provided a proper code for it.
Also corrected a bug in the code of check_join_cache_usage()
that caused a downgrade of not first join buffers of the
level 5 and 7 to level 4 and 6 correspondingly.
The cause for the bug was two-fold:
1. Incorrect detection of whether a table is the first one in a query plan -
"used_table & 1" actually checks if used_table is table with number "1".
2. Missing logic to delay the evaluation of (expensive) constant conditions
during the execution phase.
The fix adds/changes:
The patch:
- removes incorrect treatment of expensive predicates from make_cond_for_table,
and lets the caller decide when to evaluate expensive predicates.
- saves expensive constant conditions in JOIN::exec_const_cond,
which is evaluated once in the beginning of JOIN::exec.
When pushing the condition for a table in the function
JOIN_TAB::make_scan_filter the optimizer must not push
conditions from WHERE if the table is some inner table
of an outer join..
The condition over outer tables extracted from the on expression
for a outer join must be ANDed to the condition pushed to the
first inner table of this outer join only.
Nested outer joins cannot use flat join buffers. So if join_cache_level
is set to 1 then any join algorithm employing join buffers cannot be used
for nested outer joins.
A non-incremental join buffer cannot be used for inner tables of nested
outer joins. That's why when join_cache_level is set to 7 it must
be downgraded to level 6 for the inner tables of nested outer joins.
For the same reason with join_cache_level set to 3 no join buffer is
used for the inner tables of outer joins (we could downgrade it to
level 2, but this level does not support ref access).
Miscalculation of the minimum possible buffer size could trigger
an assert in JOIN_CACHE_HASHED::put_record when if join_buffer_size
was set to the values that is less than the length of one record to
stored in the join buffer.
It happened due to the following mistakes:
- underestimation of space needed for a key in the hash table
(we have to take into account that hash table can have more
buckets than the expected number of records).
- the value of maximum total length of all records stored in
the join buffer was not saved in the field max_used_fieldlength
by the function calc_used_field_length.
