Original revid: alexey.kopytov@sun.com-20100723115254-jjwmhq97b9wl932l
> Bug #54476: crash when group_concat and 'with rollup' in
> prepared statements
>
> Using GROUP_CONCAT() together with the WITH ROLLUP modifier
> could crash the server.
>
> The reason was a combination of several facts:
>
> 1. The Item_func_group_concat class stores pointers to ORDER
> objects representing the columns in the ORDER BY clause of
> GROUP_CONCAT().
>
> 2. find_order_in_list() called from
> Item_func_group_concat::setup() modifies the ORDER objects so
> that their 'item' member points to the arguments list
> allocated in the Item_func_group_concat constructor.
>
> 3. In some cases (e.g. in JOIN::rollup_make_fields) a copy of
> the original Item_func_group_concat object could be created by
> using the Item_func_group_concat::Item_func_group_concat(THD
> *thd, Item_func_group_concat *item) copy constructor. The
> latter essentially creates a shallow copy of the source
> object. Memory for the arguments array is allocated on
> thd->mem_root, but the pointers for arguments and ORDER are
> copied verbatim.
>
> What happens in the test case is that when executing the query
> for the first time, after a copy of the original
> Item_func_group_concat object has been created by
> JOIN::rollup_make_fields(), find_order_in_list() is called for
> this new object. It then resolves ORDER BY by modifying the
> ORDER objects so that they point to elements of the arguments
> array which is local to the cloned object. When thd->mem_root
> is freed upon completing the execution, pointers in the ORDER
> objects become invalid. Those ORDER objects, however, are also
> shared with the original Item_func_group_concat object which is
> preserved between executions of a prepared statement. So the
> first call to find_order_in_list() for the original object on
> the second execution tries to dereference an invalid pointer.
>
> The solution is to create copies of the ORDER objects when
> copying Item_func_group_concat to not leave any stale pointers
> in other instances with different lifecycles.
mysql-test/r/func_gconcat.result:
Test case for bug #54476.
mysql-test/t/func_gconcat.test:
Test case for bug #54476.
sql/item_sum.cc:
Copy the ORDER objects pointed to by the elements of the
'order' array in the copy constructor of
Item_func_group_concat.
sql/table.h:
Removed the unused 'item_copy' member of the ORDER class.
Improved error handling such that queries against Information_Schema.Tables won't
fail if a Federated table is unable to connect to remote host.
sql/sql_show.cc:
If Handler::Info() fails, save error text in TABLE COMMENTS column, clear error.
The problem is in the Item_func_isnull::update_used_tables() function,
bracket is at the wrong place. Because of that isnull item erroneously
is treated as const item. The fix is to set brackets in the right place.
mysql-test/r/func_isnull.result:
test case
mysql-test/t/func_isnull.test:
test case
sql/item_cmpfunc.h:
set brackets in the right place.
Some of the server implementations don't support dates later
than 2038 due to the internal time type being 32 bit.
Added checks so that the server will refuse dates that cannot
be handled by either throwing an error when setting date at
runtime or by refusing to start or shutting down the server if
the system date cannot be stored in my_time_t.
greedy_search optimizer_search_depth=0
The algorithm inside restore_prev_nj_state failed to
properly update the counters within the NESTED_JOIN
tree. The counter was decremented each time a table in the
node was removed from the QEP, the correct thing to do being
only to decrement it when the last table in the child node
was removed from the plan. This lead to node counters
getting negative values and the plan thus appeared
impossible. An assertion caught this.
Fixed by not recursing up the tree unless the last table in
the join nest node is removed from the plan
The problem was in an incorrect debug assertion. The expression
used in the failing assertion states that when finding
references matching ORDER BY expressions, there can be only one
reference to a single table. But that does not make any sense,
all test cases for this bug are valid examples with multiple
identical WHERE expressions referencing the same table which
are also present in the ORDER BY list.
Fixed by removing the failing assertion. We also have to take
care of the 'found' counter so that we count multiple
references only once. We rely on this fact later in
eq_ref_table().
mysql-test/r/join.result:
Added a test case for bug #50335.
mysql-test/t/join.test:
Added a test case for bug #50335.
sql/sql_select.cc:
Removing the assertion in eq_ref_table() as it does not make
any sense. We also have to take care of the 'found' counter so
that we count multiple references only once. We rely on this
fact later in eq_ref_table().
function on windows
When making sure that the directory path ends up with a
slash/backslash we need to check for the correct length of
the buffer and trim at the appropriate location so we don't
write past the end of the buffer.
The crash is the result of an attempt made by JOIN::optimize to evaluate
the WHERE condition when no records have been actually read.
The fix is to remove erroneous 'outer_join' variable check.
mysql-test/r/join.result:
test result
mysql-test/t/join.test:
test case
sql/sql_select.cc:
removed erroneous 'outer_join' variable check.
The crash happens because greedy_serach
can not determine best plan due to
wrong inner table dependences. These
dependences affects join table sorting
which performs before greedy_search starting.
In our case table which has real 'no dependences'
should be put on top of the list but it does not
happen as inner tables have no dependences as well.
The fix is to exclude RAND_TABLE_BIT mask from
condition which checks if table dependences
should be updated.
mysql-test/r/join.result:
test result
mysql-test/t/join.test:
test case
sql/sql_select.cc:
RAND_TABLE_BIT mask should not be counted as it
prevents update of inner table dependences.
For example it might happen if RAND() function
is used in JOIN ON clause.
SET autocommit=1 while XA transaction is active may
cause various side effects, including memory corruption
and server crash.
The problem is that SET autocommit=1 and further queries
attempt to commit local transaction, whereas XA transaction
is still active.
As local and XA transactions are mutually exclusive, this
patch forbids enabling autocommit mode while XA transaction
is active.
mysql-test/r/xa.result:
A test case for BUG#51342.
mysql-test/t/xa.test:
A test case for BUG#51342.
sql/set_var.cc:
Forbid enabling autocommit mode while XA transaction is
active.
Spatial indexes were not checking for out-of-record condition in
the handler next command when the previous command didn't found
rows.
Fixed by making the rtree index to check for end of rows condition
before re-using the key from the previous search.
Fixed another crash if the tree has changed since the last search.
Added a test case for the other error.
If an outer query is broken, a subquery might not even get set up.
EXPLAIN EXTENDED did not expect this and merrily tried to de-ref all
of the half-setup info.
We now catch this case and print as much as we have, as it doesn't cost us
anything (doesn't make regular execution slower).
backport from 5.1
mysql-test/r/explain.result:
Show that EXPLAIN EXTENDED with subquery and illegal out query doesn't crash.
Show also that SHOW WARNINGS will render an additional Note in the hope of
being, well, helpful.
mysql-test/t/explain.test:
If we have only half a query for EXPLAIN EXTENDED to print (i.e.,
incomplete subquery info as outer query is illegal), we should
provide the user with as much info as we easily can if they ask
for it. What we should not do is crash when they come asking for
help, that violates etiquette in some countries.
sql/item_subselect.cc:
If the sub-query's actually set up, print it. Otherwise, elide.
When EXPLAIN EXTENDED tries to print column names, it checks whether the
referenced table is CONST (in which case, the column's value rather than
its name will be printed). If no proper table is reference (i.e. because
a derived table was used that has since gone out of scope), this will fail
spectacularly.
This ports an equivalent of the fix for Bug 43354.
mysql-test/r/func_gconcat.result:
Show that EXPLAIN EXTENDED on a GROUP_CONCAT() on a derived table
no longer crashes the server.
mysql-test/t/func_gconcat.test:
Show that EXPLAIN EXTENDED on a GROUP_CONCAT() on a derived table
no longer crashes the server.
sql/item_sum.cc:
Do not de-ref what cannot be, that is, temp-tables that have gone away.
This is of questionable utility anyway, since our deref has the sole
purpose of checking whether the table is const (in which case, we'll
substitute the column with its value in EXPLAIN EXTENDED - that is all).
column is used for ORDER BY
Problem: filesort isn't meant for null length sort data
(e.g. char(0)), that leads to a server crash.
Fix: disregard sort order if sort data record length is 0 (nothing
to sort).
mysql-test/r/select.result:
Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL
column is used for ORDER BY
- test result.
mysql-test/t/select.test:
Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL
column is used for ORDER BY
- test case.
sql/filesort.cc:
Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL
column is used for ORDER BY
- assert added as filesort cannot handle null length sort data.
sql/sql_select.cc:
Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL
column is used for ORDER BY
- don't sort null length data e.g. in case of ORDER BY CHAR(0).
In statement-based or mixed-mode replication, use DROP TEMPORARY TABLE
to drop multiple tables causes different errors on master and slave,
when one or more of these tables do not exist. Because when executed
on slave, it would automatically add IF EXISTS to the query to ignore
all ER_BAD_TABLE_ERROR errors.
To fix the problem, do not add IF EXISTS when executing DROP TEMPORARY
TABLE on the slave, and clear the ER_BAD_TABLE_ERROR error after
execution if the query does not expect any errors.
mysql-test/r/rpl_drop_temp.result:
Updated for the patch of bug#49137.
mysql-test/t/rpl_drop_temp.test:
Added the test file to verify if DROP MULTI TEMPORARY TABLE
will cause different errors on master and slave, when one or
more of these tables do not exist.
sql/log_event.cc:
Added code to handle above cases which are
removed from sql_parse.cc
sql/sql_parse.cc:
Remove the code to issue the 'Unknown table' error,
if the temporary table does not exist when dropping
it on slave. The above cases decribed in comments
will be handled later in log_event.cc.
on re-execution of prepared statement
Problem: some (see eq_ref_table()) ORDER BY/GROUP BY optimization
is called before each PS execution. However, we don't properly
initialize its stucture every time before the call.
Fix: properly initialize the sturture used.
mysql-test/r/ps.result:
Fix for bug#49570: Assertion failed: !(order->used & map)
on re-execution of prepared statement
- test result.
mysql-test/t/ps.test:
Fix for bug#49570: Assertion failed: !(order->used & map)
on re-execution of prepared statement
- test case.
sql/sql_select.cc:
Fix for bug#49570: Assertion failed: !(order->used & map)
on re-execution of prepared statement
- set order->used to 0 before each eq_ref_table() call,
as the function relies on that.
The test allowed random coincidence of connection ids for two concurrent
sessions performing CREATE/DROP temp tables.
Fixed with correcting the test. The sessions connection ids are not changed
from their defaults anymore.
When compressed myisam files are opened, they are always memory mapped
sometimes causing memory swapping problems.
When we mmap the myisam compressed tables of size greater than the memory
available, the kswapd0 process utilization is very high consuming 30-40% of
the cpu. This happens only with linux kernels older than 2.6.9
With newer linux kernels, we don't have this problem of high cpu consumption
and this option may not be required.
The option 'myisam_mmap_size' is added to limit the amount of memory used for
memory mapping of myisam files. This option is not dynamic.
The default value on 32 bit system is 4294967295 bytes and on 64 bit system it
is 18446744073709547520 bytes.
Note: Testcase only tests the option variable. The actual bug has be to
tested manually.
include/my_global.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
define SIZE_T_MAX
include/myisam.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
declare 'myisam_mmap_size' and 'myisam_mmap_used' variables and the mutex
THR_LOCK_myisam_mmap
myisam/mi_packrec.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add 'myisam_mmap_size' option which limits the memory available to mmap of
myisam files
myisam/mi_static.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
declare 'myisam_mmap_size' and 'myisam_mmap_used' variables and the mutex
THR_LOCK_myisam_mmap
myisam/myisamdef.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
move MEMMAP_EXTRA_MARGIN to myisam.h so that it can be used in mysqld.cc
mysql-test/r/variables.result:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
Testcase for BUG#37408 to test the myisam_mmap_size option
mysql-test/t/variables.test:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
Testcase for BUG#37408 to test the myisam_mmap_size option
mysys/my_thr_init.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
intialize the lock THR_LOCK_myisam_mmap
sql/mysqld.cc:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add the 'myisam_mmap_size' option
sql/set_var.cc:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add the 'myisam_mmap_size' to the SHOW VARIABLES list
Problem: inserting a record we don't set unused null bits in the
record buffer if no default field values used.
That may lead to wrong live checksum calculation.
Fix: set unused null bits in the record buffer in such cases.
mysql-test/r/myisam.result:
Fix for bug#49465: valgrind warnings and incorrect live checksum...
- test result.
mysql-test/t/myisam.test:
Fix for bug#49465: valgrind warnings and incorrect live checksum...
- test case.
sql/sql_insert.cc:
Fix for bug#49465: valgrind warnings and incorrect live checksum...
- set unused null bits to 1 in the record buffer in case we
don't call restore_record() before a fill_record() call
(when no default values used).
At the end of execution top level join execution
we cleanup this join with true argument.
It leads to underlying join cleanup(subquery) with true argument too
and to tmp_table_param->field array cleanup which is required later.
The problem is that Item_func_set_user_var does not set
result_filed which leads to unnecessary repeated excution of subquery
on final stage.
The fix is to set result_field for Item_func_set_user_var.
mysql-test/r/count_distinct.result:
test result
mysql-test/r/user_var.result:
test result
mysql-test/t/count_distinct.test:
test case
mysql-test/t/user_var.test:
test case
sql/item_func.cc:
At the end of execution top level join execution
we cleanup this join with true argument.
It leads to underlying join cleanup(subquery) with true argument too
and to tmp_table_param->field array cleanup which is required later.
The problem is that Item_func_set_user_var does not set
result_filed which leads to unnecessary repeated excution of subquery
on final stage.
The fix is to set result_field for Item_func_set_user_var.
int join_read_key(JOIN_TAB*)
The eq_ref access method TABLE_REF (accessed through
JOIN_TAB) to save state and to track if this is the
first row it finds or not.
This state was not reset on subquery re-execution
causing an assert.
Fixed by resetting the state before the subquery
re-execution.
SPATIAL and FULLTEXT indexes don't support algorithm
selection.
Disabled by creating a special grammar rule for these
in the parser.
Added some encasulation of duplicate parser code.
Part 2 :
There was a special optimization on the ref access method for
ORDER BY ... DESC that was set without actually looking on the type of the
selected index for ORDER BY.
Fixed the SELECT ... ORDER BY .. DESC (it uses a different code path compared
to the ASC that has been fixed with the previous fix).
mysql-test/include/kill_query.inc:
Error 1034 can be generated when change MyISAM table indexes was interrupted
mysql-test/r/rpl_killed_ddl.result:
table t4 may not exists because the ALTER above was interrupted
mysql-test/t/rpl_killed_ddl.test:
table t4 may not exists because the ALTER above was interrupted
There are three issues that caused rpl_killed_ddl fails sporadically
in pb2:
1) thd->clear_error() was not called before create Query event
if operation is executed successfully.
2) DATABASE d2 might do exist because the statement to CREATE or
ALTER it was killed
3) because of bug 43353, kill the query that do DROP FUNCTION or
DROP PROCEDURE can result in SP not found
This patch fixed all above issues by:
1) Called thd->clear_error() if the operation succeeded.
2) Add IF EXISTS to the DROP DATABASE d2 statement
3) Temporarily disabled testing DROP FUNCTION/PROCEDURE IF EXISTS.
mysql-test/t/rpl_killed_ddl.test:
DATABASE d2 might not exists, add IF EXITS to the DROP statement
sql/sql_db.cc:
Called thd->clear_error() if the operation succeeded
field='const1' AND field='const2' in some cases
Building multiple equality predicates containing
a constant which is compared as a datetime (with a field)
we should take this fact into account and compare the
constant with another possible constatns as datetimes
as well.
E.g. for the
SELECT ... WHERE a='2001-01-01' AND a='2001-01-01 00:00:00'
we should compare '2001-01-01' with '2001-01-01 00:00:00' as
datetimes but not as strings.
mysql-test/r/select.result:
Fix for bug#49199: Optimizer handles incorrectly:
field='const1' AND field='const2' in some cases
- test result.
mysql-test/t/select.test:
Fix for bug#49199: Optimizer handles incorrectly:
field='const1' AND field='const2' in some cases
- test case.
sql/item_cmpfunc.cc:
Fix for bug#49199: Optimizer handles incorrectly:
field='const1' AND field='const2' in some cases
- adding a constant to Item_equal compare it as
a datetime value with stored one if there's a
date[time] field in a equality predicate.
sql/item_cmpfunc.h:
Fix for bug#49199: Optimizer handles incorrectly:
field='const1' AND field='const2' in some cases
- adding a constant to Item_equal compare it as
a datetime value with stored one if there's a
date[time] field in a equality predicate.
sql/sql_select.cc:
Fix for bug#49199: Optimizer handles incorrectly:
field='const1' AND field='const2' in some cases
- adding a constant to Item_equal compare it as
a datetime value with stored one if there's a
date[time] field in a equality predicate.
