The assertion failure was caused by an incorrectly set read_set for
functions in the ORDER BY clause in part of a union, when we are using
a mergeable view and the order by clause can be skipped (removed).
An order by clause can be skipped if it's part of one part of the UNION as
the result set is not meaningful when multiple SELECT queries are UNIONed. The
server is aware of this optimization and tries to remove the order by
clause before JOIN::prepare. The problem is that we need to throw an
error when the ORDER BY clause contains invalid columns. To do this, we
attempt resolving the ORDER BY expressions, then subsequently drop them
if resolution succeeded. However, ORDER BY resolution had the side
effect of adding the expressions to the all_fields list, which is used
to construct temporary tables to store the result. We may be ignoring
the ORDER BY statement, but the tmp table still tried to compute the
values for the expressions, even if the columns are never used.
The assertion only shows itself if the order by clause contains members
which were not previously in the select list, and are part of a
function.
There is an additional question as to why this only manifests when using
VIEWS and not when using a regular table. The difference lies with the
"reset" of the read_set for the temporary table during
SELECT_LEX::update_used_tables() in JOIN::optimize(). The changes
introduced in fdf789a7eadf864ecc0e617f25f795fafda55026 cleared the
read_set when a mergeable view is encountered in the TABLE_LIST
defintion.
Upon initial order_list resolution, the table's read_set is updated
correctly. JOIN::optimize() will only reset the read_set if it
encounters a VIEW. Since we no longer have ORDER BY clause in
JOIN::optimize() we never get to correctly update the read_set again.
Other relevant commit by Timour, which first introduced the order
resolution when we "can_skip_sort_order":
883af99e7dac91e3f258135a2053e6b8e3c05fc3
Solution:
Don't add the resolved ORDER BY elements to all_fields. We only resolve
them to check if an error should be returned for the query. Ignore them
completely otherwise.
The method subselect_union_engine::no_rows() must take
into account the fact that now unit->fake_select_lex is
NULL for for select_union_direct objects.
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
Bug #18167356: EXPLAIN W/ EXISTS(SELECT* UNION SELECT*)
WHERE ONE OF SELECT* IS DISTINCT FAILS.
the bugfix itself was not merged - MariaDB doesn't have this bug.
ISSUE:
------
For UNION of selects, rows examined by the query will be sum
of rows examined by individual select operations and rows
examined for union operation. The value of session level
global counter that is used to count the rows examined by a
select statement should be accumulated and reset before it
is used for next select statement. But we have missed to
reset the same. Because of this examined row count of a
select query is accounted more than once.
SOLUTION:
---------
In union reset the session level global counter used to
accumulate count of examined rows after its value is saved.
mysql-test/r/union.result:
Expected output of testcase added.
mysql-test/t/union.test:
Test to verify examined row count of Union operations.
sql/sql_union.cc:
Reset the value of thd->examined_row_count after
accumulating the value.
When marking used columns the function find_field_in_table_ref() erroneously
called the walk method for the real item behind a view/derived table field
with the second parameter set to TRUE.
This erroneous code was introduced in 2006.
The bug prevented acceptance of UNION queries whose non-first select
clauses contained join expressions with degenerated single-table nests
as valid queries.
The bug was introduced into mysql-5.5 code line by the patch for
bug 33204.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
We must allocate a larger ref_pointer_array. We failed to account for extra
items allocated here:
#0 find_order_in_list
uint el= all_fields.elements;
all_fields.push_front(order_item); /* Add new field to field list. */
ref_pointer_array[el]= order_item;
order->item= ref_pointer_array + el;
#1 setup_order
#2 setup_without_group
#3 JOIN::prepare
mysql-test/r/order_by.result:
New test case.
mysql-test/r/union.result:
New test case.
mysql-test/t/order_by.test:
New test case.
mysql-test/t/union.test:
New test case.
sql/sql_lex.cc:
find_order_in_list() may need some extra space, so multiply og_num by two.
sql/sql_union.cc:
For UNION, the 'n_sum_items' are accumulated in the "global_parameters" select_lex.
This number must be propagated to setup_ref_array()
When preparing a 'fake_select_lex' we need to use global_parameters->order_list
rather than fake_select_lex->order_list (see comments inside st_select_lex_unit::cleanup)
mysql-test/r/union.result:
Added test for lp:732124
mysql-test/t/union.test:
Added test for lp:732124
sql/sp_rcontext.cc:
Updated function definition for ::send_data()
sql/sp_rcontext.h:
Updated function definition for ::send_data()
sql/sql_analyse.cc:
Test if send_data() returned an error
sql/sql_class.cc:
Updated function definition for ::send_data()
sql/sql_class.h:
Changed select_result::send_data(List<Item> &items) to return -1 in case of duplicate row that should not be counted as part of LIMIT
sql/sql_cursor.cc:
Check if send_data returned error
sql/sql_delete.cc:
Updated function definition for ::send_data()
sql/sql_insert.cc:
Updated function definition for ::send_data()
sql/sql_select.cc:
Don't count rows which send_data() tells you to ignore
sql/sql_union.cc:
Inform caller that the row should be ignored. This is the real bug fix for lp:732124
sql/sql_update.cc:
Updated function definition for ::send_data()
and Order By
When having a UNION statement in a subquery, with no
referenced tables (or only a reference to the virtual
table 'dual'), the UNION did not allow an ORDER BY clause.
i.e:
SELECT(SELECT 1 AS a UNION
SELECT 0 AS a
ORDER BY a) AS b or
SELECT(SELECT 1 AS a FROM dual UNION
SELECT 0 as a
ORDER BY a) AS b
In addition, an ORDER BY / LIMIT clause was not accepted
in subqueries even for single SELECT statements with no
referenced tables (or with 'dual' as table reference)
i.e:
SELECT(SELECT 1 AS a ORDER BY a) AS b or
SELECT(SELECT 1 AS a FROM dual ORDER BY a) AS b
The fix was to allow an optional ORDER BY/LIMIT clause to
the grammar for these cases.
See also: Bug#57986
if embedded in a SELECT
An ORDER BY clause was bound to the incorrect
(sub-)statement when used in a UNION context.
In a query like:
SELECT * FROM a UNION SELECT * FROM b ORDER BY c
the result of SELECT * FROM b is sorted, and then
combined with a. The correct behaviour is that
the ORDER BY clause should be applied on the
final set. Similar behaviour was seen on LIMIT
clauses as well.
In a UNION statement, there will be a select_lex
object for each of the two selects, and a
select_lex_unit object that describes the UNION
itself. Similarly, the same behaviour was also
seen on derived tables.
The bug was caused by using a grammar rule for
ORDER BY and LIMIT that bound these elements
to thd->lex->current_select, which points to the
last of the two selects, instead of to the
fake_select_lex member of the master select_lex_unit
object.
sql/sql_yacc.yy:
Need to use (opt_)union_order_or_limit to
bind to the correct select_lex object.
'CREATE TABLE IF NOT EXISTS ... SELECT' behaviour
BUG#47132, BUG#47442, BUG49494, BUG#23992 and BUG#48814 will disappear
automatically after the this patch.
BUG#55617 is fixed by this patch too.
This is the 5.5 part.
It implements:
- 'CREATE TABLE IF NOT EXISTS ... SELECT' statement will not insert
anything and binlog anything if the table already exists.
It only generate a warning that table already exists.
- A couple of test cases for the behavior changing.
Several problems fixed :
1. Non constant expressions in UNION ... ORDER BY were not correctly cleaned up
in st_select_lex_unit::cleanup() causing crashes in EXPLAIN EXTENDED because of
fields quoted by these expressions pointing to the already freed temporary table
used to calculate the UNION.
Fixed by correctly cleaning up expressions of any depth.
2. Subqueries in the order by part of UNION ... ORDER BY ... caused a crash in
EXPLAIN EXTENDED because of a transformation attempt made during EXPLAIN EXTENDED
execution. Fixed by not doing the transformation when in EXPLAIN.
3. Fulltext functions caused crash when in the ORDER BY part of an un-parenthesized
UNION that gets "promoted" to be valid for the whole union, e.g.
SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY MATCHES (a) AGAINST ('abc' IN BOOLEAN MODE).
This is a case that demonstrates a more general problem of parts of the query being
moved to another level. When doing such transformation late in the optimization run
when most of the flags about the contents of the query are already aggregated it's possible
to "split" the flags so that they correctly reflect the new queries after the transformation.
In specific the ST_SELECT_LEX::ftfunc_list is holding all the free text function for all the
parts of the second SELECT in the UNION and we don't know what part of that is in the ORDER BY
that we're to move to the UNION level and what part is about the other parts of the second SELECT.
Fixed by throwing and error when such statements are about to be processed by adding a check
for the presence of MATCH() inside the ORDER BY clause that's going to get promoted to UNION.
To workaround this new limitation one must parenthesize the UNION SELECTs and provide a real
global ORDER BY for the UNION outside of the parenthesis.
mysql-next-mr-bugfixing.
Bug no 32858 was fixed in two different ways in what was
then called mysql 5.1 and 6.0. The fix in 6.0 was very
different since bugfix no 33204 was present. Furthermore,
the two fixes were not compatible. Hence in order to
backport Bug#33204 to the 5.1-based mysql-next-mr-bugfixing,
it was necessary to remove the 5.1 fix of 32858 and apply
the 6.0 version of the fix.
mysql-test/r/subselect.result:
Bug#33204-backport: Test result
mysql-test/r/union.result:
Bug#33204-backport:
- Reversal of test result: bugfix no 32858 for 5.1
- Application of test result: bugfix no 32858 for 6.0
mysql-test/t/subselect.test:
Bug#33204-backport:
- Changed tests
- Test case
mysql-test/t/union.test:
Bug#33204-backport:
- Reversal of test: bugfix no 32858 for 5.1
- Application of test: bugfix no 32858 for 6.0
sql/sql_class.cc:
Bug#33204-backport: Reversal of bugfix no 32858 for 5.1
sql/sql_class.h:
Bug#33204-backport: Reversal of bugfix no 32858 for 5.1
sql/sql_yacc.yy:
Bug#33204-backport:
- Reversal of bugfix no 32858 for 5.1
- Application of bugfix no 32858 for 6.0
- Application of bugfix no 33204
In UNION if we use last SELECT without braces and this
SELECT have ORDER BY clause, such clause belongs to
global UNION. It is parsed like last SELECT
part and used further as 'unit->global_parameters->order_list' value.
During DESCRIBE EXTENDED we call select_lex->print_order() for
last SELECT where order fields refer to tmp table
which already freed. It leads to crash.
The fix is clean up global_parameters->order_list
instead of fake_select_lex->order_list.
mysql-test/r/union.result:
test result
mysql-test/t/union.test:
test case
sql/sql_union.cc:
In UNION if we use last SELECT without braces and this
SELECT have ORDER BY clause, such clause belongs to
global UNION. It is parsed like last SELECT
part and used further as 'unit->global_parameters->order_list' value.
During DESCRIBE EXTENDED we call select_lex->print_order() for
last SELECT where order fields refer to tmp table
which already freed. It leads to crash.
The fix is clean up global_parameters->order_list
instead of fake_select_lex->order_list.
UNION could convert fixed-point FLOAT(M,D)/DOUBLE(M,D) columns
to FLOAT/DOUBLE when aggregating data types from the SELECT
substatements. While there is nothing particularly wrong with
this behavior, especially when M is greater than the hardware
precision limits, it could be confusing in cases when all
SELECT statements in a union have the same
FLOAT(M,D)/DOUBLE(M,D) columns with equal precision
specifications listed in the same position.
Since the manual is quite vague on what data type should be
returned in such cases, the bug was fixed by implementing the
most 'expected' behavior: do not convert FLOAT(M,D)/DOUBLE(M,D)
to anything else if all SELECT statements in a UNION have the
same precision for that column.
mysql-test/r/union.result:
Added a test case for bug #43432.
mysql-test/t/union.test:
Added a test case for bug #43432.
sql/field.cc:
Replaced FLT_DIG+6 and DBL_DIG+7 with a symbolic constant.
sql/item.cc:
Do not convert FLOAT(M,D)/DOUBLE(M,D)
to anything else if all SELECT statements in a UNION have the
same precision for that column.
sql/mysql_priv.h:
Added a symbolic constant for FLT_DIG+6 and DBL_DIG+7.
set but is ignored".
This patch makes @@session.max_allowed_packed and
@@session.net_buffer_length read-only as suggested in the bug
report. The user will have to use SET GLOBAL (and reconnect)
to alter the session values of these variables.
The error string ER_VARIABLE_IS_READONLY is introduced.
Tests are modified accordingly.
sql/set_var.cc:
The class sys_var_thd_ulong_session_readonly is introduced as
a specialization of sys_var_thd_ulong implementing a read-only
session variable. The class overrides check() and
check_default() to achieve the read-only property for the
session part of the variable.
sql/set_var.h:
The class sys_var_thd_ulong_session_readonly is introduced as
a specialization of sys_var_thd_ulong implementing a read-only
session variable. The class overrides check() and
check_default() to achieve the read-only property for the
session part of the variable.
sql/share/errmsg.txt:
New error ER_VARIABLE_IS_READONLY.
subselects into account
It is forbidden to use the SELECT INTO construction inside UNION statements
unless on the last SELECT of the union. The parser records whether it
has seen INTO or not when parsing a UNION statement. But if the INTO was
legally used in an outer query, an error is thrown if UNION is seen in a
subquery. Fixed in 5.0 by remembering the nesting level of INTO tokens and
mitigate the error unless it collides with the UNION.
mysql-test/r/union.result:
Bug#32858: Test result
mysql-test/t/union.test:
Bug#32858: Test case
sql/sql_class.cc:
Bug#32858: Initializing new member
sql/sql_class.h:
Bug#32858: Added property nest_level to select_result class.
sql/sql_yacc.yy:
Bug#32858: The fix.
