In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.
This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.
Item_in_subselect::create_single_in_to_exists_cond() should handle the
case where the subquery is a table-less select but it is not a result
of a UNION.
(Table-less subqueries like "(SELECT 1)" are "substituted" with their select
list, but table-less subqueries with WHERE or HAVING clause, like
"(SELECT 1 WHERE ...)" are not substituted. They are handled with regular
execution path)
The query causing the issue here has implicit grouping for we
have to produce one row with special values for the aggregates
(depending on each aggregate function), and NULL values for all
non-aggregate fields.
The subselect item where implicit grouping was being done,
null_value for the subselect item was not being set for
the case when the implicit grouping produces NULL values
for the items in the select list of the subquery.
This which was leading to the crash.
The fix would be to set the null_value when all the values
for the row column have NULL values.
Further changes are
1) etting null_value for Item_singlerow_subselect only
after val_* functions have been called.
2) Introduced a parameter null_value_inside to Item_cache that
would store be set to TRUE if any of the arguments of the
Item_cache are null.
Reviewed And co-authored by Monty
The issue here was the read_set bitmap was not set for a field which
was used as a reference in an inner select.
We need to make sure that if we are in an inner select and we have
references from outer select then we update the table bitmaps for
such references.
Introduced a function in the class Item_subselect that would
update bitmaps of table for the references within a
subquery that are defined in outer selects.
Follow-up fix to commit 26f5033(MDEV-23449)
The GROUP BY clause inside IN/ALL/ANY subquery is removed
when there is no aggregate function or HAVING clause in the subquery.
When the GROUP BY clause is removed, a subquery can also be removed
if it part of the GROUP BY clause. This is done inside the function
remove_redundant_subquery_clauses. Here we walk over the GROUP BY list
and remove a subselect from its unit via the callback function
eliminate_subselect_processor.
The issue here was that when the query was being re-executed it was trying
to reinitialize the select that was removed as stated above.
This is not required, so the fix would be to remove select_lex
both from tree lex structure and the global list of nodes so that
we don't do the reinitialization again.
fix printing precedence for BETWEEN, LIKE/ESCAPE, REGEXP, IN
don't use precedence for printing CASE/WHEN/THEN/ELSE/END
fix parsing precedence of BETWEEN, LIKE/ESCAPE, REGEXP, IN
support predicate arguments for IN, BETWEEN, SOUNDS LIKE, LIKE/ESCAPE,
REGEXP
use %nonassoc for unary operators
fix parsing of IS TRUE/FALSE/UNKNOWN/NULL
remove parser_precedence test as superseded by the precedence test
* Fix the crash: IN-to-EXISTS rewrite causes an error (and so
JOIN::optimize() fails with an error, too), don't call
update_used_tables(). Terminate the query execution instead.
* Fix the cause of the error in the IN-to-EXISTS rewrite: don't do
the rewrite if doing it will cause an error of this kind:
This version of MariaDB doesn't yet support 'SUBQUERY in ROW in left
expression of IN/ALL/ANY'
* Fix another issue exposed by this testcase:
JOIN::setup_subquery_caches() may be invoked before any select has
saved its query plan, and will crash because none of the SELECTs
has called create_explain_query_if_not_exists() to create the Explain
Data Structure for this SELECT.
TODO: When merging this to 10.2, remove the poorly-placed call to
create_explain_query_if_not_exists made by fix for M_D_E_V-16153
The issue here was that the left expr and right expr of the ANY subquery
had different character sets, so we were converting the left expr to utf8 character set.
So when this conversion was happening we were actually converting the item inside the cache,
it looked like <cache>(convert(t1.l1 using utf8)), which is incorrect.
To fix this problem we are going to store the reference of the left expr and convert that
to utf8 character set, it would look like convert(<cache>(`test`.`t1`.`l1`) using utf8)
For the case when the optimizer does the IN-EXISTS transformation,
the equality condition is injected in the WHERE OR HAVING clause of
the subquery. If the select list of the subquery has a reference to
the parent select make sure to use the reference and not the original
item.
set the pointer to NULL to avoid double-free
when the item is cleaned up many times
(once in JOIN_TAB::cleanup(): tmp->jtbm_subselect->cleanup()
and once at the end of the query, with all other items)
For the query having an IN subquery with no tables, we were converting the subquery with an expression between
the left part and the select list of the subquery . This can give incorrect results when we have a condition
in the subquery with a dual table (as this is treated as a no table).
The fix is that we don't do this conversion when we have conds in the subquery with a dual table.
caused an error
The function subselect_single_select_engine::print() did not print
the WITH clause attached to a subselect with single select engine.
As a result views using suqueries with attached WITH clauses lost
these clauses when saved in frm files.
Make differentiation between pullout for merge and pulout of outer field during exists2in transformation.
In last case the field was outer and so we can safely start from name resolution context of the SELECT where it was pulled.
Old behavior lead to inconsistence between list of tables and outer name resolution context (which skips one SELECT for merge purposes) which creates problem vor name resolution.
- Fix win64 pointer truncation warnings
(usually coming from misusing 0x%lx and long cast in DBUG)
- Also fix printf-format warnings
Make the above mentioned warnings fatal.
- fix pthread_join on Windows to set return value.
Item_in_subselect::pushed_cond_guards[] array is allocated only when
left_expr->maybe_null. And it is used (for row expressions) when
left_expr->element_index(i)->maybe_null.
For left_expr being a multi-column subquery, its maybe_null is
always false when the subquery doesn't use tables (see
Item_singlerow_subselect::fix_length_and_dec()
and subselect_single_select_engine::fix_length_and_dec()),
otherwise it's always true.
But row elements can be NULL regardless, so let's always allocate
pushed_cond_guards for multi-column subqueries, no matter whether
its maybe_null was forced to true or false.
Significantly reduce the amount of InnoDB, XtraDB and Mariabackup
code changes by defining pfs_os_file_t as something that is
transparently compatible with os_file_t.
In some rare cases queries with UNION ALL
using a derived table specified by
a grouping select with a subquery in WHERE and
impossible HAVING detected after constant row
substitution could hang.
The cause was not a proper return from the
function subselect_single_select_engine::exec()
in the case when the subquery was not optimized
beforehand and the optimization performed
in this function requested for a change of the
subquery engine. This was fixed.
Also a change was applied that avoided execution
of a subquery if impossible having was detected
for the main query at the optimization stage.
