1. LIMIT ROWS EXAMINED clause allows to abort a query if a certain limit
of rows involved in processing is exceeded. This limitation should be only
enforced during the execution phase and not the optimization.
2. Unions are often executed using so-called "fake_select_lex" which
collects the final result from union parts. During that finalization
LIMIT ROWS EXAMINED must be ignored to avoid producing a potentially
incomplete result. There is a workaround at `st_select_lex_unit::exec()`
which deactivates the limit before fake_select_lex processing, and
re-activates it when the processing is finished. However, this re-activation
does not take into account whether the limit was active before the start
of fake_select_lex processing.
3. `st_select_lex_unit::exec()` can be invoked during the optimization
phase for evaluation of constant conditions with unions. At that time
the limit trigger is not activated. Given that the re-activation
mentioned above does not respect the previous state of the limit trigger,
a premature activation of the limit may happen.
This commit fixes that behavior by storing the state of the trigger
before its deactivation at `st_select_lex_unit::exec()` and re-activating
it only if the limit was active before.
4. This commit also removes the call to `thd->lex->set_rows_examined()`
from `st_select_lex_unit::exec_recursive()` which was probably copied
from `st_select_lex_unit::exec()`. But in the context of
`exec_recursive()` the call does not make sense as there is no
previous deactivation of the limit as at `exec()`.
Reviewed by: Dave Gosselin, Sergei Petrunia
Cleanup sequence comes after error state when not all units were prepared.
The With_element without rec_result was created for following unit:
"select 1 AS `1` union select 'x' AS x from x" the error happened
in select "select x AS x" so above unit was not prepared. The case
seems to be legal and the presence of rec_result must be checked during
the cleanup.
For all "UNION ALL" queries, the field rows_examined in the
slow query log is always being set to 0. However, this is not
the case with UNION queries although both UNION, and UNION ALL
share the same code.
The problem is that for UNION ALL queries, rows_examined field
in the thd object is not updated to the actual processed rows,
although they are being tracked in the sql_union.cc
Later, when the thd object is used to dump the examined rows
to the slow query log, it was only writing 0, as thd object was
never updated with the processed rows count.
This PR addresses the concern to write rows_examined field correctly
for UNION_ALL queries to the slow query log.
Oracle mode has different set operator precedence and handling (not by
standard). In Oracle mode the below test case is handled as-is, in
plain order from left to right. In MariaDB default mode follows SQL
standard and makes INTERSECT prioritized, so UNION is taken from
derived table which is INTERSECT result (here and below the same
applies for EXCEPT).
Non-distinct set operator (UNION ALL/INTERSECT ALL) works via unique
key release but it can be done only once. We cannot add index to
non-empty heap table (see heap_enable_indexes()). So every UNION ALL
before rightmost UNION DISTINCT works as UNION DISTINCT. That is
common syntax, MySQL, MSSQL and Oracle work that way.
There is union_distinct property which indicates the rightmost
distinct UNION (at least, so the algorithm works simple: it releases
the unique key after union_distinct in the loop
(st_select_lex_unit::exec()).
INTERSECT ALL code (implemented by MDEV-18844 in a896beb) does not
know about Oracle mode and treats union_distinct as the last
operation, that's why it releases unique key on union_distinct
operation. INTERSECT ALL requires unique key for it to work, so before
any INTERSECT ALL unique key must not be released (see
select_unit_ext::send_data()).
The patch tweaks INTERSECT ALL code for Oracle mode. In
disable_index_if_needed() it does not allow unique key release before
the last operation and it allows unfold on the last operation. Test
case with UNION DISTINCT following INTERSECT ALL at least does not
include invalid data, but in fact the whole INTERSECT ALL code could
be refactored for better semantical triggers.
The patch fixes typo in st_select_lex_unit::prepare() where
have_except_all_or_intersect_all masked eponymous data member which
wrongly triggered unique key release in st_select_lex_unit::prepare().
The patch fixes unknown error in case ha_disable_indexes() fails.
Note: optimize_bag_operation() does some operator substitutions, but
it does not run under PS. So if there is difference in test with --ps
that means non-optimized (have_except_all_or_intersect_all == true)
code path is not good.
Note 2: VIEW is stored and executed in normal mode (see
Sql_mode_save_for_frm_handling) hence when SELECT order is different
in Oracle mode (defined by parsed_select_expr_cont()) it must be
covered by --disable_view_protocol.
The problems were that:
1) resources was freed "asimetric" normal execution in send_eof,
in case of error in destructor.
2) destructor was not called in case of SP for result objects.
(so if the last SP execution ended with error resorces was not
freeded on reinit before execution (cleanup() called before next
execution) and destructor also was not called due to lack of
delete call for the object)
Result cleanup() renamed to reset_for_next_ps_execution() to better
reflect function().
All result method revised and freeing resources made "symetric".
Destructor of result object called for SP.
Added skipped invalidation in case of error in insert.
Removed misleading naming of reset(thd) (could be mixed with
with reset()).
When UNION ALL is used with LIMIT ROWS EXAMINED, and when the limit is
exceeded for a SELECT that is not the last in the UNION, interrupt the
execution and call end_eof on the result. This makes sure that the
results are sent, and the query result status is conclusive rather
than empty, which would cause an assertion failure.
on disable_indexes(HA_KEY_SWITCH_NONUNIQ_SAVE) the engine does
not know that the long unique is logically unique, because on the
engine level it is not. And the engine disables it,
Change the disable_indexes/enable_indexes API. Instead of the enum
mode, send a key_map of indexes that should be enabled. This way the
server will decide what is unique, not the engine.
Found memory leaks were introduced by the commit
a896bebfa6
MDEV-18844 Implement EXCEPT ALL and INTERSECT ALL operations
and caused by using a statement arena instead a runtime arena for
allocation of objects having temporary life span by their nature.
Aforementioned memory leaks were produced by running queries
that typically use select with intersect, union or table values
constructors.
To fix these memory leaks use the runtime arena for allocation
of Item_field objects used by set operations.
Additionally, OOM handling added on allocation of aforementioned
Item_field objects.
This patch fixes the issue with passing the DEFAULT or IGNORE values to
positional parameters for some kind of SQL statements to be executed
as prepared statements.
The main idea of the patch is to associate an actual value being passed
by the USING clause with the positional parameter represented by
the Item_param class. Such association must be performed on execution of
UPDATE statement in PS/SP mode. Other corner cases that results in
server crash is on handling CREATE TABLE when positional parameter
placed after the DEFAULT clause or CALL statement and passing either
the value DEFAULT or IGNORE as an actual value for the positional parameter.
This case is fixed by checking whether an error is set in diagnostics
area at the function pack_vcols() on return from the function pack_expression()
MDEV-32441 SENT_ROWS shows random wrong values when stored function
is selected.
MDEV-32281 EXAMINED_ROWS is not populated in
information_schema.processlist upon SELECT.
Added ROWS_SENT to information_schema.processlist
This is to have the same information as Percona server (SENT_ROWS)
To ensure that information_schema.processlist has correct values for
sent_rows and examined_rows I introduced two new variables to hold the
total counts so far. This was needed as stored functions and stored
procedures will reset the normal counters to be able to count rows for
each statement individually for slow query log.
Other things:
- Selects with functions shows in processlist the total examined_rows
and sent_rows by the main statement and all functions.
- Stored procedures shows in processlist examined_rows and sent_rows
per stored procedure statement.
- Fixed some double accounting for sent_rows and examined_rows.
- HANDLER operations now also supports send_rows and examined_rows.
- Display sizes for MEMORY_USED, MAX_MEMORY_USED, EXAMINED_ROWS and
QUERY_ID in information_schema.processlist changed to 10 characters.
- EXAMINED_ROWS and SENT_ROWS changed to bigint.
- INSERT RETURNING and DELETE RETURNING now updates SENT_ROWS.
- As thd is always up to date with examined_rows, we do not need
to handle examined row counting for unions or filesort.
- I renamed SORT_INFO::examined_rows to m_examined_rows to ensure that
we don't get bugs in merges that tries to use examined_rows.
- Removed calls of type "thd->set_examined_row_count(0)" as they are
not needed anymore.
- Removed JOIN::join_examined_rows
- Removed not used functions:
THD::set_examined_row_count()
- Made inline some functions that where called for each row.
Fake_select_lex->join was prepared at the unit execution stage so
the validation of fake_select_lex before the unit pushdown
was incomplete. That caused pushing down of statements having
an incorrect ORDER BY clause.
This commit moves preparation of the fake_select_lex->join to the unit
prepare() method, before initializing of the pushdown handler,
so incorrect clauses error out before being pushed down
During st_select_lex_unit::prepare() the member select_unit*
st_select_lex_unit::union_result is being assigned to an instance
of one of the following classes:
- select_unit
- select_unit_ext
- select_unit_recursive
- select_union_direct
Select_union_direct used to pass the result of the query directly to
the receiving select_result without filling a temporary table. This class
wraps a select_result object and is currently used to process UNION ALL
queries. Other select_unit_* classes involve some additional result processing.
Pushed down units are processed on the engine side so the results must be
also passed directly to a select_result object. So in the case when
the unit pushdown is employed st_select_lex_unit::union_result must be
assigned to an instance of select_union_direct.
Allow queries of multiple SELECTs combined together with
UNIONs/EXCEPTs/INTERSECTs to be pushed down to foreign engines.
If the foreign engine provides an interface method "create_unit"
and the UNIT is a top-level unit of the SQL query then the server
tries to push the whole SELECT_LEX_UNIT down to the engine for execution.
The engine should perform necessary checks and if they succeed,
execute the query. If the engine is unable to execute the whole unit,
then another attempt is made to push down SELECTs composing the unit
separately using the "create_select" interface method. In this case
the results of separate SELECTs are combined at the server side
thus composing the final result
This error was discovered while working on
MDEV-30540 Wrong result with IN list length reaching
IN_PREDICATE_CONVERSION_THRESHOLD
If there is read error from handler::ha_rnd_next() during a recursive
query, st_select_lex_unit::exec_recursive() will crash as it will try to
get the error code from a structure that was deleted by the callee.
The code was using the construct:
sl->join->exec();
saved_error=sl->join->error;
This does not work as sl->join was freed by the exec() and sl->join would
be set to 0.
Fixed by having JOIN::exec() return the error code.
The included test case simulates the error in ha_rnd_next(), which causes
a crash without the patch.
scovered whle working on
MDEV-30540 Wrong result with IN list length reaching
IN_PREDICATE_CONVERSION_THRESHOLD
If there is read error from handler::ha_rnd_next() during a recursive
query, st_select_lex_unit::exec_recursive() will crash as it will try to
get the error code from a structure that was deleted by the callee.
The code was using the construct:
sl->join->exec();
saved_error=sl->join->error;
This does not work as sl->join was freed by the exec() and sl->join was
set to 0.
Fixed by having JOIN::exec() return the error code.
The included test case simulates the error in ha_rnd_next(), which causes
a crash without the patch.
