This patch adds for "--ps-protocol" second execution
of queries "SELECT".
Also in this patch it is added ability to disable/enable
(--disable_ps2_protocol/--enable_ps2_protocol) second
execution for "--ps-prototocol" in testcases.
ANALYZE FORMAT=JSON output now includes table.r_engine_stats which
has the engine statistics. Only non-zero members are printed.
Internally: EXPLAIN data structures Explain_table_acccess and
Explain_update now have handler* handler_for_stats pointer.
It is used to read statistics from handler_for_stats->handler_stats.
The following applies only to 10.9+, backport doesn't use it:
Explain data structures exist after the tables are closed. We avoid
walking invalid pointers using this:
- SQL layer calls Explain_query::notify_tables_are_closed() before
closing tables.
- After that call, printing of JSON output is disabled. Non-JSON output
can be printed but we don't access handler_for_stats when doing that.
There was no actual execution of the SQL of a pushed derived table,
which caused "r_rows" to be always displayed as 0 and "r_total_time_ms"
to show inaccurate numbers.
This commit makes a derived table SQL to be executed by the storage
engine, so the server is able to calculate the number of rows returned
and measure the execution time more accurately
Fake_select_lex->join was prepared at the unit execution stage so
the validation of fake_select_lex before the unit pushdown
was incomplete. That caused pushing down of statements having
an incorrect ORDER BY clause.
This commit moves preparation of the fake_select_lex->join to the unit
prepare() method, before initializing of the pushdown handler,
so incorrect clauses error out before being pushed down
When printing Item_direct_view_ref the printed field name must be
complemented with the view name/derived table alias.
For example, for "SELECT a FROM (SELECT a FROM t1) q" field `a`
in the select list must be printed as `q`.`a`.
But if the view was merged then the initial `q` does not make sense
any more so field `a` must be printed as `t1`.`a`
During st_select_lex_unit::prepare() the member select_unit*
st_select_lex_unit::union_result is being assigned to an instance
of one of the following classes:
- select_unit
- select_unit_ext
- select_unit_recursive
- select_union_direct
Select_union_direct used to pass the result of the query directly to
the receiving select_result without filling a temporary table. This class
wraps a select_result object and is currently used to process UNION ALL
queries. Other select_unit_* classes involve some additional result processing.
Pushed down units are processed on the engine side so the results must be
also passed directly to a select_result object. So in the case when
the unit pushdown is employed st_select_lex_unit::union_result must be
assigned to an instance of select_union_direct.
Allow queries of multiple SELECTs combined together with
UNIONs/EXCEPTs/INTERSECTs to be pushed down to foreign engines.
If the foreign engine provides an interface method "create_unit"
and the UNIT is a top-level unit of the SQL query then the server
tries to push the whole SELECT_LEX_UNIT down to the engine for execution.
The engine should perform necessary checks and if they succeed,
execute the query. If the engine is unable to execute the whole unit,
then another attempt is made to push down SELECTs composing the unit
separately using the "create_select" interface method. In this case
the results of separate SELECTs are combined at the server side
thus composing the final result
The problem was the mysql_derived_prepare() did not correctly set
'distinct' when creating a temporary derivated table.
Fixed by separating checking for distinct for queries with and without
UNION.
Other things:
- Fixed bug in generate_derived_keys_for_table() where we set the wrong
bit for join_tab->keys
- Cleaned up JOIN::drop_unused_derived_keys()
- Changed TABLE::use_index() to keep unique keys and update
share->key_parts
Author: Sergei Petrunia <sergey@mariadb.com>, monty@mariadb.org
DuplicateWeedout semi-join optimization requires that the tables in
the parent subquery provide rowids that can be compared across table
scans. Most engines support this, federated is the only exception.
DuplicateWeedout is the default catch-all semi-join strategy, which
must be always available. If it is not available for some edge case,
it's better to disable semi-join conversion altogether.
This is what was done in the fix for MDEV-30395. However that fix
has put the check before the view processing, so it didn't detect
federated tables inside mergeable VIEWs.
This patch moves the check to be done at a later phase, when mergeable
views are already merged.
The main difference in code path between EQ_REF and REF is that for
REF we have to do an extra read_next on the index to check that there
is no more matching rows.
Before this patch we added a preference of EQ_REF by ensuring that REF
would always estimate to find at least 2 rows.
This patch adds the cost of the extra key read_next to REF access and
removes the code that limited REF to at least 2 rows. For some queries
this can have a big effect as the total estimated rows will be halved
for each REF table with 1 rows.
multi_range cost calculations are also changed to take into account
the difference between EQ_REF and REF.
The effect of the patch to the test suite:
- About 80 test case changed
- Almost all changes where for EXPLAIN where estimated rows for REF
where changed from 2 to 1.
- A few test cases using explain extended had a change of 'filtered'.
This is because of the estimated rows are now closer to the
calculated selectivity.
- A very few test had a change of table order.
This is because the change of estimated rows from 2 to 1 or the small
cost change for REF
(main.subselect_sj_jcl6, main.group_by, main.dervied_cond_pushdown,
main.distinct, main.join_nested, main.order_by, main.join_cache)
- No key statistics and the estimated rows are now smaller which cased
estimated filtering to be lower.
(main.subselect_sj_mat)
- The number of total rows are halved.
(main.derived_cond_pushdown)
- Plans with 1 row changed to use RANGE instead of REF.
(main.group_min_max)
- ALL changed to REF
(main.key_diff)
- Key changed from ref + index_only to PRIMARY key for InnoDB, as
OPTIMIZER_ROW_LOOKUP_COST + OPTIMIZER_ROW_NEXT_FIND_COST is smaller than
OPTIMIZER_KEY_LOOKUP_COST + OPTIMIZER_KEY_NEXT_FIND_COST.
(main.join_outer_innodb)
- Cost changes printouts
(main.opt_trace*)
- Result order change
(innodb_gis.rtree)
SUPER privilege used to allow various actions that were alternatively
allowed by one of BINLOG ADMIN, BINLOG MONITOR, BINLOG REPLAY,
CONNECTION ADMIN, FEDERATED ADMIN, REPL MASTER ADMIN, REPL SLAVE ADMIN,
SET USER, SLAVE MONITOR.
Now SUPER no longer does that, one has to grant one of the fine-grained
privileges above to be to perform corresponding actions.
On upgrade from MariaDB versions 10.11 and below all the privileges
above are granted automatically if the user has SUPER.
As a side-effect, such an upgrade will allow SUPER-user to run SHOW
BINLOG EVENTS, SHOW RELAYLOG EVENTS, SHOW SLAVE HOSTS, even if he wasn't
able to do it before the upgrade.
The idea is that instead of marking all select_lex's with DISTINCT, we
only mark those that really need distinct result.
Benefits of this change:
- Temporary tables used with derived tables, UNION, IN are now smaller
as duplicates are removed already on the insert phase.
- The optimizer can now produce better plans with EQ_REF. This can be
seen from the tests where several queries does not anymore materialize
derived tables twice.
- Queries affected by 'in_predicate_conversion_threshold' where large IN
lists are converted to sub query produces better plans.
Other things:
- Removed on duplicate call to sel->init_select() in
LEX::add_primary_to_query_expression_body()
- I moved the testing of
tab->table->pos_in_table_list->is_materialized_derived()
in join_read_const_table() to the caller as it caused problems for
derived tables that could be proven to be const tables.
This also is likely to fix some bugs as if join_read_const_table()
was aborted, the table was left marked as JT_CONST, which cannot
be good. I added an ASSERT there for now that can be removed when
the code has been properly tested.
The problem was that federated engine does not support comparable rowids
which was not taken into account by semijoin code.
Fixed by checking that we don't use semijoin with tables that does not
support comparable rowids.
Other things:
- Fixed some typos in the code comments
Deallocation of TABLE_LIST::dt_handler and TABLE_LIST::pushdown_derived
was performed in multiple places if code. This not only made the code
more difficult to maintain but also led to memory leaks and
ASAN heap-use-after-free errors.
This commit puts deallocation of TABLE_LIST::dt_handler and
TABLE_LIST::pushdown_derived to the single point - JOIN::cleanup()
FederatedX table may refer to a table with a different name on the
remote server:
test> CREATE TABLE t2 (...) ENGINE="FEDERATEDX"
CONNECTION="mysql://user:pass@192.168.1.111:9308/federatedx/t1";
test> select * from t2 where ...;
This could cause an issue with federated_pushdown=1, because FederatedX
pushes the query (or derived table's) text to the remote server. The remote
server will try to read from table t2 (while it should read from t1).
Solution: do not allow pushing down queries with tables that have different
db_name.table name on the local and remote server.
This patch also fixes:
MDEV-29863 Server crashes in federatedx_txn::acquire after select from the
FederatedX table with partitions
Solution: disallow pushdown when partitioned FederatedX tables are used.
