cross-engine join with a ColumnStore table errors out.
ColumnStore cannot directly update a foreign table. We detect whether
a multi-table UPDATE operation is performed on a foreign table, if so,
do not create the select_handler and let the server execute the UPDATE
operation instead.
This feature allows a query execution to fallback to the server,
in case query execution using the select_handler (SH) fails. In case
of fallback, a warning message containing the original reason for
query failure using SH is generated.
To accomplish this task, SH execution is moved to an earlier step when
we create the SH in create_columnstore_select_handler(), instead of the
previous call to SH execution in ha_columnstore_select_handler::init_scan().
This requires some pre-requisite steps that occur in the server in
JOIN::optimize() and JOIN::exec() to be performed before starting SH execution.
In addition, missing test cases from MCOL-424 are also added to the MTR suite,
and the corresponding fix using disable_indices_for_CEJ() is reverted back
since the original fix now appears to be redundant.
This patch:
1. Removes the option to declare uncompressed columns (set columnstore_compression_type = 0).
2. Ignores [COMMENT '[compression=0] option at table or column level (no error messages, just disregard).
3. Removes the option to set more than 2 extents per file (ExtentsPreSegmentFile).
4. Updates rebuildEM tool to support up to 10 dictionary extent per dictionary segment file.
5. Adds check for `DBRootStorageType` for rebuildEM tool.
6. Renamed rebuildEM to mcsRebuildEM.
This is a subtask of MCOL-4525 Implement select_handler=AUTO.
Server performs outer join to inner join conversion using simplify_joins()
in sql/sql_select.cc, by updating the TABLE_LIST::outer_join variable.
In order to perform this conversion, permanent changes are made in some
cases to the SELECT_LEX::JOIN::conds and/or TABLE_LIST::on_expr.
This is undesirable for MCOL-4525 which will attemp to fallback and execute
the query inside the server, in case the query execution fails in ColumnStore
using the select_handler.
For a query such as:
SELECT * FROM t1 LEFT JOIN t2 ON expr1 LEFT JOIN t3 ON expr2
In some cases, server can update the original SELECT_LEX::JOIN::conds
and/or TABLE_LIST::on_expr and create new Item_cond_and objects
(e.g. with 2 Item's expr1 and expr2 in Item_cond_and::list).
Instead of making changes to the original query structs, we use
gp_walk_info::tableOnExprList and gp_walk_info::condList. 2 Item's,
expr1 and expr2, in the condList, mean Item_cond_and(expr1, expr2), and
hence avoid permanent transformations to the SELECT_LEX.
We also define a new member variable
ha_columnstore_select_handler::tableOuterJoinMap
which saves the original TABLE_LIST::outer_join values before they are
updated. This member variable will be used later on to restore to the original
state of TABLE_LIST::outer_join in case of a query fallback to server execution.
The original simplify_joins() implementation in the server also performs a
flattening of the JOIN nest, however we don't perform this operation in
convertOuterJoinToInnerJoin() since it is not required for ColumnStore.
InSub::handleFunc() was incorrectly exiting early for an IN subquery
containing an isnull predicate in the OR operation in the WHERE clause.
This patch properly handles the OR predicate containing an isnull/isnotnull
predicate in the WHERE clause. We don't remove the isnull operand from the
filter ParseTree in 6.x as the server no longer injects the isnull predicate
in the IN subquery due to MCOL-4617, where we moved the creation and injection
of in-to-exists predicate into the engine.
Some MTR test still failed when running without --extern
Fixing the following problems:
- Changing "--exec mariadb" to "--exect $MYSQL" to properly pass the socket
- Changing ./suite/columnstore/std_data/ to MTR_SUITE_DIR/../std_data/
- Changing "cpimport" to $MCS_CPIMPORT.
Detecting and exporting a proper $MCS_CPIMPORT in suite.pm
in the query projection list.
With this patch, we enable SELECT/DERIVED handler execution for
queries involving 'get_user_var' function. Whereas for 'set_user_var'
function, the handlers are still disabled and the query execution
fallsback to the server, except for some edges case described in
the test file mcol-4410.test, in which case, an appropriate error
message is returned to the client.
Main theme of the patch is to fix joins processing in the plugin
code. We now use SELECT_LEX::top_join_list and process the nested
joins recursively, instead of SELECT_LEX::table_list struct which
we earlier used to build the join filters. The earlier approach
did not process certain nested join ON expressions, causing certain
queries to incorrectly error out such as that described in MCOL-4680.
In addition, some legacy code is also removed.
We earlier leveraged the server functionality provided by
Item_in_subselect::create_in_to_exists_cond and
Item_in_subselect::inject_in_to_exists_cond
to create and inject the in-to-exists predicate into an IN
subquery's JOIN struct. With this patch, we leave the IN subquery's
JOIN unaltered and instead directly perform this predicate creation
and injection into ColumnStore's select execution plan.
This patch fixes failures in mtr (without --remote) in the following tests:
columnstore/basic.mcs88_import_export_csv
columnstore/basic.mcs47_cpimport_central_loc_sin_source
columnstore/basic.mcs49_cpimport_parallel_dist
columnstore/basic.mcs50_cpimport_stdin
columnstore/basic.mcs55_cpimport_binary_source
columnstore/basic.mcs56_cpimport_negative
columnstore/basic.mcs68_cpimport_datafile_separators
Also fixing the problem with missing UDFs for the affected tests.
(see MCOL-4659 for details):
When mtr runs without --remote, the UDFs must be installed inside
the test using CREATE FUNCTION.
Note, there are still more tests left with missing UDFs. They'll be
fixed separately.
