1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-11-25 20:23:16 +03:00
Commit Graph

570 Commits

Author SHA1 Message Date
Andrew Hutchings
57724e5515 Merge pull request #953 from drrtuy/MCOL-3602_MCOL-3593_3
Selected optimizer rewrites addition.
2019-11-25 21:50:38 +00:00
Roman Nozdrin
aff40f0dbc Merge pull request #949 from tntnatbry/fix-bitcount
Implement bit_count() distributed function and handle hexadecimal string literals in buildReturnedColumn().
2019-11-25 21:51:44 +03:00
Roman Nozdrin
3fabf01e93 MCOL-3593 Disabled full optimizer run and enabled copy-pasted simplify_joins.
Disabled 4th if block in buildOuterJoin to handle non-optimized MDB query
    structures.

    Broke getSelectPlan into pieces: processFrom, processWhere.

MCOL-3593 UNION processing depends on two flags isUnion that comes as
arg of getSelectPlan and unionSel that is a local variable in
getSelectPlan. Modularization of getSelectPlan broke the mechanizm.
This patch is supposed to partially fix it.

MCOL-3593 Removed unused if condition from buildOuterJoin that allows
unsupported construct subquery in ON expression.
Fixed an improper if condition that ignors tableMap entries w/o condition
in external_lock thus external_lock doesn't clean up when the query
finishes.
Fixed wrong logging for queries processed in tableMode. Now rnd_init
properly sends queryText down to ExeMgr to be logged.

MCOL-3593 Unused attribute FromSubQuery::fFromSub was removed.
 getSelectPlan has been modularized into: setExecutionParams,
 processFrom, processWhere. SELECT, HAVING, GROUP BY, ORDER BY
 still lives in getSelectPlan.
Copied optimization function simplify_joins_ into our pushdown
 code to provide the plugin code with some rewrites from MDB it
 expects.
The columnstore_processing_handlers_fallback session variable
 has been removed thus CS can't fallback from SH to partial
 execution paths, e.g. DH, GBH or plugin API.

MCOL-3602 Moved MDB optimizer rewrites into a separate file.

Add SELECT_LEX::optimize_unflattened_subqueries() call to fix IN
 into EXISTS rewrite for semi-JOINs with subqueries.

disable_indices_for_CEJ() add index related hints to disable
 index access methods in Cross Engine Joins.

create_SH() now flattens JOIN that has both physical tables and
 views. This fixes most of views related tests in the regression.
2019-11-25 10:03:32 -06:00
Gagan Goel
2e16fe674c 1. Implement bit_count() distributed function.
2. Handle hexadecimal string literals in buildReturnedColumn().
2019-11-22 18:02:47 -05:00
Gagan Goel
a76ceb5aa0 Get the associated string values for charset()/collation() functions applied to a field. 2019-11-22 01:34:29 +00:00
benthompson15
d4d1f4d578 Merge pull request #942 from LinuxJedi/MCOL-3527-2
MCOL-3527 (attempt 2) Fix proc security settings
2019-11-20 20:46:41 +01:00
Andrew Hutchings
a079cb8d59 MCOL-3527 (attempt 2) Fix proc security settings
Setting the definer didn't work and gave errors. This is because I
forgot to set the definer for the function too. This patch instead sets
the security level to INVOKER which is probably a better way of handling
this anyway.
2019-11-18 09:01:49 +00:00
Andrew Hutchings
1a94d530fd Use execvp instead execv for cpimport
execv() doesn't use PATH settings so can't find cpimport for
INSERT...SELECT and LOAD DATA INFILE. Switch to execvp() as we have done
in other parts of the codebase.
2019-11-18 07:35:48 +00:00
David Hall
11f194b5fe MCOL-3448 Corrected datatype in predicate for RowItem
RowItem compares, such as (c1,c2) IN ((v11, v21), (v12, v22)) gave bad answers due to a typo getting the wrong datatype.
2019-11-13 21:27:24 +00:00
Andrew Hutchings
8ab9ebb0f4 MCOL-3606 Make ColumnStore use generic paths
ColumnStore now uses standard bin/lib paths for pretty much everything.
Data path is now hard-coded to /var/lib/columnstore.

This patch also:

* Removes v1 decompression
* Removes a bunch of unneeded files
* Removes COLUMNSTORE_INSTALL_DIR / $INSTALLDIR
* Makes my.cnf.d work for all platforms (MCOL-3558)
* Changes configcpp to use recursive mutex (fixes possible config write deadlock)
* Fixes MCOL-3599 Fix regr functions, The library was installed in the wrong location
* Fixes a bunch of Ubuntu packaging issues
* Changes the binary names of several of the executables so as not to
clash with potential executables from other packages
2019-11-09 16:53:05 +00:00
Roman Nozdrin
7b5e5f0eb6 MCOL-894 Upmerged the fist part of the patch into develop.
MCOL-894 Add default values in Compare and CSEP ctors to activate UTF-8 sorting
    properly.

MCOL-894 Unit tests to build a framework for a new parallel sorting.

MCOL-894 Finished with parallel workers invocation.
     The implementation lacks final aggregation step.

MCOL-894 TupleAnnexStep's init and destructor are now parallel execution aware.

    Implemented final merging step for parallel execution finalizeParallelOrderBy().

    Templated unit test to use it with arbitrary number of rows, threads.

Reuse LimitedOrderBy in the final step

MCOL-894 Cleaned up finalizeParallelOrderBy.

MCOL-894 Add and propagate thread variable that controls a number of threads.

    Optimized comparators used for sorting and add corresponding UTs.

    Refactored TupleAnnexStep::finalizeParallelOrderByDistinct.

    Parallel sorting methods now preallocates memory in batches.

MCOL-894 Fixed comparator for StringCompare.
2019-11-05 15:23:43 +03:00
Roman Nozdrin
fc988a9a99 Merge pull request #907 from LinuxJedi/replication-changes
Replication improvements
2019-10-17 22:51:47 +03:00
benthompson15
2fad6a4ed3 Merge pull request #908 from LinuxJedi/proc-fixes
MCOL-3527 Fix stored proc permissions
2019-10-16 09:59:41 -05:00
Andrew Hutchings
ce9fffe0ac MCOL-3527 Fix stored proc permissions
Due to the new way our stored procedures are installed the DEFINER needs
to be set. Otherwise they are impossible to execute.
2019-10-14 13:09:39 +01:00
Andrew Hutchings
20c1949152 Replication improvements
This patch fixes:

MCOL-3557 - Row Based Replication events to ColumnStore tables will no
longer cause MariaDB to crash, it will error instead.

MCOL-3556 - Remove the Columnstore.xml variable to turn on ColumnStore
tables applying replication events and instead make it a system variable
that can be set in my.cnf called "columnstore_replication_slave". This
allows it to be set per-UM.
2019-10-14 11:54:48 +01:00
Andrew Hutchings
ebb22a96a3 MCOL-3551 Use generic MariaDB Server paths
This branch enforces the use of generic MariaDB server paths for their
binaries and data rather than custom paths.

/usr/local/mariadb/columnstore is now only for columnstore with this
patch.

It should be noted that this removes the auto-mounting of external
MariaDB UM data storage for AWS.

This is also a fix for MCOL-3510 after buildbot changes are made.

Also... MCOL-3552 Use columnstore.cnf to load plugins

The ColumnStore plugins now load using a columnstore.cnf instead of a
SQL sequence to be more in-line with MariaDB's methods.
2019-10-13 09:34:37 +01:00
Gagan Goel
c8df46ed26 Replace ha_calpont with ha_mcs in the source code and filenames in the plugin code. 2019-10-11 17:33:43 +00:00
Gagan Goel
4c8471a786 Create separate functions for saving and restoring group by list columns. 2019-10-11 14:28:01 +00:00
Gagan Goel
594ee22999 Add status variables to show ColumnStore version and commit hash. 2019-10-09 10:43:35 -04:00
Gagan Goel
ccb36e9794 Merge pull request #890 from drrtuy/separate_pushdown
MCOL-2178 Separate ha_mcs_pushdown.cpp compilation.
2019-10-07 18:50:01 -04:00
David Hall
07aaef7933 MCOL-3503 return type for moda should 4 (DECIMAL_RESULT) 2019-10-07 15:00:03 -05:00
Roman Nozdrin
12cb5201ac MCOL-2178 Separate ha_mcs_pushdown.cpp compilation.
Set a proper type for string literals on ConstantColumn ctor
to fix the regression produced by MCOL-174.

Removed OPTIMIZER_SWITCH_EXISTS_TO_IN b/c MDB produces
unsupported optimization with it and CS couldn't create
ExistsFilter.
2019-10-07 03:12:37 -05:00
Andrew Hutchings
a2abe72a89 Merge pull request #879 from dhall-MariaDB/MCOL-3503
MCOL-3503 add MODA aggregate function
2019-10-07 08:04:10 +01:00
Roman Nozdrin
00539eda34 Merge pull request #885 from tntnatbry/fix-windowfunc-orderby
Allow a window function in the ORDER BY clause.
2019-10-07 09:53:06 +03:00
Roman Nozdrin
d68b152e50 Merge pull request #880 from tntnatbry/fix-update-decimal-constant
Set constant value of type DECIMAL_RESULT/REAL_RESULT in an update statement.
2019-10-07 09:25:16 +03:00
Gagan Goel
a9eec783e6 Disable SH for a SELECT INTO query that uses interceptor classes. 2019-10-04 10:27:19 -04:00
Gagan Goel
5d585837d8 Allow a window function in the ORDER BY clause. 2019-10-03 16:12:06 -04:00
David Hall
baf31b64fc MCOL-3505 Add moda to build and mysql.func 2019-09-30 14:16:08 -05:00
Gagan Goel
6aea9e68a5 Set constant value of type DECIMAL_RESULT/REAL_RESULT in an update statement. 2019-09-27 15:09:30 -04:00
David Hall
cbef44a0be MCOL-3503 add MODA aggregate function 2019-09-27 12:22:44 -05:00
Andrew Hutchings
1f475340dc Merge pull request #876 from tntnatbry/revert-pr-859
Revert commit 98e90ed as the proper fix is applied as part of MCOL-1559.
2019-09-26 08:48:22 +01:00
Andrew Hutchings
3710d6e331 Merge pull request #875 from tntnatbry/fix-gbh-aggregate
Build an aggregate column from the extended select list in the group by handler.
2019-09-26 08:47:56 +01:00
Andrew Hutchings
b0e0d7b8b1 Merge pull request #867 from tntnatbry/fix-double-negative-zero
Preserve the sign when a double is -0 in fetchNextRow()
2019-09-26 08:47:32 +01:00
Gagan Goel
07e3262906 Revert commit 98e90ed as the proper fix is applied as part of MCOL-1559. 2019-09-25 11:52:28 -04:00
Gagan Goel
32108e69e8 Merge pull request #870 from drrtuy/fix-literals
MCOL-2178 CS now can have string literals and functions with literals
2019-09-25 11:26:09 -04:00
Gagan Goel
dac290b755 Build an aggregate column from the extended select list in
the group by handler.
2019-09-25 10:42:39 -04:00
Andrew Hutchings
70b3aa3159 Merge branch 'develop-1.2' into develop-merge-up-20190924-2 2019-09-24 14:17:57 +01:00
Roman Nozdrin
a710dff9e1 MCOL-2178 CS now can have string literals and functions with literals
in projection list of the queries with GROUP BY and ORDER BY.

Removed support for Item_equal b/c it is an internal format MDB uses
and shouldn't be used outside of MDB's optimizer.
2019-09-20 13:35:49 +03:00
Gagan Goel
abea9481b7 Preserve the sign when a double is -0 in fetchNextRow() 2019-09-19 15:48:14 -04:00
Roman Nozdrin
69c1a613fa MCOL-2178 Fix distinctAggregationAndGroupBy by allowing to traverse into
any function in projection looking for either aggregation function or
GROUP BY key column.

MDB doesn't convert Item_equal into corresponding Item_func_eq w/o
OPTIMIZER_SWITCH_COND_PUSHDOWN_FROM_HAVING optimizer flag.
Activation of the flag fixes queries from having test scenario w
equal operator in HAVING, e.g. HAVING c1 = 50.
2019-09-19 08:06:39 +03:00
Gagan Goel
7599a9379e Save and restore the group by list columns as the server
optimizer can optimize out the constant columns in this list
2019-09-17 10:36:49 -04:00
Gagan Goel
5fe8388760 Undo commit e60fbf260f 2019-09-17 00:45:04 -04:00
Gagan Goel
21a98de22a Merge pull request #864 from drrtuy/set-optimizer-flags
MCOL-2178 CS now sets optimizer flags like it did in the fork-era.
2019-09-17 00:10:39 -04:00
Gagan Goel
df1f5aa545 1. Allow IF function with an aggregate function as an argument in the order by clause.
2. Handle hexadecimal literals of the form 0xHHHH... in gp_walk().
2019-09-15 20:05:57 -04:00
Roman Nozdrin
0c58f10d12 MCOL-2178 CS now sets optimizer flags like it did in the fork-era.
This happens in external_lock() whilst locking the table.

Fixes LIMIT=1 optimization for EXISTS_SUBS subqueries.

external_lock() contains if condition that gives false positive
for SH + pushed conditions.

external_lock() now resets in_subquery_conversion_threshold
variable that governs IN_INTO_SUBQUERY optimization for
queries run in table mode.

external_lock() now purges dynamicall allocated condInfo for
SH and DH execution path.

Commented out UNION check b/c if condition gives false positives
and silently enables table mode execution for queries w/o
UNION.
2019-09-15 07:12:50 -05:00
Gagan Goel
98e90ed9a8 Follow up to commit 8513bc0646
Only trim whitespaces, not tabs
2019-09-10 13:27:05 -04:00
Gagan Goel
8513bc0646 Right trim whitespaces for a string ConstantColumn in a where clause 2019-09-09 23:33:46 -04:00
benthompson15
b658080f06 Merge pull request #854 from mariadb-corporation/remove-vpj
Remove vpj files
2019-09-06 10:01:52 -05:00
benthompson15
016523ef47 Merge pull request #851 from mariadb-corporation/fix-packaging
Fix my.cnf clash
2019-09-05 18:27:45 -05:00
Andrew Hutchings
7da5b21fe2 Add required variable back 2019-09-05 19:03:32 +01:00