1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-07 03:22:57 +03:00
Commit Graph

1086 Commits

Author SHA1 Message Date
Roman Nozdrin
7acfddddb7 Refactored MDB relation names decoding in DDL code.
SH now takes all or nothing thus we need to change if conditions that rules our GBH.

Small warning fixes for GCC8.2

Disabled GBH.
2019-12-13 11:38:19 -06:00
Andrew Hutchings
fb07a02696 Merge pull request #981 from LinuxJedi/MCOL-3672
MCOL-3672 Fix regression in deletes
2019-12-13 08:41:48 +00:00
Andrew Hutchings
b7396129c7 MCOL-3672 Fix regression in deletes
Deletes appear to only use the direct delete path. This allows that to
happen.
2019-12-13 07:33:29 +00:00
benthompson15
7cda5cd179 Merge pull request #980 from LinuxJedi/MCOL-3672
MCOL-3672 Fix double-DML execution
2019-12-13 00:18:10 +01:00
Andrew Hutchings
9734a25a35 MCOL-3672 Fix double-DML execution
Direct update/delete executed doUpdateDelete as well as the regular
execution route for doUpdateDelete.

This patch only executes doUpdateDelete the first time and direct
update/delete collects the counts.
2019-12-12 22:58:20 +00:00
Andrew Hutchings
4f03c8ac44 MCOL-3666 Cleanup columnstore.cnf
Options need to be loaded after the plugin is installed or they aren't
valid. Also remove a lot of junk from the file.
2019-12-12 13:28:17 +00:00
Roman Nozdrin
19b4702a24 This patch fllows the change introduced in the direct_update/delete API
in 10.4.11 to fix MDEV-18973 in Spider.
2019-12-11 08:05:49 -06:00
benthompson15
a119da7e54 Merge pull request #968 from LinuxJedi/MCOL-3650
MCOL-3650 delete bad UDF entries
2019-12-10 17:24:17 +01:00
Andrew Hutchings
3f5e1fd0c8 Merge pull request #964 from mariadb-corporation/MCOL-3529
MCOL-3529 This patch implements direct_update and direct_delete features
2019-12-09 09:40:33 +00:00
Andrew Hutchings
850984f590 MCOL-3650 delete bad UDF entries
In 1.4.1 and below UDFs were stored in libcalmysql.so. These have been
move to ha_columnstore.so. Unfortunately the install script ignores the
previous entries. This patch removes any left over entries before adding
the new ones.
2019-12-06 11:58:12 +00:00
Andrew Hutchings
7489d0bfd0 MCOL-3625 Rename packages
Rename packages to MariaDB-columnstore-engine, MariaDB-columnstore-libs
and MariaDB-columnstore-platform.

Also add the "columnstore-" prefix the the components so that MariaDB's
packaging system understands then and add a line to include them in
MariaDB's packaging.

In addition
* Fix S3 building for dist source build
* Fix Debian 10 dependency issue
* Fix git handling for dist builds
* Add support for MariaDB's RPM building
* Use MariaDB's PCRE and readline
* Removes a few dead files
* Fix Boost noncopyable includes
2019-12-04 11:04:39 +00:00
Andrew Hutchings
a8cd34f86d Add support for building from server 2019-12-03 21:21:28 +00:00
Andrew Hutchings
27ec629ace Remove I_S plugins from my.cnf 2019-12-03 21:21:28 +00:00
Andrew Hutchings
e072bf9e9b MCOL-3628 Move I_S tables into main handler lib
The I_S tables are all now in ha_columnstore.so
2019-12-03 21:21:28 +00:00
Andrew Hutchings
9f89ab0559 MCOL-3627 Fix library name
The ColumnStore library is now called ha_columnstore.so to be inline
with other storage engines.
2019-12-03 21:21:28 +00:00
Roman Nozdrin
5cedeb110b MCOL-3529 This patch implements direct_update and direct_delete features
to properly report the number of rows affected by UPDATE|DELETE.
2019-11-27 09:33:21 -06:00
Andrew Hutchings
57724e5515 Merge pull request #953 from drrtuy/MCOL-3602_MCOL-3593_3
Selected optimizer rewrites addition.
2019-11-25 21:50:38 +00:00
Roman Nozdrin
aff40f0dbc Merge pull request #949 from tntnatbry/fix-bitcount
Implement bit_count() distributed function and handle hexadecimal string literals in buildReturnedColumn().
2019-11-25 21:51:44 +03:00
Roman Nozdrin
3fabf01e93 MCOL-3593 Disabled full optimizer run and enabled copy-pasted simplify_joins.
Disabled 4th if block in buildOuterJoin to handle non-optimized MDB query
    structures.

    Broke getSelectPlan into pieces: processFrom, processWhere.

MCOL-3593 UNION processing depends on two flags isUnion that comes as
arg of getSelectPlan and unionSel that is a local variable in
getSelectPlan. Modularization of getSelectPlan broke the mechanizm.
This patch is supposed to partially fix it.

MCOL-3593 Removed unused if condition from buildOuterJoin that allows
unsupported construct subquery in ON expression.
Fixed an improper if condition that ignors tableMap entries w/o condition
in external_lock thus external_lock doesn't clean up when the query
finishes.
Fixed wrong logging for queries processed in tableMode. Now rnd_init
properly sends queryText down to ExeMgr to be logged.

MCOL-3593 Unused attribute FromSubQuery::fFromSub was removed.
 getSelectPlan has been modularized into: setExecutionParams,
 processFrom, processWhere. SELECT, HAVING, GROUP BY, ORDER BY
 still lives in getSelectPlan.
Copied optimization function simplify_joins_ into our pushdown
 code to provide the plugin code with some rewrites from MDB it
 expects.
The columnstore_processing_handlers_fallback session variable
 has been removed thus CS can't fallback from SH to partial
 execution paths, e.g. DH, GBH or plugin API.

MCOL-3602 Moved MDB optimizer rewrites into a separate file.

Add SELECT_LEX::optimize_unflattened_subqueries() call to fix IN
 into EXISTS rewrite for semi-JOINs with subqueries.

disable_indices_for_CEJ() add index related hints to disable
 index access methods in Cross Engine Joins.

create_SH() now flattens JOIN that has both physical tables and
 views. This fixes most of views related tests in the regression.
2019-11-25 10:03:32 -06:00
Gagan Goel
2e16fe674c 1. Implement bit_count() distributed function.
2. Handle hexadecimal string literals in buildReturnedColumn().
2019-11-22 18:02:47 -05:00
Gagan Goel
a76ceb5aa0 Get the associated string values for charset()/collation() functions applied to a field. 2019-11-22 01:34:29 +00:00
benthompson15
d4d1f4d578 Merge pull request #942 from LinuxJedi/MCOL-3527-2
MCOL-3527 (attempt 2) Fix proc security settings
2019-11-20 20:46:41 +01:00
Andrew Hutchings
a079cb8d59 MCOL-3527 (attempt 2) Fix proc security settings
Setting the definer didn't work and gave errors. This is because I
forgot to set the definer for the function too. This patch instead sets
the security level to INVOKER which is probably a better way of handling
this anyway.
2019-11-18 09:01:49 +00:00
Andrew Hutchings
1a94d530fd Use execvp instead execv for cpimport
execv() doesn't use PATH settings so can't find cpimport for
INSERT...SELECT and LOAD DATA INFILE. Switch to execvp() as we have done
in other parts of the codebase.
2019-11-18 07:35:48 +00:00
David Hall
11f194b5fe MCOL-3448 Corrected datatype in predicate for RowItem
RowItem compares, such as (c1,c2) IN ((v11, v21), (v12, v22)) gave bad answers due to a typo getting the wrong datatype.
2019-11-13 21:27:24 +00:00
Andrew Hutchings
8ab9ebb0f4 MCOL-3606 Make ColumnStore use generic paths
ColumnStore now uses standard bin/lib paths for pretty much everything.
Data path is now hard-coded to /var/lib/columnstore.

This patch also:

* Removes v1 decompression
* Removes a bunch of unneeded files
* Removes COLUMNSTORE_INSTALL_DIR / $INSTALLDIR
* Makes my.cnf.d work for all platforms (MCOL-3558)
* Changes configcpp to use recursive mutex (fixes possible config write deadlock)
* Fixes MCOL-3599 Fix regr functions, The library was installed in the wrong location
* Fixes a bunch of Ubuntu packaging issues
* Changes the binary names of several of the executables so as not to
clash with potential executables from other packages
2019-11-09 16:53:05 +00:00
Roman Nozdrin
7b5e5f0eb6 MCOL-894 Upmerged the fist part of the patch into develop.
MCOL-894 Add default values in Compare and CSEP ctors to activate UTF-8 sorting
    properly.

MCOL-894 Unit tests to build a framework for a new parallel sorting.

MCOL-894 Finished with parallel workers invocation.
     The implementation lacks final aggregation step.

MCOL-894 TupleAnnexStep's init and destructor are now parallel execution aware.

    Implemented final merging step for parallel execution finalizeParallelOrderBy().

    Templated unit test to use it with arbitrary number of rows, threads.

Reuse LimitedOrderBy in the final step

MCOL-894 Cleaned up finalizeParallelOrderBy.

MCOL-894 Add and propagate thread variable that controls a number of threads.

    Optimized comparators used for sorting and add corresponding UTs.

    Refactored TupleAnnexStep::finalizeParallelOrderByDistinct.

    Parallel sorting methods now preallocates memory in batches.

MCOL-894 Fixed comparator for StringCompare.
2019-11-05 15:23:43 +03:00
Roman Nozdrin
fc988a9a99 Merge pull request #907 from LinuxJedi/replication-changes
Replication improvements
2019-10-17 22:51:47 +03:00
benthompson15
2fad6a4ed3 Merge pull request #908 from LinuxJedi/proc-fixes
MCOL-3527 Fix stored proc permissions
2019-10-16 09:59:41 -05:00
Andrew Hutchings
ce9fffe0ac MCOL-3527 Fix stored proc permissions
Due to the new way our stored procedures are installed the DEFINER needs
to be set. Otherwise they are impossible to execute.
2019-10-14 13:09:39 +01:00
Andrew Hutchings
20c1949152 Replication improvements
This patch fixes:

MCOL-3557 - Row Based Replication events to ColumnStore tables will no
longer cause MariaDB to crash, it will error instead.

MCOL-3556 - Remove the Columnstore.xml variable to turn on ColumnStore
tables applying replication events and instead make it a system variable
that can be set in my.cnf called "columnstore_replication_slave". This
allows it to be set per-UM.
2019-10-14 11:54:48 +01:00
Andrew Hutchings
ebb22a96a3 MCOL-3551 Use generic MariaDB Server paths
This branch enforces the use of generic MariaDB server paths for their
binaries and data rather than custom paths.

/usr/local/mariadb/columnstore is now only for columnstore with this
patch.

It should be noted that this removes the auto-mounting of external
MariaDB UM data storage for AWS.

This is also a fix for MCOL-3510 after buildbot changes are made.

Also... MCOL-3552 Use columnstore.cnf to load plugins

The ColumnStore plugins now load using a columnstore.cnf instead of a
SQL sequence to be more in-line with MariaDB's methods.
2019-10-13 09:34:37 +01:00
Gagan Goel
c8df46ed26 Replace ha_calpont with ha_mcs in the source code and filenames in the plugin code. 2019-10-11 17:33:43 +00:00
Gagan Goel
4c8471a786 Create separate functions for saving and restoring group by list columns. 2019-10-11 14:28:01 +00:00
Gagan Goel
594ee22999 Add status variables to show ColumnStore version and commit hash. 2019-10-09 10:43:35 -04:00
Gagan Goel
ccb36e9794 Merge pull request #890 from drrtuy/separate_pushdown
MCOL-2178 Separate ha_mcs_pushdown.cpp compilation.
2019-10-07 18:50:01 -04:00
David Hall
07aaef7933 MCOL-3503 return type for moda should 4 (DECIMAL_RESULT) 2019-10-07 15:00:03 -05:00
Roman Nozdrin
12cb5201ac MCOL-2178 Separate ha_mcs_pushdown.cpp compilation.
Set a proper type for string literals on ConstantColumn ctor
to fix the regression produced by MCOL-174.

Removed OPTIMIZER_SWITCH_EXISTS_TO_IN b/c MDB produces
unsupported optimization with it and CS couldn't create
ExistsFilter.
2019-10-07 03:12:37 -05:00
Andrew Hutchings
a2abe72a89 Merge pull request #879 from dhall-MariaDB/MCOL-3503
MCOL-3503 add MODA aggregate function
2019-10-07 08:04:10 +01:00
Roman Nozdrin
00539eda34 Merge pull request #885 from tntnatbry/fix-windowfunc-orderby
Allow a window function in the ORDER BY clause.
2019-10-07 09:53:06 +03:00
Roman Nozdrin
d68b152e50 Merge pull request #880 from tntnatbry/fix-update-decimal-constant
Set constant value of type DECIMAL_RESULT/REAL_RESULT in an update statement.
2019-10-07 09:25:16 +03:00
Gagan Goel
a9eec783e6 Disable SH for a SELECT INTO query that uses interceptor classes. 2019-10-04 10:27:19 -04:00
Gagan Goel
5d585837d8 Allow a window function in the ORDER BY clause. 2019-10-03 16:12:06 -04:00
David Hall
baf31b64fc MCOL-3505 Add moda to build and mysql.func 2019-09-30 14:16:08 -05:00
Gagan Goel
6aea9e68a5 Set constant value of type DECIMAL_RESULT/REAL_RESULT in an update statement. 2019-09-27 15:09:30 -04:00
David Hall
cbef44a0be MCOL-3503 add MODA aggregate function 2019-09-27 12:22:44 -05:00
Andrew Hutchings
1f475340dc Merge pull request #876 from tntnatbry/revert-pr-859
Revert commit 98e90ed as the proper fix is applied as part of MCOL-1559.
2019-09-26 08:48:22 +01:00
Andrew Hutchings
3710d6e331 Merge pull request #875 from tntnatbry/fix-gbh-aggregate
Build an aggregate column from the extended select list in the group by handler.
2019-09-26 08:47:56 +01:00
Andrew Hutchings
b0e0d7b8b1 Merge pull request #867 from tntnatbry/fix-double-negative-zero
Preserve the sign when a double is -0 in fetchNextRow()
2019-09-26 08:47:32 +01:00
Gagan Goel
07e3262906 Revert commit 98e90ed as the proper fix is applied as part of MCOL-1559. 2019-09-25 11:52:28 -04:00