1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-10-31 18:30:33 +03:00
Commit Graph

183 Commits

Author SHA1 Message Date
drrtuy
80f81526eb feat(rbo,rules,QA): replace monotonic counter with column_name->col_index map and added fields to activate existing filters pushdown machinery. 2025-08-14 19:38:00 +04:00
drrtuy
0c876e1ed4 feat(rbo,rules,QA): SC in upper layer filters are updated. 2025-08-14 19:38:00 +04:00
drrtuy
7ad42678d4 chore(QA): fixed compilation and preprocessor filtering macro. 2025-08-13 13:24:30 +00:00
drrtuy
877bd85d6a chore(QA): removed commented code. 2025-08-13 13:24:30 +00:00
drrtuy
c19c49ba13 chore(rbo,rules,QA): new extractColumnStatistics, some comments with a bit of re-factoring. 2025-08-13 13:20:57 +00:00
drrtuy
c030ff4224 feat(rbo,rules,QA): index column type is now derived from the corresponding Field 2025-08-13 13:20:57 +00:00
drrtuy
e167082497 feat(rbo,rules,QA): refactored statistics storage 2025-08-13 13:20:57 +00:00
drrtuy
112ba9f162 feat(rbo,rules,QA): changed to way statistics is collected 2025-08-13 13:20:57 +00:00
drrtuy
134235b2d2 feat(rbo,rules,QA): support expressions in RCs 2025-08-13 13:20:57 +00:00
drrtuy
b07ee73fb3 feat(rbo,rules,QA): filtered RC clone for UNION units. 2025-08-13 13:20:57 +00:00
drrtuy
fa76344cfc chore(rbo): simplistic rulle application logging. 2025-08-13 14:50:58 +04:00
drrtuy
67ac7f2f75 fix(rbo,rules): MCOL-6131 predicate pushdown rule for RBO. 2025-08-13 14:50:58 +04:00
Leonid Fedorov
82421c208f chore(ci): collect asan ubsan and libc++ build with mtr and regression status ignored (#3672)
* MSan added with fixes for libc++

* libc++ sepatare build

* add libc++ to ci

* libstdc++ in CI

* libcpp and msan to external projects

* std::sqrt

* awful_hack(ci): install whole llvm instead of libc++ in terrible way for test containers

* Adding ddeb packages for teststages and repos

* libc++ more for test container

* save some money on debug

* colored coredumps

* revert ci

* chore(ci): collect asan ubsan and libc++ build with mtr and regression status ignored
2025-07-31 00:32:32 +04:00
Sergey Zefirov
3418e68b78 fix(ddl): MCOL-5974 TIMESTAMP columns are not IS NOT NULL by default in MCS (#3670)
The old behavior was to assign TIMESTAMP columns an IS NOT NULL
constraint if list of constraints was empty. Apparently, this particular
invariant was changed in 11.4 and bugfixes' mcol-5480.test started to
fail.

This patch fixes behavioral difference and reenables mcol-5480 test in
11.4.
2025-07-27 20:06:40 +01:00
drrtuy
f881bae496 chore(rbo,rules): fixes to compile MCS with ES 10.6 2025-07-21 12:54:07 +01:00
drrtuy
a01c883e07 feat(rbo,rules): mock Histogram for ES < 11.4 2025-07-21 12:54:07 +01:00
drrtuy
e600f11aa9 feat(rbo,rules): use EI statistics for filter ranges 2025-07-21 12:54:07 +01:00
drrtuy
15be33fbc5 feat(rbo,rules): refactored statistics storage in gwi and implemented statistics based UNION rewrite. 2025-07-21 12:54:07 +01:00
drrtuy
67295b4320 feat(optimizer,rules): use EI statistics for range filters 2025-07-21 12:54:07 +01:00
drrtuy
3f9ce7779e feat(optimizer): PoC for EI stats retrieval in getSelectPlan() 2025-07-21 12:54:07 +01:00
drrtuy
dfddfedfe5 feat(optimizer): collect EI statistics for a first column in existing tables indexes 2025-07-21 12:54:07 +01:00
drrtuy
70547c7358 chore(plugin): translator walks are now in separate units 2025-06-27 17:38:33 +04:00
drrtuy
e57832ee64 feat(optimizer): temporary shield optimizer with a session variable 2025-06-26 18:35:33 +01:00
drrtuy
25c7d23c21 feat(optimizer): add session switch to optionally enable optimizer 2025-06-26 18:35:33 +01:00
drrtuy
ab6063bec4 feat(optimizer): moved related code into a separate unit 2025-06-26 18:35:33 +01:00
drrtuy
e07e85b750 feat(optimizer): into derived CSEP rewrite with hardcoded tables 2025-06-26 18:35:33 +01:00
drrtuy
1baaf878d0 feat(optimizer): basic rewrite Union unit into Sub with union 2025-06-26 18:35:33 +01:00
drrtuy
021a95c683 feat(optimizer): rewrite rule refactoring 2025-06-26 18:35:33 +01:00
drrtuy
e73e5834ab feat(optimizer): first cut for rewrite foreign table into UNION rule 2025-06-26 18:35:33 +01:00
drrtuy
79008f4f69 feat(CSEP): CSEP printer with indentations to simplify reading + rewriter skeleton + some test binary to describe minimalistic CSEP localy 2025-06-26 18:35:33 +01:00
Serguey Zefirov
3a91cded27 chore(MCOL-6018) Fix incorrect Field_decimal cast
This is a fix of a problem found by UBSAN. MDB changed default type to
represent a decimal result, C-style cast did not do proper type checking
and this one-liner fixes that. Now we will have an assertion if type
changes again.
2025-06-26 19:41:58 +04:00
drrtuy
44d1698639 chore(plugin): move having and group by into separate routines 2025-06-02 12:11:41 +01:00
drrtuy
600f10c259 chore(plugin): move order by processing 2025-06-02 12:11:41 +01:00
drrtuy
bb13688ccf chore(plugin): move projection processing into a separate part. 2025-06-02 12:11:41 +01:00
Leonid Fedorov
dc4ca8d588 MCOL-5943: MCOL-4740 update rows counter for multi-table update (#3555)
* fix(plugin): MCOL-4740: This fixes update rows counter for multi-table update
For UPDATEs involving a single table, the server call to handler::direct_update_rows() is used to correctly set the count for the number of updated rows in the UPDATE statement.
However, for UPDATEs involving multi-tables, the server does not call handler::direct_update_rows(). This patch adds support to correctly report the number of updated rows to the client by setting
multi_update::updated and multi_update::found in handler::rnd_end().

* fix(plugin): MCOL-4740: this is to addres the original patch QA found in the original patch

---------

Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
Co-authored-by: drrtuy <roman.nozdrin@mariadb.com>
2025-05-29 14:23:37 +01:00
Aleksei Antipovskii
221ccfd5b6 fix(dbcon): MCOL-4756: having not() provokes an ERROR 2013
The `NOT()` function in the HAVING clause was handled
    incorrectly, which caused the server to crash.
2025-05-23 05:12:17 +04:00
Leonid Fedorov
bfe49a8345 bug(priproc) make last_day type a bit more accurate
This fixes discrepance with the server, which assigns DATE type to
last_day()'s result.

Now we also assigns DATE result type and, also, use proper
dataconvert::Day data structure to return date.

Tests agree with InnoDB.

Also, this patch includes test for MCOL-5669, to show we fixed it.
2025-05-23 05:12:17 +04:00
Sergey Zefirov
5f6080e09c fix(join): Fixes MCOL-5056, an error of joining TEXT column from InnoDB (#3160)
We incorrectly identified TEXT columns from external tables as BLOB.
Alexander Barkov suggested a way to discriminate them which I
implemented here.
2025-05-23 05:12:17 +04:00
Leonid Fedorov
0359a1fd3a chore(server_support): fixes to build columnstore branch with server >= 11.5 2025-05-22 17:14:38 +04:00
Leonid Fedorov
a0bee173f6 chore(build): fixes to satisfy clang19 warnings 2025-05-15 19:05:38 +04:00
Serguey Zefirov
bd1622f331 feat(MCOL-5886): support InnoDB's table partitions in cross-engine joins
The purpose of this changeset is to obtain list of partitions from
SELECT_LEX structure and pass it down to joblist and then to
CrossEngineStep to pass to InnoDB.
2025-04-23 08:24:10 +03:00
Leonid Fedorov
6a712dc0ad MCOL-5932: fix heap buffer overflow with minimal revert of MCOL-5776 breaking change (#3445) 2025-04-15 14:46:25 +01:00
Aleksei Antipovskii
4bea7e59a0 feat(PrimProc): MCOL-5852 disk-based GROUP_CONCAT & JSON_ARRAYAGG
* move GROUP_CONCAT/JSON_ARRAYAGG storage to the RowGroup from
  the RowAggregation*
* internal data structures (de)serialization
* get rid of a specialized classes for processing JSON_ARRAYAGG
* move the memory accounting to disk-based aggregation classes
* allow aggregation generations to be used for queries with
  GROUP_CONCAT/JSON_ARRAYAGG
* Remove the thread id from the error message as it interferes with the mtr
2025-04-11 15:21:07 +02:00
drrtuy
6b8adb822b chore(connector): remove unused and disabled group by handler (#3481) 2025-04-04 21:27:07 +01:00
Serguey Zefiov
6e539b8336 fix(MCOL-5889): Improper handle of DOUBLE result type with DECIMAL arguments
Sometimes server assigns DOUBLE type for arithmetic operations over
DECIMAL arguments. In this rare case width of result was incorrectly
adjusted and it triggered an assertion.

Now width of result gets adjusted only if result type is also DECIMAL.
2025-02-12 18:41:55 +04:00
Sergey Zefirov
60dc7550f1 fix(group by, having): MCOL-5776: GROUP BY/HAVING closer to server's (#3371)
This patch introduces an internal aggregate operator SELECT_SOME that
is automatically added to columns that are not in GROUP BY. It
"computes" some plausible value of the column (actually, last one
passed).

Along the way it fixes incorrect handling of HAVING being transferred
into WHERE, window function handling and a bit of other inconsistencies.
2024-12-20 19:11:47 +00:00
Serguey Zefirov
39a976c39a fix(ubsan): MCOL-5844 - iron out UBSAN reports
The most important fix here is the fix of possible buffer overrun in
DATEFORMAT() function. A "%W" format, repeated enough times, would
overflow the 256-bytes buffer for result. Now we use ostringstream to
construct result and we are safe.

Changes in date/time projection functions made me fix difference between
us and server behavior. The new, better behavior is reflected in changes
in tests' results.

Also, there was incorrect logic in TRUNCATE() and ROUND() functions in
computing the decimal "shift."
2024-12-10 20:30:58 +04:00
Sergey Zefirov
3bcc2e2fda fix(memory leaks): MCOL-5791 - get rid of memory leaks in plugin code (#3365)
There were numerous memory leaks in plugin's code and associated code.
During typical run of MTR tests it leaked around 65 megabytes of
objects. As a result they may severely affect long-lived connections.

This patch fixes (almost) all leaks found in the plugin. The exceptions
are two leaks associated with SHOW CREATE TABLE columnstore_table and
getting information of columns of columnstore-handled table. These
should be fixed on the server side and work is on the way.
2024-12-06 09:04:55 +00:00
drrtuy
6f6e69815d feat(bytestream,serdes): Distribute BS buf size data type change to avoid implicit data type narrowing 2024-11-08 16:28:51 +04:00
Sergey Zefirov
db4cb1d657 MCOL-4234 and MCOL 5772 cherry-picked into [stable 23.10] (#3226)
* MCOL-4234: improve GROUP BY and ORDER BY interaction (#3194)

This patch fixes the problem in MCOL-4234 and also generally improves
behavior of GROUP BY.

It does so by introducing a "dummy" aggregate and by wrapping columns
into it. This allows for columns that are not in GROUP BY to be used
more freely, for example, in SELECT * FROM tbl GROUP BY col - all
columns that are not "col" will be wrapped into an aggregate and query
will proceed to execution.

The dummy aggregate itself does nothing more than remember last value
passed into it.

There also an additional error message that tries to explain what types
of expressions can be wrapped into an aggregate.

* MCOL-5772: incorrect ORDER BY ordering for a columns not in GROUP BY (#3214)

When ORDER BY column is not in GROUP BY, is not an aggregate and there
is a SELECT column that is also not an aggregate, there was a problem:
ordering happened on the SELECTed column, not ORDERed one.

This patch fixes that particular problem and also performs some tidying
around newly added aggregate.

---------

Co-authored-by: Leonid Fedorov <79837786+mariadb-LeonidFedorov@users.noreply.github.com>
2024-06-28 00:31:53 +04:00