1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-04-29 21:36:45 +03:00

150 Commits

Author SHA1 Message Date
Leonid Fedorov
03b8b15fb2 cp1251 test 2024-03-13 15:12:08 +00:00
Leonid Fedorov
38c9b51a13 move convert to datatypes::Charset class 2024-03-13 13:24:26 +00:00
Leonid Fedorov
5104a7e1ba charsets support for regexp funcions 2024-03-11 15:45:34 +00:00
Leonid Fedorov
4c85b166ca utf-8 support for regexp functions 2024-03-11 15:45:34 +00:00
Leonid Fedorov
33b0fee5cd tests and review fixes 2024-03-11 15:45:34 +00:00
Leonid Fedorov
a9d98a8934 MTR 2024-03-11 15:45:34 +00:00
mariadb-KirillPerov
3e73af331d
Update mcs50_cpimport_stdin.test 2024-02-26 20:31:34 +06:00
Leonid Fedorov
fadb102712 fix(writeengine) MCOL-4202: use schema name when renaming table and change it's fields in syscat 2023-12-18 09:59:38 +03:00
Serguey Zefirov
1f958c9ed2 MCOL-5625: Fixes json_query implementation
Also extends func_json_value.test.
2023-12-12 15:45:03 +03:00
Leonid Fedorov
6b8c3dd918 Fix arrow linkage on external project 2023-11-30 01:47:13 +04:00
HanpyBin
fe597ec78c MCOL-5505 add parquet support for cpimport and add mcs_parquet_ddl and mcs_parquet_gen tools 2023-11-30 01:47:13 +04:00
Serguey Zefirov
792aea2a7c Fixes MCOL-5599 where LIKE operator never finishes
This is a fix of logging subsystem, nothing else.

The old code expanded an argument into string and advanced too little
and, if expansion contained argument's index, it expanded it again. And
again.
2023-11-29 19:17:16 +04:00
Sergey Zefirov
8632c85ecf
feat(primproc,aggregegation)!: Changes for ROLLUP with single-phase aggregation (#3025)
The fix is simple: enable subtotals in single-phase aggregation and
disable parallel processing when there are subtotals and aggregation is
single-phase.
2023-11-28 17:33:02 +03:00
Sergey Zefirov
9a84aa8d99
fix(plugin): Same columns fom different views in GROUP BY do not produce errors (#3035)
Fixes MCOL-5643.

The problem was that different views with same column names in GROUP BY
and on the SELECT clause produced an error about "projection column is
not an aggergate neither in GROUP BY list."

This was due to incorrect search in expressions's list that lead to
duplicate columns in GROUP BY list.
2023-11-28 17:30:56 +03:00
Sergey Zefirov
5c9770d1e6
fix(funcexp): MCOL-5607: JSON function use crashes query execution (#3028)
JSON functions were implemented violating an assumption of their
pureness, as they should not have any state. This concrete patch
fixes implementation of JSON_VALUE function.
2023-11-21 23:46:03 +03:00
Sergey Zefirov
69b8e1c779
feat(extent-elimination)!: re-enable extent-elimination for dictionary columns scanning
This is "productization" of an old code that would enable extent
elimination for dictionary columns.

This concrete patch enables it, fixes perfomance degradation (main
problem with old code) and also fixes incorrect behavior of cpimport.
2023-11-17 17:14:35 +03:00
Roman Nozdrin
f7045457f2 chore(datatypes): refactoring math ops results domain check functionality 2023-10-25 09:12:54 +00:00
Sergey Zefirov
84148cbe4c
fix(datatypes, funcexp): Overflow detection for MCOL-5568 use case (and some other) (#2987)
We add intermediate calculations in int128_t when target is UBIGINT and
check for overflow before converting into the UBIGINT. This is so
because we can overflow on addition and multiplication, with (some)
signed operands or both unsigned.
2023-10-16 16:55:02 +03:00
Sergey Zefirov
920607520c
feat(runtime)!: MCOL-678 A "GROUP BY ... WITH ROLLUP" support
Adds a special column which helps to differentiate data and rollups of
various depts and a simple logic to row aggregation to add processing of
subtotals.
2023-09-26 17:01:53 +03:00
Sergey Zefirov
4bfce51628
Fix autoincrement filtering problems with utf-8 (#2964)
MCOL-5572: Widen the autoincrement column to accomodate utf-8  encoded into weights with strnxfrm function.
2023-09-22 16:40:10 +03:00
Denis Khalikov
add3a57e8d
MCOL-5539 Put table on small side if it was involved in prev.join. (#2945) 2023-09-05 12:19:43 +03:00
Daniel Lee
1283f1fc4d
dlee mtr 23.08.1 (#2932)
* Added order by clause to keep results consistent over test runs

* Updated test result for the merging of MCOL-5519

* Updated test results for the merging of MCOL-4632

* Updated test result for the merging of MCOL-5519

* Added missing / to path

* Improved few tests cases

* Fixed test case name

---------

Co-authored-by: root <root@rocky8.localdomain>
2023-08-18 00:01:33 +03:00
drrtuy
f55d41c079
Merge pull request #2912 from tntnatbry/MCOL-5005
MCOL-5005 Add charset number to system catalog.
2023-08-15 22:22:21 +02:00
Gagan Goel
d50a0fa2e6 MCOL-5005 Add charset number to system catalog - Part 2.
1. Extend the calpontsys.syscolumn system catalog table
  with a new column, 'charsetnum'.

  'charsetnum' field is set to the 'number' member of the
  'charset_info_st' struct defined in the server in m_ctype.h.

  For CHAR/VARCHAR/TEXT column types, 'charset_info_st' is
  initialized to the charset/collation of the column, which
  is set at the column-level or at the table-level in the DDL.

  For BLOB/VARBINARY binary column types, 'charset_info_st' is
  initialized to my_charset_bin (charsetnum=63).

  For all other column types, charsetnum is set to 0.

  2. Add support for the newly added 'charsetnum' column in the
  automatic system catalog upgrade logic in dbbuilder.

  For existing table definitions, charsetnum for the column is
  defaulted to 0.

  3. Add MTR test case that creates a few table definitions with
  a range of charset/collation combinations and queries the
  calpontsys.syscolumn system catalog table with the charsetnum
  field for the columns in the table DDLs.
2023-08-15 17:21:47 +00:00
mariadb-AlexeyVorovich
64f1d541d0
MCOL-5519: new defaults in columnstore.cnf (#2894)
feat(charset)!: utf8 is a new charset default and utf8_general_ci is a new collation default in the engine configuration file shipped
---------

Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
Co-authored-by: mariadb-DanielLee <daniel.lee@mariadb.com>
2023-08-15 18:04:32 +03:00
Theresa Hradilak
48562e41f9 feat(datatypes): MCOL-4632 and MCOL-4648, fix cast leads to NULL.
Remove redundant cast.

As C-style casts with a type name in parantheses are interpreted as static_casts this literally just changes the interpretation around (and forces an implicit cast to match the return value of the function).

Switch UBIGINTNULL and UBIGINTEMPTYROW constants for consistency.

Make consistent with relation between BIGINTNULL and BIGINTEMPTYROW & make adapted cast behaviour due to NULL markers more intuitive. (After this change we can simply block the highest possible uint64_t value and if a cast results in it, print the next lower value (2^64 - 2). Previously, (2^64 - 1) was able to be printed, but (2^64 - 2) as being blocked by the UBIGINTNULL constant was not, making finding the appropiate replacement value to give out more confusing.

Introduce MAX_MCS_UBIGINT and MIN_MCS_BIGINT and adapt casts.

Adapt casting to BIGINT to remove NULL marker error.

Add bugfix regression test for MCOL 4632

Add regression test for mcol_4648

Revert "Switch UBIGINTNULL and UBIGINTEMPTYROW constants for consistency."

This reverts commit 83eac11b18937ecb0b4c754dd48e4cb47310f620.
Due to backwards compatability issues.

Refactor casting to MCS[U]Int to datatype functions.

Update regression tests to include other affected datatypes.

Apply formatting.

Refactor according to PR review

Remove redundant new constant, switch to using already existing constant.

Adapt nullstring casting to EMPTYROW markers for backwards compatability.

Adapt tests for backward compatability behaviour allowing text datatypes to be casted to EMPTYROW constant.

Adapt mcol641-functions test according to bug fix.

Update tests according to new expected behaviour.

Adapt tests to new understanding of issue.

Update comments/documentation for MCOL_4632 test.

Adapt to new cast limit logic.

Make bracketing consistent.

Adapt previous regression test to new expected behaviour.
2023-08-11 13:00:30 +00:00
Leonid Fedorov
65cde8c894
feature: pron (#2908)
* feature: Special dictionary, we can pass with session veriable to modify codepaths and behaviour for testing and debugging
2023-07-21 14:02:03 +03:00
Leonid Fedorov
77eedd1756
MCOL-5503: Fix broken quarter (#2855)
* Fix broken quarter function
2023-06-02 18:06:58 +03:00
Leonid Fedorov
8f93fc3623
MCOL-5493: First portion of UBSan fixes (#2842)
Multiple UB fixes
2023-06-02 17:02:09 +03:00
Mu He
7e2f83e39d
Merge branch 'mariadb-corporation:develop' into develop 2023-04-05 18:22:52 +02:00
MuHe03
d906974abc MCOL-4991 Solving TRUNCATE/ROUND/CEILING functions on TIME/DATETIME/TIMESTAMP
Add getDecimalVal in func_round and func_truncate for getting value while filtering

MCOL-4991 Solving TRUNCATE/ROUND/CEILING functions on TIME/DATETIME/TIMESTAMP

Update func_cast.cpp
2023-03-31 18:39:16 +02:00
Sergey Zefirov
b53c231ca6 MCOL-271 empty strings should not be NULLs (#2794)
This patch improves handling of NULLs in textual fields in ColumnStore.
Previously empty strings were considered NULLs and it could be a problem
if data scheme allows for empty strings. It was also one of major
reasons of behavior difference between ColumnStore and other engines in
MariaDB family.

Also, this patch fixes some other bugs and incorrect behavior, for
example, incorrect comparison for "column <= ''" which evaluates to
constant True for all purposes before this patch.
2023-03-30 21:18:29 +03:00
Sergey Zefirov
a2d7a21196
Fix "column1 LIKE column2" for columns from same table (#2776)
This WHERE part will be performed with FilterCommand, which did not had
LIKE operator implemented until now.
2023-03-17 12:10:53 +02:00
Roman Nozdrin
4d4e4ad30d
Merge pull request #2741 from mariadb-corporation/MDEV-25080-CS-dev
MDEV-25080 Allow pushdown of queries involving UNIONs in outer select to ColumnStore
2023-02-28 11:23:50 +00:00
Andrey Piskunov
b6808c97f1
MCOL-4530: common conjuction top rewrite (#2673)
Added logical transformation of the execplan::ParseTrees with the taking out the common factor in expression of the form "(A and B) or (A and C)" for the purposes of passing a TPCH 19 query.

Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
2023-02-27 19:23:19 +03:00
Gagan Goel
157d0f9b8c MDEV-25080 Add --sorted_result MTR qualifier to some UNION tests. 2023-02-27 06:38:31 -05:00
Gagan Goel
2f1f9c0ef0 MDEV-25080 Some fixes:
1. In TupleUnion::writeNull(), add the missing switch case for
   wide decimal with 16bytes column width.
2. MCOL-5432 Disable complete/partial pushdown of UNION operation
   if the query involves an ORDER BY or a LIMIT clause, until
   MCOL-5222 is fixed. Also add MTR test cases for this.
2023-02-27 06:38:31 -05:00
Gagan Goel
e987a766e0 MDEV-25080 Some updates to the MDEV-25080 MTR test cases. 2023-02-27 06:38:31 -05:00
Gagan Goel
86dcf92d56 MCOL-5215 Fix overflow of UNION operation involving DECIMAL datatypes.
When a UNION operation involving DECIMAL datatypes with scale and digits
before the decimal exceeds the currently supported maximum precision
of 38, we throw an error to the user:
"MCS-2060: Union operation exceeds maximum DECIMAL precision of 38".

This is until MCOL-5417 is implemented where ColumnStore will have
full parity with MariaDB server in terms of maximum supported DECIMAL
precision and scale of 65 and 38 digits respectively.
2023-02-27 06:38:31 -05:00
Gagan Goel
45a779f743 MDEV-25080 Implement ColumnStore-side changes for pushdown of SELECT_LEX_UNITs. 2023-02-27 06:38:31 -05:00
Denis Khalikov
242bc75166 MCOL-5195 Correlated subquery with equi/non-equi scalar filter and join condition
Disable check for correlated subqueries, basically those types of queries transforms
to join (aggr(table2), table1), table2) and post join scalar filter.
2022-12-23 18:33:01 +03:00
root
2ed151fa59 Recorded reference results 2022-12-09 02:24:40 +00:00
root
f090e65298 More changes 2022-12-08 21:00:01 +00:00
mariadb-DanielLee
fad0fd08f0 Disable warnings for 'drop if exists' and 'create if not exist' commands 2022-12-08 11:37:44 -06:00
root
c0d772559e Moved one more test case to 1pmonly folder 2022-12-02 14:30:43 +00:00
root
814fc37081 Moved some MTR cases to a new 1pmonly directory. Also added order by clause to few cases 2022-11-30 23:40:41 +00:00
mariadb-AndreyPiskunov
8b4be1b7f9 func json_arrayagg test adjustments 2022-11-02 16:07:45 +02:00
mariadb-AndreyPiskunov
33b09f5499 Add truncate for result consistency 2022-10-31 20:38:08 +03:00
mariadb-AndreyPiskunov
d7f4ec73c5 Small fixes + test sorting 2022-10-31 14:56:32 +02:00
mariadb-AndreyPiskunov
315e4be2d8 First working attempt for json_arrayagg 2022-10-31 14:56:32 +02:00