1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-06-12 05:01:56 +03:00
Commit Graph

197 Commits

Author SHA1 Message Date
3d068a7263 chore(codemanagement, ci): better coredumps handling, deps fixed 2025-06-09 11:56:32 +00:00
edfec4220a chore(codemanagement, build): build refactoring stage2 2025-06-09 11:56:32 +00:00
aa7e0fb9b4 Deep build refactoring phase 1 (#3562)
* configcpp refactored
* logging and datatypes refactored

* more dataconvert
* chore(build): massive removals, auto add files to debian install file
* chore(codemanagement): nodeps headers, potentioal library
* chore(build): configure before autobake
* chore(build): use custom cmake commands for components, mariadb-plugin-columnstore.install generated
* chore(build): install deps as separate step for build-packages
* more deps
* check  debian/mariadb-plugin-columnstore.install automatically
* chore(build): add option for multibracnh compilation
* Fix warning
2025-05-30 14:05:21 +04:00
3bb2496ca1 fix: MCOL-5755: incorrect handling of BLOB (and TEXT) in GROUP BY
BLOB fields did not work as grouping keys at all, they were assigned
value NULL for any value, be it NULL or not. The fix is in the
rowaggregation.cpp in the initMapping(), a switch/case branch was added
to handle BLOB field copying there.

Also, TEXT columns did not distinguish between NULL and empty string in
the grouping algorithm, now they do. The fix is in the equals()
function, now we specifically check for isNull() equality between
values.
2025-05-23 05:12:17 +04:00
6db2dc668f stubs and cmake formatting 2025-05-20 18:22:59 +04:00
2036e521c7 named linkage 2025-05-20 18:22:59 +04:00
af1864f544 feat(memory): custom OOM exception 2025-05-20 18:22:16 +04:00
16303bef2b chore(build): clang-20 warnings fixed 2025-05-15 19:05:38 +04:00
fc028983b9 chore(build): more fixes for the new warnings 2025-05-15 19:05:38 +04:00
a0bee173f6 chore(build): fixes to satisfy clang19 warnings 2025-05-15 19:05:38 +04:00
d434b6587a fix(perf,allocator): increased CountingAllocator checkpoint step size up
to 100K
2025-05-02 10:11:40 +01:00
42417764d8 chore(): cleanup. 2025-05-02 10:11:40 +01:00
4d32a59775 fix(perf,allocator): reduce CountingAllocator step size to improve its memory consumption reaction speed. 2025-05-02 10:11:40 +01:00
01cc73d416 fix(perf,allocator): test build with reduced CountingAllocator parameter values. 2025-05-02 10:11:40 +01:00
252b3ddbef fix(perf,allocator): adding unit test to check CountingAllocator behavior when it is wrapped in STLPoolAllocator 2025-05-02 10:11:40 +01:00
3fd90d3270 chore(compilation): static analyser flag went mad. 2025-03-27 22:12:48 +00:00
729db829a2 feat(RM,allocators): use atomic counters math abstraction 2025-03-27 22:12:48 +00:00
a4c4d33ee7 feat(): zerocopy TNS case and JOIN results RGData with CountingAllocator 2025-03-27 22:12:48 +00:00
3dfc8cd454 feat(): first cleanup 2025-03-27 22:12:48 +00:00
4c1d9bceb7 feat(): Replacing STLPoolAllocator with CountingAllocator for in-memory joins 2025-03-27 22:12:48 +00:00
bb4cd40ca4 feat(): aggregating CountingAllocator 2025-03-27 22:12:48 +00:00
90b4322470 feat(): propagated changes into SLTPoolAllocator and friends 2025-03-27 22:12:48 +00:00
a6de8ec1ac feat(): dangling pointer/ref issue has been solved for both RGData and BS 2025-03-27 22:12:48 +00:00
397b3ff729 feat(): change ref to atomic with ptr to atomic 2025-03-27 22:12:48 +00:00
4e86123a5a feat(): use boost::make_shared b/c most distros can't do allocate_shared for array types. 2025-03-27 22:12:48 +00:00
5f1bd3be12 feat(RGData,StringStore): add counting allocator capabilities to those ctors used in BPP::execute() 2025-03-27 22:12:48 +00:00
02b8ea1331 feat(PP,ByteStream): new counting memory allocator 2025-03-27 22:12:48 +00:00
0ab03c7258 chore(codestyle): mark virtual methods as override 2025-02-21 20:01:34 +04:00
88e80c1542 fix(utils): MCOL-5881 set/getThreadName use FreeBSD API (#3384)
Taken from FreeBSD ports, this uses the FreeBSD
APIs rather than the Linux specific prctl to change
and retreive the thread names.

Co-authored-by: Bernard Spil <brnrd@FreeBSD.org>
2025-02-20 16:32:59 +00:00
eaba4d33b4 fix(DEC):MCOL-5805,5808 to resolve UM-only node crash inside DEC when there is no local PP to send the local requests to. (#3350)
* Revert "fix(DEC): MCOL-5602 fixing potentially endless loop in DEC (#3049)"

This reverts commit 1d416bc6ed.

* fix(DEC):MCOL-5805,5808 to resolve UM-only node crash inside DEC when there is no local PP to send the local requests to.
2024-11-11 18:31:15 +00:00
4b411b3968 MCOL-4696: get rid of boost::iequals 2024-08-21 21:35:52 +04:00
d089171637 fix(compilation): add explicit comparison operators and explicitly cast a type (#3229) 2024-06-29 07:53:54 +01:00
6c6fa7d5a4 MCOL-5328: PCRE based regexp regexp_substr regexp_instr regexp_replace [stable-23.10] (#3215)
* MCOL-5328: PCRE based regexp regexp_substr regexp_instr regexp_replace

* Add qa test for MCOL-5328

---------

Co-authored-by: Susil Behera <susil.behera@mariadb.com>
2024-06-27 14:20:08 +04:00
cce0f6ab0c fix(cgroups)!: Containers memory limits for CI (#3108) (#3209)
Limit test containers by memory, fix cgroup path inside the containers by introducing new ugly setting name 

---------

Co-authored-by: drrtuy <roman.nozdrin@mariadb.com>
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2024-06-20 10:48:20 +01:00
1d416bc6ed fix(DEC): MCOL-5602 fixing potentially endless loop in DEC (#3049) 2023-12-05 18:39:24 +03:00
2b01ab1b15 fix(primproc,stringstore): MCOL-5597 Set length for nullptr string to 0. (#3034) 2023-11-23 15:37:37 +03:00
fd94ab5042 chore(logging): move cgroup /cgroup version log from constructor to getTotalMemory to avoid duplicate log as constructor is called per query 2023-09-25 22:17:09 +03:00
7f9c624626 MCOL-5573 Fix cpimport truncation of TEXT columns.
1. Restore the utf8_truncate_point() function in utils/common/utils_utf8.h
that I removed as part of the patch for MCOL-4931.

2. As per the definition of TEXT columns, the default column width represents
the maximum number of bytes that can be stored in the TEXT column. So the
effective maximum length is less if the value contains multi-byte characters.
However, if the user explicitly specifies the length of the TEXT column in a
table DDL, such as TEXT(65535), then the DDL logic ensures that enough number
of bytes are allocated (upto a system maximum) to allow upto that many number
of characters (multi-byte characters if the charset for the column is multi-byte,
such as utf8mb3).
2023-09-20 12:23:22 -04:00
931f2b36a1 MCOL-4931 Make cpimport charset-aware. (#2938)
1. Extend the following CalpontSystemCatalog member functions to
   set CalpontSystemCatalog::ColType::charsetNumber, after the
   system catalog update to add charset number to calpontsys.syscolumn
   in MCOL-5005:
     CalpontSystemCatalog::lookupOID
     CalpontSystemCatalog::colType
     CalpontSystemCatalog::columnRIDs
     CalpontSystemCatalog::getSchemaInfo

2. Update cpimport to use the CHARSET_INFO object associated with the
   charset number retrieved from the system catalog, for a
   dictionary/non-dictionary CHAR/VARCHAR/TEXT column, to truncate
   long strings that exceed the target column character length.

3. Add MTR test cases.
2023-09-05 17:17:20 +03:00
5b4f06bf0d Logging of memory (#2930)
* -logging of memory WIP

* -better log for cgroup case

* -fix log

* -display in GIB

* add log for freememory for non CGROUP
(to be discussed)

* test repeated log entries

* -added counter for every 1000 call. effectivly 15m

* Name logginng period and inrease it, clear config files from PR, add .gitignore

---------

Co-authored-by: pgmabv99 <alexey.vorovich@gmail.com>
Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
2023-09-05 15:46:29 +03:00
48562e41f9 feat(datatypes): MCOL-4632 and MCOL-4648, fix cast leads to NULL.
Remove redundant cast.

As C-style casts with a type name in parantheses are interpreted as static_casts this literally just changes the interpretation around (and forces an implicit cast to match the return value of the function).

Switch UBIGINTNULL and UBIGINTEMPTYROW constants for consistency.

Make consistent with relation between BIGINTNULL and BIGINTEMPTYROW & make adapted cast behaviour due to NULL markers more intuitive. (After this change we can simply block the highest possible uint64_t value and if a cast results in it, print the next lower value (2^64 - 2). Previously, (2^64 - 1) was able to be printed, but (2^64 - 2) as being blocked by the UBIGINTNULL constant was not, making finding the appropiate replacement value to give out more confusing.

Introduce MAX_MCS_UBIGINT and MIN_MCS_BIGINT and adapt casts.

Adapt casting to BIGINT to remove NULL marker error.

Add bugfix regression test for MCOL 4632

Add regression test for mcol_4648

Revert "Switch UBIGINTNULL and UBIGINTEMPTYROW constants for consistency."

This reverts commit 83eac11b18937ecb0b4c754dd48e4cb47310f620.
Due to backwards compatability issues.

Refactor casting to MCS[U]Int to datatype functions.

Update regression tests to include other affected datatypes.

Apply formatting.

Refactor according to PR review

Remove redundant new constant, switch to using already existing constant.

Adapt nullstring casting to EMPTYROW markers for backwards compatability.

Adapt tests for backward compatability behaviour allowing text datatypes to be casted to EMPTYROW constant.

Adapt mcol641-functions test according to bug fix.

Update tests according to new expected behaviour.

Adapt tests to new understanding of issue.

Update comments/documentation for MCOL_4632 test.

Adapt to new cast limit logic.

Make bracketing consistent.

Adapt previous regression test to new expected behaviour.
2023-08-11 13:00:30 +00:00
6d44d2e850 MCOL-5500 Remove another noisy printout. (#2886)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-06-29 15:46:00 +03:00
79b636d853 MCOL-5500 Remove noisy printout from CGroupConfigurator method 2023-06-19 11:24:41 +00:00
375d162376 MCOL-5500 This patch adds cgroup v2 support with some sanity checks for (#2849)
values reported by cgroups v1
2023-06-09 17:37:21 +03:00
c2d0fa24da replace boost::shared_array<T> to std::shared_ptr<T[]> 2023-04-14 10:33:27 +00:00
a508b86091 remove boost/shared_array include 2023-04-14 09:42:50 +00:00
c38d98a510 Merge pull request #2762 from mariadb-corporation/MCOL-5191_Dist
MCOL-5191 Refacator statistics.
2023-04-06 21:09:42 +01:00
2e1394149b MCOL-5464: Fixes of bugs from ASAN warnings, part one (#2792)
* Fixes of bugs from ASAN warnings, part one

* MQC as static library, with nifty counter for global map and mutex

* Switch clang to 16

* link messageqcpp to execplan
2023-04-04 02:33:23 +03:00
b53c231ca6 MCOL-271 empty strings should not be NULLs (#2794)
This patch improves handling of NULLs in textual fields in ColumnStore.
Previously empty strings were considered NULLs and it could be a problem
if data scheme allows for empty strings. It was also one of major
reasons of behavior difference between ColumnStore and other engines in
MariaDB family.

Also, this patch fixes some other bugs and incorrect behavior, for
example, incorrect comparison for "column <= ''" which evaluates to
constant True for all purposes before this patch.
2023-03-30 21:18:29 +03:00
bca300cd11 MCOL-5191 Refacator statistics.
Move uniform distribution to Statitistics constructor, remove rowcount.
2023-03-06 15:06:40 +03:00