1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2026-10-23 21:49:05 +03:00
Commit Graph

430 Commits

Author SHA1 Message Date
Aleksei Antipovskii
cdb6c8a29f fix(writeengine): overwrite command name 2026-09-25 17:16:09 +02:00
Vasily Kozhukhovskiy
9a9f371f02 MCOL-6442 Fix memory leak of IDBDataFile* in cpimport on abort 2026-07-23 15:34:17 +02:00
Vasily Kozhukhovskiy
bb5f7addc8 chore(infer): MCOL-6423 re-add USE_AFTER_DELETE suppressions for node-based containers
The nightly scan showed the loop refactors fix the false positives for
std::vector but not for std::list/std::set, which Infer has no precise
model for. Re-add targeted @infer-ignore comments at the five surviving
sites (threadpool.h, we_chunkmanager.cpp x2, we_dctnry.cpp,
lbidresourcegraph.cpp), keeping the refactored loops.
2026-07-23 15:20:58 +02:00
Vasily Kozhukhovskiy
a3d167a22b chore(infer): MCOL-6423 refactor loops instead of suppressing USE_AFTER_DELETE
Per review feedback, replace the @infer-ignore comments with refactors
that remove the patterns Infer trips on:

- destructor delete-loops: use range-based for
  (altertable, tabledef, calpontsystemcatalog, threadpool, we_chunkmanager)
- we_dctnry freeStringCache(): iterate by const reference, dropping the
  local Signature copy and its dead store
- LBIDResourceGraph::releaseResources(): scope rNode per iteration and
  advance the iterator before the erase, removing the dummy_sit juggling
2026-07-22 15:36:17 +02:00
Vasily Kozhukhovskiy
628c1fe1fe chore(infer): MCOL-6423 suppress false-positive USE_AFTER_DELETE Infer findings 2026-07-22 15:36:17 +02:00
Vasily Kozhukhovskiy
5eb1644166 chore(infer): MCOL-6428 suppress false-positive MEMORY_LEAK_C in Cache::init() 2026-07-22 09:33:27 +02:00
Akhmad O.
d101692e77 fix(MCOL-4628): SET double_col=decimal_col returns an error (#3531) 2026-04-29 10:45:55 +01:00
Serguey Zefirov
9c3452797c fix(EM rebuild): MCOL-6302, MCOL-6321 - restore EM in case of header overflow, with test
This patch adds logic to read block offsets of a dictionary column
from tokens in a pseudo-column. These block offsets then are used to
restore LBIDs in extent map and set proper HWM for a proper LBID.

An accompanying test tests rebuild process and integrity of the
restored data.
2026-04-08 22:07:39 +04:00
Leonid Fedorov
72762d043b Fix Infer reports #1484-#1494: Initialize dataFbo/dataBio in writeRow* functions
#1484,#1487,#1490,#1492 we_colop.cpp - dataFbo uninitialized
#1485,#1486,#1488,#1489,#1491,#1493,#1494 we_colop.cpp - dataBio uninitialized

Both are output params from calculateRowId(), but initializing to 0
ensures safety if the while loop condition changes.
2026-04-08 21:55:13 +04:00
Leonid Fedorov
5c95ece4c5 Fix Infer report #1482: Initialize rc in BRMWrapper::writeVB
#1482 we_brm.cpp:1723 Uninitialized Value - rc could be read without init
if skipBeginVBCopy is true and freeList is empty (loop doesn't execute).
2026-04-08 21:55:13 +04:00
Leonid Fedorov
b1a2092386 Fix Infer reports #1474-#1479: Initialize DDLColumn.oid and DctnryTuple fields
#1474 we_ddlcommandproc.cpp:2064 Uninitialized Value - column.oid
#1475 we_ddlcommandproc.cpp:2253 Uninitialized Value - column.oid
#1476 we_ddlcommandproc.cpp:2684 Uninitialized Value - column.oid
#1477 we_ddlcommandproc.cpp:2713 Uninitialized Value - DctnryTuple.sigValue
#1478 we_ddlcommandproc.cpp:3842 Uninitialized Value - column.oid
#1479 we_ddlcommandproc.cpp:3943 Uninitialized Value - DctnryTuple.sigValue

Added default constructors to DDLColumn (both in we_ddlcommon.h and
ddlpackageprocessor.h) and DctnryTuple in we_type.h.
2026-04-08 21:55:13 +04:00
Leonid Fedorov
1abfd09c12 Fix Infer reports #1468-#1470: Initialize fields in ColExtInfEntry and output params
#1468 we_colextinf.cpp:80 Uninitialized Value - ColExtInfEntry constructors
missed initializing fbigMinVal/fbigMaxVal or fMinVal/fMaxVal depending on
which constructor was called. Fixed all 4 non-default constructors.
#1469 we_columninfo.cpp:631 Uninitialized Value - startLbid output param
#1470 we_columninfocompressed.cpp:191 Uninitialized Value - startFileOffset
2026-04-08 21:55:13 +04:00
Sergey Zefirov
086044c4c1 fix(ASAN): MCOL-5604 - fix ASAN diagnostics (#3913)
This simple patch removes use of type handler of column operations which
can be incorrect.
2026-03-01 10:12:41 +00:00
Sergey Zefirov
a4044413b9 fix(cpimport): MCOL-6243 - cpimport fails at fields longer than 1M (#3910)
* fix(cpimport): MCOL-6243 - cpimport fails at fields longer than 1M

This patch makes cpimport buffers resizeable, thus cpimport's
command-line buffer size argument is nothing but a hint now.

* No read past EOF
2026-01-28 11:19:10 +00:00
Leonid Fedorov
2f0f5a79b6 fix(correctness): replace volatiles with atomics 2025-10-31 15:29:58 +04:00
Leonid Fedorov
7d23514315 fix(perf): add reserves for usage of vector filled after creation 2025-10-29 17:11:17 +04:00
Serguey Zefirov
bb631dcffb feat(JSON,data_type): MCOL-6197 - support for JSON type
This patch does exactly this, it implements support for JSON in DDL.

Right now we use server's check for JSON validity on INSERT. We do not implement
JSON validity check during updates, it is postponed for later work.
2025-10-17 11:48:46 +03:00
Serguey Zefirov
a9f23729a9 feat(MCOL-6082) Cluster with read-only nodes should correctly work with DML
This patch changes logic from counting all nodes to counting only
read-write nodes when messaging about DML operations.

feat(MCOL-6082): Multiple readers of dbroots using OamCache logic

This patch introduces centralized logic of selecting what dbroot is
accessible in PrimProc on what node. The logic is in OamCache for time
being and can be moved later.

Fix build
2025-09-26 21:55:10 +04:00
Leonid Fedorov
0f3f73ddb5 fix(crash): libc++ + clang generate segfault on addressing null pointer here 2025-08-04 19:58:49 +04:00
Leonid Fedorov
8a535e872c Revert "Deep build refactoring phase 2 (#3564)" (#3678)
This reverts commit 449029a827.
2025-08-01 02:56:31 +04:00
Leonid Fedorov
82421c208f chore(ci): collect asan ubsan and libc++ build with mtr and regression status ignored (#3672)
* MSan added with fixes for libc++

* libc++ sepatare build

* add libc++ to ci

* libstdc++ in CI

* libcpp and msan to external projects

* std::sqrt

* awful_hack(ci): install whole llvm instead of libc++ in terrible way for test containers

* Adding ddeb packages for teststages and repos

* libc++ more for test container

* save some money on debug

* colored coredumps

* revert ci

* chore(ci): collect asan ubsan and libc++ build with mtr and regression status ignored
2025-07-31 00:32:32 +04:00
Kristina Pavlova
72eb0bbaf9 chore(cpimport): MCOL-6033 Change batch max size (#3652) 2025-07-27 20:10:27 +01:00
Leonid Fedorov
449029a827 Deep build refactoring phase 2 (#3564)
* configcpp refactored

* chore(build): massive removals, auto add files to debian install file

* chore(build): configure before autobake

* chore(build): use custom cmake commands for components, mariadb-plugin-columnstore.install generated

* chore(build): install deps as separate step for build-packages

* more deps

* chore(codemanagement, build): build refactoring stage2

* chore(safety): Locked Map for MessageqCpp with a simpler way

 Please enter the commit message for your changes. Lines starting

* chore(codemanagement, ci): better coredumps handling, deps fixed

* Delete build/bootstrap_mcs.py

* Update charset.cpp (add license)
2025-07-17 16:14:10 +04:00
Leonid Fedorov
05bbfcae57 One more try to fix autobake build 2025-07-16 06:59:55 +04:00
Leonid Fedorov
11e0492561 chore(build): replace contains method of map with count for gcc 8.5 2025-07-15 23:54:46 +04:00
Aleksei Antipovskii
1ce46b5e0b feature(cpimport): MCOL-5164 ignore all errors (-e all) 2025-07-15 20:52:29 +04:00
Aleksei Antipovskii
7dca1da8f2 fix(cpimport): MCOL-4882 fix -L arg default value 2025-07-13 21:33:59 +04:00
Alexey Antipovsky
78c1b5034d Feature/mcol 4882 cpimport skip rows (#3594)
* feat(cpimport): MCOL-4882 add a parameter to skip header rows

* chore(cpimport): MCOL-4882 Use boost::program_options to arguments parsing

* feat(cpimport.bin): MCOL-4882 Add missing changes

* add test

* fix clang

* add missing cmdline argument

* fix bug

* Fix double lines skipping

* Fix incorrect --silent (-N) parsing

* fix default --max-errors processing

* fix overwriting default username

* move initialization to members declaration
2025-07-11 23:35:43 +04:00
mariadb-KristinaPavlova
d031239844 add args to lost connection error when getting system data 2025-07-04 19:57:49 +04:00
mariadb-KristinaPavlova
6831b82515 fix err syntax - disk join 2025-07-04 19:57:49 +04:00
mariadb-KristinaPavlova
52351f2f29 add args to try larger read buffer error 2025-07-04 19:57:49 +04:00
mariadb-KristinaPavlova
3083eb6d95 add args to max file system disk usage error 2025-07-04 19:57:49 +04:00
Leonid Fedorov
aa7e0fb9b4 Deep build refactoring phase 1 (#3562)
* configcpp refactored
* logging and datatypes refactored

* more dataconvert
* chore(build): massive removals, auto add files to debian install file
* chore(codemanagement): nodeps headers, potentioal library
* chore(build): configure before autobake
* chore(build): use custom cmake commands for components, mariadb-plugin-columnstore.install generated
* chore(build): install deps as separate step for build-packages
* more deps
* check  debian/mariadb-plugin-columnstore.install automatically
* chore(build): add option for multibracnh compilation
* Fix warning
2025-05-30 14:05:21 +04:00
Leonid Fedorov
4a8f5830fd chore(build) update boost to 1.88 2025-05-20 21:43:43 +04:00
Leonid Fedorov
127dd09a7a fix back statics 2025-05-20 18:22:59 +04:00
Leonid Fedorov
6db2dc668f stubs and cmake formatting 2025-05-20 18:22:59 +04:00
Leonid Fedorov
2036e521c7 named linkage 2025-05-20 18:22:59 +04:00
Leonid Fedorov
a1019b7c0e chore(build): refactor main CMakeLists.txt (#3543)
* chore(build): refactor main CMakeLists.txt

* chore(build): fix boost version for packages, set clang-20 only for amd and arm

* chore(build): boost 4 sm

* chore(build): boost dep for rowgroup

* chore(build): toolset for boost

* chore(build): suppress clang warnings for boost

* chore(ci, build): use ASAN for unittest on ubuntu 24.04 only, added custom cmake flag option for bootstrap, custom params for new and existing pipelines

* chore(build): sort bootstrap flags

* chore(CI): remove publish pkg step, adding clickable link instead to publish steps, fix customenv
2025-05-20 05:00:48 +04:00
Leonid Fedorov
a0bee173f6 chore(build): fixes to satisfy clang19 warnings 2025-05-15 19:05:38 +04:00
Denis Khalikov
80277a5e7d DBBuilder: check syscat files instead dirs 2025-04-01 19:53:28 +04:00
drrtuy
02b8ea1331 feat(PP,ByteStream): new counting memory allocator 2025-03-27 22:12:48 +00:00
drrtuy
87981483e2 chore(compilation): to resolve unexpected boost warnings 2025-03-27 21:35:11 +04:00
Aleksei Antipovskii
0ab03c7258 chore(codestyle): mark virtual methods as override 2025-02-21 20:01:34 +04:00
Sergey Zefirov
3bcc2e2fda fix(memory leaks): MCOL-5791 - get rid of memory leaks in plugin code (#3365)
There were numerous memory leaks in plugin's code and associated code.
During typical run of MTR tests it leaked around 65 megabytes of
objects. As a result they may severely affect long-lived connections.

This patch fixes (almost) all leaks found in the plugin. The exceptions
are two leaks associated with SHOW CREATE TABLE columnstore_table and
getting information of columns of columnstore-handled table. These
should be fixed on the server side and work is on the way.
2024-12-06 09:04:55 +00:00
drrtuy
6f6e69815d feat(bytestream,serdes): Distribute BS buf size data type change to avoid implicit data type narrowing 2024-11-08 16:28:51 +04:00
Leonid Fedorov
5824a0cebe MCOL-5746 Cpimport: convert blob data from ascii hex when reads from STDIN (#3217)
Co-authored-by: Denis Khalikov <dennis.khalikov@gmail.com>
2024-06-27 14:23:07 +04:00
Leonid Fedorov
3f8758ba4c MCOL-5747 gcc-14.1.1 compile error - calloc - transposed args (#3216)
The arguments of calloc are the number of members and the sizeof the
member. Gcc-14.1.1 worked out how to tell the difference.

We correct this by transposing to gcc's will.

Co-authored-by: Daniel Black <daniel@mariadb.org>
2024-06-27 14:21:26 +04:00
Serguey Zefirov
2cd8f716c1 Fix MCOL-5035, a difference in INSERT and UPDATE behavior
The UPDATE statement wrote NULL when the column set is DATETIME and
value is '0000-00-00 00:00:00'. The problem was inside WriteEngine's
handling of UPDATE statements and this is where heart of change is.

Other changes are related to some obsolete data structures in DML/DDL
handling that just hanging around there, doing nothing.
2024-06-27 13:07:49 +03:00
Denis Khalikov
55ffacf546 MCOL-5555 Add support for startreadonly command. (#3081)
This patch adds support for `startreadonly` command which waits
until all active cpimport jobs are done and then puts controller node to readonly
mode.
2023-12-26 15:11:33 +04:00
Denis Khalikov
6315546557 Revert "MCOL-5555 Add support for startreadonly command."
This reverts commit 441cd9d34f.
2023-12-20 01:05:37 +03:00