1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-29 08:21:15 +03:00
Commit Graph

187 Commits

Author SHA1 Message Date
dd3ce850b7 add filename to ExtentMap::save():open err log 2025-07-04 19:57:49 +04:00
c4d60f66c4 fix(asan): fix ASAN warnings spoted 2025-06-13 23:36:10 +01:00
aa7e0fb9b4 Deep build refactoring phase 1 (#3562)
* configcpp refactored
* logging and datatypes refactored

* more dataconvert
* chore(build): massive removals, auto add files to debian install file
* chore(codemanagement): nodeps headers, potentioal library
* chore(build): configure before autobake
* chore(build): use custom cmake commands for components, mariadb-plugin-columnstore.install generated
* chore(build): install deps as separate step for build-packages
* more deps
* check  debian/mariadb-plugin-columnstore.install automatically
* chore(build): add option for multibracnh compilation
* Fix warning
2025-05-30 14:05:21 +04:00
806c3abc78 feat(BRM,oids): add a tool to reclaim OID space parts. (#3524)
* feat(BRM,oids): add a tool to reclaim OID space parts.


---------

Co-authored-by: = <=>
Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
Co-authored-by: Leonid Fedorov <79837786+mariadb-LeonidFedorov@users.noreply.github.com>
2025-05-23 05:53:09 +04:00
4a8f5830fd chore(build) update boost to 1.88 2025-05-20 21:43:43 +04:00
6db2dc668f stubs and cmake formatting 2025-05-20 18:22:59 +04:00
2036e521c7 named linkage 2025-05-20 18:22:59 +04:00
fc028983b9 chore(build): more fixes for the new warnings 2025-05-15 19:05:38 +04:00
a0bee173f6 chore(build): fixes to satisfy clang19 warnings 2025-05-15 19:05:38 +04:00
727627a012 fix(BRM): removes load_brm extra printouts (#3489) 2025-04-14 15:14:09 +01:00
0ab03c7258 chore(codestyle): mark virtual methods as override 2025-02-21 20:01:34 +04:00
d5dcae52cb fix(BRM): MCOL-5879 DBRM::clearShm runs crit sections w/o sync mechanism (#3391) 2025-02-20 16:33:44 +00:00
9763b12651 MCOL-5880: get rid of CLI11 dep in favour of boost::program_options 2025-01-14 14:58:22 +04:00
2b404fff1f fix(syscat): MCOL-5816 23.02 -> 23.10 upgrade issues (#3346)
* feat(BRM,tools): couple utilities to watch/operate shared memory locks and extent map

* feat(BRM,tools): merged two utilities and added some extra dbbuilder output in case of upgrade

* fix(dbbuilder): extra output to log upgrade detection.
2024-12-23 23:01:55 +04:00
e0a01c6cf4 Reapply "fix(aggregation, disk-based) MCOL-5691 distinct aggregate disk based (#3145)"
This reverts commit a5c12b98d7.
2024-12-11 12:02:24 +00:00
3bcc2e2fda fix(memory leaks): MCOL-5791 - get rid of memory leaks in plugin code (#3365)
There were numerous memory leaks in plugin's code and associated code.
During typical run of MTR tests it leaked around 65 megabytes of
objects. As a result they may severely affect long-lived connections.

This patch fixes (almost) all leaks found in the plugin. The exceptions
are two leaks associated with SHOW CREATE TABLE columnstore_table and
getting information of columns of columnstore-handled table. These
should be fixed on the server side and work is on the way.
2024-12-06 09:04:55 +00:00
6f6e69815d feat(bytestream,serdes): Distribute BS buf size data type change to avoid implicit data type narrowing 2024-11-08 16:28:51 +04:00
9e9ac32879 fix(logging): MCOL-5789 Add logs on failure. (#3307) 2024-09-10 17:43:25 +03:00
a5c12b98d7 Revert "fix(aggregation, disk-based) MCOL-5691 distinct aggregate disk based (#3145)"
This reverts commit c7caa4374f.
2024-07-07 13:09:56 +00:00
c7caa4374f fix(aggregation, disk-based) MCOL-5691 distinct aggregate disk based (#3145)
* fix(aggregation, disk-based): MCOL-5689 this fixes disk-based distinct aggregation functions
Previously disk-based distinct aggregation functions produced incorrect results b/c there was no finalization applied for previous generations stored on disk.

*  fix(aggregation, disk-based): Fix disk-based COUNT(DISTINCT ...) queries. (Case 2). (Distinct & Multi-Distinct, Single- & Multi-Threaded).

* fix(aggregation, disk-based): Fix disk-based DISTINCT & GROUP BY queries. (Case 1). (Distinct & Multi-Distinct, Single- & Multi-Threaded).

---------

Co-authored-by: Theresa Hradilak <theresa.hradilak@gmail.com>
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2024-03-24 18:04:37 +03:00
14882dd617 feat(brm): MCOL-5623 This patch changes the logic applied to save function for EM (#3080) 2023-12-27 09:47:58 +03:00
575852d2e0 feat(brm): MCOL-5631 Optimize BRM load operation. (#3090) 2023-12-27 09:45:11 +03:00
55ffacf546 MCOL-5555 Add support for startreadonly command. (#3081)
This patch adds support for `startreadonly` command which waits
until all active cpimport jobs are done and then puts controller node to readonly
mode.
2023-12-26 15:11:33 +04:00
6315546557 Revert "MCOL-5555 Add support for startreadonly command."
This reverts commit 441cd9d34f.
2023-12-20 01:05:37 +03:00
441cd9d34f MCOL-5555 Add support for startreadonly command.
This patch adds support for `startreadonly` command which waits
until all active cpimport jobs are done and then puts controller node to readonly
mode.
2023-10-16 16:11:50 +03:00
fce971dcd2 feat(IO): MCOL-5555 Reduce number of direct writes to the file storing BRM journal and tablelocks (#2974) 2023-09-30 19:00:39 +03:00
3be86a7131 Revert "feat(BRM) MCOL-5555 Reduce a number of direct writes to BRM journal/tablelocks files" (#2972)
This reverts commit 02114b5b7c.
2023-09-26 05:52:59 +03:00
02114b5b7c feat(BRM) MCOL-5555 Reduce a number of direct writes to BRM journal/tablelocks files 2023-09-25 23:39:49 +03:00
5f4d2705fd fix(brm): This refactors shmem RWLocks operations for EM, EMIndex and FreeList so that re-map operation always takes shmem lock in Exclusive mode
---------

Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-09-06 21:07:44 +03:00
426557380c fix(cmapi, brm): MCOL-5535 Use an accurate glob when forcing owner of files in /dev/shm on startup (#2923)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-08-11 20:04:12 +03:00
8f93fc3623 MCOL-5493: First portion of UBSan fixes (#2842)
Multiple UB fixes
2023-06-02 17:02:09 +03:00
4a8e53f14d Atomic bools for RWLockMonitor 2023-04-21 13:23:58 +00:00
c2d0fa24da replace boost::shared_array<T> to std::shared_ptr<T[]> 2023-04-14 10:33:27 +00:00
a508b86091 remove boost/shared_array include 2023-04-14 09:42:50 +00:00
2e1394149b MCOL-5464: Fixes of bugs from ASAN warnings, part one (#2792)
* Fixes of bugs from ASAN warnings, part one

* MQC as static library, with nifty counter for global map and mutex

* Switch clang to 16

* link messageqcpp to execplan
2023-04-04 02:33:23 +03:00
70124ecc01 Fix trivial spelling errors
- occured -> occurred
- reponse -> response
- seperated -> separated

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2023-03-11 11:59:47 -08:00
56f2346083 Remove windows ifdefs 2023-03-02 15:59:42 +00:00
a1d89d8f31 Merge pull request #2630 from dhall-MariaDB/sergchanges
Sergchanges
2022-12-02 19:08:08 +03:00
b936ed8b2e Fix some GCC-12 Build errors 2022-11-22 03:28:17 +03:00
246a4db8de fix C API includes
ColumnStore used to include server's mysql.h
but link all tools with libmariadb.so

There's no guarantee that this would work, even with workarounds
it had in dbcon/mysql/sm.cpp

Fix:
* tools (linked with libmariadb.so) *must* include libmariadb's mysql.h
* as a hack prevent service_thd_timezone.h from being loaded into tools,
  as it conflicts with libmariadb's mysql.h
* server plugin *must* include server's mysql.h
* also don't link every tool with libmariadb.so, link the helper library
  (liblibmysqlclient.so) that actually needs it, tools use this
  helper library, not libmariadb.so directly
2022-11-17 12:02:07 -06:00
37fd915a08 Serg`s patch for develop-6 revised for develop https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/2614 2022-11-09 22:41:38 +00:00
b5d8e0324b extentmap iterator (#2540) 2022-08-29 13:30:03 +03:00
c049d85d63 MCOL-5206 This patch brings some EMIndex and treeish EM leftovers from develop-5 2022-08-24 19:13:34 +00:00
2f5f9cdbe1 MCOL-5091/MCOL-4917 ExtentMap as RBTree.
This patch replaces a `linear array` to `Red black tree` for ExtentMap.
2022-08-23 17:33:16 +03:00
cbfdae3481 MCOL-5021 Code changes based on review feedback. 2022-08-05 14:40:50 -04:00
94e9f55940 MCOL-5021 Add a new member function to the DBRM class, DBRM::addToLBIDList().
This function iterates over lbidList (populated by an earlier call to
DBRM::getUncommittedExtentLBIDs()) to find those LBIDs which belong to
the AUX column. It then finds the corresponding LBIDs for all other columns
which belong to the same table as the AUX LBID and appends them to lbidList.
The updated lbidList is used by invalidateUncommittedExtentLBIDs() to update
the casual partitioning information.

DBRM::addToLBIDList() only comes into play in case of a transaction ROLLBACK.
2022-08-05 14:40:50 -04:00
de522ed15a Fix: grabEMIndex for loadFromBinaryBlob 2022-07-28 16:47:27 +03:00
8bf4e0eaa5 MCOL-5050 EM Index dbroot container out-of-bound access checks doing EM deletes 2022-05-17 12:53:33 +00:00
6aadc6cc17 boost::interprocess::vector instead of std::vector 2022-05-04 12:59:40 +00:00
4c26e4f960 MCOL-4912 This patch introduces Extent Map index to improve EM scaleability
EM scaleability project has two parts: phase1 and phase2.
        This is phase1 that brings EM index to speed up(from O(n) down
        to the speed of boost::unordered_map) EM lookups looking for
        <dbroot, oid, partition> tuple to turn it into LBID,
        e.g. most bulk insertion meta info operations.
        The basis is boost::shared_managed_object where EMIndex is
        stored. Whilst it is not debug-friendly it allows to put a
        nested structs into shmem. EMIndex has 3 tiers. Top down description:
        vector of dbroots, map of oids to partition vectors, partition
        vectors that have EM indices.
        Separate EM methods now queries index before they do EM run.
        EMIndex has a separate shmem file with the fixed id
        MCS-shm-00060001.
2022-05-04 12:59:16 +00:00