database/mariadb-columnstore-engine
1
0
Fork 0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-05 16:15:50 +03:00
Code Activity
6,391 Commits 350 Branches 89 Tags
MCOL-5995-add-custom-bootstrap-params
Commit Graph

52 Commits

Author SHA1 Message Date
drrtuy
a16fbd137b fix(disk-based-join): this fixes multiple SEGV for disk-based join algo 2025-04-15 15:06:49 +01:00
drrtuy
6e98ef3037 chore(): removed extra printout 2025-04-15 15:06:49 +01:00
drrtuy
e4ca7425e2 feat(stats): CES reports table name and HJS reports total matched rows 2025-04-15 15:06:49 +01:00
drrtuy
c64d4b950f chore(): fix compilation issue 2025-03-31 22:47:46 +01:00
drrtuy
481267dca8 chore(): reduced error-prone boilerplate for STS functionality 2025-03-31 22:47:46 +01:00
drrtuy
d1de121476 fix(): periodically return from UM JOIN loop that can produce a lot of RGData-s and thus overcommit for RAM 2025-03-27 22:12:48 +00:00
drrtuy
b649af5a0c chore(): merge cleanup 2025-03-27 22:12:48 +00:00
drrtuy
b14613a66b fix(aggregation): replaced instances with references 2025-03-27 22:12:48 +00:00
drrtuy
7ad4022d96 fix(): remove an additional and errenous CountingAllocator for RGData so it does not throw at an unexpected spot 2025-03-27 22:12:48 +00:00
drrtuy
a4c4d33ee7 feat(): zerocopy TNS case and JOIN results RGData with CountingAllocator 2025-03-27 22:12:48 +00:00
drrtuy
3dfc8cd454 feat(): first cleanup 2025-03-27 22:12:48 +00:00
drrtuy
4c1d9bceb7 feat(): Replacing STLPoolAllocator with CountingAllocator for in-memory joins 2025-03-27 22:12:48 +00:00
drrtuy
1aa2f3a42b feat(): TupleHashJoin now handles bad_alloc case switching to disk-based if it is enabled 2025-03-27 22:12:48 +00:00
drrtuy
f594d27685 feat(): accounts hash tables RAM allocations/removes STLPoolAllocator 2025-03-27 22:12:48 +00:00
drrtuy
90b4322470 feat(): propagated changes into SLTPoolAllocator and friends 2025-03-27 22:12:48 +00:00
drrtuy
aa4bbc0152 feat(joblist,runtime): this is the first part of the execution model that produces a workload that can be predicted for a given query. 
* feat(joblist,runtime): this is the first part of the execution model that produces a workload that can be predicted for a given query.
  - forces to UM join converter to use a value from a configuration
  - replaces a constant used to control a number of outstanding requests with a value depends on column width
  - modifies related Columnstore.xml values
 2024-12-03 22:18:21 +00:00
Alexey Antipovsky
11136b3545 fix(PrimProc): MCOL-5651 Add a workaround to avoid choosing an incorrect TupleHashJoinStep as a joiner [stable-23.10] (#3331) 
* fix(PrimProc): MCOL-5651 Add a workaround to avoid choosing an incorrect TupleHashJoinStep as a joiner
 2024-11-08 12:51:25 +00:00
drrtuy
dc03621e9d fix(rowgroup): RGData now uses uint64_t counter for the fixed sizes columns data buf. 
The buffer can utilize > 4GB RAM that is necessary for PM side join.
	RGData ctor uses uint32_t allocating data buffer.
 	This fact causes implicit heap overflow.
 2024-11-08 16:28:51 +04:00
drrtuy
6757535b6e fix(join, UM, perf): UM join is multi-threaded now (#3286) 
* chore: UM join is multi-threaded now

* fix(UMjoin):  replace TR1 maps with stdlib versions
 2024-09-04 18:56:35 +04:00
Denis Khalikov
2a66ae2ed1 MCOL-5514 Parallel disk join step. 2023-07-11 14:05:14 +03:00
Denis Khalikov
2aba28d855 Merge pull request #2851 from denis0x0D/MCOL-5477 
MCOL-5477 Disk join step improvement.
 2023-06-26 11:02:20 +03:00
Denis Khalikov
1f190a6e75 MCOL-5477 Disk join step improvement. 
This patch:
1. Handles corner case when the bucket exceeded the memory limit, but we cannot redistribute the data in this bucket into new buckets based on a hash algorithm, because the rows have the same values.
2. Adds force option for disk join step.
3. Add a option to contol the depth of the partition tree.
 2023-06-23 18:40:15 +03:00
Denis Khalikov
024e6bd358 MCOL-5512 Fix for post join filter. 
This patch fixes certain situations where post join filter is not applying.
 2023-06-09 11:15:05 +03:00
Leonid Fedorov
8f93fc3623 MCOL-5493: First portion of UBSan fixes (#2842) 
Multiple UB fixes
 2023-06-02 17:02:09 +03:00
Roman Nozdrin
4fe9cd64a3 Revert "No boost condition (#2822)" (#2828) 
This reverts commit f916e64927.
 2023-04-22 15:49:50 +03:00
Leonid Fedorov
f916e64927 No boost condition (#2822) 
This patch replaces boost primitives with stdlib counterparts.
 2023-04-22 00:42:45 +03:00
Leonid Fedorov
c2d0fa24da replace boost::shared_array<T> to std::shared_ptr<T[]> 2023-04-14 10:33:27 +00:00
Leonid Fedorov
6c32c658d5 MCOL-5385: Delete RowGroup::setData and make Pointer ctor explicit (#2808) 
* Delete RowGroup::setData and make Pointer ctor explicit

* some push_backs replaced with emplace_backs

* Fixes of review notes
 2023-04-13 03:55:30 +03:00
Denis Khalikov
e09d24cb8d [MCOL-5265] Change boost:shared_ptr to std::shared_ptr. 
This is attempt to make some part of the code more stable.
For some reason we can get a spurious nullptr for boost::shared_ptr
which cause an assert and abort.
 2022-11-14 18:53:53 +03:00
Denis Khalikov
636e60b5f9 [MCOL-4699] Add support for circular outer joins. 2022-07-19 21:47:36 +03:00
david.hall
3b6449842f Merge branch 'develop' into MCOL-4841 
# Conflicts:
#	exemgr/main.cpp
#	oam/etc/Columnstore.xml.singleserver
#	primitives/primproc/primproc.cpp
 2022-06-09 10:07:26 -05:00
Leonid Fedorov
3919c541ac New warnfixes (#2254) 
* Fix clang warnings

* Remove vim tab guides

* initialize variables

* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length

* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison

* chars are unsigned on ARM, having  if (ival < 0) always false

* chars are unsigned by default on ARM and comparison with -1 if always true
 2022-02-17 13:08:58 +03:00
David Hall
27dea733c5 MCOL4841 dev port run large join without OOM 2022-02-09 17:33:55 -06:00
Leonid Fedorov
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00
Roman Nozdrin
bed0b7c6bc MCOL-4173 This patch adds support for wide-DECIMAL INNER, OUTER, SEMI, functional JOINs 
based on top of TypelessData
 2021-06-24 08:07:23 +00:00
Alexander Barkov
129d5b5a0f MCOL-4174 Review/refactor frontend/connector code 2020-11-18 13:53:15 +00:00
Gagan Goel
d3bc68b02f MCOL-641 Refactor initial extent elimination support. 
This commit also adds support in TupleHashJoinStep::forwardCPData,
although we currently do not support wide decimals as join keys.

Row estimation to determine large-side of the join is also updated.
 2020-11-18 13:52:19 +00:00
Alexey Antipovsky
b25fee320a Remove variable-length arrays (-Wvla) 2020-11-17 15:03:10 +03:00
Roman Nozdrin
cd48df99e5 MCOL-4368 Unified exceptions handling code in dbcon/joblist 2020-10-21 18:17:32 +00:00
Patrick LeBlanc
a3409e1a94 Merge branch 'develop' into join-abort-fix 2020-01-08 15:19:12 -06:00
Patrick LeBlanc
04807c3ef8 MCOL-3713: Join abort problem 
Noticed a race on djsJoiners in abort() and segregateJoiners().
Re-added the mutex lock, but after risk of double lock grab.
 2020-01-08 16:05:22 -05:00
Patrick LeBlanc
0dbf3280bb MCOL-3713: UM join mem overflow abort problem 
A little cleanup.  It's working.
 2020-01-07 10:58:23 -05:00
Patrick LeBlanc
a55495a73f MCOL-3713 - A UM join mem overflow abort bug 
Seems that segregateJoiners would double-lock a mutex.  Not sure why
we're only seeing it now.

This is a checkpoint commit, probably not final for this bug.
 2020-01-06 18:03:45 -05:00
Patrick LeBlanc
80ff0bfa56 Fixed a couple minor things I ran into working on another task 
- Made TupleUnion not allocate 64MB per frame.
 - Saw a crash, where a THJS would be destroyed w/o being used.
   Fixed the bad reference.
 2020-01-06 14:17:43 -05:00
Sergei Golubchik
586391e1ca compilation failure 
error: reference to 'mutex' is ambiguous
note: candidates are: 'class boost::mutex'
note:                 'class std::mutex'
 2019-12-19 18:13:39 +01:00
Patrick LeBlanc
acdd8ca9f1 Removed a debugging printout, used RM to get core count. 2019-11-22 10:38:05 -05:00
Patrick LeBlanc
0d26dc447c Squash merge of the threaded UM hash table construction feature. 
Conflicts:
	oam/etc/Columnstore.xml.singleserver
 2019-11-21 14:41:00 -05:00
Andrew Hutchings
01446d1e22 Reformat all code to coding standard 2017-10-26 17:18:17 +01:00
David Hall
55d006de1a MCOL-513 use thread pool for jobsteps 2017-02-03 15:25:21 -06:00
Andrew Hutchings
ffcfc41563 MCOL-507 Further ExeMgr performance improvements 
This does the following:

* Switch resource manager to a singleton which reduces the amount of
times the XML data is scanned and objects allocated.
* Make the I_S tables use the FE implementation of the system catalog
* Make the I_S.columnstore_columns table use the RID list cache
* Make the extentmap pre-allocate a vector instead of many small allocs
 2017-01-16 12:33:27 +00:00