1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-07 03:22:57 +03:00
Commit Graph

1782 Commits

Author SHA1 Message Date
drrtuy
d1de121476 fix(): periodically return from UM JOIN loop that can produce a lot of RGData-s and thus overcommit for RAM 2025-03-27 22:12:48 +00:00
drrtuy
b649af5a0c chore(): merge cleanup 2025-03-27 22:12:48 +00:00
drrtuy
729db829a2 feat(RM,allocators): use atomic counters math abstraction 2025-03-27 22:12:48 +00:00
drrtuy
b14613a66b fix(aggregation): replaced instances with references 2025-03-27 22:12:48 +00:00
drrtuy
8ca8119f36 fix(TNS): fix for duplicate DISTINCT output 2025-03-27 22:12:48 +00:00
drrtuy
09926b3157 feat(TNS, sorting, distinct): TNS now accounts data used by RGDatas and distinct maps. 2025-03-27 22:12:48 +00:00
drrtuy
c6dabe7eb5 feat(TNS): distribute SortingPQ that supports CountingAllocator 2025-03-27 22:12:48 +00:00
drrtuy
be5711cf0d feat(): replace getMaxDataSize with getMaxDataSizeWithStrings to accurately account for mem 2025-03-27 22:12:48 +00:00
drrtuy
da28750ffc fix(TNS): removed optimization 2025-03-27 22:12:48 +00:00
drrtuy
0bfe10a7d0 feat(): use CountingAllocator for DISTINCT 2025-03-27 22:12:48 +00:00
drrtuy
7ad4022d96 fix(): remove an additional and errenous CountingAllocator for RGData so it does not throw at an unexpected spot 2025-03-27 22:12:48 +00:00
drrtuy
a4c4d33ee7 feat(): zerocopy TNS case and JOIN results RGData with CountingAllocator 2025-03-27 22:12:48 +00:00
drrtuy
3dfc8cd454 feat(): first cleanup 2025-03-27 22:12:48 +00:00
drrtuy
4c1d9bceb7 feat(): Replacing STLPoolAllocator with CountingAllocator for in-memory joins 2025-03-27 22:12:48 +00:00
drrtuy
1aa2f3a42b feat(): TupleHashJoin now handles bad_alloc case switching to disk-based if it is enabled 2025-03-27 22:12:48 +00:00
drrtuy
f594d27685 feat(): accounts hash tables RAM allocations/removes STLPoolAllocator 2025-03-27 22:12:48 +00:00
drrtuy
90b4322470 feat(): propagated changes into SLTPoolAllocator and friends 2025-03-27 22:12:48 +00:00
drrtuy
397b3ff729 feat(): change ref to atomic with ptr to atomic 2025-03-27 22:12:48 +00:00
drrtuy
5f1bd3be12 feat(RGData,StringStore): add counting allocator capabilities to those ctors used in BPP::execute() 2025-03-27 22:12:48 +00:00
drrtuy
02b8ea1331 feat(PP,ByteStream): new counting memory allocator 2025-03-27 22:12:48 +00:00
drrtuy
16904ab97d chore(compilation): fix to satisfy specific warning 2025-03-27 21:35:11 +04:00
drrtuy
87981483e2 chore(compilation): to resolve unexpected boost warnings 2025-03-27 21:35:11 +04:00
drrtuy
9b272f59d2 fix(DEC): DEC now sends multiple acks with the correct number of msgs acked to facilitate ControlFlow using same node communication bypass. (#3345) 2025-03-19 12:24:05 +00:00
Serguey Zefirov
230506b996 fix(MCOL-5396): Fix possible infinite loop in plugin--PrimProc communication
If you manage to shut down PrimProc just before plugin is trying to send
Calpont Select Execution Plan to PrimProc, you now get a nice error
message about PrimProc being down instead of endless logs of failed
reconnection attempts.
2025-03-11 09:38:18 +03:00
Sergey Zefirov
3b2f9f5344 fix(plugin): MCOL-4942 No-table-SELECT now can return empty set (#3413)
The query like "SELECT 1 WHERE 1=0" was returning a row despite
unsatisfiable condition in WHERE. Now it returns an empty set.
2025-03-05 07:35:00 +00:00
Aleksei Antipovskii
cbf4fef47c more overrides 2025-02-21 20:01:34 +04:00
Aleksei Antipovskii
a4c1e2cbcb add missing override 2025-02-21 20:01:34 +04:00
Aleksei Antipovskii
0ab03c7258 chore(codestyle): mark virtual methods as override 2025-02-21 20:01:34 +04:00
drrtuy
d5dcae52cb fix(BRM): MCOL-5879 DBRM::clearShm runs crit sections w/o sync mechanism (#3391) 2025-02-20 16:33:44 +00:00
Serguey Zefiov
6e539b8336 fix(MCOL-5889): Improper handle of DOUBLE result type with DECIMAL arguments
Sometimes server assigns DOUBLE type for arithmetic operations over
DECIMAL arguments. In this rare case width of result was incorrectly
adjusted and it triggered an assertion.

Now width of result gets adjusted only if result type is also DECIMAL.
2025-02-12 18:41:55 +04:00
Sergei Golubchik
9618ff5e61 compilation fixes for MDEV-34811 handlerton (#3341)
Co-authored-by: Leonid Fedorov <79837786+mariadb-LeonidFedorov@users.noreply.github.com>
2025-01-12 16:08:34 +04:00
mariadb-AlanMologorsky
d7cfa15d2a fix(client): MCOL-5587: Add quick-max-column-width for maridb clients.
This changeset enables quick (mariadb -q) mode when columnstore is
installed. Quick mode precludes client CLI program from storing too
much data in memory, preventing out of memory conditions.
Add quick-max-column-width=0 to prevent extra garbage dashes in output.
2024-12-22 16:59:13 +04:00
Sergey Zefirov
60dc7550f1 fix(group by, having): MCOL-5776: GROUP BY/HAVING closer to server's (#3371)
This patch introduces an internal aggregate operator SELECT_SOME that
is automatically added to columns that are not in GROUP BY. It
"computes" some plausible value of the column (actually, last one
passed).

Along the way it fixes incorrect handling of HAVING being transferred
into WHERE, window function handling and a bit of other inconsistencies.
2024-12-20 19:11:47 +00:00
Aleksei Antipovskii
23048e9749 fix(aggregation): remove double returnMemory() 2024-12-11 12:02:24 +00:00
Denis Khalikov
928678499a fix(aggregation, RAM): MCOL-5715 Changes the second phase aggregation. (#3171)
This patch changes the second phase aggregation pipeline - takes into
account current memory consumption.

Co-authored-by: Leonid Fedorov <79837786+mariadb-LeonidFedorov@users.noreply.github.com>
Co-authored-by: drrtuy <roman.nozdrin@mariadb.com>
2024-12-11 12:02:24 +00:00
Aleksei Antipovskii
e0a01c6cf4 Reapply "fix(aggregation, disk-based) MCOL-5691 distinct aggregate disk based (#3145)"
This reverts commit a5c12b98d7.
2024-12-11 12:02:24 +00:00
Serguey Zefirov
5e5d328269 Fix build 2024-12-10 20:30:58 +04:00
Serguey Zefirov
50a31d1296 Fix build 2024-12-10 20:30:58 +04:00
Serguey Zefirov
6b2334cecf Fix build 2024-12-10 20:30:58 +04:00
Serguey Zefirov
39a976c39a fix(ubsan): MCOL-5844 - iron out UBSAN reports
The most important fix here is the fix of possible buffer overrun in
DATEFORMAT() function. A "%W" format, repeated enough times, would
overflow the 256-bytes buffer for result. Now we use ostringstream to
construct result and we are safe.

Changes in date/time projection functions made me fix difference between
us and server behavior. The new, better behavior is reflected in changes
in tests' results.

Also, there was incorrect logic in TRUNCATE() and ROUND() functions in
computing the decimal "shift."
2024-12-10 20:30:58 +04:00
Sergey Zefirov
3bcc2e2fda fix(memory leaks): MCOL-5791 - get rid of memory leaks in plugin code (#3365)
There were numerous memory leaks in plugin's code and associated code.
During typical run of MTR tests it leaked around 65 megabytes of
objects. As a result they may severely affect long-lived connections.

This patch fixes (almost) all leaks found in the plugin. The exceptions
are two leaks associated with SHOW CREATE TABLE columnstore_table and
getting information of columns of columnstore-handled table. These
should be fixed on the server side and work is on the way.
2024-12-06 09:04:55 +00:00
drrtuy
aa4bbc0152 feat(joblist,runtime): this is the first part of the execution model that produces a workload that can be predicted for a given query.
* feat(joblist,runtime): this is the first part of the execution model that produces a workload that can be predicted for a given query.
  - forces to UM join converter to use a value from a configuration
  - replaces a constant used to control a number of outstanding requests with a value depends on column width
  - modifies related Columnstore.xml values
2024-12-03 22:18:21 +00:00
drrtuy
eaba4d33b4 fix(DEC):MCOL-5805,5808 to resolve UM-only node crash inside DEC when there is no local PP to send the local requests to. (#3350)
* Revert "fix(DEC): MCOL-5602 fixing potentially endless loop in DEC (#3049)"

This reverts commit 1d416bc6ed.

* fix(DEC):MCOL-5805,5808 to resolve UM-only node crash inside DEC when there is no local PP to send the local requests to.
2024-11-11 18:31:15 +00:00
Denis Khalikov
4cba8301d5 fix(logging): Add setddldebuglevel command (#3312) (#3344) 2024-11-09 21:38:19 +00:00
Alexey Antipovsky
11136b3545 fix(PrimProc): MCOL-5651 Add a workaround to avoid choosing an incorrect TupleHashJoinStep as a joiner [stable-23.10] (#3331)
* fix(PrimProc): MCOL-5651 Add a workaround to avoid choosing an incorrect TupleHashJoinStep as a joiner
2024-11-08 12:51:25 +00:00
drrtuy
6f6e69815d feat(bytestream,serdes): Distribute BS buf size data type change to avoid implicit data type narrowing 2024-11-08 16:28:51 +04:00
drrtuy
dc03621e9d fix(rowgroup): RGData now uses uint64_t counter for the fixed sizes columns data buf.
The buffer can utilize > 4GB RAM that is necessary for PM side join.
	RGData ctor uses uint32_t allocating data buffer.
 	This fact causes implicit heap overflow.
2024-11-08 16:28:51 +04:00
Aleksei Antipovskii
ca6c35abdd fix(dbcon) MCOL-5812 server crash related to stored functions
Using the stored function's return value as an argument
for another function was handled incorrectly, leading
to a server crash.
2024-11-05 20:33:41 +04:00
drrtuy
ce86d1025a feat(joblist): better dot graphs that represents joblist steps execution tree. 2024-09-26 18:51:49 +04:00
drrtuy
6757535b6e fix(join, UM, perf): UM join is multi-threaded now (#3286)
* chore: UM join is multi-threaded now

* fix(UMjoin):  replace TR1 maps with stdlib versions
2024-09-04 18:56:35 +04:00