1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-05 16:15:50 +03:00
Commit Graph

1250 Commits

Author SHA1 Message Date
drrtuy
3fd90d3270 chore(compilation): static analyser flag went mad. 2025-03-27 22:12:48 +00:00
drrtuy
04b44a835e fix(rowgroup): fix for the forgotten attributes assignment 2025-03-27 22:12:48 +00:00
drrtuy
db2c82519d chore(): another cleanup round 2025-03-27 22:12:48 +00:00
drrtuy
b649af5a0c chore(): merge cleanup 2025-03-27 22:12:48 +00:00
drrtuy
729db829a2 feat(RM,allocators): use atomic counters math abstraction 2025-03-27 22:12:48 +00:00
drrtuy
b14613a66b fix(aggregation): replaced instances with references 2025-03-27 22:12:48 +00:00
drrtuy
09926b3157 feat(TNS, sorting, distinct): TNS now accounts data used by RGDatas and distinct maps. 2025-03-27 22:12:48 +00:00
drrtuy
c6dabe7eb5 feat(TNS): distribute SortingPQ that supports CountingAllocator 2025-03-27 22:12:48 +00:00
drrtuy
be5711cf0d feat(): replace getMaxDataSize with getMaxDataSizeWithStrings to accurately account for mem 2025-03-27 22:12:48 +00:00
drrtuy
0bfe10a7d0 feat(): use CountingAllocator for DISTINCT 2025-03-27 22:12:48 +00:00
drrtuy
bb94343080 fix(): allocate Pointer vector in both TupleJoiner ctor 2025-03-27 22:12:48 +00:00
drrtuy
a4c4d33ee7 feat(): zerocopy TNS case and JOIN results RGData with CountingAllocator 2025-03-27 22:12:48 +00:00
drrtuy
3dfc8cd454 feat(): first cleanup 2025-03-27 22:12:48 +00:00
drrtuy
4c1d9bceb7 feat(): Replacing STLPoolAllocator with CountingAllocator for in-memory joins 2025-03-27 22:12:48 +00:00
drrtuy
bb4cd40ca4 feat(): aggregating CountingAllocator 2025-03-27 22:12:48 +00:00
drrtuy
cf9b5eb32c feat(): restore user-space mem allocator 2025-03-27 22:12:48 +00:00
drrtuy
1aa2f3a42b feat(): TupleHashJoin now handles bad_alloc case switching to disk-based if it is enabled 2025-03-27 22:12:48 +00:00
drrtuy
f594d27685 feat(): accounts hash tables RAM allocations/removes STLPoolAllocator 2025-03-27 22:12:48 +00:00
drrtuy
90b4322470 feat(): propagated changes into SLTPoolAllocator and friends 2025-03-27 22:12:48 +00:00
drrtuy
a6de8ec1ac feat(): dangling pointer/ref issue has been solved for both RGData and BS 2025-03-27 22:12:48 +00:00
drrtuy
397b3ff729 feat(): change ref to atomic with ptr to atomic 2025-03-27 22:12:48 +00:00
drrtuy
71ed9cabe0 feat(): propagate long strings SP type change 2025-03-27 22:12:48 +00:00
drrtuy
4e86123a5a feat(): use boost::make_shared b/c most distros can't do allocate_shared for array types. 2025-03-27 22:12:48 +00:00
drrtuy
5f1bd3be12 feat(RGData,StringStore): add counting allocator capabilities to those ctors used in BPP::execute() 2025-03-27 22:12:48 +00:00
drrtuy
02b8ea1331 feat(PP,ByteStream): new counting memory allocator 2025-03-27 22:12:48 +00:00
Amr Elmohamady
2d69b49ba0 fix(utils): MCOL-4605 handle negative decimals in FROM_UNIXTIME (#3466) 2025-03-27 22:11:16 +00:00
Leonid Fedorov
1fdeb740c9 MCOL-5992: fix NULLIF if first param is NULL 2025-03-28 00:03:32 +04:00
Sergey Zefirov
3b2f9f5344 fix(plugin): MCOL-4942 No-table-SELECT now can return empty set (#3413)
The query like "SELECT 1 WHERE 1=0" was returning a row despite
unsatisfiable condition in WHERE. Now it returns an empty set.
2025-03-05 07:35:00 +00:00
Serguey Zefirov
e37d621a12 fix(MCOL-5842): Fix JSON_OBJECT's handling of empty strings
JSON_OBJECT() (and probably some other JSON functions) now properly
handle empty strings in their arguments - JSON_OBJECT used to return
NULL, now it returns empty string.
2025-02-28 17:17:16 +04:00
Aleksei Antipovskii
0ab03c7258 chore(codestyle): mark virtual methods as override 2025-02-21 20:01:34 +04:00
Daniel Black
88e80c1542 fix(utils): MCOL-5881 set/getThreadName use FreeBSD API (#3384)
Taken from FreeBSD ports, this uses the FreeBSD
APIs rather than the Linux specific prctl to change
and retreive the thread names.

Co-authored-by: Bernard Spil <brnrd@FreeBSD.org>
2025-02-20 16:32:59 +00:00
Sergey Zefirov
60dc7550f1 fix(group by, having): MCOL-5776: GROUP BY/HAVING closer to server's (#3371)
This patch introduces an internal aggregate operator SELECT_SOME that
is automatically added to columns that are not in GROUP BY. It
"computes" some plausible value of the column (actually, last one
passed).

Along the way it fixes incorrect handling of HAVING being transferred
into WHERE, window function handling and a bit of other inconsistencies.
2024-12-20 19:11:47 +00:00
Aleksei Antipovskii
e0a01c6cf4 Reapply "fix(aggregation, disk-based) MCOL-5691 distinct aggregate disk based (#3145)"
This reverts commit a5c12b98d7.
2024-12-11 12:02:24 +00:00
Serguey Zefirov
39a976c39a fix(ubsan): MCOL-5844 - iron out UBSAN reports
The most important fix here is the fix of possible buffer overrun in
DATEFORMAT() function. A "%W" format, repeated enough times, would
overflow the 256-bytes buffer for result. Now we use ostringstream to
construct result and we are safe.

Changes in date/time projection functions made me fix difference between
us and server behavior. The new, better behavior is reflected in changes
in tests' results.

Also, there was incorrect logic in TRUNCATE() and ROUND() functions in
computing the decimal "shift."
2024-12-10 20:30:58 +04:00
drrtuy
aa4bbc0152 feat(joblist,runtime): this is the first part of the execution model that produces a workload that can be predicted for a given query.
* feat(joblist,runtime): this is the first part of the execution model that produces a workload that can be predicted for a given query.
  - forces to UM join converter to use a value from a configuration
  - replaces a constant used to control a number of outstanding requests with a value depends on column width
  - modifies related Columnstore.xml values
2024-12-03 22:18:21 +00:00
drrtuy
eaba4d33b4 fix(DEC):MCOL-5805,5808 to resolve UM-only node crash inside DEC when there is no local PP to send the local requests to. (#3350)
* Revert "fix(DEC): MCOL-5602 fixing potentially endless loop in DEC (#3049)"

This reverts commit 1d416bc6ed.

* fix(DEC):MCOL-5805,5808 to resolve UM-only node crash inside DEC when there is no local PP to send the local requests to.
2024-11-11 18:31:15 +00:00
Alexey Antipovsky
11136b3545 fix(PrimProc): MCOL-5651 Add a workaround to avoid choosing an incorrect TupleHashJoinStep as a joiner [stable-23.10] (#3331)
* fix(PrimProc): MCOL-5651 Add a workaround to avoid choosing an incorrect TupleHashJoinStep as a joiner
2024-11-08 12:51:25 +00:00
drrtuy
0a71892d97 feat(rowgroup): this returns bits lost during cherry-pick. The bits lost caused the first RGData::serialize to crash a process 2024-11-08 16:28:51 +04:00
drrtuy
6f6e69815d feat(bytestream,serdes): Distribute BS buf size data type change to avoid implicit data type narrowing 2024-11-08 16:28:51 +04:00
drrtuy
a947f7341c feat(bytestream,serdes): BS buffer size type is uint64_t
This necessary to handle 64bit RGData, that comes as
	a separate patch. The pair of patches would allow to
	have PM joins when SmallSide size > 4GB.
2024-11-08 16:28:51 +04:00
drrtuy
dc03621e9d fix(rowgroup): RGData now uses uint64_t counter for the fixed sizes columns data buf.
The buffer can utilize > 4GB RAM that is necessary for PM side join.
	RGData ctor uses uint32_t allocating data buffer.
 	This fact causes implicit heap overflow.
2024-11-08 16:28:51 +04:00
drrtuy
6757535b6e fix(join, UM, perf): UM join is multi-threaded now (#3286)
* chore: UM join is multi-threaded now

* fix(UMjoin):  replace TR1 maps with stdlib versions
2024-09-04 18:56:35 +04:00
Leonid Fedorov
4b411b3968 MCOL-4696: get rid of boost::iequals 2024-08-21 21:35:52 +04:00
Aleksei Antipovskii
4aa281645e feat(SM): MCOL-5785 S3Storage improvements
Update libmarias3
fix build with the recent libmarias3

feat(SM): MCOL-5785 Add timeout options for S3Storage

    In some unfortunate situations StorageManager may get stuck on
    network operations. This commit adds the ability to set network
    timeouts which will help to ensure that the system is more
    responsive.

feat(SM): MCOL-5785 Add smps & smkill tools

    * `smps` shows all active S3 network operations
    * `smkill` terminates S3 network operations

    NB! At the moment smkill is able to terminate operations
    that are stuck on retries, but not hang inside the libcurl
    call. In other words if you want to terminate all operations
    you should configure `connect_timeout` & `timeout`

Install smkill & smps

Add install for new binaries
2024-08-21 20:45:38 +04:00
Roman Nozdrin
a5c12b98d7 Revert "fix(aggregation, disk-based) MCOL-5691 distinct aggregate disk based (#3145)"
This reverts commit c7caa4374f.
2024-07-07 13:09:56 +00:00
drrtuy
d089171637 fix(compilation): add explicit comparison operators and explicitly cast a type (#3229) 2024-06-29 07:53:54 +01:00
Sergey Zefirov
db4cb1d657 MCOL-4234 and MCOL 5772 cherry-picked into [stable 23.10] (#3226)
* MCOL-4234: improve GROUP BY and ORDER BY interaction (#3194)

This patch fixes the problem in MCOL-4234 and also generally improves
behavior of GROUP BY.

It does so by introducing a "dummy" aggregate and by wrapping columns
into it. This allows for columns that are not in GROUP BY to be used
more freely, for example, in SELECT * FROM tbl GROUP BY col - all
columns that are not "col" will be wrapped into an aggregate and query
will proceed to execution.

The dummy aggregate itself does nothing more than remember last value
passed into it.

There also an additional error message that tries to explain what types
of expressions can be wrapped into an aggregate.

* MCOL-5772: incorrect ORDER BY ordering for a columns not in GROUP BY (#3214)

When ORDER BY column is not in GROUP BY, is not an aggregate and there
is a SELECT column that is also not an aggregate, there was a problem:
ordering happened on the SELECTed column, not ORDERed one.

This patch fixes that particular problem and also performs some tidying
around newly added aggregate.

---------

Co-authored-by: Leonid Fedorov <79837786+mariadb-LeonidFedorov@users.noreply.github.com>
2024-06-28 00:31:53 +04:00
Denis Khalikov
9f4231f87f MCOL-5708 Calculate precision and scale for constant decimal. (#3227)
This patch calculates precision and scale for constant decimal
value for SUM aggregation function.
2024-06-28 00:31:03 +04:00
Denis Khalikov
985cd94402 fix(join, disk-based): MCOL-5597: large side read errors (#3117) (#3225)
The large side read errors mentioned there can be due to failure to
close file stream properly. Some of the data may still reside in the
file stream buffers, closing must flush it. The flush is an I/O
operation and can fail, leading to partial write and subsequent partial
read.

This patch tries to provide better diagnostics.

Co-authored-by: Sergey Zefirov <72864488+mariadb-SergeyZefirov@users.noreply.github.com>
2024-06-27 17:24:45 +04:00
Denis Khalikov
d6db3552c3 MCOL-5597 Rollback changes introduced for DJS. (#3224)
This patch changes:
1. The number of buckets created on each split.
2. The heuristic which calculates the bucket size.
2024-06-27 17:22:11 +04:00