1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-06-07 19:22:02 +03:00

100 Commits

Author SHA1 Message Date
Leonid Fedorov
108701f3e7 Tricky immortal singleton 2025-05-30 03:27:04 +00:00
Leonid Fedorov
e7c6b860e4 locked map for messageqcpp 2025-05-30 03:27:04 +00:00
Leonid Fedorov
1f64934041 add message where needed, mpve pseudosocket tp primitiveserver 2025-05-30 03:27:04 +00:00
Leonid Fedorov
7af81b932e check debian/mariadb-plugin-columnstore.install automatically 2025-05-30 03:27:04 +00:00
Leonid Fedorov
6804e3003d more deps 2025-05-30 03:27:04 +00:00
Leonid Fedorov
0984e35709 configcpp refactored 2025-05-30 03:27:04 +00:00
Leonid Fedorov
6db2dc668f stubs and cmake formatting 2025-05-20 18:22:59 +04:00
Leonid Fedorov
a0bee173f6 chore(build): fixes to satisfy clang19 warnings 2025-05-15 19:05:38 +04:00
Aleksei Antipovskii
21ebd1ac20 feat(bytestream): serialize long strings in the common way 2025-04-11 15:21:07 +02:00
Aleksei Antipovskii
4bea7e59a0 feat(PrimProc): MCOL-5852 disk-based GROUP_CONCAT & JSON_ARRAYAGG
* move GROUP_CONCAT/JSON_ARRAYAGG storage to the RowGroup from
  the RowAggregation*
* internal data structures (de)serialization
* get rid of a specialized classes for processing JSON_ARRAYAGG
* move the memory accounting to disk-based aggregation classes
* allow aggregation generations to be used for queries with
  GROUP_CONCAT/JSON_ARRAYAGG
* Remove the thread id from the error message as it interferes with the mtr
2025-04-11 15:21:07 +02:00
drrtuy
db2c82519d chore(): another cleanup round 2025-03-27 22:12:48 +00:00
drrtuy
3dfc8cd454 feat(): first cleanup 2025-03-27 22:12:48 +00:00
drrtuy
a6de8ec1ac feat(): dangling pointer/ref issue has been solved for both RGData and BS 2025-03-27 22:12:48 +00:00
drrtuy
71ed9cabe0 feat(): propagate long strings SP type change 2025-03-27 22:12:48 +00:00
drrtuy
02b8ea1331 feat(PP,ByteStream): new counting memory allocator 2025-03-27 22:12:48 +00:00
Aleksei Antipovskii
0ab03c7258 chore(codestyle): mark virtual methods as override 2025-02-21 20:01:34 +04:00
drrtuy
6f6e69815d feat(bytestream,serdes): Distribute BS buf size data type change to avoid implicit data type narrowing 2024-11-08 16:28:51 +04:00
drrtuy
a947f7341c feat(bytestream,serdes): BS buffer size type is uint64_t
This necessary to handle 64bit RGData, that comes as
	a separate patch. The pair of patches would allow to
	have PM joins when SmallSide size > 4GB.
2024-11-08 16:28:51 +04:00
Leonid Fedorov
9a9b5f8036
fix(join,threadpool): MCOL-5565: MCOL-5636: MCOL-5645: port from develop-23.02 to [stable-23.10] (#3127)
* fix(threadpool): MCOL-5565 queries stuck in FairThreadScheduler. (#3100)

Meta Primitive Jobs, .e.g ADD_JOINER, LAST_JOINER stuck
	in Fair scheduler without out-of-band scheduler. Add OOB
	scheduler back to remedy the issue.

* fix(messageqcpp): MCOL-5636 same node communication crashes transmiting PP errors to EM b/c error messaging leveraged socket that was a nullptr. (#3106)

* fix(threadpool): MCOL-5645 errenous threadpool Job ctor implictly sets socket shared_ptr to nullptr causing sigabrt when threadpool returns an error (#3125)

---------

Co-authored-by: drrtuy <roman.nozdrin@mariadb.com>
2024-02-14 14:56:07 +03:00
Leonid Fedorov
1f71847e99 fix(packaging) dh_missing: warning are treated as errors for buildbot debians
dh_missing: warning: Compatibility levels before 10 are deprecated (level 9 in use)
dh_missing: warning: usr/lib/x86_64-linux-gnu/libmessageqcpp.a exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/lib/x86_64-linux-gnu/libpron.a exists in debian/tmp but is not installed to anywhere

so do not install static libraries as targets on CMake
2023-10-04 13:20:24 -04:00
Roman Nozdrin
176eefcc57 MCOL-5456 codebase preparation for colab with CS 11.1 2023-05-08 18:54:47 +00:00
Roman Nozdrin
4fe9cd64a3
Revert "No boost condition (#2822)" (#2828)
This reverts commit f916e64927cd81569327014f20c4cc0b8aca40ff.
2023-04-22 15:49:50 +03:00
Leonid Fedorov
f916e64927
No boost condition (#2822)
This patch replaces boost primitives with stdlib counterparts.
2023-04-22 00:42:45 +03:00
Leonid Fedorov
c2d0fa24da replace boost::shared_array<T> to std::shared_ptr<T[]> 2023-04-14 10:33:27 +00:00
Leonid Fedorov
a508b86091 remove boost/shared_array include 2023-04-14 09:42:50 +00:00
Leonid Fedorov
2e1394149b
MCOL-5464: Fixes of bugs from ASAN warnings, part one (#2792)
* Fixes of bugs from ASAN warnings, part one

* MQC as static library, with nifty counter for global map and mutex

* Switch clang to 16

* link messageqcpp to execplan
2023-04-04 02:33:23 +03:00
Sergey Zefirov
b53c231ca6 MCOL-271 empty strings should not be NULLs (#2794)
This patch improves handling of NULLs in textual fields in ColumnStore.
Previously empty strings were considered NULLs and it could be a problem
if data scheme allows for empty strings. It was also one of major
reasons of behavior difference between ColumnStore and other engines in
MariaDB family.

Also, this patch fixes some other bugs and incorrect behavior, for
example, incorrect comparison for "column <= ''" which evaluates to
constant True for all purposes before this patch.
2023-03-30 21:18:29 +03:00
Leonid Fedorov
a0cb59f5bc This patch resolves UB with non-existing mutex usage in MQC on runtime shutdown 2023-03-09 20:15:28 +00:00
Leonid Fedorov
56f2346083 Remove windows ifdefs 2023-03-02 15:59:42 +00:00
Roman Nozdrin
a9d8924683 MCOL-5166 This patch adds support for in-memory communication b/w EM to PP via a shared queue in DEC class
JobList low-level code relateod to primitive jobs now uses shared pointers instead of ByteStream refs talking to DEC
b/c same-node EM-PP communication now goes over a queue in DEC instead of a network hop.
PP now has a separate thread that processes the primitive job messages from that DEC queue.
2022-08-04 18:51:31 +03:00
Roman Nozdrin
1624c347f6 MCOL-5152 This patch enables PP to put ByteStreams into DEC input queue directly for a local PP-EM connection 2022-07-04 09:06:40 +00:00
david.hall
9a24934728 MCOL-4841 remove BOOST_BIND_GLOBAL_PLACEHOLDERS
drone has this defined on the command line
2022-06-14 16:16:38 -05:00
david.hall
d4cf894edc MCOL-4841 fix some compiler issues 2022-06-14 14:32:01 -05:00
Leonid Fedorov
3919c541ac
New warnfixes (#2254)
* Fix clang warnings

* Remove vim tab guides

* initialize variables

* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length

* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison

* chars are unsigned on ARM, having  if (ival < 0) always false

* chars are unsigned by default on ARM and comparison with -1 if always true
2022-02-17 13:08:58 +03:00
Leonid Fedorov
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00
Leonid Fedorov
01f3ceb437 replace header guards with #pragma once 2022-01-21 15:24:58 +00:00
Denis Khalikov
1b6d278404 Handle error during parsing of bytestream.
The error can occur in case we send the bytestream with `old` format header.
2021-12-30 12:30:21 +03:00
Denis Khalikov
6393c6d019 MCOL-4810 Add support for missed operation for longStrings. 2021-10-28 10:02:02 +03:00
Denis Khalikov
7bda598fbf MCOL-4810 Redundant copying and wasting memory in PrimProc
This patch eliminates a copying `long string`s into the bytestream.
2021-08-26 12:16:23 +03:00
Roman Nozdrin
a292585b8c MCOL-4815 ColumnCommand was replaced with a set of derived classes specified by
column width

RTSCommand was modified to use a fabric that produces CC class based on column width

NB this patch doesn't affect PseudoCC that also leverages ColumnCommand
2021-07-21 12:54:14 +00:00
Roman Nozdrin
866dc25729
Merge pull request #1842 from denis0x0D/MCOL-987_LZ
MCOL-987 LZ4 compression support.
2021-07-07 13:13:18 +03:00
Roman Nozdrin
fb5ba84212 MCOL-4802 Removed ByteStream methods for bool manipulations and add some logging into I_S.columnstore_files 2021-07-07 07:16:30 +00:00
Denis Khalikov
cc1c3629c5 MCOL-987 Add LZ4 compression.
* Adds CompressInterfaceLZ4 which uses LZ4 API for compress/uncompress.
* Adds CMake machinery to search LZ4 on running host.
* All methods which use static data and do not modify any internal data - become `static`,
  so we can use them without creation of the specific object. This is possible, because
  the header specification has not been modified. We still use 2 sections in header, first
  one with file meta data, the second one with pointers for compressed chunks.
* Methods `compress`, `uncompress`, `maxCompressedSize`, `getUncompressedSize` - become
  pure virtual, so we can override them for the other compression algos.
* Adds method `getChunkMagicNumber`, so we can verify chunk magic number
  for each compression algo.
* Renames "s/IDBCompressInterface/CompressInterface/g" according to requirement.
2021-07-06 18:04:37 +03:00
Roman Nozdrin
bed0b7c6bc MCOL-4173 This patch adds support for wide-DECIMAL INNER, OUTER, SEMI, functional JOINs
based on top of TypelessData
2021-06-24 08:07:23 +00:00
Roman Nozdrin
90397dfed0 MCOL-4675 DMLProc now automatically and gracefully shutdowns when a cluster state is set to
SS_SHUTDOWN_PENDING | SS_ROLLBACK
2021-05-27 11:07:32 +00:00
Roman Nozdrin
c9b353e975 Simplify PMS connection entries configuration 2021-04-16 10:52:13 +00:00
Roman Nozdrin
3eb26c0d4a MCOL-4313 Introduced TSInt128 that is a storage class for int128
Removed uint128 from joblist/lbidlist.*

Another toString() method for wide-decimal that is EMPTY/NULL aware

Unified decimal processing in WF functions

Fixed a potential issue in EqualCompData::operator() for
    wide-decimal processing

Fixed some signedness warnings
2020-11-18 13:53:15 +00:00
Alexander Barkov
d5c6645ba1 Adding mcs_basic_types.h
For now it consists of only:

using int128_t = __int128;
using uint128_t = unsigned __int128;

All new privitive data types should go into this file in the future.
2020-11-18 13:53:15 +00:00
Roman Nozdrin
844472d812 MCOL-4313 Very fragile but high speed approach with inline ASM
GCC compiler uses aligned versions of SIMD instructions expecting
aligned memory blocks that is hard to implement now
2020-11-18 13:52:20 +00:00
David Hall
2d044fd7f1 MCOL-4171 Fix comments in bytestream.h 2020-11-18 13:52:19 +00:00