1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-01 06:46:55 +03:00
Commit Graph

958 Commits

Author SHA1 Message Date
d7d5e2211b MCOL-5175 Bump overlooked libmariaS3 and added more ENV variables to fine tune initial installation on S3 (#2869)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-06-18 13:35:26 +03:00
0a4478a7cd [serg changes] mcs-savebrm.py: don't abort with "[Errno 1] Unknown host". 2023-05-01 16:04:21 -04:00
2f153184c3 Fixes of bugs from ASAN warnings, part one (#2796) 2023-03-30 18:29:04 +03:00
56f2346083 Remove windows ifdefs 2023-03-02 15:59:42 +00:00
b6808c97f1 MCOL-4530: common conjuction top rewrite (#2673)
Added logical transformation of the execplan::ParseTrees with the taking out the common factor in expression of the form "(A and B) or (A and C)" for the purposes of passing a TPCH 19 query.

Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
2023-02-27 19:23:19 +03:00
e6127ab9bc MCOL-5242: fix Columnstore tmp_files dir owner (#2731)
* When dbrmroot created, it has columnstore_tmp_files as parent, and this parent was still with root owner

* Change f-string to format
2023-02-21 22:04:20 +03:00
6d81cde000 Typo errpr (#2671) 2022-12-20 21:21:09 +03:00
0ad8c7e758 MCOL-5302: Fix brm-save to prevent overwrites Extent Map files multiple times with shared(non-S3) storage setup.
[fix] oam/install_scripts/mcs-savebrm.py.in , now everytime using method to detect primary node
[fix] condition style in oam/install_scripts/mcs-loadbrm.py.in
2022-11-24 21:59:02 +04:00
96c40d5081 Add core dumps and stack trace (#2604)
* Add core dumps and stack trace
2022-11-22 03:29:34 +03:00
0ee1d94f71 MCOL-5218: Additional fix to mcs-loadbrm and mcs-savebrm + postinstall.
[fix] bug in detecting primary.
[fix] minor bugs.
[add] not getting meta on secondaries if no cmapi available on primary node.
[fix] postinstall script to mask mariadb-columnstore service on multinode configurations.
2022-09-23 22:01:16 +03:00
70c850ca71 MCOL-5218: Fix mcs-loadbrm nd mcs-savebrm service run while reisntall.
Removed "requests" external python dependency to keep working with preinstalled python3.
Add fallback to detect primary if CMAPI is unreachable with hostname resolving.
Fix various minor python code styling.

f
2022-09-23 04:11:36 +03:00
56bbef62e6 Merge pull request #2406 from tntnatbry/MCOL-5021-dev
MCOL-5021 AUX column implementation to improve DELETE performance.
2022-08-15 19:03:42 +03:00
8b15e2f6a4 Merge pull request #2490 from benthompson15/MCOL-5163-dev
MCOL-5163: remove systemd coupling of processes.
2022-08-09 14:44:10 +03:00
cbfdae3481 MCOL-5021 Code changes based on review feedback. 2022-08-05 14:40:50 -04:00
c8b6b154bf MCOL-5021 Add an option in Columnstore.xml, fastdelete (disabled
by default), which when enabled, indiscriminately invalidates all
column extents and performs the actual DELETE only on the AUX
column. The trade-off with this approach would now be that the
first SELECT for certain query patterns (those containing a WHERE
predicate) after the DELETE operation will slow down as the
invalidated column extent would need to be scanned again to set
the min/max values.
2022-08-05 14:40:49 -04:00
7de8589139 MCOL-5174: Create bash script to restart processes in a loop.
[add] loop_process_starter.sh to oam/install_scripts
[fix] oam/install_scripts/CMakeLists.txt
2022-08-05 17:37:52 +03:00
fd2ebba6de MCOL-5163: remove systemd coupling of processes. 2022-08-04 18:04:42 -05:00
dc82c080f6 MCOL-5097 This patch fixes bash syntax issue and removes ExeMgr systemd unit on post-install 2022-07-08 07:53:34 +02:00
272246e9fa Merge branch 'develop' into MCOL-4841 2022-06-09 16:58:33 -05:00
3b6449842f Merge branch 'develop' into MCOL-4841
# Conflicts:
#	exemgr/main.cpp
#	oam/etc/Columnstore.xml.singleserver
#	primitives/primproc/primproc.cpp
2022-06-09 10:07:26 -05:00
652c9299b7 MCOL-4989: Fix error handling for misconfigured StorageManager startup. (#2385) 2022-06-02 08:50:41 -05:00
4e46645124 Merge pull request #2398 from drrtuy/MCOL-5060-post-install-containers-dev
MCOL-5060 Fixes post-install script failures in containers
2022-05-31 14:20:32 +05:30
c89864c37d MCOL-5060 Fixes post-install script failures in containers 2022-05-30 11:41:09 +00:00
965ca1ca26 MCOL-5105 DMLProc startup/shutdown timeouts affect rollback and graceful shutdown cases, workernode shutdown timeout affects save_brm (#2391) 2022-05-27 13:35:56 -05:00
bbb168a846 Mcol 4560 (#2337)
* MCOL-4560 remove unused xml entries and code that references it.
There is reader code and variables for some of these settings, but nobody uses them.
2022-04-18 18:00:17 -04:00
7453db6b02 MCOL-5001 This patch removes ExeMgr traces 2022-04-04 12:50:42 +00:00
e174696351 MCOL-5001 This patch merges ExeMgr and PrimProc runtimes
EM and PP are most resource-hungry runtimes.
        The merge enables to control their cummulative
        resource consumption, thread allocation + enables
        zero-copy data exchange b/w local EM and PP facilities.
2022-04-04 11:46:33 +00:00
3919c541ac New warnfixes (#2254)
* Fix clang warnings

* Remove vim tab guides

* initialize variables

* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length

* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison

* chars are unsigned on ARM, having  if (ival < 0) always false

* chars are unsigned by default on ARM and comparison with -1 if always true
2022-02-17 13:08:58 +03:00
c4db823ba7 Merge branch 'develop' of github.com:mariadb-corporation/mariadb-columnstore-engine into develop 2022-02-07 12:20:17 -06:00
64a6b8a041 MCOL-4965: upgrade breaks trying to use deprecated autoConfigure executable. (#2243) 2022-02-03 16:41:43 -06:00
b5d500e97b MCOL-4094 shell test argument quotation fix (#2246)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2022-02-03 16:14:47 -06:00
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00
01f3ceb437 replace header guards with #pragma once 2022-01-21 15:24:58 +00:00
22b0e4addc MCOL-4943 Moved SQL script call into columnstore-post-install 2021-12-18 03:59:58 +00:00
199d89f681 MCOL-4832 Removing needless error output 2021-12-15 10:35:03 +00:00
4ee1057374 Remove commented line
Changes to Columnstore.xml are causing issues with automated install scripts. Removing these minor changes will get us through the release. The problem shall be fixed with MCOL-4928
2021-11-19 11:28:59 -06:00
7fea3c988e [MCOL-4829] Compression for the temp disk-based aggregation files 2021-09-02 19:30:25 +03:00
1d2114a0d5 MCOL-4751: done run mycnfUpgrade anymore just reuse existing cnf if possible 2021-08-05 11:12:17 -05:00
21e6e88d0f MCOL-4751: autoConfigure to run in post-install on files in correct paths. 2021-08-05 11:11:54 -05:00
c5c82c1d4f MCOL-4751 resurrect autoConfigure
A need to set Columnstore.xml to the new version with any user changed settings preserved
2021-08-05 11:11:36 -05:00
f4f6cf8b09 MCOL-4773: fix grep for mcsapikey value to allow '=' characters in key (#2069) 2021-08-05 10:31:23 -05:00
866dc25729 Merge pull request #1842 from denis0x0D/MCOL-987_LZ
MCOL-987 LZ4 compression support.
2021-07-07 13:13:18 +03:00
cc1c3629c5 MCOL-987 Add LZ4 compression.
* Adds CompressInterfaceLZ4 which uses LZ4 API for compress/uncompress.
* Adds CMake machinery to search LZ4 on running host.
* All methods which use static data and do not modify any internal data - become `static`,
  so we can use them without creation of the specific object. This is possible, because
  the header specification has not been modified. We still use 2 sections in header, first
  one with file meta data, the second one with pointers for compressed chunks.
* Methods `compress`, `uncompress`, `maxCompressedSize`, `getUncompressedSize` - become
  pure virtual, so we can override them for the other compression algos.
* Adds method `getChunkMagicNumber`, so we can verify chunk magic number
  for each compression algo.
* Renames "s/IDBCompressInterface/CompressInterface/g" according to requirement.
2021-07-06 18:04:37 +03:00
b24c7d0c62 MCOL-4781 change smcat to SMCAT
The case was mistakenly left lower
2021-06-29 08:18:07 -05:00
36c6c3fd69 MCOL-4770 Installation.PMwithUM is enabled by default so a plugin communicates with a local EM only 2021-06-23 10:38:46 +00:00
c040956190 MCOL-4744 Revert "MCOL-4541 CMAPI API version change"
This reverts commit 8cf2724e4d.
2021-06-18 12:58:37 +00:00
7a152c6a19 Merge pull request #1944 from mariadb-AlexeyAntipovsky/MCOL-563-dev
[MCOL-4709] Disk-based aggregation
2021-06-08 20:42:58 +03:00
475104e4d3 [MCOL-4709] Disk-based aggregation
* Introduce multigeneration aggregation

* Do not save unused part of RGDatas to disk
* Add IO error explanation (strerror)

* Reduce memory usage while aggregating
* introduce in-memory generations to better memory utilization

* Try to limit the qty of buckets at a low limit

* Refactor disk aggregation a bit
* pass calculated hash into RowAggregation
* try to keep some RGData with free space in memory

* do not dump more than half of rowgroups to disk if generations are
  allowed, instead start a new generation
* for each thread shift the first processed bucket at each iteration,
  so the generations start more evenly

* Unify temp data location

* Explicitly create temp subdirectories
  whether disk aggregation/join are enabled or not
2021-06-06 16:09:15 +03:00
606194e6e4 MCOL-4685: Eliminate some irrelevant settings (uncompressed data and extents per file).
This patch:
1. Removes the option to declare uncompressed columns (set columnstore_compression_type = 0).
2. Ignores [COMMENT '[compression=0] option at table or column level (no error messages, just disregard).
3. Removes the option to set more than 2 extents per file (ExtentsPreSegmentFile).
4. Updates rebuildEM tool to support up to 10 dictionary extent per dictionary segment file.
5. Adds check for `DBRootStorageType` for rebuildEM tool.
6. Renamed rebuildEM to mcsRebuildEM.
2021-06-03 14:44:33 +03:00
12e3368ebf MCOL-4597: logic from postConfigure migrated to post-install to upgrade existing columnstore.cnf 2021-05-26 10:41:39 -05:00