1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-04-18 21:44:02 +03:00

976 Commits

Author SHA1 Message Date
drrtuy
6445f4dff3
feat(joblist,runtime): this is the first part of the execution model that produces a workload that can be predicted for a given query.
* feat(joblist,runtime): this is the first part of the execution model that produces a workload that can be predicted for a given query.
  - forces to UM join converter to use a value from a configuration
  - replaces a constant used to control a number of outstanding requests with a value depends on column width
  - modifies related Columnstore.xml values
2024-12-03 22:17:49 +00:00
Alan Mologorsky
932546f47c
fix(CMAPI): reduce timeout value for couple CMAPI endpoints (#3153)
* Minor fix in timeout for /notfound "endpoint" call.

- [fix] timeout to urlopen /notfound "endpoint" in mcs-savebrm and mcs-loadbrm

* chore(CMAPI): reduced a second Time Out asking MDB for a async replication status. The latency must be small b/c MDB Server to be requested is local.

---------

Co-authored-by: drrtuy <drrtuy@gmail.com>
2024-11-12 10:56:16 +00:00
Allen Herrera
bd9dfdf38c
feat(installation): MCOL-4886 skip chown if top folder permissions good (#3093)
* MCOL-4886 skip chown if top folder permissions good

* remove '-type f' check permissions of directories too
2024-09-11 14:50:41 +03:00
Sergei Golubchik
aaa39e6860 don't add new build dependencies
and particularly, *not for unit tests*
2024-08-06 18:19:06 +04:00
Roman Nozdrin
e7c1582910 fix(mcs-savebrm): replace debug with error to put mcs-savebrm.py log messages in order 2024-07-25 21:05:43 +04:00
Sergey Zefirov
f5089c7d80
fix(client): MCOL-5587: enable quick mode for predictable performance (#3240)
This changeset enables quick (mariadb -q) mode when columnstore is
installed. Quick mode precludes client CLI program from storing too
much data in memory, preventing out of memory conditions.
2024-07-07 13:52:21 +01:00
Alan Mologorsky
116711e9c1
fix(save-brm): MCOL-5709: Review fix. (#3207)
[fix] if file_path_prefix is None throwing TypeError
2024-06-20 10:50:22 +01:00
Alan Mologorsky
4da9bcdccf
feat(save_brm)!: MCOL-5709: protect from S3/NFS IO errors (#3206)
* feat(save_brm)!: protect from S3/NFS IO errors

* feat(save_brm)!: future refactoring

* cleanup

* feat(save_brm)!: forgotten template

* feat(save-brm,ci)!: python3 package for rocky8

---------

Co-authored-by: Roman Nozdrin <roman.nozdrin@mariadb.com>
2024-06-18 18:42:01 +01:00
Alan Mologorsky
1e8172f74b
MCOL-5338: Fixed issue related to mcs-loadbrm. (#3107)
[fix] eventually wrong logging messages format
2024-02-05 21:27:57 +03:00
Denis Khalikov
a8c4a7673c fix(build): Fixing build for S3 with "LocalStorage" type. 2023-10-11 13:44:11 +03:00
mariadb-AndreyPiskunov
05547f2342 Add a limit (as runtime value) for long in queries 2023-08-21 10:38:46 +03:00
drrtuy
426557380c
fix(cmapi, brm): MCOL-5535 Use an accurate glob when forcing owner of files in /dev/shm on startup (#2923)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-08-11 20:04:12 +03:00
mariadb-AlanMologorsky
66ec9f039b MCOL-5553: Not running save_brm on a single node without CMAPI.
[add] LOCALHOSTS tuple to mcs-savebrm script
[fix] detecting primary when no working CMAPI found
2023-08-11 18:22:59 +03:00
Leonid Fedorov
aead012f9f Simpler OAM 2023-07-12 18:15:51 +03:00
drrtuy
385e580e9c
MCOL-5175 Bump overlooked libmariaS3 and added more ENV variables to fine tune initial installation on S3 (#2870)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-06-27 17:33:39 +03:00
Leonid Fedorov
8f93fc3623
MCOL-5493: First portion of UBSan fixes (#2842)
Multiple UB fixes
2023-06-02 17:02:09 +03:00
Gagan Goel
a388d691ac [serg changes] mcs-savebrm.py: don't abort with "[Errno 1] Unknown host". 2023-05-08 18:56:29 +00:00
Roman Nozdrin
4fe9cd64a3
Revert "No boost condition (#2822)" (#2828)
This reverts commit f916e64927cd81569327014f20c4cc0b8aca40ff.
2023-04-22 15:49:50 +03:00
Leonid Fedorov
f916e64927
No boost condition (#2822)
This patch replaces boost primitives with stdlib counterparts.
2023-04-22 00:42:45 +03:00
Leonid Fedorov
3ce19abdae
Options to build with TSAN, UBSAN and skipping smoke (#2826) 2023-04-21 21:24:48 +03:00
Leonid Fedorov
2e1394149b
MCOL-5464: Fixes of bugs from ASAN warnings, part one (#2792)
* Fixes of bugs from ASAN warnings, part one

* MQC as static library, with nifty counter for global map and mutex

* Switch clang to 16

* link messageqcpp to execplan
2023-04-04 02:33:23 +03:00
Leonid Fedorov
56f2346083 Remove windows ifdefs 2023-03-02 15:59:42 +00:00
Andrey Piskunov
b6808c97f1
MCOL-4530: common conjuction top rewrite (#2673)
Added logical transformation of the execplan::ParseTrees with the taking out the common factor in expression of the form "(A and B) or (A and C)" for the purposes of passing a TPCH 19 query.

Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
2023-02-27 19:23:19 +03:00
Leonid Fedorov
e6127ab9bc
MCOL-5242: fix Columnstore tmp_files dir owner (#2731)
* When dbrmroot created, it has columnstore_tmp_files as parent, and this parent was still with root owner

* Change f-string to format
2023-02-21 22:04:20 +03:00
Leonid Fedorov
6d81cde000
Typo errpr (#2671) 2022-12-20 21:21:09 +03:00
mariadb-AlanMologorsky
0ad8c7e758 MCOL-5302: Fix brm-save to prevent overwrites Extent Map files multiple times with shared(non-S3) storage setup.
[fix] oam/install_scripts/mcs-savebrm.py.in , now everytime using method to detect primary node
[fix] condition style in oam/install_scripts/mcs-loadbrm.py.in
2022-11-24 21:59:02 +04:00
Andrey Piskunov
96c40d5081
Add core dumps and stack trace (#2604)
* Add core dumps and stack trace
2022-11-22 03:29:34 +03:00
mariadb-AlanMologorsky
0ee1d94f71 MCOL-5218: Additional fix to mcs-loadbrm and mcs-savebrm + postinstall.
[fix] bug in detecting primary.
[fix] minor bugs.
[add] not getting meta on secondaries if no cmapi available on primary node.
[fix] postinstall script to mask mariadb-columnstore service on multinode configurations.
2022-09-23 22:01:16 +03:00
mariadb-AlanMologorsky
70c850ca71 MCOL-5218: Fix mcs-loadbrm nd mcs-savebrm service run while reisntall.
Removed "requests" external python dependency to keep working with preinstalled python3.
Add fallback to detect primary if CMAPI is unreachable with hostname resolving.
Fix various minor python code styling.

f
2022-09-23 04:11:36 +03:00
Roman Nozdrin
56bbef62e6
Merge pull request #2406 from tntnatbry/MCOL-5021-dev
MCOL-5021 AUX column implementation to improve DELETE performance.
2022-08-15 19:03:42 +03:00
Roman Nozdrin
8b15e2f6a4
Merge pull request #2490 from benthompson15/MCOL-5163-dev
MCOL-5163: remove systemd coupling of processes.
2022-08-09 14:44:10 +03:00
Gagan Goel
cbfdae3481 MCOL-5021 Code changes based on review feedback. 2022-08-05 14:40:50 -04:00
Gagan Goel
c8b6b154bf MCOL-5021 Add an option in Columnstore.xml, fastdelete (disabled
by default), which when enabled, indiscriminately invalidates all
column extents and performs the actual DELETE only on the AUX
column. The trade-off with this approach would now be that the
first SELECT for certain query patterns (those containing a WHERE
predicate) after the DELETE operation will slow down as the
invalidated column extent would need to be scanned again to set
the min/max values.
2022-08-05 14:40:49 -04:00
mariadb-AlanMologorsky
7de8589139 MCOL-5174: Create bash script to restart processes in a loop.
[add] loop_process_starter.sh to oam/install_scripts
[fix] oam/install_scripts/CMakeLists.txt
2022-08-05 17:37:52 +03:00
benthompson15
fd2ebba6de MCOL-5163: remove systemd coupling of processes. 2022-08-04 18:04:42 -05:00
Roman Nozdrin
dc82c080f6 MCOL-5097 This patch fixes bash syntax issue and removes ExeMgr systemd unit on post-install 2022-07-08 07:53:34 +02:00
David.Hall
272246e9fa
Merge branch 'develop' into MCOL-4841 2022-06-09 16:58:33 -05:00
david.hall
3b6449842f Merge branch 'develop' into MCOL-4841
# Conflicts:
#	exemgr/main.cpp
#	oam/etc/Columnstore.xml.singleserver
#	primitives/primproc/primproc.cpp
2022-06-09 10:07:26 -05:00
benthompson15
652c9299b7
MCOL-4989: Fix error handling for misconfigured StorageManager startup. (#2385) 2022-06-02 08:50:41 -05:00
Roman Nozdrin
4e46645124
Merge pull request #2398 from drrtuy/MCOL-5060-post-install-containers-dev
MCOL-5060 Fixes post-install script failures in containers
2022-05-31 14:20:32 +05:30
Roman Nozdrin
c89864c37d MCOL-5060 Fixes post-install script failures in containers 2022-05-30 11:41:09 +00:00
Roman Nozdrin
965ca1ca26
MCOL-5105 DMLProc startup/shutdown timeouts affect rollback and graceful shutdown cases, workernode shutdown timeout affects save_brm (#2391) 2022-05-27 13:35:56 -05:00
David.Hall
bbb168a846
Mcol 4560 (#2337)
* MCOL-4560 remove unused xml entries and code that references it.
There is reader code and variables for some of these settings, but nobody uses them.
2022-04-18 18:00:17 -04:00
Roman Nozdrin
7453db6b02 MCOL-5001 This patch removes ExeMgr traces 2022-04-04 12:50:42 +00:00
Roman Nozdrin
e174696351 MCOL-5001 This patch merges ExeMgr and PrimProc runtimes
EM and PP are most resource-hungry runtimes.
        The merge enables to control their cummulative
        resource consumption, thread allocation + enables
        zero-copy data exchange b/w local EM and PP facilities.
2022-04-04 11:46:33 +00:00
Leonid Fedorov
3919c541ac
New warnfixes (#2254)
* Fix clang warnings

* Remove vim tab guides

* initialize variables

* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length

* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison

* chars are unsigned on ARM, having  if (ival < 0) always false

* chars are unsigned by default on ARM and comparison with -1 if always true
2022-02-17 13:08:58 +03:00
David Hall
c4db823ba7 Merge branch 'develop' of github.com:mariadb-corporation/mariadb-columnstore-engine into develop 2022-02-07 12:20:17 -06:00
benthompson15
64a6b8a041
MCOL-4965: upgrade breaks trying to use deprecated autoConfigure executable. (#2243) 2022-02-03 16:41:43 -06:00
Roman Nozdrin
b5d500e97b
MCOL-4094 shell test argument quotation fix (#2246)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2022-02-03 16:14:47 -06:00
Leonid Fedorov
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00