1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-07 03:22:57 +03:00
Commit Graph

6391 Commits

Author SHA1 Message Date
Sergei Golubchik
0482308b45 disable pch when building boost with gcc 8.x
according to https://bugzilla.redhat.com/show_bug.cgi?id=1806545

  GCC's PCH mechanism is fundamentally incompatible with the security
  hardening requirements for RHEL (PIE in particular).  We are currently
  evaluating the best way to present a more friendly diagnostic for this
  issue.

and also

  we don't turn on hardening the compiler and its binaries in DTS 9.
2023-07-04 12:58:18 -04:00
Leonid Fedorov
501da394ca Replace std::set contains method with count
to support Rocky/RHEL/Alma 8 where the std::set
in the stock STL does not have contains method
2023-07-04 12:58:18 -04:00
Sergei Golubchik
a8be4a3787 compiler warnings
like

dbcon/joblist/batchprimitiveprocessor-jl.cpp:893:54: error: pointer used after ‘void operator delete [](void*, std::size_t)’ [-Werror=use-after-free]
  893 |           joinResults.reset(new vector<uint32_t>[8192]);
      |                                                      ^
2023-07-04 12:58:18 -04:00
Denis Khalikov
cfcda69b1d Add description for Disk Join Step. (#2882) 2023-07-03 04:03:29 +03:00
Alan Mologorsky
b2a573cf82 MCOL-5521: Use Columnstore VERSION file for CMAPI versioning. (#2885)
[fix] columnstore_version.cmake setting extra variables for CMAPI versioning
[fix] CMAPI CmakeLists.txt using columnstore_version.cmake file
[fix] CMAPI VERSION file to template file due to using it inside CMAPI applicatio
n
2023-06-29 15:47:55 +03:00
drrtuy
6d44d2e850 MCOL-5500 Remove another noisy printout. (#2886)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-06-29 15:46:00 +03:00
drrtuy
385e580e9c MCOL-5175 Bump overlooked libmariaS3 and added more ENV variables to fine tune initial installation on S3 (#2870)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-06-27 17:33:39 +03:00
Denis Khalikov
2aba28d855 Merge pull request #2851 from denis0x0D/MCOL-5477
MCOL-5477 Disk join step improvement.
2023-06-26 11:02:20 +03:00
Roman Nozdrin
535b7d5b61 Merge pull request #2877 from tntnatbry/my_init_resource_leak_fix-dev
Fix resource leak in DDLProc/DMLProc/PrimProc/WriteengineServer processes.
2023-06-25 20:55:30 +01:00
Roman Nozdrin
18f4131176 Merge pull request #2871 from drrtuy/MCOL-5500-dev
MCOL-5500 Remove noisy printout from CGroupConfigurator method
2023-06-25 20:51:38 +01:00
Gagan Goel
ab7dfaa25b Fix resource leak in DDLProc/DMLProc/PrimProc/WriteengineServer processes.
As part of the charset support, a call to MY_INIT() was added at the
initialization of the above processes. This call initializes the MySQL
thread environment required by the charset library. However, the
accompanying my_end() call required to terminate this thread environment
was not added at the termination of these process, hence leaking
resources. As a fix, we move the MY_INIT() calls to the Child()
functions of these services and also add the missing my_end() call.
2023-06-23 20:18:32 +00:00
Roman Nozdrin
669c88007e Merge pull request #2875 from phoeinx/update-documentation
Update BUILD.md.
2023-06-23 19:06:51 +01:00
Denis Khalikov
1f190a6e75 MCOL-5477 Disk join step improvement.
This patch:
1. Handles corner case when the bucket exceeded the memory limit, but we cannot redistribute the data in this bucket into new buckets based on a hash algorithm, because the rows have the same values.
2. Adds force option for disk join step.
3. Add a option to contol the depth of the partition tree.
2023-06-23 18:40:15 +03:00
Roman Nozdrin
cc08929578 Merge pull request #2865 from denis0x0D/MCOL-5512
MCOL-5512 Fix for post join filter.
2023-06-21 10:45:13 +01:00
Roman Nozdrin
79b636d853 MCOL-5500 Remove noisy printout from CGroupConfigurator method 2023-06-19 11:24:41 +00:00
Leonid Fedorov
d2d0e08690 Fixed stdout and stderr forking porcs handling 2023-06-16 04:38:09 +03:00
Theresa Hradilak
5dba0ba53a Merge branch 'mariadb-corporation:develop' into update-documentation 2023-06-13 21:53:03 +02:00
Roman Nozdrin
780232b29f Merge pull request #2862 from mariadb-corporation/remove-bind-mount-cgroup
ci: try to remove bind mount host sys fs cgroup
2023-06-12 10:57:53 +01:00
mariadb-RomanNavrotskiy
a71b21a491 ci: remove bind mount sys/fs/cgroup 2023-06-11 17:27:59 +02:00
Roman Nozdrin
375d162376 MCOL-5500 This patch adds cgroup v2 support with some sanity checks for (#2849)
values reported by cgroups v1
2023-06-09 17:37:21 +03:00
Denis Khalikov
024e6bd358 MCOL-5512 Fix for post join filter.
This patch fixes certain situations where post join filter is not applying.
2023-06-09 11:15:05 +03:00
Leonid Fedorov
ab9b95e368 Add one more unit to logs (#2861) 2023-06-07 16:32:21 +03:00
Roman Nozdrin
62dc392476 MCOL-5499 Enable ControlFlow for same node communication processing path to avoid DEC queue overloading (#2848) 2023-06-07 15:41:59 +03:00
Roman Nozdrin
cacbbee1c2 MCOL-5175 Increase the maximum effective length of S3 secret used as SHA256 key producing S3 signature (#2859)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-06-07 15:24:02 +03:00
mariadb-RomanNavrotskiy
1eca477485 MCOL-5496: Fix CI adding CMAPI steps.
[fix] deb packages deps commands
[add] several additional local variables
[fix] packages url
[fix] smoke step
[fix] mtr step
[fix] regression step
[add] cmapipython, cmapibuild, cmapitest and cmapilog steps
[fix] dockerfile step
[fix] build step to include cmapi package on repodata creating
[fix] multi_node_mtr step
[fix] pkg step
[add] cmapi steps to pipelines
[fix] cmapi/CMakeLists.txt to prevent cmake byte-compile .py files for rpm packages
[add] setup-repo.sh file
[fix] now use packages from the repos in tests steps
2023-06-07 10:00:16 +03:00
mariadb-AlanMologorsky
a079a2c944 MCOL-5496: Merge CMAPI code to engine repo.
[add] cmapi code to engine
2023-06-07 10:00:16 +03:00
Leonid Fedorov
77eedd1756 MCOL-5503: Fix broken quarter (#2855)
* Fix broken quarter function
2023-06-02 18:06:58 +03:00
Leonid Fedorov
8f93fc3623 MCOL-5493: First portion of UBSan fixes (#2842)
Multiple UB fixes
2023-06-02 17:02:09 +03:00
Theresa Hradilak
5ede14c9c0 Fix indentation in BUILD.md. 2023-06-01 19:09:11 +02:00
Theresa Hradilak
44633f40ec Update BUILD.md and delete duplicate build/README.md.
This commit updates the BUILD.md with some commments for contributors building from a fork and regarding dependency installation.
2023-06-01 17:03:48 +00:00
Sergey Zefirov
0a2e9760ee Fix for JSON_VALUE function to remove OOB stack access (#2852)
MCOL-271 introduced a bug in JSON_VALUE that was discovered during
implementation of ASAN builds. The changes here restore normal
functionality.

In short, changes in MCOL-271 introduced a local variable instead of
reference to a string in ConstantColumn's fResult.strVal. The handling
of ConstantColumn is different because ConstantColumn's value is used
to initialize JSON path once. JSON path value holds pointer to data it
does not own and if there are two or more rows the data can be corrupted
and/or be out of stack bounds.

The changes here introduce reference to a NullString that is held in the
ConstantColumn's fResult.strVal and uses appropriate functions to obtain
data from the NullString. CC's fResult is held by CC and strVal is also
neither changing nor moving during operation, which allow JSON path to
hold correct pointers during multi-row operation.
2023-05-31 15:30:40 +03:00
Gagan Goel
32482cd928 Add an option in build/bootstrap_mcs.sh to NOT perform a clean install. (#2854)
A new option is added (-n/--no-clean-install) that allows a user to
re-install ColumnStore without deleting the existing db files. This is
useful for testing new code changes in the engine without the need to
re-create the database tables to test the code changes.
2023-05-30 21:04:51 +03:00
Roman Nozdrin
d083650f75 Merge pull request #2846 from tntnatbry/MCOL-5480
MCOL-5480 LOAD DATA INFILE incorrectly loads values for MEDIUMINT datatype.
2023-05-26 14:39:06 +01:00
Leonid Fedorov
f18c556311 Fix gcc-13 warning and add support for building Fedora (#2845) 2023-05-26 16:30:53 +03:00
Gagan Goel
c598a9bbed MCOL-5480 LOAD DATA INFILE incorrectly loads values for MEDIUMINT datatype.
Internal memory representation of MEDIUMINT datatype uses 24 bits. This is
true for both MariaDB server as well as ColumnStore. MCS plugin code uses
TypeHandlerSInt24 and TypeHandlerUInt24 classes to respectively convert the
binary representation of the signed and unsigned MEDIUMINT values passed by
the server to the plugin. The plugin then outputs the text representation
of these values into an open file descriptor which is piped to cpimport
for the final load into the MCS db files.

The TypeHandlerXInt24 classes were earlier incorrectly using
WriteBatchField::ColWriteBatchXInt32() functions which operate on a 4 byte
buffer. This resulted in incorrect parsing of MEDIUMINT values. As a fix,
we implement WriteBatchField::ColWriteBatchXInt24() functions which
correctly handle the 24 bit input buffer used for MEDIUMINT datatype.
2023-05-23 16:00:05 -04:00
Roman Nozdrin
a90535e1a7 Merge pull request #2844 from drrtuy/obsidian
An initial Obsidian vault as a knowledge base.
2023-05-20 16:01:42 +01:00
Roman Nozdrin
26aa9abaf0 An initial Obsidian vault as a knowledge base. 2023-05-20 15:46:19 +03:00
Roman Nozdrin
de7ba854bd Merge pull request #2840 from tntnatbry/MCOL-5491
MCOL-5491 Enable StringStore for long strings in JSON_ARRAYAGG processing.
2023-05-17 12:12:03 +01:00
Roman Nozdrin
d3778e391e Merge pull request #2827 from mariadb-corporation/rwlockmonitor_datarace
Atomic bools for RWLockMonitor
2023-05-17 12:09:11 +01:00
Roman Nozdrin
fed115c9ac Merge pull request #2837 from tntnatbry/MCOL-5357-dev
MCOL-5357 Fix TPC-DS query error "MCS-3009: Unknown column '.<colname>'".
2023-05-17 12:06:49 +01:00
Roman Nozdrin
5a21440cd5 Merge pull request #2839 from tntnatbry/develop-23.02-changes-for-develop
develop-23.02 changes for develop.
2023-05-12 22:01:16 +01:00
Gagan Goel
87eb875379 MCOL-5491 Enable StringStore for long strings in JSON_ARRAYAGG processing.
This patch is the JSON_ARRAYAGG clone of the changes done in MCOL-5429
where we enabled usage of StringStore for long strings in
GROUP_CONCAT() processing to reduce memory footprint of PrimProc and
thus avoiding a potential OS triggered OOM crash.
2023-05-12 19:45:02 +00:00
Gagan Goel
f4af756af1 Bump VERSION to 23.08.1-1. 2023-05-08 19:00:31 +00:00
Roman Nozdrin
e6e74c0be7 MCOL-5437 Fixes to follow the charset_info api change introduced by MDEV-30661 2023-05-08 18:57:36 +00:00
Gagan Goel
3364ceb2f0 cmake fixes related to mpark_patterns. 2023-05-08 18:56:48 +00:00
Gagan Goel
a388d691ac [serg changes] mcs-savebrm.py: don't abort with "[Errno 1] Unknown host". 2023-05-08 18:56:29 +00:00
Gagan Goel
9f3b191efb [serg changes] fix for debian builds. 2023-05-08 18:56:06 +00:00
Gagan Goel
3985aac871 [serg changes] skip ColumnStore if old cmake. 2023-05-08 18:55:45 +00:00
Gagan Goel
e9bc708c98 [serg changes] fix cmake warning. 2023-05-08 18:55:33 +00:00
Gagan Goel
f2c90a8dae [serg changes] skip ColumnStore if STL is too old. 2023-05-08 18:55:01 +00:00