1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-04-18 21:44:02 +03:00

6347 Commits

Author SHA1 Message Date
Leonid Fedorov
7ea79ecc18 fix mtr according server updates 2023-12-07 00:29:41 +03:00
Denis Khalikov
9119f6f7b8
fix(aggregate): MCOL-5467 Add support for duplicate expressions in group by. (#3045)
This patch adds support for duplicate expressions (builtin_functions) with
one argument in select statement and group by statement.
2023-12-05 15:04:53 +03:00
Denis Khalikov
b25dce3181 Refactor - add "all_off" instead cycle. 2023-11-30 01:47:13 +04:00
Denis Khalikov
0747099456 Update on review comments 2023-11-30 01:47:13 +04:00
Denis Khalikov
865cca11c9 MCOL-5505 Add TypeHandler functions. 2023-11-30 01:47:13 +04:00
Leonid Fedorov
491ba6e0aa Segfault dummy fix
There are two different variables binaryArray and fixedSizeBinaryArray
initilized accordingly to columnData->type_id()

for columnData->type_id() equaled to  arrow::Type::type::FIXED_SIZE_BINARY
not initilized binaryArray was used below. The code, taking care about this two branches was commented.
This part looks quite messy and needs refactoring
2023-11-30 01:47:13 +04:00
Leonid Fedorov
6b8c3dd918 Fix arrow linkage on external project 2023-11-30 01:47:13 +04:00
HanpyBin
fe597ec78c MCOL-5505 add parquet support for cpimport and add mcs_parquet_ddl and mcs_parquet_gen tools 2023-11-30 01:47:13 +04:00
drrtuy
94a680ea60
fix(cmake): raise cmake version to satisfy all the requirements for the current cmake code (#3036)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-11-29 19:33:31 +03:00
drrtuy
45d42239f6
fix(build): MCOL-5612 const func pointer assignement must happen after func definition to satisfy modern compilers static analyzer (#3038)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2023-11-29 19:33:12 +03:00
Leonid Fedorov
df92d5ee9c chore(ci) Centos7 packaging: Very hard bug with not matching
MariaDB-shared-10.1-kvm-rpm-centos74-amd64.rpm
MariaDB-shared-5.3-amd64.rpm
to
FILE(GLOB compat53 RELATIVE ${CMAKE_SOURCE_DIR}
    "${CMAKE_SOURCE_DIR}/../MariaDB-shared-5.3.*.rpm")
FILE(GLOB compat101 RELATIVE ${CMAKE_SOURCE_DIR}
    "${CMAKE_SOURCE_DIR}/../MariaDB-shared-10.1.*.rpm")

files renamed to
MariaDB-shared-10.1.kvm-rpm-centos74-amd64.rpm'
MariaDB-shared-5.3.amd64.rpm'

(dash changed to dot after numbers)
to match uncontrollable glob in the server code

Added server 10.6.14-9 for upgrade testing
2023-11-29 19:40:52 +04:00
Leonid Fedorov
95185eff7e chore(ci) refactor reports from tests stage, add it to upgrade 2023-11-29 19:40:52 +04:00
Serguey Zefirov
792aea2a7c Fixes MCOL-5599 where LIKE operator never finishes
This is a fix of logging subsystem, nothing else.

The old code expanded an argument into string and advanced too little
and, if expansion contained argument's index, it expanded it again. And
again.
2023-11-29 19:17:16 +04:00
Sergey Zefirov
8632c85ecf
feat(primproc,aggregegation)!: Changes for ROLLUP with single-phase aggregation (#3025)
The fix is simple: enable subtotals in single-phase aggregation and
disable parallel processing when there are subtotals and aggregation is
single-phase.
2023-11-28 17:33:02 +03:00
Sergey Zefirov
9a84aa8d99
fix(plugin): Same columns fom different views in GROUP BY do not produce errors (#3035)
Fixes MCOL-5643.

The problem was that different views with same column names in GROUP BY
and on the SELECT clause produced an error about "projection column is
not an aggergate neither in GROUP BY list."

This was due to incorrect search in expressions's list that lead to
duplicate columns in GROUP BY list.
2023-11-28 17:30:56 +03:00
Denis Khalikov
76e4e13b80
fix(rowgroup,stringstore): MCOL-5597 Set length for nullptr string to 0. (#3027) 2023-11-28 17:18:52 +03:00
drrtuy
26f5f8fe5c
fix(plugin): this is to addres the original patch QA found in the original patch 2023-11-22 17:20:37 +03:00
Sergey Zefirov
5c9770d1e6
fix(funcexp): MCOL-5607: JSON function use crashes query execution (#3028)
JSON functions were implemented violating an assumption of their
pureness, as they should not have any state. This concrete patch
fixes implementation of JSON_VALUE function.
2023-11-21 23:46:03 +03:00
Sergey Zefirov
69b8e1c779
feat(extent-elimination)!: re-enable extent-elimination for dictionary columns scanning
This is "productization" of an old code that would enable extent
elimination for dictionary columns.

This concrete patch enables it, fixes perfomance degradation (main
problem with old code) and also fixes incorrect behavior of cpimport.
2023-11-17 17:14:35 +03:00
drrtuy
f5ff63b52f
Merge pull request #3011 from mariadb-corporation/MCOL-4740-2
MCOL-4740: This fixes update rows counter for multi-table update
2023-11-07 21:30:57 +02:00
Roman Nozdrin
6579180810 fix(plugin): MCOL-4740: This fixes update rows counter for multi-table update
For UPDATEs involving a single table, the server call to handler::direct_update_rows() is used to correctly set the count for the number of updated rows in the UPDATE statement.
However, for UPDATEs involving multi-tables, the server does not call handler::direct_update_rows(). This patch adds support to correctly report the number of updated rows to the client by setting
multi_update::updated and multi_update::found in handler::rnd_end().
2023-11-02 14:18:06 +00:00
drrtuy
67c842e792
Merge pull request #3017 from drrtuy/fix/MCOL-5472-urandom-mutex
fix(rowstorage): MCOL-5472 SplitMix64 PRNG implementation to replace stdlib MT PRNG that uses /dev/urandom guarded by spinlock
2023-11-02 16:10:57 +02:00
drrtuy
95bedb8334
Merge pull request #3018 from drrtuy/chore/ignore-upgrade-tests
chore(CI): this allows CI to ignore the results of upgrade tests that are not production ready yet
2023-11-02 14:02:52 +02:00
drrtuy
2c3071c4c2 chore(CI): this allows CI to ignore the results of upgrade tests that are not production ready yet 2023-11-02 13:38:29 +03:00
Roman Nozdrin
dfc9e89496 fix(rowstorage): SplitMix64 PRNG implementation to replace stdlib MT PRNG that uses /dev/urandom guarded by spinlock 2023-11-01 18:19:45 +00:00
drrtuy
2f0bbed165
Merge pull request #3015 from drrtuy/chore/MCOL-5568-math-with-domain-range-check
chore(datatypes): refactoring math ops results domain check functionality
2023-10-31 22:28:58 +02:00
mariadb-AlanMologorsky
7aa6e4c107 MCOL-5595: Wrong metadata in CMAPI packages.
[fix] cmapi packages metadata version format for debian systems and version + release in RHEL systems
2023-10-27 04:07:05 +03:00
Roman Nozdrin
f7045457f2 chore(datatypes): refactoring math ops results domain check functionality 2023-10-25 09:12:54 +00:00
Roman Nozdrin
eb744eafed chore(datatypes): this refactors the placement of the main SQL data types enum to enable templates that are parametrized with this enum(see mcs_datatype_basic.h changes for more details). 2023-10-24 18:44:35 +03:00
Allen Herrera
244e0adc8e
feat(package_manager): improved cs_package_manager functionality for better UX
- add apt support for community and enterprise install
- add enterprise_staging install
- filter check results down to os/arch
- add --token flag for enterprise (#3002)
2023-10-20 17:13:10 +03:00
drrtuy
242408f751
fix(datatypes, funcexp): static_cast typo fix (#3001) 2023-10-17 23:58:59 +03:00
Sergey Zefirov
84148cbe4c
fix(datatypes, funcexp): Overflow detection for MCOL-5568 use case (and some other) (#2987)
We add intermediate calculations in int128_t when target is UBIGINT and
check for overflow before converting into the UBIGINT. This is so
because we can overflow on addition and multiplication, with (some)
signed operands or both unsigned.
2023-10-16 16:55:02 +03:00
Denis Khalikov
3fcb9b66f5 MCOL-5555 Add support for startreadonly command.
This patch adds support for `startreadonly` command which waits
until all active cpimport jobs are done and then puts controller node to readonly
mode.
2023-10-16 16:11:12 +03:00
Roman Nozdrin
2b20e1de25 chore(docs): rename the doc that affects git clone on Windows 2023-10-16 16:07:14 +03:00
Leonid Fedorov
fea5479650 Revert "chore(docs): rename the doc that affects git clone on Windows"
This reverts commit e9f8a6a2f6f1dc39b452e3f46a0e978fb2d8b15f.
2023-10-16 16:06:20 +03:00
Roman Nozdrin
e9f8a6a2f6 chore(docs): rename the doc that affects git clone on Windows 2023-10-16 16:05:45 +03:00
Leonid Fedorov
c479407f39 feat(build) external project for Apache Arrow 13.0 2023-10-11 11:56:22 -04:00
Denis Khalikov
a8c4a7673c fix(build): Fixing build for S3 with "LocalStorage" type. 2023-10-11 13:44:11 +03:00
Leonid Fedorov
97f58e8592 Stable branch for .drone 2023-10-10 12:28:35 -04:00
Leonid Fedorov
1f71847e99 fix(packaging) dh_missing: warning are treated as errors for buildbot debians
dh_missing: warning: Compatibility levels before 10 are deprecated (level 9 in use)
dh_missing: warning: usr/lib/x86_64-linux-gnu/libmessageqcpp.a exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/lib/x86_64-linux-gnu/libpron.a exists in debian/tmp but is not installed to anywhere

so do not install static libraries as targets on CMake
vcolumnstore-23.10.0-1
2023-10-04 13:20:24 -04:00
Allen Herrera
221dea1788 fixed on premise s3 backup & restore --endpoint-url flag 2023-10-03 01:58:55 +03:00
Allen Herrera
d2b05afdc1 init columnstore_backup.sh 2023-10-03 01:58:55 +03:00
Leonid Fedorov
b2f2a0c78a Revert "Disable Debian 12 due to mirrors errors (#2952)"
This reverts commit 11b3aed91bc6313a16acf35ea40c1dae3ff2e27d.
2023-10-02 15:12:24 -04:00
Denis Khalikov
fce971dcd2
feat(IO): MCOL-5555 Reduce number of direct writes to the file storing BRM journal and tablelocks (#2974) 2023-09-30 19:00:39 +03:00
Leonid Fedorov
86c1c5d537
fix(rgdata)!: Fix assertion failure leading to disk-based aggregation failure
The new added invariant checking that RGData knows the number of columns and fixed size columns was failing for disk-based aggregation workloads, leading them to provide a wrong result. (The assertion failure happened in RGData::getRow(uint32_t num, Row* row) which is called in the finalization of sub-aggregation results, necessary for merging part results. As the merging failed, duplicate results were output for disk-based aggregation queries.
The assertion failure was caused by RGData::deserialize(ByteStream& bs,
uint32_t defAmount) not setting rowSize and colCount if necessary (e.g.
when the deserialization happens into a new, default RGData, which
doesn't know anything about its structure yet. This is the case when the
default constructor for RGData() is used, which sets rowSize and
columnCount to 0 each.
There are three code parts that make use of the default RGData() ctor.
The fix is for the use in RowGroupStorage::loadRG(uint64_t rgid,
std::unique_ptr<RGData>& rgdata, bool unlinkDump = false), where the
default RGData object is used to directly deserialize a ByteStream into
it. The deserialize method now checks if both rowSize and columnCount
are 0 and if yes sets the read values from the ByteStream for both.
We should probably check the other two code parts making use of the
default RGData ctor, too. This happens in joinpartition.cpp and
tuplejoiner.cpp.

---------

Co-authored-by: Theresa Hradilak <34538290+phoeinx@users.noreply.github.com>
2023-09-30 00:02:31 +03:00
Gagan Goel
6f52039658
Bump VERSION to 23.10.0-1. (#2979) 2023-09-29 20:07:23 +03:00
Gagan Goel
320df831c6
MCOL-5572 Force the charset on the autoincrement column of (#2976)
calpontsys.syscolumn syscat table to be latin1.

This change is done in one of the ctors of pColStep which is
initiated while building the job list from the execution plan.
2023-09-28 22:03:39 +03:00
Sergey Zefirov
920607520c
feat(runtime)!: MCOL-678 A "GROUP BY ... WITH ROLLUP" support
Adds a special column which helps to differentiate data and rollups of
various depts and a simple logic to row aggregation to add processing of
subtotals.
2023-09-26 17:01:53 +03:00
Leonid Fedorov
5013717730
fix(plugin): Fix wrong ask for stat call for table mode 2023-09-26 14:43:06 +03:00
Allen Herrera
a97ff73cea
init columnstore_review.sh & cs_package_manager.sh (#2969) 2023-09-26 05:59:00 +03:00