1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-08 14:22:09 +03:00
Commit Graph

6247 Commits

Author SHA1 Message Date
Roman Nozdrin
6a8a464afc Merge pull request #2512 from mariadb-corporation/MCOL-5013-params
MCOL-5013 : Changed function name, schema and params order
2022-08-18 20:50:11 +03:00
Roman Nozdrin
4fc43c3c93 Merge pull request #2506 from mariadb-corporation/MCOL-5092-addendum
MCOL-5092 addendum modify moda() mtr tests with order by
2022-08-18 20:49:01 +03:00
Alexey Antipovsky
dca359c2ab Fix excessive memory consumption at the last stage of aggregation 2022-08-18 14:00:53 +03:00
Leonid Fedorov
d02b3403b7 Changed function name, schema and params order to achieve
columnstore_info.load_from_s3("<bucket>", "<file_name>", "<db_name>", "<table_name>");
2022-08-18 10:41:22 +00:00
Leonid Fedorov
3798dd8d38 MDEV-28711: add libmariadb to columnstore deps 2022-08-18 10:02:09 +00:00
root
471b8e3e09 Updated test case 2022-08-17 16:56:26 +00:00
mariadb-DanielLee
8166e0284f Disabled warnings and add drop table commands 2022-08-17 11:29:04 -05:00
mariadb-DanielLee
10a8d274cd Updated reference results to match that of innodb tables 2022-08-17 11:28:05 -05:00
Roman Nozdrin
4e2d4984cf Merge pull request #2508 from denis0x0D/MCOL-5172
MCOL-5172 RebuildEM: improve internal representation of sys.cat files
2022-08-17 18:52:31 +03:00
Roman Nozdrin
0f2f0fc0ba Merge pull request #2507 from denis0x0D/MCOL-4715
MCOL-4715 Mixed inner and outer joins with "null" filter for the table which is not involved into the outer join produces wrong results.
2022-08-17 18:50:43 +03:00
david.hall
5c62d96b71 MCOL-5092 Use sorted_result to get consistent results in mtr 2022-08-17 09:07:47 -05:00
Denis Khalikov
ab25337c0a MCOL-5172 RebuildEM: improve internal representation of sys.cat files 2022-08-17 15:59:32 +03:00
Roman Nozdrin
edbd477528 Merge pull request #2472 from tntnatbry/MCOL-5162
MCOL-5162 This patch allows for an automatic upgrade of syscat tables via dbbuilder.
2022-08-17 14:39:03 +03:00
david.hall
5099133642 MCOL-5092 correct mtr result 2022-08-16 09:14:16 -05:00
Denis Khalikov
59166608b1 MCOL-4715 Mixed inner and outer joins with "null" filter for the table which is not involved into the outer join produces wrong results. 2022-08-16 17:13:03 +03:00
Gagan Goel
1a849c88e8 MCOL-5162 This patch allows for an automatic upgrade of syscat tables via dbbuilder.
We have added a new function, SystemCatalog::upgrade(), which will
create a new column in the database and fill it with default values.
It uses a reference column to fill the column, which is passed as a
std::unordered_map<key, value> value to SystemCatalog::upgrade()
function, where key is the target column which needs to be created.
In addition, we pass additional std::unordered_map's to
SystemCatalog::upgrade() containing information such as the datatypes
and colwidths of the target and reference columns, as well as the
default value which will be used to fill the target column.
2022-08-15 21:03:13 +00:00
david.hall
05bdddf3a9 MCOL-5092 addendum modify moda() mtr tests with order by
fix an issue in moda() evaluate for string types
2022-08-15 15:50:15 -05:00
Roman Nozdrin
56bbef62e6 Merge pull request #2406 from tntnatbry/MCOL-5021-dev
MCOL-5021 AUX column implementation to improve DELETE performance.
2022-08-15 19:03:42 +03:00
Sergei Golubchik
a7a9ccf889 Serg dev (#2504)
* more build dependencies

* fix for cmake < 3.11

It cannot do ADD_LIBRARY(... ALIAS ...) on IMPORTED targets

* another fix for cmake 3.10.2

It doesn't know about CMAKE_CXX_STANDARD=20,
let's add the correct flag manually

* gcc 8 on aarch64

utils/common/simd_arm.h:241:16: error: need ‘typename’ before ‘simd::TypeToVecWrapperType<T>::WrapperType’ because ‘simd::TypeToVecWrapperType<T>’ is a dependent scope
2022-08-15 13:35:30 +03:00
Sergei Golubchik
dee50318ad Serg dev (#2502)
* build boost during build phase, not during configure

* add dependency for generated header files errorids.h messageids.h

see 7e868bc588

* set explicit dependencies on external_boost for #include's

* clang-14 compatibility fix
2022-08-13 07:18:30 +03:00
Roman Nozdrin
35f0f7bf0a Merge pull request #2498 from drrtuy/unittests-failure
Looking into unittests.
2022-08-12 01:19:31 +03:00
David.Hall
2020f35e88 Mcol 5092 MODA uses wrong column width for some types (#2450)
* MCOL-5092 Ensure column width is correct for datatype
                       Change MODA return type to STRING
                       Modify MODA to handle every numeric type
* MCOL-5162 MODA to support char and varchar with collation support

Fixes to the aggregate bit functions
When we fixed the storage sign issue for MCOL-5092, it uncovered a problem in the bit aggregates (bit_and, bit_or and bit_xor). These aggregates should always return UBIGINT, but they relied on the type of the argument column, which gave bad results.
2022-08-11 15:16:11 -05:00
Roman Nozdrin
f40175dc32 This patch solves the ocassional issues with the FairThreadPool unit test 2022-08-11 16:56:17 +00:00
Andrey Piskunov
c906172bf5 MCOL-5180 Check CPU vector instructions set in installer and PrimProc on start (#2499)
* Add check for simd acrh support

* Updates

* More polite and detailed error messages

* Updates

* Always true to conditional

Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
2022-08-11 15:28:22 +03:00
Roman Nozdrin
0174c8ec08 Merge pull request #2497 from drrtuy/MCOL-5167
[MCOL-5167] Add support for on clause filter for a table which is not…
2022-08-11 12:19:51 +03:00
Denis Khalikov
61cf18b92d [MCOL-5167] Add support for on clause filter for a table which is not involved in join.
This patch adds support for on clause filter for a table which is not involved in particular join
by disabling an `merge optimization` for those particular cases.
The `merge optimization` is optimization when CS
tries to create a one BPP join with one `large side` table and multiple `small sides` tables, in this
case we cannot apply a FE filter if this filter requires a columns from `small side` table which is not
involved in particular join.
2022-08-10 10:46:43 +00:00
Daniel Lee
4c9d6e39ac Dlee mtr restructure (#2494)
* Restructured test suites and added autopilot and extended suites

* Updated autopilot with correct branch - develop

* Moved setup test case to a 'setup' directory, for consistency

* Fixed a path issue

* Updated some tests cases to keep up with development

Co-authored-by: root <root@rocky8.localdomain>
2022-08-09 21:20:56 +03:00
Roman Nozdrin
8b15e2f6a4 Merge pull request #2490 from benthompson15/MCOL-5163-dev
MCOL-5163: remove systemd coupling of processes.
2022-08-09 14:44:10 +03:00
Roman Nozdrin
dd96e686c0 MCOL-5153 This patch replaces MDB collation aware hash function with the (#2488)
exact functionality that does not use MDB hash function.
This patch also takes a bit from Robin Hood hash map implementation forgotten
that reduces hash function collision rate.
2022-08-07 02:36:03 +03:00
Gagan Goel
cbfdae3481 MCOL-5021 Code changes based on review feedback. 2022-08-05 14:40:50 -04:00
Gagan Goel
11b7ee2f11 MCOL-5021 Disallow the following ALTER TABLE ADD COLUMN statement:
ALTER TABLE calpontsys.systable ADD COLUMN (auxcolumnoid INT NOT NULL DEFAULT 0);
2022-08-05 14:40:50 -04:00
Gagan Goel
1355237ca3 MCOL-5021 Some minor fixes. 2022-08-05 14:40:50 -04:00
Gagan Goel
94e9f55940 MCOL-5021 Add a new member function to the DBRM class, DBRM::addToLBIDList().
This function iterates over lbidList (populated by an earlier call to
DBRM::getUncommittedExtentLBIDs()) to find those LBIDs which belong to
the AUX column. It then finds the corresponding LBIDs for all other columns
which belong to the same table as the AUX LBID and appends them to lbidList.
The updated lbidList is used by invalidateUncommittedExtentLBIDs() to update
the casual partitioning information.

DBRM::addToLBIDList() only comes into play in case of a transaction ROLLBACK.
2022-08-05 14:40:50 -04:00
Gagan Goel
0818b95caa MCOL-5021 Update MTR tests to reflect changes introduced by this feature. 2022-08-05 14:40:50 -04:00
Gagan Goel
9b6d3c3870 MCOL-5021 Add support for AUX column in the client code calling
CalpontSystemCatalog::columnRIDs().
2022-08-05 14:40:49 -04:00
Gagan Goel
439db48c5a MCOL-5021 Add support for the AUX column in TRUNCATE table processing. 2022-08-05 14:40:49 -04:00
Gagan Goel
ea1861fdb5 MCOL-5021 Add a new function to CalpontSystemCatalog class,
isAUXColumnOID(), to check if a given OID is an auxilliary
column OID.
2022-08-05 14:40:49 -04:00
Gagan Goel
262cd5c501 MCOL-5021 Remove hard-coded values for data type, column width
and compression type for the AUX column, and replace them with
constants defined in the execplan namespace.
2022-08-05 14:40:49 -04:00
Gagan Goel
c8b6b154bf MCOL-5021 Add an option in Columnstore.xml, fastdelete (disabled
by default), which when enabled, indiscriminately invalidates all
column extents and performs the actual DELETE only on the AUX
column. The trade-off with this approach would now be that the
first SELECT for certain query patterns (those containing a WHERE
predicate) after the DELETE operation will slow down as the
invalidated column extent would need to be scanned again to set
the min/max values.
2022-08-05 14:40:49 -04:00
Gagan Goel
2280b1dd25 MCOL-5021 Add support for the AUX column in ExeMgr and PrimProc.
In the joblist code, in addition to sending the lbid of the SCAN
column, we also send the corresponding lbid of the AUX column to PrimProc.

In the primitives processor code in PrimProc, we load the AUX column
block (8192 rows since the AUX column is implemented as a 1-byte
UNSIGNED TINYINT) into memory and then pass it down to the low-level
scanning (vectorized scanning as applicable) routine to build a non-Empty
mask for the block being processed to filter out DELETED rows based on
comparison of the AUX block row to the empty magic value for the AUX column.
2022-08-05 14:40:49 -04:00
Gagan Goel
60eb0f86ec MCOL-5021 non-AUX column files are opened in read-only mode during
the DELETE operation. ColumnOp::readBlock() calls can cause writes
to database files when the active chunk list in ChunkManager is full.
Since non-AUX columns are read-only for the DELETE operation, we prevent
writes of compressed chunks and header for these columns by passing
an isReadOnly flag to CompFileData which indicates whether the column
is read-only or read-write.
2022-08-05 14:40:49 -04:00
Gagan Goel
35a3a93964 MCOL-5021 For the DELETE operation, empty magic values are only
written to database files for AUX column. Perform read-only operation
for other columns in the table to update the Casual Partitioning information.
2022-08-05 14:40:49 -04:00
Gagan Goel
86df9a972c MCOL-5021 Add prototype support for the AUX column in CREATE/DROP
DDL commands, single and multi-value INSERTs, cpimport, and
DELETE.
2022-08-05 14:40:49 -04:00
Roman Nozdrin
af9caf8d6e Merge pull request #2493 from mariadb-corporation/feature/MCOL-5174
MCOL-5174: Create bash script to restart processes in a loop.
2022-08-05 18:12:33 +03:00
mariadb-AlanMologorsky
7de8589139 MCOL-5174: Create bash script to restart processes in a loop.
[add] loop_process_starter.sh to oam/install_scripts
[fix] oam/install_scripts/CMakeLists.txt
2022-08-05 17:37:52 +03:00
Andrey Piskunov
82bfcd2b80 Restructure benchmarks (#2492)
* Add chargen

* Restructure

* Remove redundant line

* Correct directories

* Scale up the numbers
2022-08-05 16:30:18 +03:00
benthompson15
fd2ebba6de MCOL-5163: remove systemd coupling of processes. 2022-08-04 18:04:42 -05:00
Andrey Piskunov
8089690593 Add chargen 2022-08-04 20:11:08 +03:00
Andrey Piskunov
e51adb71b8 Add bench.lua for KIND_TEXT and fix bench.sh 2022-08-04 20:11:08 +03:00
Andrey Piskunov
af661c06ac Use utils.sh parsing 2022-08-04 20:11:08 +03:00