1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-08 14:22:09 +03:00
Commit Graph

6318 Commits

Author SHA1 Message Date
Gagan Goel
cbfdae3481 MCOL-5021 Code changes based on review feedback. 2022-08-05 14:40:50 -04:00
Gagan Goel
11b7ee2f11 MCOL-5021 Disallow the following ALTER TABLE ADD COLUMN statement:
ALTER TABLE calpontsys.systable ADD COLUMN (auxcolumnoid INT NOT NULL DEFAULT 0);
2022-08-05 14:40:50 -04:00
Gagan Goel
1355237ca3 MCOL-5021 Some minor fixes. 2022-08-05 14:40:50 -04:00
Gagan Goel
94e9f55940 MCOL-5021 Add a new member function to the DBRM class, DBRM::addToLBIDList().
This function iterates over lbidList (populated by an earlier call to
DBRM::getUncommittedExtentLBIDs()) to find those LBIDs which belong to
the AUX column. It then finds the corresponding LBIDs for all other columns
which belong to the same table as the AUX LBID and appends them to lbidList.
The updated lbidList is used by invalidateUncommittedExtentLBIDs() to update
the casual partitioning information.

DBRM::addToLBIDList() only comes into play in case of a transaction ROLLBACK.
2022-08-05 14:40:50 -04:00
Gagan Goel
0818b95caa MCOL-5021 Update MTR tests to reflect changes introduced by this feature. 2022-08-05 14:40:50 -04:00
Gagan Goel
9b6d3c3870 MCOL-5021 Add support for AUX column in the client code calling
CalpontSystemCatalog::columnRIDs().
2022-08-05 14:40:49 -04:00
Gagan Goel
439db48c5a MCOL-5021 Add support for the AUX column in TRUNCATE table processing. 2022-08-05 14:40:49 -04:00
Gagan Goel
ea1861fdb5 MCOL-5021 Add a new function to CalpontSystemCatalog class,
isAUXColumnOID(), to check if a given OID is an auxilliary
column OID.
2022-08-05 14:40:49 -04:00
Gagan Goel
262cd5c501 MCOL-5021 Remove hard-coded values for data type, column width
and compression type for the AUX column, and replace them with
constants defined in the execplan namespace.
2022-08-05 14:40:49 -04:00
Gagan Goel
c8b6b154bf MCOL-5021 Add an option in Columnstore.xml, fastdelete (disabled
by default), which when enabled, indiscriminately invalidates all
column extents and performs the actual DELETE only on the AUX
column. The trade-off with this approach would now be that the
first SELECT for certain query patterns (those containing a WHERE
predicate) after the DELETE operation will slow down as the
invalidated column extent would need to be scanned again to set
the min/max values.
2022-08-05 14:40:49 -04:00
Gagan Goel
2280b1dd25 MCOL-5021 Add support for the AUX column in ExeMgr and PrimProc.
In the joblist code, in addition to sending the lbid of the SCAN
column, we also send the corresponding lbid of the AUX column to PrimProc.

In the primitives processor code in PrimProc, we load the AUX column
block (8192 rows since the AUX column is implemented as a 1-byte
UNSIGNED TINYINT) into memory and then pass it down to the low-level
scanning (vectorized scanning as applicable) routine to build a non-Empty
mask for the block being processed to filter out DELETED rows based on
comparison of the AUX block row to the empty magic value for the AUX column.
2022-08-05 14:40:49 -04:00
Gagan Goel
60eb0f86ec MCOL-5021 non-AUX column files are opened in read-only mode during
the DELETE operation. ColumnOp::readBlock() calls can cause writes
to database files when the active chunk list in ChunkManager is full.
Since non-AUX columns are read-only for the DELETE operation, we prevent
writes of compressed chunks and header for these columns by passing
an isReadOnly flag to CompFileData which indicates whether the column
is read-only or read-write.
2022-08-05 14:40:49 -04:00
Gagan Goel
35a3a93964 MCOL-5021 For the DELETE operation, empty magic values are only
written to database files for AUX column. Perform read-only operation
for other columns in the table to update the Casual Partitioning information.
2022-08-05 14:40:49 -04:00
Gagan Goel
86df9a972c MCOL-5021 Add prototype support for the AUX column in CREATE/DROP
DDL commands, single and multi-value INSERTs, cpimport, and
DELETE.
2022-08-05 14:40:49 -04:00
Roman Nozdrin
af9caf8d6e Merge pull request #2493 from mariadb-corporation/feature/MCOL-5174
MCOL-5174: Create bash script to restart processes in a loop.
2022-08-05 18:12:33 +03:00
mariadb-AlanMologorsky
7de8589139 MCOL-5174: Create bash script to restart processes in a loop.
[add] loop_process_starter.sh to oam/install_scripts
[fix] oam/install_scripts/CMakeLists.txt
2022-08-05 17:37:52 +03:00
Andrey Piskunov
82bfcd2b80 Restructure benchmarks (#2492)
* Add chargen

* Restructure

* Remove redundant line

* Correct directories

* Scale up the numbers
2022-08-05 16:30:18 +03:00
benthompson15
fd2ebba6de MCOL-5163: remove systemd coupling of processes. 2022-08-04 18:04:42 -05:00
Andrey Piskunov
8089690593 Add chargen 2022-08-04 20:11:08 +03:00
Andrey Piskunov
e51adb71b8 Add bench.lua for KIND_TEXT and fix bench.sh 2022-08-04 20:11:08 +03:00
Andrey Piskunov
af661c06ac Use utils.sh parsing 2022-08-04 20:11:08 +03:00
Andrey Piskunov
e8c9cd0502 Switch to with_benchmarks every time 2022-08-04 20:11:08 +03:00
Andrey Piskunov
1ec0262d79 Allow custom file with data 2022-08-04 20:11:08 +03:00
Andrey Piskunov
a88e335cb7 Don't hardocde table name 2022-08-04 20:11:08 +03:00
Andrey Piskunov
8290374f5b First attempt to automate my benchmark 2022-08-04 20:11:08 +03:00
David.Hall
d3b57ec767 MCOL-4800 emit error if IN filter > 65535 entries (#2480)
* MCOL-4800 emit error if IN filter > 65535 entries
2022-08-04 19:21:58 +03:00
Roman Nozdrin
a9d8924683 MCOL-5166 This patch adds support for in-memory communication b/w EM to PP via a shared queue in DEC class
JobList low-level code relateod to primitive jobs now uses shared pointers instead of ByteStream refs talking to DEC
b/c same-node EM-PP communication now goes over a queue in DEC instead of a network hop.
PP now has a separate thread that processes the primitive job messages from that DEC queue.
2022-08-04 18:51:31 +03:00
Roman Nozdrin
9ef16c6ded Merge pull request #2433 from mariadb-corporation/vect_min_max
Vectorized update min max
2022-08-04 16:39:36 +03:00
Roman Nozdrin
3fb02ca22f Merge pull request #2479 from denis0x0D/port_from_dev6
MCOL-5106 Port patches for develop from develop-6 related to rebuildEM tool.
2022-08-04 16:38:55 +03:00
Andrey Piskunov
c3a5731890 Rename cmpGt2 2022-08-04 16:16:38 +03:00
Andrey Piskunov
24b2c1c283 Vectorizing min/max for KIND_TEXT 2022-08-04 16:16:38 +03:00
NTH19
c4798ce585 fix 2022-08-04 16:16:38 +03:00
NTH19
231930b71d update 2022-08-04 16:16:38 +03:00
NTH19
19ca844cd1 support_max_min 2022-08-04 16:16:38 +03:00
Andrey Piskunov
9f5a1b559d Yes to unittests 2022-08-04 16:16:38 +03:00
Andrey Piskunov
bcb89e00f4 Remove include 2022-08-04 16:16:38 +03:00
Andrey Piskunov
589b786fda Don't ignore null or empty in calculation 2022-08-04 16:16:38 +03:00
Andrey Piskunov
04ac04ff74 Temporary test fix 2022-08-04 16:16:38 +03:00
Andrey Piskunov
5c6cd2cca3 use vect update for everything except TEXT 2022-08-04 16:16:38 +03:00
Andrey Piskunov
225f54fd79 Tests for simd min/max 2022-08-04 16:16:38 +03:00
Andrey Piskunov
20f48fd730 Vectorized update min max 2022-08-04 16:16:38 +03:00
Andrey Piskunov
f88a3bfc65 Remove include 2022-08-04 16:16:38 +03:00
Andrey Piskunov
b8200acd3b Don't ignore null or empty in calculation 2022-08-04 16:16:38 +03:00
Andrey Piskunov
2a7da39610 Temporary test fix 2022-08-04 16:16:38 +03:00
Andrey Piskunov
c4df7925d1 use vect update for everything except TEXT 2022-08-04 16:16:38 +03:00
Andrey Piskunov
1681edaca0 Tests for simd min/max 2022-08-04 16:16:38 +03:00
Andrey Piskunov
9930d0dedd Vectorized update min max 2022-08-04 16:16:38 +03:00
Roman Nozdrin
37d8f14250 Merge pull request #2483 from mariadb-corporation/rocky9
Rockylinux9 CI support
2022-08-04 15:35:18 +03:00
Denis Khalikov
2e2fc7d4a3 [MCOL-5106] Support 48 extents per file for rebuildEM.
This patch increases the number of extents per segment file
for rebuildEM tool.
2022-08-04 13:58:17 +03:00
Leonid Fedorov
8b2a78a553 RockyLinux 9 CI builds support 2022-08-04 01:05:04 +03:00