1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-05 16:15:50 +03:00
Commit Graph

329 Commits

Author SHA1 Message Date
Sergei Golubchik
246a4db8de fix C API includes
ColumnStore used to include server's mysql.h
but link all tools with libmariadb.so

There's no guarantee that this would work, even with workarounds
it had in dbcon/mysql/sm.cpp

Fix:
* tools (linked with libmariadb.so) *must* include libmariadb's mysql.h
* as a hack prevent service_thd_timezone.h from being loaded into tools,
  as it conflicts with libmariadb's mysql.h
* server plugin *must* include server's mysql.h
* also don't link every tool with libmariadb.so, link the helper library
  (liblibmysqlclient.so) that actually needs it, tools use this
  helper library, not libmariadb.so directly
2022-11-17 12:02:07 -06:00
Leonid Fedorov
37fd915a08 Serg`s patch for develop-6 revised for develop https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/2614 2022-11-09 22:41:38 +00:00
Roman Nozdrin
d22627af7d Merge pull request #2566 from denis0x0D/MCOL-5191_1
MCOL-5191 Add MCV statistics.
2022-10-30 15:49:46 +03:00
Roman Nozdrin
878a8ab857 Compilation error fixes for the recent updates in container images 2022-10-14 17:53:48 +03:00
Denis Khalikov
e299a8409d MCOL-5191 Add MCV statistics.
This patch adds:
1. Initial version of random sampling.
2. Initial version of MCV statistics.
2022-10-09 22:26:40 +03:00
NTH19
7d76dc4534 AUX column scan(MCOL-5021) effectively disables vectorized scanning on
ARM platforms. This patch resolves this issue and unifies AUX column
processing at x86 and ARM using tempate class SimdProcessor.
The patch also replaces uint16_t mask previously used in column.cpp and
SimProcessor code with a native masks that platform uses, e.g. __m128i
or __m128 on x86 and variety of masks on ARM.
To unify the processing I introduced a new filtering Compare Operator - COMPARE_NULLEQ.
with a 'c1 IS NULL semantics'.
2022-10-07 10:32:54 +00:00
Roman Nozdrin
eb57103c06 This patch fixes nullptr dereferencing in the same node queue thread 2022-08-31 18:55:48 +00:00
NTH19
df7c967d54 when eq Filtercount <6 ,the speed of for loop is faster than hashmap
add threshold for eqFilter
2022-08-23 18:45:36 +08:00
Gagan Goel
6a6fee5969 MCOL-5021 Followup.
Allow the compiler to inline the call to nextColValue() in column.cpp.
2022-08-18 19:35:35 +00:00
Roman Nozdrin
56bbef62e6 Merge pull request #2406 from tntnatbry/MCOL-5021-dev
MCOL-5021 AUX column implementation to improve DELETE performance.
2022-08-15 19:03:42 +03:00
Andrey Piskunov
c906172bf5 MCOL-5180 Check CPU vector instructions set in installer and PrimProc on start (#2499)
* Add check for simd acrh support

* Updates

* More polite and detailed error messages

* Updates

* Always true to conditional

Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
2022-08-11 15:28:22 +03:00
Gagan Goel
cbfdae3481 MCOL-5021 Code changes based on review feedback. 2022-08-05 14:40:50 -04:00
Gagan Goel
11b7ee2f11 MCOL-5021 Disallow the following ALTER TABLE ADD COLUMN statement:
ALTER TABLE calpontsys.systable ADD COLUMN (auxcolumnoid INT NOT NULL DEFAULT 0);
2022-08-05 14:40:50 -04:00
Gagan Goel
1355237ca3 MCOL-5021 Some minor fixes. 2022-08-05 14:40:50 -04:00
Gagan Goel
262cd5c501 MCOL-5021 Remove hard-coded values for data type, column width
and compression type for the AUX column, and replace them with
constants defined in the execplan namespace.
2022-08-05 14:40:49 -04:00
Gagan Goel
c8b6b154bf MCOL-5021 Add an option in Columnstore.xml, fastdelete (disabled
by default), which when enabled, indiscriminately invalidates all
column extents and performs the actual DELETE only on the AUX
column. The trade-off with this approach would now be that the
first SELECT for certain query patterns (those containing a WHERE
predicate) after the DELETE operation will slow down as the
invalidated column extent would need to be scanned again to set
the min/max values.
2022-08-05 14:40:49 -04:00
Gagan Goel
2280b1dd25 MCOL-5021 Add support for the AUX column in ExeMgr and PrimProc.
In the joblist code, in addition to sending the lbid of the SCAN
column, we also send the corresponding lbid of the AUX column to PrimProc.

In the primitives processor code in PrimProc, we load the AUX column
block (8192 rows since the AUX column is implemented as a 1-byte
UNSIGNED TINYINT) into memory and then pass it down to the low-level
scanning (vectorized scanning as applicable) routine to build a non-Empty
mask for the block being processed to filter out DELETED rows based on
comparison of the AUX block row to the empty magic value for the AUX column.
2022-08-05 14:40:49 -04:00
Roman Nozdrin
a9d8924683 MCOL-5166 This patch adds support for in-memory communication b/w EM to PP via a shared queue in DEC class
JobList low-level code relateod to primitive jobs now uses shared pointers instead of ByteStream refs talking to DEC
b/c same-node EM-PP communication now goes over a queue in DEC instead of a network hop.
PP now has a separate thread that processes the primitive job messages from that DEC queue.
2022-08-04 18:51:31 +03:00
Andrey Piskunov
c3a5731890 Rename cmpGt2 2022-08-04 16:16:38 +03:00
Andrey Piskunov
24b2c1c283 Vectorizing min/max for KIND_TEXT 2022-08-04 16:16:38 +03:00
NTH19
231930b71d update 2022-08-04 16:16:38 +03:00
NTH19
19ca844cd1 support_max_min 2022-08-04 16:16:38 +03:00
Andrey Piskunov
bcb89e00f4 Remove include 2022-08-04 16:16:38 +03:00
Andrey Piskunov
589b786fda Don't ignore null or empty in calculation 2022-08-04 16:16:38 +03:00
Andrey Piskunov
5c6cd2cca3 use vect update for everything except TEXT 2022-08-04 16:16:38 +03:00
Andrey Piskunov
20f48fd730 Vectorized update min max 2022-08-04 16:16:38 +03:00
Andrey Piskunov
f88a3bfc65 Remove include 2022-08-04 16:16:38 +03:00
Andrey Piskunov
b8200acd3b Don't ignore null or empty in calculation 2022-08-04 16:16:38 +03:00
Andrey Piskunov
c4df7925d1 use vect update for everything except TEXT 2022-08-04 16:16:38 +03:00
Andrey Piskunov
9930d0dedd Vectorized update min max 2022-08-04 16:16:38 +03:00
Roman Nozdrin
3b87532413 Revert "This patch disables FairThreadPool to double check if this feature contributes to multiple strange side-effects and ocassional failed MTR tests"
This reverts commit b78cbffa93.
2022-07-22 14:04:06 +00:00
Roman Nozdrin
b78cbffa93 This patch disables FairThreadPool to double check if this feature contributes to multiple strange side-effects and ocassional failed MTR tests 2022-07-20 11:17:19 +00:00
Roman Nozdrin
4d41a945db This patch removes Out-Of-Band pool from PP 2022-07-09 12:38:52 +00:00
Roman Nozdrin
2c2e1f9924 This patch disables OOB pool in PP 2022-07-09 12:38:52 +00:00
Roman Nozdrin
a3c582d9fe WIP with map clean-up 2022-07-09 12:38:52 +00:00
Roman Nozdrin
0e8014db02 MCOL-5044 Adding EXTRA thread logic into FairThreadPool 2022-07-09 12:38:52 +00:00
Roman Nozdrin
6cff14997d Revert "This reverts MCOL-5044 AKA FairThreadPool that breaks regr test002"
This reverts commit 61359119ad.
2022-07-09 12:38:51 +00:00
Roman Nozdrin
1624c347f6 MCOL-5152 This patch enables PP to put ByteStreams into DEC input queue directly for a local PP-EM connection 2022-07-04 09:06:40 +00:00
Roman Nozdrin
fcf8596089 Merge pull request #2403 from denis0x0D/MCOL-5109
[MCOL-5109] Make PPS as singleton
2022-06-21 16:17:05 +03:00
david.hall
a227fccfa4 MCOL-4841 compiler fixes 2022-06-14 15:02:05 -05:00
david.hall
6d47529499 Merge branch 'develop' into MCOL-4841 2022-06-14 14:41:41 -05:00
david.hall
d4cf894edc MCOL-4841 fix some compiler issues 2022-06-14 14:32:01 -05:00
Roman Nozdrin
61359119ad This reverts MCOL-5044 AKA FairThreadPool that breaks regr test002
This reverts commit e40c16bd56, reversing
changes made to 18e6b1d77b.
2022-06-10 14:17:59 +00:00
David.Hall
272246e9fa Merge branch 'develop' into MCOL-4841 2022-06-09 16:58:33 -05:00
david.hall
3b6449842f Merge branch 'develop' into MCOL-4841
# Conflicts:
#	exemgr/main.cpp
#	oam/etc/Columnstore.xml.singleserver
#	primitives/primproc/primproc.cpp
2022-06-09 10:07:26 -05:00
Denis Khalikov
467fe0b401 [MCOL-5109] Make a singleton from ServicePrimProc.
This patch makes a singleton from ServicePrimProc.
2022-06-07 13:27:45 +03:00
Roman Nozdrin
fd8ba33f21 MCOL-5044 This patch replaces PriorityThreadPool with FairThreadPool that uses a simple
operations + morsel size weight model to equally allocate CPU b/w parallel query morsels.
This patch delivers better parallel query timings distribution(timings graph resembles normal
distribution with a bigger left side thus more queries runs faster comparing with PrioThreadPool-based
single-node installation).
See changes in batchprimitiveprocessor-jl.h and comments in fair_threadpool.h for
important implementation details
2022-06-03 10:08:12 +00:00
Denis Khalikov
94ca6fdd34 [MCOL-5001] Explicitly initialize startupRaceFlag.
Explicitly initialize `startupRaceFlag_` to avoid PrimProc freezing on some platforms.
2022-05-18 13:38:31 +03:00
David.Hall
bbb168a846 Mcol 4560 (#2337)
* MCOL-4560 remove unused xml entries and code that references it.
There is reader code and variables for some of these settings, but nobody uses them.
2022-04-18 18:00:17 -04:00
Leonid Fedorov
5820a21e19 Merge pull request #2331 from drrtuy/MCOL-5001-pp-em-combo-merge-1
Mcol 5001 pp em combo merge 1
2022-04-13 15:16:18 +03:00