1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-07 03:22:57 +03:00
Commit Graph

1737 Commits

Author SHA1 Message Date
Roman Nozdrin
15f65eff15 Merge pull request #2655 from denis0x0D/MCOL-5263_2
MCOL-5263 Add support to ROLLBACK when PP were restarted.
2022-12-13 21:24:01 +03:00
Roman Nozdrin
d43e418a8c MCOL-5346 This patch forces TreeNode::getIntValue to use conversion for dict-based CHAR/VARCHAR and TEXT columns (#2657)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2022-12-13 18:13:22 +03:00
Denis Khalikov
d61780cab1 MCOL-5263 Add support to ROLLBACK when PP were restarted.
DMLProc starts ROLLBACK when SELECT part of UPDATE fails b/c EM facility in PP were restarted.
Unfortunately this ROLLBACK stuck if EM/PP are not yet available.
DMLProc must have a t/o with re-try doing ROLLBACK.
2022-12-13 16:18:53 +03:00
benthompson15
635a9fdb56 Merge pull request #2658 from dhall-MariaDB/patch_out_of_band
patch_out_of_band
2022-12-12 17:14:41 -06:00
david.hall
10e2834033 patch_out_of_band
Some changes made to 10.6-enterprise make a build using the out-of-band method of compiling columnstore not work. Out-of band means the source for the engine is not in the storage subdir of server, but rather in a stand alone directory. This is used by developers for easier develop work. In the case of out-of-band, INSTALL_LAYOUT is false in CMakeLists.txt
2022-12-12 14:17:09 -06:00
david.hall
369aea884e MCOL-5311 Add timezone to jobList in subquerytransformer
TimeZone was uninitialized in this scenario and led to undefined behavior.
2022-12-07 09:52:00 -06:00
Roman Nozdrin
a1d89d8f31 Merge pull request #2630 from dhall-MariaDB/sergchanges
Sergchanges
2022-12-02 19:08:08 +03:00
david.hall
7e3ad24437 Serge changes -- Add static joblist lib 2022-11-30 12:46:26 -06:00
Roman Nozdrin
bfbe5bf315 MCOL-5264 This patch replaces boost mutex locks with std analogs
boost::uniqie_lock dtor calls a fancy unlock logic that throws twice.
First if the mutex is 0 and second lock doesn't own the mutex.
The first condition failure causes unhandled exception for one of the clients
in DEC::writeToClient(). I was unable to find out why Linux can have a 0
mutex and replaced boost::mutex with std::mutex b/c stdlibc++ should
be more stable comparing with boost.
2022-11-24 17:19:24 +00:00
Leonid Fedorov
b936ed8b2e Fix some GCC-12 Build errors 2022-11-22 03:28:17 +03:00
Roman Nozdrin
84bb4e56b8 Merge pull request #2624 from mariadb-corporation/columnstore-22.08.4-1
Columnstore 22.08.4 1
2022-11-18 19:01:33 +03:00
Sergei Golubchik
246a4db8de fix C API includes
ColumnStore used to include server's mysql.h
but link all tools with libmariadb.so

There's no guarantee that this would work, even with workarounds
it had in dbcon/mysql/sm.cpp

Fix:
* tools (linked with libmariadb.so) *must* include libmariadb's mysql.h
* as a hack prevent service_thd_timezone.h from being loaded into tools,
  as it conflicts with libmariadb's mysql.h
* server plugin *must* include server's mysql.h
* also don't link every tool with libmariadb.so, link the helper library
  (liblibmysqlclient.so) that actually needs it, tools use this
  helper library, not libmariadb.so directly
2022-11-17 12:02:07 -06:00
Sergei Golubchik
21c3bbce16 do *not* link ha_columnstore.so with libmariadb.so
this means some libraries have to be compiled twice -
for tools with libmariadb.so and for plugin, without.
2022-11-17 11:53:42 -06:00
david.hall
9b84bf57c9 From serg: add dependency for generated header files errorids.h messageids.h 2022-11-17 11:46:10 -06:00
Roman Nozdrin
4b15a7e8a9 Merge pull request #2628 from denis0x0D/MCOL-5265
[MCOL-5265] Change boost:shared_ptr to std::shared_ptr.
2022-11-15 16:45:19 +03:00
Denis Khalikov
e09d24cb8d [MCOL-5265] Change boost:shared_ptr to std::shared_ptr.
This is attempt to make some part of the code more stable.
For some reason we can get a spurious nullptr for boost::shared_ptr
which cause an assert and abort.
2022-11-14 18:53:53 +03:00
David.Hall
61d5f80aa0 MCOL-5279 This approach executes same node DEC::writeToClient the last taking multiple ExeMgrs into account (#2623)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2022-11-11 13:17:45 -06:00
Roman Nozdrin
e71bb49267 Merge pull request #2617 from mariadb-corporation/spetrunia-tmp
Add ability to compile against MariaDB with new cost model
2022-11-10 20:49:08 +03:00
Sergei Petrunia
b7fc8e4609 Add ability to compile against MariaDB with new cost model
If MariaDB defines MARIADB_NEW_COST_MODEL, then ha_mcs::scan_time()
has a different signature.
2022-11-10 18:16:44 +03:00
Leonid Fedorov
37fd915a08 Serg`s patch for develop-6 revised for develop https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/2614 2022-11-09 22:41:38 +00:00
David.Hall
f5aeb1c1a7 Merge branch 'develop' into columnstore-22.08.3-1 2022-11-04 16:12:56 -05:00
Roman Nozdrin
09d785fc9b MCOL-5279 This commit fixes the corner case of the issue that can rarely happen when flow control is used to backpressure PPs that overflows EM with Primitive messages. It also adds a relevant test that makes sense in a multi-node scenario only. (#2605)
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
2022-11-03 16:36:54 -05:00
David.Hall
e3ab19e639 Remove unreferenced variable 2022-11-03 11:29:24 -05:00
David.Hall
4ccab75720 Mcol 5279 2 (#2606)
* MCOL-5279 Send ByteStream with Primitive message to remote PPs first
and to the local PP last

MCOL-5166 forces EM to interact with PP over a messaging queue when they are in the same process. When ByteStream is taken from the queue it is drained so it is impossible to re-use the same ByteStream to send to the other nodes.

* MCOL-5279 Fix a corner case when sending PP messages
Multi-node can break when flow control is used to backpressure PPs that overflows EM with Primitive messages.
2022-11-03 09:42:12 -05:00
Leonid Fedorov
88404f70f1 MCOL-5013: added terminated_by, enclosed_by, escaped_by for s3dataload 2022-11-03 15:46:08 +03:00
David.Hall
a1c8f519c1 MCOL-5279 Send ByteStream with Primitive message to remote PPs first (#2603)
and to the local PP last

MCOL-5166 forces EM to interact with PP over a messaging queue when they are in the same process. When ByteStream is taken from the queue it is drained so it is impossible to re-use the same ByteStream to send to the other nodes.
2022-11-02 16:09:45 -05:00
Roman Nozdrin
76a33ac246 MCOL-5279 This patch forces to send ByteStream with a Primitive message to the remote PPs first and to the local PP last
MCOL-5166 forces EM to interact with PP over a messaging queue when they are in the same process. When ByteStream is taken from the queue it is drained so it is impossible to re-use the same ByteStream to send to the other nodes.
2022-11-01 19:54:52 +00:00
David.Hall
6d680ceb8c MCOL-603 Add error message for sum(a=1) (#2597)
* MCOL-603 Add error message for sum(a=1)
This isn't currently supported, but rather than emitting an error, it asserted and crashed.
2022-11-01 10:13:40 -05:00
mariadb-AndreyPiskunov
d7f4ec73c5 Small fixes + test sorting 2022-10-31 14:56:32 +02:00
mariadb-AndreyPiskunov
b57d2c30fe Minor fixes 2022-10-31 14:56:32 +02:00
mariadb-AndreyPiskunov
315e4be2d8 First working attempt for json_arrayagg 2022-10-31 14:56:32 +02:00
mariadb-AndreyPiskunov
1714b75434 Non working attempt to do MCOL-5227 2022-10-31 14:56:32 +02:00
Roman Nozdrin
a0086bc561 Adding NULL flag into ConstString class 2022-10-21 18:13:18 +00:00
Gagan Goel
8a9b6b32e7 MCOL-5000 Disable ALTER TABLE statement execution on replicas.
Exit early from the plugin execution of ALTER TABLE statements
on the replica nodes. This is to prevent re-execution of syscat
table population from the replica nodes which should only be
executed once by the primary node in a CS cluster setup.
2022-10-14 17:20:55 +00:00
NTH19
7d76dc4534 AUX column scan(MCOL-5021) effectively disables vectorized scanning on
ARM platforms. This patch resolves this issue and unifies AUX column
processing at x86 and ARM using tempate class SimdProcessor.
The patch also replaces uint16_t mask previously used in column.cpp and
SimProcessor code with a native masks that platform uses, e.g. __m128i
or __m128 on x86 and variety of masks on ARM.
To unify the processing I introduced a new filtering Compare Operator - COMPARE_NULLEQ.
with a 'c1 IS NULL semantics'.
2022-10-07 10:32:54 +00:00
Jigao Luo
7f97a66184 [MCOL-4590] UNION Performance Improvement with the focus on the normalize functions.
This patch improves the runtime performance of UNION processing in CS, as reported JIRA issue MCOL 4590. The idea of the optimization is to infer the normalize seperate functions beforehand and perform the normalization individually later, instead of a huge switch body of all normalization. This patch also cover engineering optimization, removing the hotspots in UNION processing. After application of this patch, the normalize part takes only about 25% of the whole UNION query in our experiment avg case.

Signed-off-by: Jigao Luo <luojigao@outlook.com>
2022-09-09 22:49:18 +02:00
david.hall
28a12eda82 Fix up CmakeLists.txt
A better way to fix the dependencies
2022-09-06 16:15:07 -05:00
david.hall
bcaf867731 Fix up cmake to build out of band
The main CmakeLists.txt was using MY_CHECK_AND_SET_COMPILER_FLAG before the include. This works in-band with server because it was already included in server's CmakeLists.txt.
dbcon/mysql included curl as a build dependency. We don't build curl. It's a lib dependency. Not sure why it works in-band. One wouldn't think it should.
2022-09-06 16:08:47 -05:00
Ziy1-Tan
cdd41f05f3 MCOL-785 Implement DISTRIBUTED JSON functions
The following functions are created:
Create function JSON_VALID and test cases
Create function JSON_DEPTH and test cases
Create function JSON_LENGTH and test cases
Create function JSON_EQUALS and test cases
Create function JSON_NORMALIZE and test cases
Create function JSON_TYPE and test cases
Create function JSON_OBJECT and test cases
Create function JSON_ARRAY and test cases
Create function JSON_KEYS and test cases
Create function JSON_EXISTS and test cases
Create function JSON_QUOTE/JSON_UNQUOTE and test cases
Create function JSON_COMPACT/DETAILED/LOOSE and test cases
Create function JSON_MERGE and test cases
Create function JSON_MERGE_PATCH and test cases
Create function JSON_VALUE and test cases
Create function JSON_QUERY and test cases
Create function JSON_CONTAINS and test cases
Create function JSON_ARRAY_APPEND and test cases
Create function JSON_ARRAY_INSERT and test cases
Create function JSON_INSERT/REPLACE/SET and test cases
Create function JSON_REMOVE and test cases
Create function JSON_CONTAINS_PATH and test cases
Create function JSON_OVERLAPS and test cases
Create function JSON_EXTRACT and test cases
Create function JSON_SEARCH and test cases

Note:
Some functions output differs from MDB because session variables that affects functions output,e.g JSON_QUOTE/JSON_UNQUOTE
This depends on MCOL-5212
2022-08-30 22:22:23 +08:00
Roman Nozdrin
568ac5ba7b Merge pull request #2535 from mariadb-corporation/int128Fields
Int128 fields
2022-08-28 17:42:15 +01:00
Leonid Fedorov
d2432f9bf6 get rid of pointers for 128 fields 2022-08-26 15:12:22 +00:00
mariadb-AndreyPiskunov
0863ecd279 Replace getBinaryField 2022-08-25 18:21:43 +03:00
Roman Nozdrin
72e264e8ef MCOL-5199 This patch solves the overal performance degradation introduced with a new way of char columns hashing
in aggregation code
The patch disables padding that forces hasher to calculate over the whole 2k buffer. This patch also moves hashing code
into the common place where it belongs.
2022-08-24 19:07:06 +00:00
NTH19
df7c967d54 when eq Filtercount <6 ,the speed of for loop is faster than hashmap
add threshold for eqFilter
2022-08-23 18:45:36 +08:00
Roman Nozdrin
14bbba2c97 Merge pull request #2510 from tntnatbry/MCOL-5021-followup
MCOL-5021 Followup.
2022-08-19 20:44:56 +03:00
Gagan Goel
6a6fee5969 MCOL-5021 Followup.
Allow the compiler to inline the call to nextColValue() in column.cpp.
2022-08-18 19:35:35 +00:00
Leonid Fedorov
d02b3403b7 Changed function name, schema and params order to achieve
columnstore_info.load_from_s3("<bucket>", "<file_name>", "<db_name>", "<table_name>");
2022-08-18 10:41:22 +00:00
Roman Nozdrin
0f2f0fc0ba Merge pull request #2507 from denis0x0D/MCOL-4715
MCOL-4715 Mixed inner and outer joins with "null" filter for the table which is not involved into the outer join produces wrong results.
2022-08-17 18:50:43 +03:00
Denis Khalikov
59166608b1 MCOL-4715 Mixed inner and outer joins with "null" filter for the table which is not involved into the outer join produces wrong results. 2022-08-16 17:13:03 +03:00
Roman Nozdrin
56bbef62e6 Merge pull request #2406 from tntnatbry/MCOL-5021-dev
MCOL-5021 AUX column implementation to improve DELETE performance.
2022-08-15 19:03:42 +03:00