1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-07 03:22:57 +03:00
Commit Graph

413 Commits

Author SHA1 Message Date
Serguey Zefirov
cb326ba5fd Progress keeping commit 2025-07-04 11:00:41 +00:00
Serguey Zefirov
dc17e7cb3d Work for almost three days - progress keeping commit 2025-07-03 13:41:22 +00:00
Serguey Zefirov
0283598a38 Debug logs 2025-06-09 18:17:56 +03:00
Serguey Zefirov
09f949a73b A theory to test 2025-06-09 18:17:56 +03:00
Serguey Zefirov
649558a4d2 Debug logs 2025-06-09 18:17:56 +03:00
Serguey Zefirov
254efc0b45 A fix??? 2025-06-09 18:17:55 +03:00
Serguey Zefirov
8303560d82 Debug logs 2025-06-09 18:17:55 +03:00
Serguey Zefirov
72f8094db9 Debug logs 2025-06-09 18:17:55 +03:00
Serguey Zefirov
66812840a5 debug logs 2025-06-09 18:17:55 +03:00
Serguey Zefirov
cd0fa95054 debug logs 2025-06-09 18:17:55 +03:00
Serguey Zefirov
30501fefe0 Fix build 2025-06-09 18:17:55 +03:00
Serguey Zefirov
7118b6cdc1 Fix build 2025-06-09 18:17:55 +03:00
Serguey Zefirov
1b43cbd65d Bigger better fix for access to R/W nodes only 2025-06-09 18:17:55 +03:00
Serguey Zefirov
53b9086cb2 A small fix
Fix of .drone.jsonnet
2025-06-09 18:17:55 +03:00
Alexander Presnyakov
68d742d8e7 Try not to create connection to WES if it was not enabled in the config 2025-06-09 18:17:55 +03:00
Leonid Fedorov
aa7e0fb9b4 Deep build refactoring phase 1 (#3562)
* configcpp refactored
* logging and datatypes refactored

* more dataconvert
* chore(build): massive removals, auto add files to debian install file
* chore(codemanagement): nodeps headers, potentioal library
* chore(build): configure before autobake
* chore(build): use custom cmake commands for components, mariadb-plugin-columnstore.install generated
* chore(build): install deps as separate step for build-packages
* more deps
* check  debian/mariadb-plugin-columnstore.install automatically
* chore(build): add option for multibracnh compilation
* Fix warning
2025-05-30 14:05:21 +04:00
Leonid Fedorov
4a8f5830fd chore(build) update boost to 1.88 2025-05-20 21:43:43 +04:00
Leonid Fedorov
127dd09a7a fix back statics 2025-05-20 18:22:59 +04:00
Leonid Fedorov
6db2dc668f stubs and cmake formatting 2025-05-20 18:22:59 +04:00
Leonid Fedorov
2036e521c7 named linkage 2025-05-20 18:22:59 +04:00
Leonid Fedorov
a1019b7c0e chore(build): refactor main CMakeLists.txt (#3543)
* chore(build): refactor main CMakeLists.txt

* chore(build): fix boost version for packages, set clang-20 only for amd and arm

* chore(build): boost 4 sm

* chore(build): boost dep for rowgroup

* chore(build): toolset for boost

* chore(build): suppress clang warnings for boost

* chore(ci, build): use ASAN for unittest on ubuntu 24.04 only, added custom cmake flag option for bootstrap, custom params for new and existing pipelines

* chore(build): sort bootstrap flags

* chore(CI): remove publish pkg step, adding clickable link instead to publish steps, fix customenv
2025-05-20 05:00:48 +04:00
Leonid Fedorov
a0bee173f6 chore(build): fixes to satisfy clang19 warnings 2025-05-15 19:05:38 +04:00
Denis Khalikov
80277a5e7d DBBuilder: check syscat files instead dirs 2025-04-01 19:53:28 +04:00
drrtuy
02b8ea1331 feat(PP,ByteStream): new counting memory allocator 2025-03-27 22:12:48 +00:00
drrtuy
87981483e2 chore(compilation): to resolve unexpected boost warnings 2025-03-27 21:35:11 +04:00
Aleksei Antipovskii
0ab03c7258 chore(codestyle): mark virtual methods as override 2025-02-21 20:01:34 +04:00
Sergey Zefirov
3bcc2e2fda fix(memory leaks): MCOL-5791 - get rid of memory leaks in plugin code (#3365)
There were numerous memory leaks in plugin's code and associated code.
During typical run of MTR tests it leaked around 65 megabytes of
objects. As a result they may severely affect long-lived connections.

This patch fixes (almost) all leaks found in the plugin. The exceptions
are two leaks associated with SHOW CREATE TABLE columnstore_table and
getting information of columns of columnstore-handled table. These
should be fixed on the server side and work is on the way.
2024-12-06 09:04:55 +00:00
drrtuy
6f6e69815d feat(bytestream,serdes): Distribute BS buf size data type change to avoid implicit data type narrowing 2024-11-08 16:28:51 +04:00
Leonid Fedorov
5824a0cebe MCOL-5746 Cpimport: convert blob data from ascii hex when reads from STDIN (#3217)
Co-authored-by: Denis Khalikov <dennis.khalikov@gmail.com>
2024-06-27 14:23:07 +04:00
Leonid Fedorov
3f8758ba4c MCOL-5747 gcc-14.1.1 compile error - calloc - transposed args (#3216)
The arguments of calloc are the number of members and the sizeof the
member. Gcc-14.1.1 worked out how to tell the difference.

We correct this by transposing to gcc's will.

Co-authored-by: Daniel Black <daniel@mariadb.org>
2024-06-27 14:21:26 +04:00
Serguey Zefirov
2cd8f716c1 Fix MCOL-5035, a difference in INSERT and UPDATE behavior
The UPDATE statement wrote NULL when the column set is DATETIME and
value is '0000-00-00 00:00:00'. The problem was inside WriteEngine's
handling of UPDATE statements and this is where heart of change is.

Other changes are related to some obsolete data structures in DML/DDL
handling that just hanging around there, doing nothing.
2024-06-27 13:07:49 +03:00
Denis Khalikov
55ffacf546 MCOL-5555 Add support for startreadonly command. (#3081)
This patch adds support for `startreadonly` command which waits
until all active cpimport jobs are done and then puts controller node to readonly
mode.
2023-12-26 15:11:33 +04:00
Denis Khalikov
6315546557 Revert "MCOL-5555 Add support for startreadonly command."
This reverts commit 441cd9d34f.
2023-12-20 01:05:37 +03:00
Leonid Fedorov
97abd9866b fix(writeengine) MCOL-4202: use schema name when renaming table and change it's fields in syscat 2023-12-18 09:59:44 +03:00
Denis Khalikov
441cd9d34f MCOL-5555 Add support for startreadonly command.
This patch adds support for `startreadonly` command which waits
until all active cpimport jobs are done and then puts controller node to readonly
mode.
2023-10-16 16:11:50 +03:00
Gagan Goel
7f9c624626 MCOL-5573 Fix cpimport truncation of TEXT columns.
1. Restore the utf8_truncate_point() function in utils/common/utils_utf8.h
that I removed as part of the patch for MCOL-4931.

2. As per the definition of TEXT columns, the default column width represents
the maximum number of bytes that can be stored in the TEXT column. So the
effective maximum length is less if the value contains multi-byte characters.
However, if the user explicitly specifies the length of the TEXT column in a
table DDL, such as TEXT(65535), then the DDL logic ensures that enough number
of bytes are allocated (upto a system maximum) to allow upto that many number
of characters (multi-byte characters if the charset for the column is multi-byte,
such as utf8mb3).
2023-09-20 12:23:22 -04:00
Gagan Goel
e0d9b82705 MCOL-5021 Fix a minor bug related to the AUX column support in cpimport. 2023-09-06 16:48:54 -04:00
Gagan Goel
931f2b36a1 MCOL-4931 Make cpimport charset-aware. (#2938)
1. Extend the following CalpontSystemCatalog member functions to
   set CalpontSystemCatalog::ColType::charsetNumber, after the
   system catalog update to add charset number to calpontsys.syscolumn
   in MCOL-5005:
     CalpontSystemCatalog::lookupOID
     CalpontSystemCatalog::colType
     CalpontSystemCatalog::columnRIDs
     CalpontSystemCatalog::getSchemaInfo

2. Update cpimport to use the CHARSET_INFO object associated with the
   charset number retrieved from the system catalog, for a
   dictionary/non-dictionary CHAR/VARCHAR/TEXT column, to truncate
   long strings that exceed the target column character length.

3. Add MTR test cases.
2023-09-05 17:17:20 +03:00
Gagan Goel
d50a0fa2e6 MCOL-5005 Add charset number to system catalog - Part 2.
1. Extend the calpontsys.syscolumn system catalog table
  with a new column, 'charsetnum'.

  'charsetnum' field is set to the 'number' member of the
  'charset_info_st' struct defined in the server in m_ctype.h.

  For CHAR/VARCHAR/TEXT column types, 'charset_info_st' is
  initialized to the charset/collation of the column, which
  is set at the column-level or at the table-level in the DDL.

  For BLOB/VARBINARY binary column types, 'charset_info_st' is
  initialized to my_charset_bin (charsetnum=63).

  For all other column types, charsetnum is set to 0.

  2. Add support for the newly added 'charsetnum' column in the
  automatic system catalog upgrade logic in dbbuilder.

  For existing table definitions, charsetnum for the column is
  defaulted to 0.

  3. Add MTR test case that creates a few table definitions with
  a range of charset/collation combinations and queries the
  calpontsys.syscolumn system catalog table with the charsetnum
  field for the columns in the table DDLs.
2023-08-15 17:21:47 +00:00
Sergei Golubchik
30064e3d4a cpimport.bin doesn't really need libboost_program_options
linking with unused libraries creates a difference in dependencies
between --no-as-needed (gcc default) and --as-needed (default on
Fedora rpmbuild) builds.
2023-07-04 12:58:18 -04:00
Gagan Goel
ab7dfaa25b Fix resource leak in DDLProc/DMLProc/PrimProc/WriteengineServer processes.
As part of the charset support, a call to MY_INIT() was added at the
initialization of the above processes. This call initializes the MySQL
thread environment required by the charset library. However, the
accompanying my_end() call required to terminate this thread environment
was not added at the termination of these process, hence leaking
resources. As a fix, we move the MY_INIT() calls to the Child()
functions of these services and also add the missing my_end() call.
2023-06-23 20:18:32 +00:00
Leonid Fedorov
8f93fc3623 MCOL-5493: First portion of UBSan fixes (#2842)
Multiple UB fixes
2023-06-02 17:02:09 +03:00
Roman Nozdrin
4fe9cd64a3 Revert "No boost condition (#2822)" (#2828)
This reverts commit f916e64927.
2023-04-22 15:49:50 +03:00
Leonid Fedorov
f916e64927 No boost condition (#2822)
This patch replaces boost primitives with stdlib counterparts.
2023-04-22 00:42:45 +03:00
Leonid Fedorov
3ce19abdae Options to build with TSAN, UBSAN and skipping smoke (#2826) 2023-04-21 21:24:48 +03:00
Leonid Fedorov
c2d0fa24da replace boost::shared_array<T> to std::shared_ptr<T[]> 2023-04-14 10:33:27 +00:00
Leonid Fedorov
2e1394149b MCOL-5464: Fixes of bugs from ASAN warnings, part one (#2792)
* Fixes of bugs from ASAN warnings, part one

* MQC as static library, with nifty counter for global map and mutex

* Switch clang to 16

* link messageqcpp to execplan
2023-04-04 02:33:23 +03:00
Sergey Zefirov
b53c231ca6 MCOL-271 empty strings should not be NULLs (#2794)
This patch improves handling of NULLs in textual fields in ColumnStore.
Previously empty strings were considered NULLs and it could be a problem
if data scheme allows for empty strings. It was also one of major
reasons of behavior difference between ColumnStore and other engines in
MariaDB family.

Also, this patch fixes some other bugs and incorrect behavior, for
example, incorrect comparison for "column <= ''" which evaluates to
constant True for all purposes before this patch.
2023-03-30 21:18:29 +03:00
Otto Kekäläinen
70124ecc01 Fix trivial spelling errors
- occured -> occurred
- reponse -> response
- seperated -> separated

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2023-03-11 11:59:47 -08:00
Leonid Fedorov
56f2346083 Remove windows ifdefs 2023-03-02 15:59:42 +00:00