1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-04-26 11:48:52 +03:00

124 Commits

Author SHA1 Message Date
Roman Nozdrin
63dcaa387f MCOL-641 Simple INSERT with one record works with this commit. 2020-11-18 13:47:00 +00:00
Roman Nozdrin
df65543dd4 MCOL-641 This commit contains fixes for the rebase that mostly adds
WE_BINARY and WE_INT128 into switch-case blocks.
2020-11-18 13:47:00 +00:00
Roman Nozdrin
c9f42fb5cc MCOL-641 PoC version for DECIMAL(38) using BINARY as a basis. 2020-11-18 13:47:00 +00:00
Gagan Goel
32f6167067 MCOL-641 Work of Ivan Zuniga on basic read and write support for Binary16 2020-11-18 13:47:00 +00:00
Alexey Antipovsky
f4a6294c95 Fix ignored qualifiers warnings (-Wignored-qualifiers) 2020-11-17 15:03:10 +03:00
Roman Nozdrin
6f120d2637 MCOL-4328 MCS avoids chown() calls for files that are on S3
MCS now chowns created directories hierarchy not only files and
immediate parent directories

Minor changes to cpimport's help printout

cpimport's -f option is now mandatory with mode 2
2020-10-09 11:02:31 +00:00
Roman Nozdrin
328ae25650 MCOL-4328 There is a new option in both cpimport and cpimport.bin to asign
an owner for all data files created by cpimport

The patch consists of two parts: cpimport.bin changes, cpimport splitter
changes

cpimport.bin computes uid_t and gid_t early and propagates it down the stack
where MCS creates data files
2020-10-03 14:05:29 +00:00
Roman Nozdrin
7e868bc588 add dependency for generated header files errorids.h messageids.h patch made by Ben. 2020-07-10 18:38:49 +00:00
benthompson15
eac7dab096 MCOL-4030: first commit of warning removals unneed const and missing virtual dtors. 2020-06-23 13:51:36 -05:00
Patrick LeBlanc
5440977f45
Merge pull request #1197 from tntnatbry/develop-batchinserts-improve
Improve batch inserts.
2020-06-02 14:05:47 -05:00
Jose
98abf95eae MCOL-3991 MCS is now single package and properly uninstalls 2020-05-12 13:36:24 +00:00
Gagan Goel
d4d0ebdf5d Improve batch inserts.
1) Instead of making dbrm calls to writeVBEntry() per block,
     we make these calls per batch. This can have non-trivial
     reductions in the overhead of these calls if the batch size
     is large.

  2) In dmlproc, do not deserialize the whole insertpackage, which
     consists of the complete record set per column, which would be
     wasteful as we only need some metadata fields from insertpackage
     here. This is only done for batch inserts at the moment, this
     should also be applied to single inserts.
2020-05-10 19:38:06 -04:00
Patrick LeBlanc
a65c8e4544 MCOL-3251
Moved the new logic into we_fileop to maintain existing encapsulation.
2020-03-13 15:18:55 -04:00
Patrick LeBlanc
ffcd527c4e Merge pull request #1052 from mariadb-corporation/MCOL-3791
MCOL-3791 This patch implicitly enables disk space preallocation for …
2020-02-28 16:20:07 -05:00
Sergei Golubchik
586391e1ca compilation failure
error: reference to 'mutex' is ambiguous
note: candidates are: 'class boost::mutex'
note:                 'class std::mutex'
2019-12-19 18:13:39 +01:00
Andrew Hutchings
49994f7bc3 Fix warnings found in DEBUG combined build
Fixes:
* Irrelevant where conditions
* Irrelevant const
* A potential infinite loop in treenode
* Bad implicit case fallthroughs
* Explicit markings for required case fallthroughs
* Unused variables
* Unused function

Also disabled some warnings for now which we should fix later.
2019-12-10 16:33:08 +00:00
benthompson15
207fb3f588
Merge pull request #960 from LinuxJedi/unified_build
Unified build for ES and MCS
2019-12-05 01:19:23 +01:00
Roman Nozdrin
766d84ada8 MCOL-3585 Enabled MCOL-498 by default for all dbroots. 2019-12-04 06:49:31 -06:00
Andrew Hutchings
7489d0bfd0 MCOL-3625 Rename packages
Rename packages to MariaDB-columnstore-engine, MariaDB-columnstore-libs
and MariaDB-columnstore-platform.

Also add the "columnstore-" prefix the the components so that MariaDB's
packaging system understands then and add a line to include them in
MariaDB's packaging.

In addition
* Fix S3 building for dist source build
* Fix Debian 10 dependency issue
* Fix git handling for dist builds
* Add support for MariaDB's RPM building
* Use MariaDB's PCRE and readline
* Removes a few dead files
* Fix Boost noncopyable includes
2019-12-04 11:04:39 +00:00
Andrew Hutchings
a8cd34f86d Add support for building from server 2019-12-03 21:21:28 +00:00
Andrew Hutchings
8ab9ebb0f4 MCOL-3606 Make ColumnStore use generic paths
ColumnStore now uses standard bin/lib paths for pretty much everything.
Data path is now hard-coded to /var/lib/columnstore.

This patch also:

* Removes v1 decompression
* Removes a bunch of unneeded files
* Removes COLUMNSTORE_INSTALL_DIR / $INSTALLDIR
* Makes my.cnf.d work for all platforms (MCOL-3558)
* Changes configcpp to use recursive mutex (fixes possible config write deadlock)
* Fixes MCOL-3599 Fix regr functions, The library was installed in the wrong location
* Fixes a bunch of Ubuntu packaging issues
* Changes the binary names of several of the executables so as not to
clash with potential executables from other packages
2019-11-09 16:53:05 +00:00
Patrick LeBlanc
9c7e2e923e MCOL-3488. Found places with similar code. 2019-09-09 13:58:12 -05:00
Patrick LeBlanc
d47980ceab MCOL-3488. Chunk shifting failed with SM enabled.
The ChunkManager class was getting an IDBFileSystem instance in
a different way than seemingly everything else.  Added code
to allow it to get an SMFileSystem if cloud storage is specified.
2019-09-09 09:54:38 -05:00
Roman Nozdrin
e2f1b07e70 MCOL-3317 Moved fill-next-block from writeRow() into allocRowId.
Intro* INSERT statements could face a non-existant block when MCOL-498 feature
    is enabled. writeRow() guard blocks was supposed to proactively create empty
    blocks. The pre-patch logic failed when first value in the block has been
    removed by DELETE and this overwrites the whole valid block with empty magics.
    This patch moves proactive creation logic into allocRowId().
2019-08-16 21:28:07 +03:00
Andrew Hutchings
5e4f1b9933
Merge branch 'develop' into MCOL-265 2019-06-10 13:58:03 +01:00
Roman Nozdrin
e12a2acd53 MCOL-537 Regression test doesn't tolerate 'failed' in stderr, stdout.
I reformulate the messages.

    Changed version in preprocessor conditions to avoid compilation
    warnings in Debian 9.

    Disabled sign-compare check for generated files in DML/DDL.
2019-05-20 18:30:52 +03:00
Andrew Hutchings
020b211bb7 Merge branch 'develop-1.2' into develop-merge-up-20190514 2019-05-14 13:58:33 +01:00
Roman Nozdrin
3c89a4bba4 MCOL-537 Preprocessor if blocks with pragmas now have else branch.
DMLProc exits on setupCwd failure.
2019-05-09 20:25:21 +03:00
Roman Nozdrin
b2436502cb MCOL-537 Enabled -Wno-unused-result for OAM code.
Fixed pragmas that disables compilation checks.

    DDLProc now returns an error if it couldn't cwd.

    Use either auto_ptr or unique_ptr depending on GCC version.
2019-05-08 19:44:01 +03:00
Roman Nozdrin
7e2cb05624 MCOL-537 There are no CS-specific warnings building with gcc 8.2. 2019-05-07 16:00:05 +03:00
Roman Nozdrin
9dc33c4e82 Another try to cope with warnings under gcc 8.2. 2019-04-29 11:05:03 +03:00
Gagan Goel
e89d1ac3cf MCOL-265 Add support for TIMESTAMP data type 2019-04-23 00:00:09 -04:00
Roman Nozdrin
22c0c98e61 MCOL-498 Reduced number of blocks created for abbreviated extents
thus reduced IO load when creating a table.
    Uncompressed abbreviated segment and dicts aren't affected by
    this b/c CS'es system catalog uses uncompressed dict files. CS
    now doesn't work with empty dicts files.
2019-04-22 20:02:04 +03:00
Roman Nozdrin
bc3c780e35 MCOL-498 Revived unit tests for writeengine/shared and add new tests
for extent extention.
Added a getter, moved some methods from protected into public to use
with unit tests, e.g createFile, setPreallocSpace. Added code stub in
FileOp::oid2FileName to use with UT.
2019-04-22 20:02:00 +03:00
Roman Nozdrin
abf7ef80c2 MCOL-498 Changes made according with review suggestions.
Add more comments.
    Changed return value for HDFS'es fallocate.
    Removed unnecessary code in ColumnBufferCompressed::writeToFile
    Replaced Nulls with Empties in variable names.
2019-04-22 20:01:50 +03:00
Roman Nozdrin
cbdcdb9f10 MCOL-498 Add DBRootX.PreallocSpace setting in the XML. Dict files extents now contain a correct number of blocks available. 2019-04-22 20:01:43 +03:00
Roman Nozdrin
29becc2971 MCOL-498 Passed test100. 2019-04-22 20:01:36 +03:00
Roman Nozdrin
8037af5161 MCOL-498 Fill up next block with empty values if insert values up to the block boundary. 2019-04-22 20:01:26 +03:00
Roman Nozdrin
7cf0d55dd0 MCOL-498: Fill up the block with NULLs when CS touches for the first time it with INSERT..VALUES. 2019-04-22 20:01:18 +03:00
Roman Nozdrin
81fe7fa1a9 MCOL-498. Add the knob to disable segment|dict file preallocation. Dict files extension uses fallocate() if possible. 2019-04-22 20:01:14 +03:00
Roman Nozdrin
1d9f47a55c MCOL-498. Segment files extension uses fallocate() now to optimize load put on SSD disks. 2019-04-22 20:01:09 +03:00
Andrew Hutchings
a955b56f4d Merge branch 'develop-1.2' into develop-merge-up-20190218 2019-02-18 15:55:11 +00:00
Andrew Hutchings
d4a5c2823b Merge branch 'develop-1.1' into develop-1.2-merge-up-20190214 2019-02-14 15:55:15 +00:00
Andrew Hutchings
159df155a7 Merge branch 'develop-1.0' into develop-1.1-merge-up-20190214 2019-02-14 14:15:07 +00:00
Andrew Hutchings
f805478eb9 MCOL-2149 Fix cpimport decimal saturation
If we saturate int64_t during string -> decimal conversion then end
processing there instead of continuing. This preserves a good saturation
value.
2019-02-08 17:55:50 +00:00
Andrew Hutchings
ab74bec10b Merge branch 'develop-1.1' into 1.2-merge-up-20190111 2019-01-11 20:16:38 +00:00
Andrew Hutchings
8f5fb1a71e Merge branch 'develop-1.0' into 1.1-merge-up-20190111 2019-01-11 18:44:15 +00:00
Patrick LeBlanc
fe94776ac6 MCOL-2062: bug parsing scientific notation in cpimport
Made it use strtold instead of the 'manual' string parsing
stuff it was doing.
2019-01-09 16:31:38 -06:00
Gagan Goel
d1ada75395 MCOL-270 Add support for MEDIUMINT data type 2018-12-30 19:13:16 -05:00
david hill
101b36acb6
Merge branch 'develop' into MCOL-520 2018-10-08 14:23:48 -05:00