1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-01 06:21:41 +03:00
Commit Graph

123 Commits

Author SHA1 Message Date
df65543dd4 MCOL-641 This commit contains fixes for the rebase that mostly adds
WE_BINARY and WE_INT128 into switch-case blocks.
2020-11-18 13:47:00 +00:00
c9f42fb5cc MCOL-641 PoC version for DECIMAL(38) using BINARY as a basis. 2020-11-18 13:47:00 +00:00
32f6167067 MCOL-641 Work of Ivan Zuniga on basic read and write support for Binary16 2020-11-18 13:47:00 +00:00
f4a6294c95 Fix ignored qualifiers warnings (-Wignored-qualifiers) 2020-11-17 15:03:10 +03:00
6f120d2637 MCOL-4328 MCS avoids chown() calls for files that are on S3
MCS now chowns created directories hierarchy not only files and
immediate parent directories

Minor changes to cpimport's help printout

cpimport's -f option is now mandatory with mode 2
2020-10-09 11:02:31 +00:00
328ae25650 MCOL-4328 There is a new option in both cpimport and cpimport.bin to asign
an owner for all data files created by cpimport

The patch consists of two parts: cpimport.bin changes, cpimport splitter
changes

cpimport.bin computes uid_t and gid_t early and propagates it down the stack
where MCS creates data files
2020-10-03 14:05:29 +00:00
7e868bc588 add dependency for generated header files errorids.h messageids.h patch made by Ben. 2020-07-10 18:38:49 +00:00
eac7dab096 MCOL-4030: first commit of warning removals unneed const and missing virtual dtors. 2020-06-23 13:51:36 -05:00
5440977f45 Merge pull request #1197 from tntnatbry/develop-batchinserts-improve
Improve batch inserts.
2020-06-02 14:05:47 -05:00
98abf95eae MCOL-3991 MCS is now single package and properly uninstalls 2020-05-12 13:36:24 +00:00
d4d0ebdf5d Improve batch inserts.
1) Instead of making dbrm calls to writeVBEntry() per block,
     we make these calls per batch. This can have non-trivial
     reductions in the overhead of these calls if the batch size
     is large.

  2) In dmlproc, do not deserialize the whole insertpackage, which
     consists of the complete record set per column, which would be
     wasteful as we only need some metadata fields from insertpackage
     here. This is only done for batch inserts at the moment, this
     should also be applied to single inserts.
2020-05-10 19:38:06 -04:00
a65c8e4544 MCOL-3251
Moved the new logic into we_fileop to maintain existing encapsulation.
2020-03-13 15:18:55 -04:00
ffcd527c4e Merge pull request #1052 from mariadb-corporation/MCOL-3791
MCOL-3791 This patch implicitly enables disk space preallocation for …
2020-02-28 16:20:07 -05:00
586391e1ca compilation failure
error: reference to 'mutex' is ambiguous
note: candidates are: 'class boost::mutex'
note:                 'class std::mutex'
2019-12-19 18:13:39 +01:00
49994f7bc3 Fix warnings found in DEBUG combined build
Fixes:
* Irrelevant where conditions
* Irrelevant const
* A potential infinite loop in treenode
* Bad implicit case fallthroughs
* Explicit markings for required case fallthroughs
* Unused variables
* Unused function

Also disabled some warnings for now which we should fix later.
2019-12-10 16:33:08 +00:00
207fb3f588 Merge pull request #960 from LinuxJedi/unified_build
Unified build for ES and MCS
2019-12-05 01:19:23 +01:00
766d84ada8 MCOL-3585 Enabled MCOL-498 by default for all dbroots. 2019-12-04 06:49:31 -06:00
7489d0bfd0 MCOL-3625 Rename packages
Rename packages to MariaDB-columnstore-engine, MariaDB-columnstore-libs
and MariaDB-columnstore-platform.

Also add the "columnstore-" prefix the the components so that MariaDB's
packaging system understands then and add a line to include them in
MariaDB's packaging.

In addition
* Fix S3 building for dist source build
* Fix Debian 10 dependency issue
* Fix git handling for dist builds
* Add support for MariaDB's RPM building
* Use MariaDB's PCRE and readline
* Removes a few dead files
* Fix Boost noncopyable includes
2019-12-04 11:04:39 +00:00
a8cd34f86d Add support for building from server 2019-12-03 21:21:28 +00:00
8ab9ebb0f4 MCOL-3606 Make ColumnStore use generic paths
ColumnStore now uses standard bin/lib paths for pretty much everything.
Data path is now hard-coded to /var/lib/columnstore.

This patch also:

* Removes v1 decompression
* Removes a bunch of unneeded files
* Removes COLUMNSTORE_INSTALL_DIR / $INSTALLDIR
* Makes my.cnf.d work for all platforms (MCOL-3558)
* Changes configcpp to use recursive mutex (fixes possible config write deadlock)
* Fixes MCOL-3599 Fix regr functions, The library was installed in the wrong location
* Fixes a bunch of Ubuntu packaging issues
* Changes the binary names of several of the executables so as not to
clash with potential executables from other packages
2019-11-09 16:53:05 +00:00
9c7e2e923e MCOL-3488. Found places with similar code. 2019-09-09 13:58:12 -05:00
d47980ceab MCOL-3488. Chunk shifting failed with SM enabled.
The ChunkManager class was getting an IDBFileSystem instance in
a different way than seemingly everything else.  Added code
to allow it to get an SMFileSystem if cloud storage is specified.
2019-09-09 09:54:38 -05:00
e2f1b07e70 MCOL-3317 Moved fill-next-block from writeRow() into allocRowId.
Intro* INSERT statements could face a non-existant block when MCOL-498 feature
    is enabled. writeRow() guard blocks was supposed to proactively create empty
    blocks. The pre-patch logic failed when first value in the block has been
    removed by DELETE and this overwrites the whole valid block with empty magics.
    This patch moves proactive creation logic into allocRowId().
2019-08-16 21:28:07 +03:00
5e4f1b9933 Merge branch 'develop' into MCOL-265 2019-06-10 13:58:03 +01:00
e12a2acd53 MCOL-537 Regression test doesn't tolerate 'failed' in stderr, stdout.
I reformulate the messages.

    Changed version in preprocessor conditions to avoid compilation
    warnings in Debian 9.

    Disabled sign-compare check for generated files in DML/DDL.
2019-05-20 18:30:52 +03:00
020b211bb7 Merge branch 'develop-1.2' into develop-merge-up-20190514 2019-05-14 13:58:33 +01:00
3c89a4bba4 MCOL-537 Preprocessor if blocks with pragmas now have else branch.
DMLProc exits on setupCwd failure.
2019-05-09 20:25:21 +03:00
b2436502cb MCOL-537 Enabled -Wno-unused-result for OAM code.
Fixed pragmas that disables compilation checks.

    DDLProc now returns an error if it couldn't cwd.

    Use either auto_ptr or unique_ptr depending on GCC version.
2019-05-08 19:44:01 +03:00
7e2cb05624 MCOL-537 There are no CS-specific warnings building with gcc 8.2. 2019-05-07 16:00:05 +03:00
9dc33c4e82 Another try to cope with warnings under gcc 8.2. 2019-04-29 11:05:03 +03:00
e89d1ac3cf MCOL-265 Add support for TIMESTAMP data type 2019-04-23 00:00:09 -04:00
22c0c98e61 MCOL-498 Reduced number of blocks created for abbreviated extents
thus reduced IO load when creating a table.
    Uncompressed abbreviated segment and dicts aren't affected by
    this b/c CS'es system catalog uses uncompressed dict files. CS
    now doesn't work with empty dicts files.
2019-04-22 20:02:04 +03:00
bc3c780e35 MCOL-498 Revived unit tests for writeengine/shared and add new tests
for extent extention.
Added a getter, moved some methods from protected into public to use
with unit tests, e.g createFile, setPreallocSpace. Added code stub in
FileOp::oid2FileName to use with UT.
2019-04-22 20:02:00 +03:00
abf7ef80c2 MCOL-498 Changes made according with review suggestions.
Add more comments.
    Changed return value for HDFS'es fallocate.
    Removed unnecessary code in ColumnBufferCompressed::writeToFile
    Replaced Nulls with Empties in variable names.
2019-04-22 20:01:50 +03:00
cbdcdb9f10 MCOL-498 Add DBRootX.PreallocSpace setting in the XML. Dict files extents now contain a correct number of blocks available. 2019-04-22 20:01:43 +03:00
29becc2971 MCOL-498 Passed test100. 2019-04-22 20:01:36 +03:00
8037af5161 MCOL-498 Fill up next block with empty values if insert values up to the block boundary. 2019-04-22 20:01:26 +03:00
7cf0d55dd0 MCOL-498: Fill up the block with NULLs when CS touches for the first time it with INSERT..VALUES. 2019-04-22 20:01:18 +03:00
81fe7fa1a9 MCOL-498. Add the knob to disable segment|dict file preallocation. Dict files extension uses fallocate() if possible. 2019-04-22 20:01:14 +03:00
1d9f47a55c MCOL-498. Segment files extension uses fallocate() now to optimize load put on SSD disks. 2019-04-22 20:01:09 +03:00
a955b56f4d Merge branch 'develop-1.2' into develop-merge-up-20190218 2019-02-18 15:55:11 +00:00
d4a5c2823b Merge branch 'develop-1.1' into develop-1.2-merge-up-20190214 2019-02-14 15:55:15 +00:00
159df155a7 Merge branch 'develop-1.0' into develop-1.1-merge-up-20190214 2019-02-14 14:15:07 +00:00
f805478eb9 MCOL-2149 Fix cpimport decimal saturation
If we saturate int64_t during string -> decimal conversion then end
processing there instead of continuing. This preserves a good saturation
value.
2019-02-08 17:55:50 +00:00
ab74bec10b Merge branch 'develop-1.1' into 1.2-merge-up-20190111 2019-01-11 20:16:38 +00:00
8f5fb1a71e Merge branch 'develop-1.0' into 1.1-merge-up-20190111 2019-01-11 18:44:15 +00:00
fe94776ac6 MCOL-2062: bug parsing scientific notation in cpimport
Made it use strtold instead of the 'manual' string parsing
stuff it was doing.
2019-01-09 16:31:38 -06:00
d1ada75395 MCOL-270 Add support for MEDIUMINT data type 2018-12-30 19:13:16 -05:00
101b36acb6 Merge branch 'develop' into MCOL-520 2018-10-08 14:23:48 -05:00
f1f13a09d1 Merge branch 'develop-1.1' into 1.1-merge-up-2018-10-05 2018-10-05 18:40:07 +01:00