1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-08 14:22:09 +03:00
Commit Graph

105 Commits

Author SHA1 Message Date
Andrew Hutchings
a8cd34f86d Add support for building from server 2019-12-03 21:21:28 +00:00
Andrew Hutchings
8ab9ebb0f4 MCOL-3606 Make ColumnStore use generic paths
ColumnStore now uses standard bin/lib paths for pretty much everything.
Data path is now hard-coded to /var/lib/columnstore.

This patch also:

* Removes v1 decompression
* Removes a bunch of unneeded files
* Removes COLUMNSTORE_INSTALL_DIR / $INSTALLDIR
* Makes my.cnf.d work for all platforms (MCOL-3558)
* Changes configcpp to use recursive mutex (fixes possible config write deadlock)
* Fixes MCOL-3599 Fix regr functions, The library was installed in the wrong location
* Fixes a bunch of Ubuntu packaging issues
* Changes the binary names of several of the executables so as not to
clash with potential executables from other packages
2019-11-09 16:53:05 +00:00
Patrick LeBlanc
9c7e2e923e MCOL-3488. Found places with similar code. 2019-09-09 13:58:12 -05:00
Patrick LeBlanc
d47980ceab MCOL-3488. Chunk shifting failed with SM enabled.
The ChunkManager class was getting an IDBFileSystem instance in
a different way than seemingly everything else.  Added code
to allow it to get an SMFileSystem if cloud storage is specified.
2019-09-09 09:54:38 -05:00
Roman Nozdrin
e2f1b07e70 MCOL-3317 Moved fill-next-block from writeRow() into allocRowId.
Intro* INSERT statements could face a non-existant block when MCOL-498 feature
    is enabled. writeRow() guard blocks was supposed to proactively create empty
    blocks. The pre-patch logic failed when first value in the block has been
    removed by DELETE and this overwrites the whole valid block with empty magics.
    This patch moves proactive creation logic into allocRowId().
2019-08-16 21:28:07 +03:00
Andrew Hutchings
5e4f1b9933 Merge branch 'develop' into MCOL-265 2019-06-10 13:58:03 +01:00
Roman Nozdrin
e12a2acd53 MCOL-537 Regression test doesn't tolerate 'failed' in stderr, stdout.
I reformulate the messages.

    Changed version in preprocessor conditions to avoid compilation
    warnings in Debian 9.

    Disabled sign-compare check for generated files in DML/DDL.
2019-05-20 18:30:52 +03:00
Andrew Hutchings
020b211bb7 Merge branch 'develop-1.2' into develop-merge-up-20190514 2019-05-14 13:58:33 +01:00
Roman Nozdrin
3c89a4bba4 MCOL-537 Preprocessor if blocks with pragmas now have else branch.
DMLProc exits on setupCwd failure.
2019-05-09 20:25:21 +03:00
Roman Nozdrin
b2436502cb MCOL-537 Enabled -Wno-unused-result for OAM code.
Fixed pragmas that disables compilation checks.

    DDLProc now returns an error if it couldn't cwd.

    Use either auto_ptr or unique_ptr depending on GCC version.
2019-05-08 19:44:01 +03:00
Roman Nozdrin
7e2cb05624 MCOL-537 There are no CS-specific warnings building with gcc 8.2. 2019-05-07 16:00:05 +03:00
Roman Nozdrin
9dc33c4e82 Another try to cope with warnings under gcc 8.2. 2019-04-29 11:05:03 +03:00
Gagan Goel
e89d1ac3cf MCOL-265 Add support for TIMESTAMP data type 2019-04-23 00:00:09 -04:00
Roman Nozdrin
22c0c98e61 MCOL-498 Reduced number of blocks created for abbreviated extents
thus reduced IO load when creating a table.
    Uncompressed abbreviated segment and dicts aren't affected by
    this b/c CS'es system catalog uses uncompressed dict files. CS
    now doesn't work with empty dicts files.
2019-04-22 20:02:04 +03:00
Roman Nozdrin
bc3c780e35 MCOL-498 Revived unit tests for writeengine/shared and add new tests
for extent extention.
Added a getter, moved some methods from protected into public to use
with unit tests, e.g createFile, setPreallocSpace. Added code stub in
FileOp::oid2FileName to use with UT.
2019-04-22 20:02:00 +03:00
Roman Nozdrin
abf7ef80c2 MCOL-498 Changes made according with review suggestions.
Add more comments.
    Changed return value for HDFS'es fallocate.
    Removed unnecessary code in ColumnBufferCompressed::writeToFile
    Replaced Nulls with Empties in variable names.
2019-04-22 20:01:50 +03:00
Roman Nozdrin
cbdcdb9f10 MCOL-498 Add DBRootX.PreallocSpace setting in the XML. Dict files extents now contain a correct number of blocks available. 2019-04-22 20:01:43 +03:00
Roman Nozdrin
29becc2971 MCOL-498 Passed test100. 2019-04-22 20:01:36 +03:00
Roman Nozdrin
8037af5161 MCOL-498 Fill up next block with empty values if insert values up to the block boundary. 2019-04-22 20:01:26 +03:00
Roman Nozdrin
7cf0d55dd0 MCOL-498: Fill up the block with NULLs when CS touches for the first time it with INSERT..VALUES. 2019-04-22 20:01:18 +03:00
Roman Nozdrin
81fe7fa1a9 MCOL-498. Add the knob to disable segment|dict file preallocation. Dict files extension uses fallocate() if possible. 2019-04-22 20:01:14 +03:00
Roman Nozdrin
1d9f47a55c MCOL-498. Segment files extension uses fallocate() now to optimize load put on SSD disks. 2019-04-22 20:01:09 +03:00
Andrew Hutchings
a955b56f4d Merge branch 'develop-1.2' into develop-merge-up-20190218 2019-02-18 15:55:11 +00:00
Andrew Hutchings
d4a5c2823b Merge branch 'develop-1.1' into develop-1.2-merge-up-20190214 2019-02-14 15:55:15 +00:00
Andrew Hutchings
159df155a7 Merge branch 'develop-1.0' into develop-1.1-merge-up-20190214 2019-02-14 14:15:07 +00:00
Andrew Hutchings
f805478eb9 MCOL-2149 Fix cpimport decimal saturation
If we saturate int64_t during string -> decimal conversion then end
processing there instead of continuing. This preserves a good saturation
value.
2019-02-08 17:55:50 +00:00
Andrew Hutchings
ab74bec10b Merge branch 'develop-1.1' into 1.2-merge-up-20190111 2019-01-11 20:16:38 +00:00
Andrew Hutchings
8f5fb1a71e Merge branch 'develop-1.0' into 1.1-merge-up-20190111 2019-01-11 18:44:15 +00:00
Patrick LeBlanc
fe94776ac6 MCOL-2062: bug parsing scientific notation in cpimport
Made it use strtold instead of the 'manual' string parsing
stuff it was doing.
2019-01-09 16:31:38 -06:00
Gagan Goel
d1ada75395 MCOL-270 Add support for MEDIUMINT data type 2018-12-30 19:13:16 -05:00
david hill
101b36acb6 Merge branch 'develop' into MCOL-520 2018-10-08 14:23:48 -05:00
Andrew Hutchings
f1f13a09d1 Merge branch 'develop-1.1' into 1.1-merge-up-2018-10-05 2018-10-05 18:40:07 +01:00
David Hill
f25082bc9a MCOL-520 2018-09-25 17:20:46 -05:00
David Hill
b7b98a3e1a MCOL-520 2018-09-25 11:32:56 -05:00
David Hill
81e35e35eb MCOL-520 2018-09-21 11:23:43 -05:00
Andrew Hutchings
24c5e93756 MCOL-1737 Add debug logging options for LRU cache
This adds options which are user enabled to debug the LRU cache inside
ColumnStore. Specifically cache flushing.

It adds the following:
* PrimProc flush information when SIGUSR2 mode is enabled
* cpimport dictionary flush information when -d2 is used
* WriteEngineServer DML flush information to STDERR
2018-09-21 09:50:10 +01:00
Andrew Hutchings
dba04e8b72 MCOL-392 Fix cpimport and >8bit positive hour 2018-04-30 09:42:41 +01:00
Andrew Hutchings
3c1ebd8b94 MCOL-392 Add initial TIME datatype support 2018-04-30 09:42:41 +01:00
Andrew Hutchings
17f077012d Merge branch 'develop-1.1' into 1.1-merge-up 2017-12-13 09:09:39 +00:00
Andrew Hutchings
8babe4a35d Merge branch 'develop-1.0' into 1.0-merge-up 2017-12-12 10:01:14 +00:00
David Hall
34799d8d30 MCOL-994 handle a second abbreviated extent in case it moved dbroots because of redistribute remove. 2017-12-07 10:49:51 -06:00
Andrew Hutchings
01446d1e22 Reformat all code to coding standard 2017-10-26 17:18:17 +01:00
Andrew Hutchings
3330495a2e MCOL-777 Cleanup source
Clean out autotools and some other things from the source tree.
2017-08-07 15:59:56 +01:00
Andrew Hutchings
f251ebccb2 MCOL-669 TEXT cpimport fixes
* 64KB TEXT column had off-by-one length pointer counting
* TEXT I_S/LDI was looping where it shouldn't causing pointer issues
* TEXT data type wasn't fully understood by cpimport
2017-04-19 14:11:21 +01:00
Andrew Hutchings
e9db44424c MCOL-642 Separate TEXT from BLOB
* TEXT and BLOB now have separate identifiers internally
* TEXT columns are identified as such in system catalog
* cpimport only requires hex input for BLOB, not TEXT
2017-03-27 21:36:27 +01:00
Andrew Hutchings
aea729fe7d MCOL-267 DML support
* DML writes for multi-block dictionary (blob) now works
* PrimProc fixed so that the first block in multi-block is read
correctly
* Performance optimisation (removed string copy into stack) for new
dictionary entries
2017-03-18 14:31:29 +00:00
Andrew Hutchings
c08d03fba4 MCOL-267 Make BLOB DDL/DML work
Currently limited to 8KB inserts.
2017-03-17 12:10:02 +00:00
Ben Thompson
424628349b Add CMake build tree files 2016-07-15 10:49:57 -05:00
Ben Thompson
d90af9496e Remove Makefile.in and update gitignore 2016-07-15 10:49:57 -05:00
david hill
7d8de28b43 MCOL-59, change calpont.xml 2016-06-22 16:00:00 -05:00