1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-04 04:42:30 +03:00
Commit Graph

205 Commits

Author SHA1 Message Date
59d1caae02 Fix CMake dependencies
Some odd quirks with CMake mean that marias3 has to be in the top level
CMake. Also made submodule checkout happen at cmake time.
2019-09-25 11:30:32 +01:00
7fb1a7bd34 Fix early EOF for file reading
Bad 'if' statement caused an EOF to trigger on the first buffer fill for
regular file reading.
2019-09-24 18:08:28 +01:00
fe8228e50a Fix build order so ms3 is checked out 2019-09-24 16:47:00 +01:00
8633859dd4 MCOL-3514 Add support for S3 to cpimport
cpimport now has the ability to use libmarias3 to read an object from an
S3 bucket instead of a file on local disk.

This also moves libmarias3 to utils/libmarias3.
2019-09-24 10:31:22 +01:00
24e1c3ff2d Merge pull request #855 from mariadb-corporation/move-etc
Move config files
2019-09-09 15:54:30 -05:00
b0e934dd2d MCOL-3488. Found another one. 2019-09-09 13:58:12 -05:00
9c7e2e923e MCOL-3488. Found places with similar code. 2019-09-09 13:58:12 -05:00
d47980ceab MCOL-3488. Chunk shifting failed with SM enabled.
The ChunkManager class was getting an IDBFileSystem instance in
a different way than seemingly everything else.  Added code
to allow it to get an SMFileSystem if cloud storage is specified.
2019-09-09 09:54:38 -05:00
97bda78c3b Move config files
This patch:

* Moves config files from /usr/local/mariadb/columnstore/etc to
ENGINE_SYSCONFDIR/columnstore (ENGINE_SYSCONFDIR is /etc by default)
* Sets a define called MCSSYSCONFDIR whic contains the
ENGINE_SYSCONFDIR compile time setting
* Modifies scripts and code to use the new paths
* Removes a whole bunch of files we don't use
2019-09-09 14:13:56 +01:00
3fef0f21d3 Remove vpj files
They shouldn't be here
2019-09-05 17:38:03 +01:00
e2f1b07e70 MCOL-3317 Moved fill-next-block from writeRow() into allocRowId.
Intro* INSERT statements could face a non-existant block when MCOL-498 feature
    is enabled. writeRow() guard blocks was supposed to proactively create empty
    blocks. The pre-patch logic failed when first value in the block has been
    removed by DELETE and this overwrites the whole valid block with empty magics.
    This patch moves proactive creation logic into allocRowId().
2019-08-16 21:28:07 +03:00
b1bc995420 Merge branch 'develop' into remove-infinidb 2019-08-13 12:32:01 +03:00
6cdca1330b Merge pull request #808 from mariadb-corporation/develop-merge-up-20190729
Merge develop-1.2 into develop
2019-08-13 11:55:22 +03:00
9d83b49fca MCOL-104 First pass of InfiniDB rename in code 2019-08-12 09:41:28 +01:00
7b006b6e74 MCOL-104 Remove InfiniDB Engine
Remove the InfiniDB engine which is a duplicate of the ColumnStore
engine.
2019-08-09 11:51:55 +01:00
a09a9d5d0f Mass substitution 'Corporaton' -> 'Corporation' 2019-08-07 14:43:25 -05:00
811909aa72 Merge branch 'develop-1.2' into develop-merge-up-20190729 2019-07-29 12:19:26 +01:00
6bee5f9f61 MCOL-3395 Always clear dctnry cache on close
The dictionary cache needs to be cleared every time it is closed so that
on bulk insert we don't mix the cache between columns.
2019-07-03 08:46:43 +01:00
cddb776bd4 Merge branch 'develop-1.1' into develop-1.2-merge-up-20190619 2019-06-19 18:34:43 +01:00
1db56546ee TIMESTAMP namespace fixes
Namespace issues in the TIMESTAMP code caused compiling to break. This
patch fixes that.
2019-06-12 15:05:20 +01:00
e3cd205388 MCOL-1968 Fix UTF char/varchar min/max handling
If the first byte of a char/varchar was > 0x80 then it will break the
min/max values for an extent during cpimport. This patch makes the
min/max compare unsigned and only switches to signed when storing.

In addition send all the LDI / INSERT...SELECT data to cpimport, not
truncated. Let cpimport figure out the truncation point.
2019-06-11 10:37:04 +01:00
5e4f1b9933 Merge branch 'develop' into MCOL-265 2019-06-10 13:58:03 +01:00
e12a2acd53 MCOL-537 Regression test doesn't tolerate 'failed' in stderr, stdout.
I reformulate the messages.

    Changed version in preprocessor conditions to avoid compilation
    warnings in Debian 9.

    Disabled sign-compare check for generated files in DML/DDL.
2019-05-20 18:30:52 +03:00
020b211bb7 Merge branch 'develop-1.2' into develop-merge-up-20190514 2019-05-14 13:58:33 +01:00
0dd33c6f75 Merge pull request #751 from drrtuy/mcol-537-compiler-warnings
MCOL-537 Address existing compiler warnings.
2019-05-10 09:27:18 -05:00
3c89a4bba4 MCOL-537 Preprocessor if blocks with pragmas now have else branch.
DMLProc exits on setupCwd failure.
2019-05-09 20:25:21 +03:00
067b1bd3d0 Merge pull request #752 from mariadb-corporation/MCOL-1495
MCOL-1495 DML operations created two fCatalogMap entries per session.
2019-05-09 09:44:05 +01:00
b2436502cb MCOL-537 Enabled -Wno-unused-result for OAM code.
Fixed pragmas that disables compilation checks.

    DDLProc now returns an error if it couldn't cwd.

    Use either auto_ptr or unique_ptr depending on GCC version.
2019-05-08 19:44:01 +03:00
7e2cb05624 MCOL-537 There are no CS-specific warnings building with gcc 8.2. 2019-05-07 16:00:05 +03:00
b2bf6dece5 missing semicol 2019-05-01 13:38:02 -05:00
9e7d852804 unneeded k for loop 2019-05-01 10:29:57 -05:00
a8adef8820 MCOL-1495 DML operations created two fCatalogMap entries per session.
These entries were never deleted so WE leaks about 7MB per 100 DML
    sessions. I add purge operations in the very end of four functions
    involved in DML.
2019-05-01 18:03:32 +03:00
9dc33c4e82 Another try to cope with warnings under gcc 8.2. 2019-04-29 11:05:03 +03:00
cbbf267e88 MCOL-537, cleanup compiler warnings. Checkpointing a bunch of fixes.
Work in progress...
2019-04-29 10:56:48 +03:00
56767ae793 Add a few missing qualifiers 2019-04-29 01:28:55 -05:00
4b9d046c6e Fully resolve potentially ambiguous symbols by removing using namespace statements from headers which have a cascading effect. This causes potential behavior changes when switching to c++11 since symbols can be exported from std and boost while both have been imported into the global namespace. 2019-04-29 01:21:15 -05:00
1813add162 Merge pull request #743 from davidjmott/develop
Add -DSERVER_BUILD_DIR configure parameter to interrogate the server …
2019-04-26 14:02:20 +01:00
e65f80f493 delete visual c++ project files. cmake can generate these if needed 2019-04-25 23:35:03 -05:00
784bbe09d4 Merge branch 'develop-1.2' into develop-merge-up-20190425 2019-04-25 10:27:59 +01:00
723ca53669 Merge branch 'develop' into MCOL-498_5 2019-04-24 09:04:39 +01:00
e89d1ac3cf MCOL-265 Add support for TIMESTAMP data type 2019-04-23 00:00:09 -04:00
22c0c98e61 MCOL-498 Reduced number of blocks created for abbreviated extents
thus reduced IO load when creating a table.
    Uncompressed abbreviated segment and dicts aren't affected by
    this b/c CS'es system catalog uses uncompressed dict files. CS
    now doesn't work with empty dicts files.
2019-04-22 20:02:04 +03:00
bc3c780e35 MCOL-498 Revived unit tests for writeengine/shared and add new tests
for extent extention.
Added a getter, moved some methods from protected into public to use
with unit tests, e.g createFile, setPreallocSpace. Added code stub in
FileOp::oid2FileName to use with UT.
2019-04-22 20:02:00 +03:00
ecbf6b7606 MCOL-498 Returned changes in we_colbufcompressed.* b/c they are neccesary
to track compressed data size.
2019-04-22 20:01:55 +03:00
abf7ef80c2 MCOL-498 Changes made according with review suggestions.
Add more comments.
    Changed return value for HDFS'es fallocate.
    Removed unnecessary code in ColumnBufferCompressed::writeToFile
    Replaced Nulls with Empties in variable names.
2019-04-22 20:01:50 +03:00
cbdcdb9f10 MCOL-498 Add DBRootX.PreallocSpace setting in the XML. Dict files extents now contain a correct number of blocks available. 2019-04-22 20:01:43 +03:00
29becc2971 MCOL-498 Passed test100. 2019-04-22 20:01:36 +03:00
46a46aa6b1 MCOL-498 Support for non dict compressed columns. 2019-04-22 20:01:30 +03:00
8037af5161 MCOL-498 Fill up next block with empty values if insert values up to the block boundary. 2019-04-22 20:01:26 +03:00
6db8b1f432 MCOL-498: Fill up the last used block with empty values, whilst doing bulk insertion with uncompressed files. 2019-04-22 20:01:23 +03:00