1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-04-20 09:07:44 +03:00

18 Commits

Author SHA1 Message Date
Leonid Fedorov
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00
Denis Khalikov
cc1c3629c5 MCOL-987 Add LZ4 compression.
* Adds CompressInterfaceLZ4 which uses LZ4 API for compress/uncompress.
* Adds CMake machinery to search LZ4 on running host.
* All methods which use static data and do not modify any internal data - become `static`,
  so we can use them without creation of the specific object. This is possible, because
  the header specification has not been modified. We still use 2 sections in header, first
  one with file meta data, the second one with pointers for compressed chunks.
* Methods `compress`, `uncompress`, `maxCompressedSize`, `getUncompressedSize` - become
  pure virtual, so we can override them for the other compression algos.
* Adds method `getChunkMagicNumber`, so we can verify chunk magic number
  for each compression algo.
* Renames "s/IDBCompressInterface/CompressInterface/g" according to requirement.
2021-07-06 18:04:37 +03:00
Alexander Barkov
b3d6f62964 MCOL-4753 Performance problem in Typeless join 2021-06-10 09:26:26 +00:00
Roman Nozdrin
c6d0b46bc6
Merge pull request #1984 from denis0x0D/MCOL-4685_fix_warn
MCOL-4685: Fix GCC warnings.
2021-06-10 11:39:10 +03:00
Alexey Antipovsky
0dedb7e628 Fix compilation warnings 2021-06-09 16:51:00 +03:00
Denis Khalikov
2cd024145c MCOL-4685: Fix GCC warnings.
This patch fixes GCC warnings:
1.'const long int' and 'long unsigned int' [-Werror=sign-compare]
2. unused variable 'cf' [-Werror=unused-variable]
2021-06-09 14:33:32 +03:00
Denis Khalikov
caa02a383a MCOL-4685 Fix bug after rebase and add comments. 2021-06-07 13:00:11 +03:00
Denis Khalikov
606194e6e4 MCOL-4685: Eliminate some irrelevant settings (uncompressed data and extents per file).
This patch:
1. Removes the option to declare uncompressed columns (set columnstore_compression_type = 0).
2. Ignores [COMMENT '[compression=0] option at table or column level (no error messages, just disregard).
3. Removes the option to set more than 2 extents per file (ExtentsPreSegmentFile).
4. Updates rebuildEM tool to support up to 10 dictionary extent per dictionary segment file.
5. Adds check for `DBRootStorageType` for rebuildEM tool.
6. Renamed rebuildEM to mcsRebuildEM.
2021-06-03 14:44:33 +03:00
Denis Khalikov
5d497e8821 MCOL-4566: Add rebuildEM tool support to work with compressed files.
* This patch adds rebuildEM tool support to work with compressed files.
* This patch increases a version of the file header.

Note: Default version of the `rebuildEM` tool was using very old API,
those functions are not present currently. So `rebuildEM` will not work with
files created without compression, because we cannot deduce some info which are
needed to create column extent.
2021-04-02 10:55:01 +03:00
Denis Khalikov
a2efa1efeb MCOL-4566: Extend CompressedDBFileHeader struct with new fields.
* This patch extends CompressedDBFileHeader struct with new fields:
  `fColumWidth`, `fColDataType`, which are necessary to rebuild extent map
  from the given file. Note: new fields do not change the memory
  layout of the struct, because the size is calculated as
  max(sizeof(CompressedDBFileHeader), HDR_BUF_LEN)).

* This patch changes API of some functions, by adding new function
  argument `colDataType` when needed, to be able to call `initHdr`
  function with colDataType value.
2021-03-05 22:15:34 +03:00
Roman Nozdrin
5fce19df0a MCOL-4412 Introduce TypeHandler::getEmptyValueForType to return const ptr for an empty value
WE changes for SQL DML and DDL operations

Changes for bulk operations

Changes for scanning operations

Cleanup
2021-01-18 12:30:17 +00:00
Gagan Goel
824615a55b MCOL-641 Refactor empty value implementation in writeengine. 2020-11-18 13:47:44 +00:00
Roman Nozdrin
ecbf6b7606 MCOL-498 Returned changes in we_colbufcompressed.* b/c they are neccesary
to track compressed data size.
2019-04-22 20:01:55 +03:00
Roman Nozdrin
abf7ef80c2 MCOL-498 Changes made according with review suggestions.
Add more comments.
    Changed return value for HDFS'es fallocate.
    Removed unnecessary code in ColumnBufferCompressed::writeToFile
    Replaced Nulls with Empties in variable names.
2019-04-22 20:01:50 +03:00
Roman Nozdrin
46a46aa6b1 MCOL-498 Support for non dict compressed columns. 2019-04-22 20:01:30 +03:00
Roman Nozdrin
6db8b1f432 MCOL-498: Fill up the last used block with empty values, whilst doing bulk insertion with uncompressed files. 2019-04-22 20:01:23 +03:00
Andrew Hutchings
01446d1e22 Reformat all code to coding standard 2017-10-26 17:18:17 +01:00
david hill
f6afc42dd0 the begginning 2016-01-06 14:08:59 -06:00