1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-04-21 19:45:56 +03:00

16 Commits

Author SHA1 Message Date
Leonid Fedorov
43de58f80d
MCOL-5746 Cpimport: convert blob data from ascii hex when reads from STDIN (#3218)
Co-authored-by: Denis Khalikov <dennis.khalikov@gmail.com>
2024-06-27 14:24:16 +04:00
Leonid Fedorov
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00
Denis Khalikov
cc1c3629c5 MCOL-987 Add LZ4 compression.
* Adds CompressInterfaceLZ4 which uses LZ4 API for compress/uncompress.
* Adds CMake machinery to search LZ4 on running host.
* All methods which use static data and do not modify any internal data - become `static`,
  so we can use them without creation of the specific object. This is possible, because
  the header specification has not been modified. We still use 2 sections in header, first
  one with file meta data, the second one with pointers for compressed chunks.
* Methods `compress`, `uncompress`, `maxCompressedSize`, `getUncompressedSize` - become
  pure virtual, so we can override them for the other compression algos.
* Adds method `getChunkMagicNumber`, so we can verify chunk magic number
  for each compression algo.
* Renames "s/IDBCompressInterface/CompressInterface/g" according to requirement.
2021-07-06 18:04:37 +03:00
Denis Khalikov
5d497e8821 MCOL-4566: Add rebuildEM tool support to work with compressed files.
* This patch adds rebuildEM tool support to work with compressed files.
* This patch increases a version of the file header.

Note: Default version of the `rebuildEM` tool was using very old API,
those functions are not present currently. So `rebuildEM` will not work with
files created without compression, because we cannot deduce some info which are
needed to create column extent.
2021-04-02 10:55:01 +03:00
Denis Khalikov
a2efa1efeb MCOL-4566: Extend CompressedDBFileHeader struct with new fields.
* This patch extends CompressedDBFileHeader struct with new fields:
  `fColumWidth`, `fColDataType`, which are necessary to rebuild extent map
  from the given file. Note: new fields do not change the memory
  layout of the struct, because the size is calculated as
  max(sizeof(CompressedDBFileHeader), HDR_BUF_LEN)).

* This patch changes API of some functions, by adding new function
  argument `colDataType` when needed, to be able to call `initHdr`
  function with colDataType value.
2021-03-05 22:15:34 +03:00
Roman Nozdrin
5fce19df0a MCOL-4412 Introduce TypeHandler::getEmptyValueForType to return const ptr for an empty value
WE changes for SQL DML and DDL operations

Changes for bulk operations

Changes for scanning operations

Cleanup
2021-01-18 12:30:17 +00:00
Gagan Goel
f6b55c1e18 MCOL-4177 Add support for bulk insertion for wide decimals.
1. This patch adds support for wide decimals with/without scale
     to cpimport. In addition, INSERT ... SELECT and LDI are also
     now supported.
  2. Logic to compute the number of bytes to convert a binary
     representation in the buffer to a narrow decimal is also
     simplified.
2020-12-15 22:14:54 +00:00
Gagan Goel
824615a55b MCOL-641 Refactor empty value implementation in writeengine. 2020-11-18 13:47:44 +00:00
Roman Nozdrin
328ae25650 MCOL-4328 There is a new option in both cpimport and cpimport.bin to asign
an owner for all data files created by cpimport

The patch consists of two parts: cpimport.bin changes, cpimport splitter
changes

cpimport.bin computes uid_t and gid_t early and propagates it down the stack
where MCS creates data files
2020-10-03 14:05:29 +00:00
Gagan Goel
d1ada75395 MCOL-270 Add support for MEDIUMINT data type 2018-12-30 19:13:16 -05:00
Andrew Hutchings
17f077012d Merge branch 'develop-1.1' into 1.1-merge-up 2017-12-13 09:09:39 +00:00
Andrew Hutchings
8babe4a35d Merge branch 'develop-1.0' into 1.0-merge-up 2017-12-12 10:01:14 +00:00
David Hall
34799d8d30 MCOL-994 handle a second abbreviated extent in case it moved dbroots because of redistribute remove. 2017-12-07 10:49:51 -06:00
Andrew Hutchings
01446d1e22 Reformat all code to coding standard 2017-10-26 17:18:17 +01:00
Andrew Hutchings
b7a01ce02e MCOL-267 Add blob support for INSERT_SELECT
* Note there is a 1MB buffer limit, rows longer than 512KB will fail (2x
due to hex of blob data)
* cpimport needs to use hex of blob data
2017-03-23 14:04:14 +00:00
david hill
f6afc42dd0 the begginning 2016-01-06 14:08:59 -06:00