* Adds CompressInterfaceLZ4 which uses LZ4 API for compress/uncompress.
* Adds CMake machinery to search LZ4 on running host.
* All methods which use static data and do not modify any internal data - become `static`,
so we can use them without creation of the specific object. This is possible, because
the header specification has not been modified. We still use 2 sections in header, first
one with file meta data, the second one with pointers for compressed chunks.
* Methods `compress`, `uncompress`, `maxCompressedSize`, `getUncompressedSize` - become
pure virtual, so we can override them for the other compression algos.
* Adds method `getChunkMagicNumber`, so we can verify chunk magic number
for each compression algo.
* Renames "s/IDBCompressInterface/CompressInterface/g" according to requirement.
This patch:
1. Removes the option to declare uncompressed columns (set columnstore_compression_type = 0).
2. Ignores [COMMENT '[compression=0] option at table or column level (no error messages, just disregard).
3. Removes the option to set more than 2 extents per file (ExtentsPreSegmentFile).
4. Updates rebuildEM tool to support up to 10 dictionary extent per dictionary segment file.
5. Adds check for `DBRootStorageType` for rebuildEM tool.
6. Renamed rebuildEM to mcsRebuildEM.
* This patch adds rebuildEM tool support to work with compressed files.
* This patch increases a version of the file header.
Note: Default version of the `rebuildEM` tool was using very old API,
those functions are not present currently. So `rebuildEM` will not work with
files created without compression, because we cannot deduce some info which are
needed to create column extent.
* This patch extends CompressedDBFileHeader struct with new fields:
`fColumWidth`, `fColDataType`, which are necessary to rebuild extent map
from the given file. Note: new fields do not change the memory
layout of the struct, because the size is calculated as
max(sizeof(CompressedDBFileHeader), HDR_BUF_LEN)).
* This patch changes API of some functions, by adding new function
argument `colDataType` when needed, to be able to call `initHdr`
function with colDataType value.
MCS now chowns created directories hierarchy not only files and
immediate parent directories
Minor changes to cpimport's help printout
cpimport's -f option is now mandatory with mode 2
an owner for all data files created by cpimport
The patch consists of two parts: cpimport.bin changes, cpimport splitter
changes
cpimport.bin computes uid_t and gid_t early and propagates it down the stack
where MCS creates data files
Intro* INSERT statements could face a non-existant block when MCOL-498 feature
is enabled. writeRow() guard blocks was supposed to proactively create empty
blocks. The pre-patch logic failed when first value in the block has been
removed by DELETE and this overwrites the whole valid block with empty magics.
This patch moves proactive creation logic into allocRowId().
thus reduced IO load when creating a table.
Uncompressed abbreviated segment and dicts aren't affected by
this b/c CS'es system catalog uses uncompressed dict files. CS
now doesn't work with empty dicts files.
for extent extention.
Added a getter, moved some methods from protected into public to use
with unit tests, e.g createFile, setPreallocSpace. Added code stub in
FileOp::oid2FileName to use with UT.
Add more comments.
Changed return value for HDFS'es fallocate.
Removed unnecessary code in ColumnBufferCompressed::writeToFile
Replaced Nulls with Empties in variable names.
