The nightly scan showed the loop refactors fix the false positives for
std::vector but not for std::list/std::set, which Infer has no precise
model for. Re-add targeted @infer-ignore comments at the five surviving
sites (threadpool.h, we_chunkmanager.cpp x2, we_dctnry.cpp,
lbidresourcegraph.cpp), keeping the refactored loops.
Per review feedback, replace the @infer-ignore comments with refactors
that remove the patterns Infer trips on:
- destructor delete-loops: use range-based for
(altertable, tabledef, calpontsystemcatalog, threadpool, we_chunkmanager)
- we_dctnry freeStringCache(): iterate by const reference, dropping the
local Signature copy and its dead store
- LBIDResourceGraph::releaseResources(): scope rNode per iteration and
advance the iterator before the erase, removing the dummy_sit juggling
This patch adds logic to read block offsets of a dictionary column
from tokens in a pseudo-column. These block offsets then are used to
restore LBIDs in extent map and set proper HWM for a proper LBID.
An accompanying test tests rebuild process and integrity of the
restored data.
#1474 we_ddlcommandproc.cpp:2064 Uninitialized Value - column.oid
#1475 we_ddlcommandproc.cpp:2253 Uninitialized Value - column.oid
#1476 we_ddlcommandproc.cpp:2684 Uninitialized Value - column.oid
#1477 we_ddlcommandproc.cpp:2713 Uninitialized Value - DctnryTuple.sigValue
#1478 we_ddlcommandproc.cpp:3842 Uninitialized Value - column.oid
#1479 we_ddlcommandproc.cpp:3943 Uninitialized Value - DctnryTuple.sigValue
Added default constructors to DDLColumn (both in we_ddlcommon.h and
ddlpackageprocessor.h) and DctnryTuple in we_type.h.
* configcpp refactored
* chore(build): massive removals, auto add files to debian install file
* chore(build): configure before autobake
* chore(build): use custom cmake commands for components, mariadb-plugin-columnstore.install generated
* chore(build): install deps as separate step for build-packages
* more deps
* chore(codemanagement, build): build refactoring stage2
* chore(safety): Locked Map for MessageqCpp with a simpler way
Please enter the commit message for your changes. Lines starting
* chore(codemanagement, ci): better coredumps handling, deps fixed
* Delete build/bootstrap_mcs.py
* Update charset.cpp (add license)
There were numerous memory leaks in plugin's code and associated code.
During typical run of MTR tests it leaked around 65 megabytes of
objects. As a result they may severely affect long-lived connections.
This patch fixes (almost) all leaks found in the plugin. The exceptions
are two leaks associated with SHOW CREATE TABLE columnstore_table and
getting information of columns of columnstore-handled table. These
should be fixed on the server side and work is on the way.
This patch adds support for `startreadonly` command which waits
until all active cpimport jobs are done and then puts controller node to readonly
mode.
This patch adds support for `startreadonly` command which waits
until all active cpimport jobs are done and then puts controller node to readonly
mode.
1. Extend the following CalpontSystemCatalog member functions to
set CalpontSystemCatalog::ColType::charsetNumber, after the
system catalog update to add charset number to calpontsys.syscolumn
in MCOL-5005:
CalpontSystemCatalog::lookupOID
CalpontSystemCatalog::colType
CalpontSystemCatalog::columnRIDs
CalpontSystemCatalog::getSchemaInfo
2. Update cpimport to use the CHARSET_INFO object associated with the
charset number retrieved from the system catalog, for a
dictionary/non-dictionary CHAR/VARCHAR/TEXT column, to truncate
long strings that exceed the target column character length.
3. Add MTR test cases.
* Fixes of bugs from ASAN warnings, part one
* MQC as static library, with nifty counter for global map and mutex
* Switch clang to 16
* link messageqcpp to execplan
This patch improves handling of NULLs in textual fields in ColumnStore.
Previously empty strings were considered NULLs and it could be a problem
if data scheme allows for empty strings. It was also one of major
reasons of behavior difference between ColumnStore and other engines in
MariaDB family.
Also, this patch fixes some other bugs and incorrect behavior, for
example, incorrect comparison for "column <= ''" which evaluates to
constant True for all purposes before this patch.
- occured -> occurred
- reponse -> response
- seperated -> separated
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
DMLProc starts ROLLBACK when SELECT part of UPDATE fails b/c EM facility in PP were restarted.
Unfortunately this ROLLBACK stuck if EM/PP are not yet available.
DMLProc must have a t/o with re-try doing ROLLBACK.
by default), which when enabled, indiscriminately invalidates all
column extents and performs the actual DELETE only on the AUX
column. The trade-off with this approach would now be that the
first SELECT for certain query patterns (those containing a WHERE
predicate) after the DELETE operation will slow down as the
invalidated column extent would need to be scanned again to set
the min/max values.
the DELETE operation. ColumnOp::readBlock() calls can cause writes
to database files when the active chunk list in ChunkManager is full.
Since non-AUX columns are read-only for the DELETE operation, we prevent
writes of compressed chunks and header for these columns by passing
an isReadOnly flag to CompFileData which indicates whether the column
is read-only or read-write.
The idea is relatively simple - encode prefixes of collated strings as
integers and use them to compute extents' ranges. Then we can eliminate
extents with strings.
The actual patch does have all the code there but miss one important
step: we do not keep collation index, we keep charset index. Because of
this, some of the tests in the bugfix suite fail and thus main
functionality is turned off.
The reason of this patch to be put into PR at all is that it contains
changes that made CHAR/VARCHAR columns unsigned. This change is needed in
vectorization work.
* Fix clang warnings
* Remove vim tab guides
* initialize variables
* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length
* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison
* chars are unsigned on ARM, having if (ival < 0) always false
* chars are unsigned by default on ARM and comparison with -1 if always true
data types TEXT, CHAR, VARCHAR, FLOAT and DOUBLE are not yet supported by vectorized path
This patch introduces an example for Google benchmarking suite to measure a perf diff
b/w legacy scan/filtering code and the templated version
* Adds CompressInterfaceLZ4 which uses LZ4 API for compress/uncompress.
* Adds CMake machinery to search LZ4 on running host.
* All methods which use static data and do not modify any internal data - become `static`,
so we can use them without creation of the specific object. This is possible, because
the header specification has not been modified. We still use 2 sections in header, first
one with file meta data, the second one with pointers for compressed chunks.
* Methods `compress`, `uncompress`, `maxCompressedSize`, `getUncompressedSize` - become
pure virtual, so we can override them for the other compression algos.
* Adds method `getChunkMagicNumber`, so we can verify chunk magic number
for each compression algo.
* Renames "s/IDBCompressInterface/CompressInterface/g" according to requirement.
