BLOB fields did not work as grouping keys at all, they were assigned
value NULL for any value, be it NULL or not. The fix is in the
rowaggregation.cpp in the initMapping(), a switch/case branch was added
to handle BLOB field copying there.
Also, TEXT columns did not distinguish between NULL and empty string in
the grouping algorithm, now they do. The fix is in the equals()
function, now we specifically check for isNull() equality between
values.
Limit test containers by memory, fix cgroup path inside the containers by introducing new ugly setting name
---------
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
The fix is simple: enable subtotals in single-phase aggregation and
disable parallel processing when there are subtotals and aggregation is
single-phase.
This is "productization" of an old code that would enable extent
elimination for dictionary columns.
This concrete patch enables it, fixes perfomance degradation (main
problem with old code) and also fixes incorrect behavior of cpimport.
1. Restore the utf8_truncate_point() function in utils/common/utils_utf8.h
that I removed as part of the patch for MCOL-4931.
2. As per the definition of TEXT columns, the default column width represents
the maximum number of bytes that can be stored in the TEXT column. So the
effective maximum length is less if the value contains multi-byte characters.
However, if the user explicitly specifies the length of the TEXT column in a
table DDL, such as TEXT(65535), then the DDL logic ensures that enough number
of bytes are allocated (upto a system maximum) to allow upto that many number
of characters (multi-byte characters if the charset for the column is multi-byte,
such as utf8mb3).
1. Extend the following CalpontSystemCatalog member functions to
set CalpontSystemCatalog::ColType::charsetNumber, after the
system catalog update to add charset number to calpontsys.syscolumn
in MCOL-5005:
CalpontSystemCatalog::lookupOID
CalpontSystemCatalog::colType
CalpontSystemCatalog::columnRIDs
CalpontSystemCatalog::getSchemaInfo
2. Update cpimport to use the CHARSET_INFO object associated with the
charset number retrieved from the system catalog, for a
dictionary/non-dictionary CHAR/VARCHAR/TEXT column, to truncate
long strings that exceed the target column character length.
3. Add MTR test cases.
* -logging of memory WIP
* -better log for cgroup case
* -fix log
* -display in GIB
* add log for freememory for non CGROUP
(to be discussed)
* test repeated log entries
* -added counter for every 1000 call. effectivly 15m
* Name logginng period and inrease it, clear config files from PR, add .gitignore
---------
Co-authored-by: pgmabv99 <alexey.vorovich@gmail.com>
Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
Remove redundant cast.
As C-style casts with a type name in parantheses are interpreted as static_casts this literally just changes the interpretation around (and forces an implicit cast to match the return value of the function).
Switch UBIGINTNULL and UBIGINTEMPTYROW constants for consistency.
Make consistent with relation between BIGINTNULL and BIGINTEMPTYROW & make adapted cast behaviour due to NULL markers more intuitive. (After this change we can simply block the highest possible uint64_t value and if a cast results in it, print the next lower value (2^64 - 2). Previously, (2^64 - 1) was able to be printed, but (2^64 - 2) as being blocked by the UBIGINTNULL constant was not, making finding the appropiate replacement value to give out more confusing.
Introduce MAX_MCS_UBIGINT and MIN_MCS_BIGINT and adapt casts.
Adapt casting to BIGINT to remove NULL marker error.
Add bugfix regression test for MCOL 4632
Add regression test for mcol_4648
Revert "Switch UBIGINTNULL and UBIGINTEMPTYROW constants for consistency."
This reverts commit 83eac11b18937ecb0b4c754dd48e4cb47310f620.
Due to backwards compatability issues.
Refactor casting to MCS[U]Int to datatype functions.
Update regression tests to include other affected datatypes.
Apply formatting.
Refactor according to PR review
Remove redundant new constant, switch to using already existing constant.
Adapt nullstring casting to EMPTYROW markers for backwards compatability.
Adapt tests for backward compatability behaviour allowing text datatypes to be casted to EMPTYROW constant.
Adapt mcol641-functions test according to bug fix.
Update tests according to new expected behaviour.
Adapt tests to new understanding of issue.
Update comments/documentation for MCOL_4632 test.
Adapt to new cast limit logic.
Make bracketing consistent.
Adapt previous regression test to new expected behaviour.
* Fixes of bugs from ASAN warnings, part one
* MQC as static library, with nifty counter for global map and mutex
* Switch clang to 16
* link messageqcpp to execplan
This patch improves handling of NULLs in textual fields in ColumnStore.
Previously empty strings were considered NULLs and it could be a problem
if data scheme allows for empty strings. It was also one of major
reasons of behavior difference between ColumnStore and other engines in
MariaDB family.
Also, this patch fixes some other bugs and incorrect behavior, for
example, incorrect comparison for "column <= ''" which evaluates to
constant True for all purposes before this patch.
ARM platforms. This patch resolves this issue and unifies AUX column
processing at x86 and ARM using tempate class SimdProcessor.
The patch also replaces uint16_t mask previously used in column.cpp and
SimProcessor code with a native masks that platform uses, e.g. __m128i
or __m128 on x86 and variety of masks on ARM.
To unify the processing I introduced a new filtering Compare Operator - COMPARE_NULLEQ.
with a 'c1 IS NULL semantics'.
in aggregation code
The patch disables padding that forces hasher to calculate over the whole 2k buffer. This patch also moves hashing code
into the common place where it belongs.
* more build dependencies
* fix for cmake < 3.11
It cannot do ADD_LIBRARY(... ALIAS ...) on IMPORTED targets
* another fix for cmake 3.10.2
It doesn't know about CMAKE_CXX_STANDARD=20,
let's add the correct flag manually
* gcc 8 on aarch64
utils/common/simd_arm.h:241:16: error: need ‘typename’ before ‘simd::TypeToVecWrapperType<T>::WrapperType’ because ‘simd::TypeToVecWrapperType<T>’ is a dependent scope
* MCOL-5092 Ensure column width is correct for datatype
Change MODA return type to STRING
Modify MODA to handle every numeric type
* MCOL-5162 MODA to support char and varchar with collation support
Fixes to the aggregate bit functions
When we fixed the storage sign issue for MCOL-5092, it uncovered a problem in the bit aggregates (bit_and, bit_or and bit_xor). These aggregates should always return UBIGINT, but they relied on the type of the argument column, which gave bad results.
exact functionality that does not use MDB hash function.
This patch also takes a bit from Robin Hood hash map implementation forgotten
that reduces hash function collision rate.
