BLOB fields did not work as grouping keys at all, they were assigned
value NULL for any value, be it NULL or not. The fix is in the
rowaggregation.cpp in the initMapping(), a switch/case branch was added
to handle BLOB field copying there.
Also, TEXT columns did not distinguish between NULL and empty string in
the grouping algorithm, now they do. The fix is in the equals()
function, now we specifically check for isNull() equality between
values.
This fixes discrepance with the server, which assigns DATE type to
last_day()'s result.
Now we also assigns DATE result type and, also, use proper
dataconvert::Day data structure to return date.
Tests agree with InnoDB.
Also, this patch includes test for MCOL-5669, to show we fixed it.
This patch fixes the problem in MCOL-4234 and also generally improves
behavior of GROUP BY.
It does so by introducing a "dummy" aggregate and by wrapping columns
into it. This allows for columns that are not in GROUP BY to be used
more freely, for example, in SELECT * FROM tbl GROUP BY col - all
columns that are not "col" will be wrapped into an aggregate and query
will proceed to execution.
The dummy aggregate itself does nothing more than remember last value
passed into it.
There also an additional error message that tries to explain what types
of expressions can be wrapped into an aggregate.
Limit test containers by memory, fix cgroup path inside the containers by introducing new ugly setting name
---------
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
This changeset contains fixes in Oracle mode tests and for the
implementation of the CONCAT_ORACLE. Also, we harmonise our
translation process with the recent changes in the server.
Due to changed behavior of the server, some CREATE VIEW/EXPLAIN
statements' results begun to output unexpected results and need to be
fixed.
Also, concatenation operation's name also changed. This lead to
disabled func_concat_oracle test to be enabled to test it and it
turned out that our implementation of this function was broken
and need to be fixed too.
This changeset contains fixes in Oracle mode tests and for the
implementation of the CONCAT_ORACLE. Also, we harmonise our
translation process with the recent changes in the server.
Due to changed behavior of the server, some CREATE VIEW/EXPLAIN
statements' results begun to output unexpected results and need to be
fixed.
Also, concatenation operation's name also changed. This lead to
disabled func_concat_oracle test to be enabled to test it and it
turned out that our implementation of this function was broken
and need to be fixed too.
* fix(threadpool): MCOL-5565 queries stuck in FairThreadScheduler. (#3100)
Meta Primitive Jobs, .e.g ADD_JOINER, LAST_JOINER stuck
in Fair scheduler without out-of-band scheduler. Add OOB
scheduler back to remedy the issue.
* fix(messageqcpp): MCOL-5636 same node communication crashes transmiting PP errors to EM b/c error messaging leveraged socket that was a nullptr. (#3106)
* fix(threadpool): MCOL-5645 errenous threadpool Job ctor implictly sets socket shared_ptr to nullptr causing sigabrt when threadpool returns an error (#3125)
---------
Co-authored-by: drrtuy <roman.nozdrin@mariadb.com>
The large side read errors mentioned there can be due to failure to
close file stream properly. Some of the data may still reside in the
file stream buffers, closing must flush it. The flush is an I/O
operation and can fail, leading to partial write and subsequent partial
read.
This patch tries to provide better diagnostics.
This is a fix of logging subsystem, nothing else.
The old code expanded an argument into string and advanced too little
and, if expansion contained argument's index, it expanded it again. And
again.
The fix is simple: enable subtotals in single-phase aggregation and
disable parallel processing when there are subtotals and aggregation is
single-phase.
JSON functions were implemented violating an assumption of their
pureness, as they should not have any state. This concrete patch
fixes implementation of JSON_VALUE function.
This is "productization" of an old code that would enable extent
elimination for dictionary columns.
This concrete patch enables it, fixes perfomance degradation (main
problem with old code) and also fixes incorrect behavior of cpimport.
dh_missing: warning: Compatibility levels before 10 are deprecated (level 9 in use)
dh_missing: warning: usr/lib/x86_64-linux-gnu/libmessageqcpp.a exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/lib/x86_64-linux-gnu/libpron.a exists in debian/tmp but is not installed to anywhere
so do not install static libraries as targets on CMake
The new added invariant checking that RGData knows the number of columns and fixed size columns was failing for disk-based aggregation workloads, leading them to provide a wrong result. (The assertion failure happened in RGData::getRow(uint32_t num, Row* row) which is called in the finalization of sub-aggregation results, necessary for merging part results. As the merging failed, duplicate results were output for disk-based aggregation queries.
The assertion failure was caused by RGData::deserialize(ByteStream& bs,
uint32_t defAmount) not setting rowSize and colCount if necessary (e.g.
when the deserialization happens into a new, default RGData, which
doesn't know anything about its structure yet. This is the case when the
default constructor for RGData() is used, which sets rowSize and
columnCount to 0 each.
There are three code parts that make use of the default RGData() ctor.
The fix is for the use in RowGroupStorage::loadRG(uint64_t rgid,
std::unique_ptr<RGData>& rgdata, bool unlinkDump = false), where the
default RGData object is used to directly deserialize a ByteStream into
it. The deserialize method now checks if both rowSize and columnCount
are 0 and if yes sets the read values from the ByteStream for both.
We should probably check the other two code parts making use of the
default RGData ctor, too. This happens in joinpartition.cpp and
tuplejoiner.cpp.
---------
Co-authored-by: Theresa Hradilak <34538290+phoeinx@users.noreply.github.com>
Adds a special column which helps to differentiate data and rollups of
various depts and a simple logic to row aggregation to add processing of
subtotals.
1. Restore the utf8_truncate_point() function in utils/common/utils_utf8.h
that I removed as part of the patch for MCOL-4931.
2. As per the definition of TEXT columns, the default column width represents
the maximum number of bytes that can be stored in the TEXT column. So the
effective maximum length is less if the value contains multi-byte characters.
However, if the user explicitly specifies the length of the TEXT column in a
table DDL, such as TEXT(65535), then the DDL logic ensures that enough number
of bytes are allocated (upto a system maximum) to allow upto that many number
of characters (multi-byte characters if the charset for the column is multi-byte,
such as utf8mb3).
1. Extend the following CalpontSystemCatalog member functions to
set CalpontSystemCatalog::ColType::charsetNumber, after the
system catalog update to add charset number to calpontsys.syscolumn
in MCOL-5005:
CalpontSystemCatalog::lookupOID
CalpontSystemCatalog::colType
CalpontSystemCatalog::columnRIDs
CalpontSystemCatalog::getSchemaInfo
2. Update cpimport to use the CHARSET_INFO object associated with the
charset number retrieved from the system catalog, for a
dictionary/non-dictionary CHAR/VARCHAR/TEXT column, to truncate
long strings that exceed the target column character length.
3. Add MTR test cases.
* -logging of memory WIP
* -better log for cgroup case
* -fix log
* -display in GIB
* add log for freememory for non CGROUP
(to be discussed)
* test repeated log entries
* -added counter for every 1000 call. effectivly 15m
* Name logginng period and inrease it, clear config files from PR, add .gitignore
---------
Co-authored-by: pgmabv99 <alexey.vorovich@gmail.com>
Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
CI ocassionaly stuck running test001 b/c PP threadpool endlessly reschedules
meta jobs, e.g. BATCH_PRIMITIVE_CREATE, which ByteStreams were somehow damaged or read out.
Co-authored-by: Leonid Fedorov <leonid.fedorov@mariadb.com>
Remove redundant cast.
As C-style casts with a type name in parantheses are interpreted as static_casts this literally just changes the interpretation around (and forces an implicit cast to match the return value of the function).
Switch UBIGINTNULL and UBIGINTEMPTYROW constants for consistency.
Make consistent with relation between BIGINTNULL and BIGINTEMPTYROW & make adapted cast behaviour due to NULL markers more intuitive. (After this change we can simply block the highest possible uint64_t value and if a cast results in it, print the next lower value (2^64 - 2). Previously, (2^64 - 1) was able to be printed, but (2^64 - 2) as being blocked by the UBIGINTNULL constant was not, making finding the appropiate replacement value to give out more confusing.
Introduce MAX_MCS_UBIGINT and MIN_MCS_BIGINT and adapt casts.
Adapt casting to BIGINT to remove NULL marker error.
Add bugfix regression test for MCOL 4632
Add regression test for mcol_4648
Revert "Switch UBIGINTNULL and UBIGINTEMPTYROW constants for consistency."
This reverts commit 83eac11b18937ecb0b4c754dd48e4cb47310f620.
Due to backwards compatability issues.
Refactor casting to MCS[U]Int to datatype functions.
Update regression tests to include other affected datatypes.
Apply formatting.
Refactor according to PR review
Remove redundant new constant, switch to using already existing constant.
Adapt nullstring casting to EMPTYROW markers for backwards compatability.
Adapt tests for backward compatability behaviour allowing text datatypes to be casted to EMPTYROW constant.
Adapt mcol641-functions test according to bug fix.
Update tests according to new expected behaviour.
Adapt tests to new understanding of issue.
Update comments/documentation for MCOL_4632 test.
Adapt to new cast limit logic.
Make bracketing consistent.
Adapt previous regression test to new expected behaviour.
