Part 1:
As part of MCOL-3776 to address synchronization issue while accessing
the fTimeZone member of the Func class, mutex locks were added to the
accessor and mutator methods. However, this slows down processing
of TIMESTAMP columns in PrimProc significantly as all threads across
all concurrently running queries would serialize on the mutex. This
is because PrimProc only has a single global object for the functor
class (class derived from Func in utils/funcexp/functor.h) for a given
function name. To fix this problem:
(1) We remove the fTimeZone as a member of the Func derived classes
(hence removing the mutexes) and instead use the fOperationType
member of the FunctionColumn class to propagate the timezone values
down to the individual functor processing functions such as
FunctionColumn::getStrVal(), FunctionColumn::getIntVal(), etc.
(2) To achieve (1), a timezone member is added to the
execplan::CalpontSystemCatalog::ColType class.
Part 2:
Several functors in the Funcexp code call dataconvert::gmtSecToMySQLTime()
and dataconvert::mySQLTimeToGmtSec() functions for conversion between seconds
since unix epoch and broken-down representation. These functions in turn call
the C library function localtime_r() which currently has a known bug of holding
a global lock via a call to __tz_convert. This significantly reduces performance
in multi-threaded applications where multiple threads concurrently call
localtime_r(). More details on the bug:
https://sourceware.org/bugzilla/show_bug.cgi?id=16145
This bug in localtime_r() caused processing of the Functors in PrimProc to
slowdown significantly since a query execution causes Functors code to be
processed in a multi-threaded manner.
As a fix, we remove the calls to localtime_r() from gmtSecToMySQLTime()
and mySQLTimeToGmtSec() by performing the timezone-to-offset conversion
(done in dataconvert::timeZoneToOffset()) during the execution plan
creation in the plugin. Note that localtime_r() is only called when the
time_zone system variable is set to "SYSTEM".
This fix also required changing the timezone type from a std::string to
a long across the system.
non-wide decimal column in the HAVING clause.
In buildAggregateColumn(), if an aggregate function (such as avg)
is applied on a non-wide decimal column, we were setting the precision
of the resulting column as -1. This later down in the execution got
converted to 255 as in some cases, precision is stored as uint8_t.
The predicate operations on a DECIMAL column has logic that uses
the wide Decimal::s128value field if precision > 18. This logic incorrectly
used the Decimal::s128value instead of the correct value stored in the
narrow Decimal::value field, since precision of the Decimal column
was 255. The fix is to set the aggregate column precision to
datatypes::INT64MAXPRECISION (18) in buildAggregateColumn() when the
aggregate is applied on a non-wide decimal column.
This commit also partially fixes -Wstrict-aliasing GCC warnings.
In this patch, we set the unioned type to a wide decimal, if any of the
numeric columns involved in the union operation have a precision > 18
(which is also possible for BIGINT/UBIGINT types) and <= 38.
1. This patch adds support for wide decimals with/without scale
to cpimport. In addition, INSERT ... SELECT and LDI are also
now supported.
2. Logic to compute the number of bytes to convert a binary
representation in the buffer to a narrow decimal is also
simplified.
Since we now perform type promotion to wide decimals for aggregations
involving numeric fields, we need to check for wide decimal in
in and out ROWs and call the appropriate setter and getter functions.
For now it consists of only:
using int128_t = __int128;
using uint128_t = unsigned __int128;
All new privitive data types should go into this file in the future.
This commit also adds support in TupleHashJoinStep::forwardCPData,
although we currently do not support wide decimals as join keys.
Row estimation to determine large-side of the join is also updated.
2. Set Decimal precision in SimpleColumn::evaluate().
3. Add support for int128_t in ConstantColumn.
4. Set IDB_Decimal::s128Value in buildDecimalColumn().
5. Use width 16 as first if predicate for branching based on decimal width.
This commit introduces DataConvert UTs.
DataConvert::decimalToString now can negative values.
Next version for Row::toString(), applyMapping UT checks.
Row:equals() is now wide-DECIMAL aware.
Replaced BINARYEMPTYROW and BINARYNULL values. We need to have
separate magic values for numeric and non-numeric binary types
b/c numeric cant tolerate losing 0 used for magics previously.
atoi128() now parses minus sign and produces negative values.
RowAggregation::isNull() now uses Row::isNull() for DECIMAL.
There are multiple overloaded version of the low level DML write methods to
push down CSC column type. WE needs the type to convert values correctly.
Replaced WE_INT128 with CSC data type that is more informative.
Removed commented and obsolete code.
Replaced switch-case blocks with oneliners.
Fixes:
* Irrelevant where conditions
* Irrelevant const
* A potential infinite loop in treenode
* Bad implicit case fallthroughs
* Explicit markings for required case fallthroughs
* Unused variables
* Unused function
Also disabled some warnings for now which we should fix later.
Rename packages to MariaDB-columnstore-engine, MariaDB-columnstore-libs
and MariaDB-columnstore-platform.
Also add the "columnstore-" prefix the the components so that MariaDB's
packaging system understands then and add a line to include them in
MariaDB's packaging.
In addition
* Fix S3 building for dist source build
* Fix Debian 10 dependency issue
* Fix git handling for dist builds
* Add support for MariaDB's RPM building
* Use MariaDB's PCRE and readline
* Removes a few dead files
* Fix Boost noncopyable includes
Found the following:
* Potential stack explosions with alloca() usage on potentially large
strings
* Memory leaks in WriteEngineServer
* Stack usage out of scope in dataconvert
* A typo in an 'if' statement in dataconvert
