mcsconfig.h and my_config.h have the following
pre-processor definitions:
1. Conflicting definitions coming from the standard cmake definitions:
- PACKAGE
- PACKAGE_BUGREPORT
- PACKAGE_NAME
- PACKAGE_STRING
- PACKAGE_TARNAME
- PACKAGE_VERSION
- VERSION
2. Conflicting definitions of other kinds:
- HAVE_STRTOLL - this is a dirt in MariaDB headers.
Should be fixed in the server code. my_config.h erroneously
performs "#define HAVE_STRTOLL" instead of "#define HAVE_STRTOLL 1".
in some cases. The former is not CMake compatible style. The latter is.
3. Non-conflicting definitions:
Otherwise, mcsconfig.h and my_config.h should be mutually compatible,
because both are generated by cmake on the same host machine. So
they should have exactly equal definitions like "HAVE_XXX", "SIZEOF_XXX", etc.
Observations:
- It's OK to include both mcsconfig.h and my_config.h providing that we
suppress duplicate definition of the above conflicting types #1 and #2.
- There is no a need to suppress duplicate definitions mentioned in #3,
as they are compatible!
- my_sys.h and m_ctype.h must always follow a CMake configuation header,
either my_config.h or mcsconfig.h (or both).
They must never be included without any preceeding configuration header.
This change make sure that we resolve conflicts by:
- either disallowing inclusion of mcsconfig.h and my_config.h
at the same time
- or by hiding conflicting definitions #1 and #2
(with their later restoring).
- also, by making sure that my_sys.h and m_ctype.h always follow
a CMake configuration file.
Details:
- idb_mysql.h can now only be included only after my_config.h
An attempt to use idb_mysql.h with mcsconfig.h instead of
my_config.h is caught by the "#error" preprocessor directive.
- mariadb_my_sys.h can now be only included after mcsconfig.h.
An attempt to use mariadb_my_sys.h without mcscofig.h
(e.g. with my_config.h) is also caught by "#error".
- collation.h now can now be included in two ways.
It now has the following effective structure:
#if defined(PREFER_MY_CONFIG_H) && defined(MY_CONFIG_H)
// Remember current conflicting definitions on the preprocessor stack
// Undefine current conflicting definitions
#endif
#include "mcsconfig.h"
#include "m_ctype.h"
#if defined(PREFER_MY_CONFIG_H) && defined(MY_CONFIG_H)
# Restore conflicting definitions from the preprocessor stack
#endif
and can be included as follows:
a. using only mcsconfig.h as a configuration header:
// my_config.h must not be included so far
#include "collation.h"
b. using my_config.h as the first included configuration file:
#define PREFER_MY_CONFIG_H // Force conflict resolution
#include "my_config.h" // can be included directly or indirectly
...
#include "collation.h"
Other changes:
- Adding helper header files
utils/common/mcsconfig_conflicting_defs_remember.h
utils/common/mcsconfig_conflicting_defs_restore.h
utils/common/mcsconfig_conflicting_defs_undef.h
to perform conflict resolution easier.
- Removing `#include "collation.h"` from a number of files,
as it's automatically included from rowgroup.h.
- Removing redundant `#include "utils_utf8.h"`.
This change is not directly related to the problem being fixed,
but it's nice to remove redundant directives for both collation.h
and utils_utf8.h from all the files that do not really need them.
(this change could probably have gone as a separate commit)
- Changing my_init() to MY_INIT(argv[0]) in the MCS services sources.
After the fix of the complitation failure it appeared that ColumnStore
services compiled with the debug build crash due to recent changes in
safemalloc. The crash happened in strcmp() with `my_progname` as an argument
(where my_progname is a mysys global variable). This problem should
probably be fixed on the server side as well to avoid passing NULL.
But, the majority of MariaDB executable programs also use MY_INIT(argv[0])
rather than my_init(). So let's make MCS do like the other programs do.
This patch is fixing the following bugs:
- MCOL-4609 TreeNode::getIntVal() does not round: implicit DECIMAL->INT cast is not MariaDB compatible
- MCOL-4610 TreeNode::getUintVal() looses precision for narrow decimal
- MCOL-4619 TreeNode::getUintVal() does not round: Implicit DECIMAL->UINT conversion is not like in InnoDB
- MCOL-4650 TreeNode::getIntVal() looses precision for narrow decimal
- MCOL-4651 SEC_TO_TIME(hugePositiveDecimal) returns a negative time
For a query of the form:
SELECT COUNT(c2) FROM (SELECT * FROM t1) q;
where t1 contains 10 columns c1, c2, ... , c10.
We currently create an intermediate RowGroup in ExeMgr with
a row of the form (1, c2_value1, 1, 1, 1, 1, 1, 1, 1, 1), i.e.
for all the columns of the subquery which are not referenced in
the outer query, we substitute a constant value, which is wasteful.
With this optimization, we are trimming the RowGroup to a row
of the form (1, c2_value1). This can have non-trivial query
execution time improvements if the subquery contains large number
of columns (such as a "select *" on a very wide table) and the outer
query is only referencing a subset of these columns with lower
index values from the subquery (as an example, c1 or c2 above).
That is, the current limitation of this optimization is we are not
removing those non-referenced subquery columns (c1 in the query above)
which are to the left of a referenced column.
CI with RelWithDebInfo builds revealed a problem in the main
patch for MCOL-4464, which did not show up with Debug builds.
Methods like:
- getDoubleVal()
- getDateIntVal()
- getDatetimeIntVal()
- getTimestampIntVal()
- getTimeIntVal()
- getUintVal()
- getIntVal()
- getStrVal()
require the caller to initialize the isNull argument to false.
This fact was not taken into account in MCOL-4464.
Adding proper initializations.
1. Add wide decimal support to AggregateColumn::evaluate
and TreeNode::getDecimalVal().
2. Use the pm aggregate attributes to determine um aggregate
attributes in TupleAggregateStep::prep2PhasesAggregate.
MCOL-4409 This patch combines VDecimal and Decimal and makes
IDB_Decimal an alias for the result class
MCOL-4409 More boilerplate reduction in Func_mod
Removed couple TSInt128::toType() methods
1. Make PredicateOperator::setOpType() function wide decimal aware.
2. Added support for wide decimal in jlf_subquery.cpp::getColumnValue()
used in scalar subqueries.
3. Fixed the column index used for fetching wide decimal values from a Row
when a wide decimal field is used in the order by clause.
1. In TupleAggregateStep::configDeliveredRowGroup(), use
jobInfo.projectionCols instead of jobInfo.nonConstCols
for setting scale and precision if the source column is
wide decimal.
2. Tighten rules for wide decimal processing. Specifically:
a. Replace (precision > INT64MAXPRECISION) checks with
(precision > INT64MAXPRECISION && precision <= INT128MAXPRECISION)
b. At places where (colWidth == MAXDECIMALWIDTH) is not enough to
determine if a column is wide decimal or not, also add a check on
type being DECIMAL/UDECIMAL.
In addition, a regression in a WHERE clause with a WF field
as the LHS and an addition operation on two WF fields on the RHS
is also fixed. The issue was SimpleColumn::getDecimalVal() was
setting precision = 19, with the value of one of the operands of the
addition operation being set in VDecimal::value instead of
VDecimal::s128Value. addSubtractExecute() in mcs_decimal.cpp makes the
assumption that if precision > 18 and precision <= 38, we need to
fetch the wide s128Value, not the narrow value field. So we are
fixing the precision set in SimpleColumn::getDecimalVal().
Removed uint128 from joblist/lbidlist.*
Another toString() method for wide-decimal that is EMPTY/NULL aware
Unified decimal processing in WF functions
Fixed a potential issue in EqualCompData::operator() for
wide-decimal processing
Fixed some signedness warnings
For now it consists of only:
using int128_t = __int128;
using uint128_t = unsigned __int128;
All new privitive data types should go into this file in the future.
WF::percentile runtime threw an exception b/c of wrong DT deduced from its argument
Replaced literals with constants
Tought WF_sum_avg::checkSumLimit to use refs instead of values
Introduced fDecimalOverflowCheck to enable/disable overflow check.
Add support into a FunctionColumn.
Low level scanning crashes on medium sized data sets.
2. Set Decimal precision in SimpleColumn::evaluate().
3. Add support for int128_t in ConstantColumn.
4. Set IDB_Decimal::s128Value in buildDecimalColumn().
5. Use width 16 as first if predicate for branching based on decimal width.
