database/mariadb-columnstore-engine
1
0
Fork 0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-02 17:22:27 +03:00
Code Activity

Fixing DOUBLE-to-[U]INT conversion (MCOL-4649, MCOL-4631, MCOL-4647)

Browse Source 
Bugs fixed:
- MCOL-4649 CAST(double AS UNSIGNED) returns 0
- MCOL-4631 CAST(double AS SIGNED) returns 0 or NULL
- MCOL-4647 SEC_TO_TIME(double_or_float) returns a wrong result

Problems:
- The code in Func_cast_unsigned::getUintVal() and
  Func_cast_signed::getIntVal() did not properly check
  the double value to fit inside a uint64_t/int64_t range.
  So the corner cases:
  - numeric_limits<uint64_t>::max()-2 for uint64_t
  - numeric_limits<int64_t>::max() for int64_t
  produced unexpected results.

  The problem was in tests like this:
    if (value > (double) numeric_limits<int64_t>::max())
  A correct test would be:
    if (value >= (double) numeric_limits<int64_t>::max())

- The code in Func_sec_to_time::getStrVal() searched for the decimal
  dot character, assuming that the next character after the dot
  was the leftmost fractional digit.
  This assumption was wrong because huge double numbers use
  scientific notation. So for example in "2.5e-40" the
  digit "5" following the dot is NOT the leftmost fractional digit.
  Also, the code in Func_sec_to_time::getStrVal() was slow
  because of using non necessary to-string and from-string
  data conversion.
  Also, the code in Func_sec_to_time::getStrVal() evaluated
  the argument two times: using getStrVal() then using getIntVal().

Solution:
- Adding new classes TDouble and TLongDouble.
- Adding a few function templates to reuse the code easier.
- Moving the conversion code inside TDouble and TLongDouble
  methods toMCSSInt64Round() and toMCSUInt64Round().
- Reusing new classes and their methods in func_cast.cc and
  func_sec_to_time.cc.
This commit is contained in:
Alexander Barkov
2021-04-01 21:32:31 +04:00
parent f3766e40e4
commit a86f432f35
10 changed files with 302 additions and 99 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
utils/funcexp/func_sec_to_time.cpp
View File

@ -75,47 +75,14 @@ string Func_sec_to_time::getStrVal(rowgroup::Row& row,
case execplan::CalpontSystemCatalog::DOUBLE:
{
const string& valStr = parm[0]->data()->getStrVal(row, isNull);
val = parm[0]->data()->getIntVal(row, isNull);
size_t x = valStr.find(".");
if (x < string::npos)
{
string tmp = valStr.substr(x + 1, 1);
char* ptr = &tmp[0];
int i = atoi(ptr);
if (i >= 5)
{
if (val > 0)
val += 1;
else
val -= 1;
}
}
datatypes::TDouble d(parm[0]->data()->getDoubleVal(row, isNull));
val = d.toMCSSInt64Round();
break;
}
break;
case execplan::CalpontSystemCatalog::FLOAT:
{
const string& valStr = parm[0]->data()->getStrVal(row, isNull);
val = parm[0]->data()->getIntVal(row, isNull);
size_t x = valStr.find(".");
if (x < string::npos)
{
string tmp = valStr.substr(x + 1, 1);
char* ptr = &tmp[0];
int i = atoi(ptr);
if (i >= 5)
{
if (val > 0)
val += 1;
else
val -= 1;
}
}
datatypes::TDouble d(parm[0]->data()->getFloatVal(row, isNull));
val = d.toMCSSInt64Round();
}
break;