You've already forked mariadb-columnstore-engine
mirror of
https://github.com/mariadb-corporation/mariadb-columnstore-engine.git
synced 2025-07-29 08:21:15 +03:00
MCOL-4600 CAST(decimal AS SIGNED/UNSIGNED) returns a wrong result
The "SIGNED" part of the problem was previously fixed by MCOL-4640. Fixing the "UNSIGNED" part. - Adding TDecimal64::toUInt64Round() and Decimal::decimal64ToUInt64Round() - Renaming Decimal::narrowRound() to decimal64ToSInt64Round(), for a more self-descriptive name, and for symmetry with decimal64ToUInt64Round() - Reusing TDecimal64::toSInt64Round() inside decimal64ToSInt64Round(). This change was forgotten in MCOL-4640 :( - Removing the old code in Func_cast_unsigned::getUintVal with pow(). It caused precision loss, hence the bug. Adding a call for Decimal::decimal64ToUInt64Round() instead. - Adding tests for both SIGNED and UNSIGNED casts. Additional change: - Moving the wide-decimal-to-uint64_t rounding code from Func_cast_unsigned::getUintVal() to TDecimal128::toUInt64Round() (with refactoring). Adding TDecimal::toUInt64Round() for symmetry with TDecimal::toSInt64Round(). It will be easier to reuse the code with way.
This commit is contained in:
@ -100,7 +100,7 @@ datatypes::TUInt64Null DecimalToBitOperand(Row& row,
|
||||
return ConvertToBitOperand<int128_t>(val);
|
||||
}
|
||||
|
||||
return datatypes::TUInt64Null((uint64_t) d.narrowRound());
|
||||
return datatypes::TUInt64Null((uint64_t) d.decimal64ToSInt64Round());
|
||||
}
|
||||
|
||||
|
||||
|
Reference in New Issue
Block a user