1
0
mirror of https://github.com/MariaDB/server.git synced 2025-07-30 16:24:05 +03:00

MDEV-30932 UBSAN: negation of -X cannot be represented in type ..

'long long int'; cast to an unsigned type to negate this value ..
  to itself in Item_func_mul::int_op and Item_func_round::int_op

Problems:

  The code in multiple places in the following methods:
    - Item_func_mul::int_op()
    - longlong Item_func_int_div::val_int()
    - Item_func_mod::int_op()
    - Item_func_round::int_op()

  did not properly check for corner values LONGLONG_MIN
  and (LONGLONG_MAX+1) before doing negation.
  This cuased UBSAN to complain about undefined behaviour.

Fix summary:

  - Adding helper classes ULonglong, ULonglong_null, ULonglong_hybrid
    (in addition to their signed couterparts in sql/sql_type_int.h).

  - Moving the code performing multiplication of ulonglong numbers
    from Item_func_mul::int_op() to ULonglong_hybrid::ullmul().

  - Moving the code responsible for extracting absolute values
    from negative numbers to Longlong::abs().
    It makes sure to perform negation without undefinite behavior:
    LONGLONG_MIN is handled in a special way.

  - Moving negation related code to ULonglong::operator-().
    It makes sure to perform negation without undefinite behavior:
    (LONGLONG_MAX + 1) is handled in a special way.

  - Moving signed<=>unsigned conversion code to
    Longlong_hybrid::val_int() and ULonglong_hybrid::val_int().

  - Reusing old and new sql_type_int.h classes in multiple
    places in Item_func_xxx::int_op().

Fix details (explain how sql_type_int.h classes are reused):

  - Instead of straight negation of negative "longlong" arguments
    *before* performing unsigned multiplication,
    Item_func_mul::int_op() now calls ULonglong_null::ullmul()
    using Longlong_hybrid_null::abs() to pass arguments.
    This fixes undefined behavior N1.

  - Instead of straight negation of "ulonglong" result
    *after* performing unsigned multiplication,
    Item_func_mul::int_op() now calls ULonglong_hybrid::val_int(),
    which recursively calls ULonglong::operator-().
    This fixes undefined behavior N2.

  - Removing duplicate negating code from Item_func_mod::int_op().
    Using ULonglong_hybrid::val_int() instead.
    This fixes undefinite behavior N3.

  - Removing literal "longlong" negation from Item_func_round::int_op().
    Using Longlong::abs() instead, which correctly handler LONGLONG_MIN.
    This fixes undefinite behavior N4.

  - Removing the duplicate (negation related) code from
    Item_func_int_div::val_int(). Reusing class ULonglong_hybrid.
    There were no undefinite behavior in here.
    However, this change allowed to reveal a bug in
    "-9223372036854775808 DIV 1".
    The removed negation code appeared to be incorrect when
    negating +9223372036854775808. It returned the "out of range" error.
    ULonglong_hybrid::operator-() now handles all values correctly
    and returns +9223372036854775808 as a negation for -9223372036854775808.

    Re-recording wrong results for
      SELECT -9223372036854775808 DIV  1;
    Now instead of "out of range", it returns -9223372036854775808,
    which is the smallest possible value for the expression data type
    (signed) BIGINT.

  - Removing "no UBSAN" branch from Item_func_splus::int_opt()
    and Item_func_minus::int_opt(), as it made UBSAN happy but
    in RelWithDebInfo some MTR tests started to fail.
This commit is contained in:
Alexander Barkov
2023-06-28 16:47:27 +04:00
parent 428c7964a2
commit 67657a01bf
5 changed files with 245 additions and 97 deletions

View File

@ -710,7 +710,6 @@ DROP TABLE t1;
SELECT 9223372036854775808 DIV 1;
--error ER_DATA_OUT_OF_RANGE
SELECT 9223372036854775808 DIV -1;
--error ER_DATA_OUT_OF_RANGE
SELECT -9223372036854775808 DIV 1;
--error ER_DATA_OUT_OF_RANGE
SELECT -9223372036854775808 DIV -1;
@ -1867,6 +1866,32 @@ SELECT * FROM t2;
SHOW CREATE TABLE t2;
DROP TABLE t1,t2;
--echo #
--echo # MDEV-30932 UBSAN: negation of -X cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself in Item_func_mul::int_op and Item_func_round::int_op
--echo #
--error ER_DATA_OUT_OF_RANGE
SELECT (1 DIV(-1/POW(807,14))*1);
DO((-9223372036854775808)*(1));
SELECT (-9223372036854775808)*(1);
--error ER_DATA_OUT_OF_RANGE
SELECT (GET_FORMAT(TIME,'JIS'))DIV(POW(-40,65)DIV(1)*2);
SELECT -9223372036854775808 MOD 9223372036854775810;
CREATE TABLE t1 (c INT);
INSERT INTO t1 VALUES(TRUNCATE(0,-1.e+30));
DROP TABLE t1;
SELECT TRUNCATE(0, -9223372036854775808);
--disable_warnings
SELECT GET_FORMAT(TIME,'JIS') DIV ATAN (TRUNCATE (0,'2000000000000000' DIV SIN(1500)*NOW(5)));
SELECT (GET_FORMAT(TIME,'JIS') DIV ATAN (TRUNCATE (0,'2000000000000000' DIV SIN(1500)*NOW(5))/ROUND(-1)))DIV(-1-LOG2(1))-(-1*POWER(-1,0));
--enable_warnings
--echo #
--echo # End of 10.4 tests
--echo #