Problem: SUM(DISTINCT b'1100') returns 9 instead of 12. The hybrid
hex/bit literal handler uses the default DOUBLE temporary field, so
its byte length becomes a DOUBLE(M,0) display width and clips the
numeric value.
Fix: Override numeric DISTINCT field construction for hybrid hex and
bit literals to allocate an unsigned Field_longlong directly. Preserve
argument-based field selection for other types so DISTINCT retains the
input expression's precision and scale, including decimal AVG inputs.
Extend main.sum_distinct to cover bit and hex literals, zero, byte and
unsigned 64-bit boundaries, duplicates, AVG, COUNT and decimal scale.
Recreated-by: Georgi "Joro" Kodinov <joro@mariadb.org>
Applying HEX encoding write writting an ENUM/SET TYPELIB to FRM
if the TYPELIB has 0x00 bytes in the value.
This HEX encoding was earlier used only to write UCS2/UTF16/UTF32 TYPELIBs.
A new flag FIELDFLAG_FRM_HEX_ENCODED_TYPELIB was added to indicate
that the TYPELIB is hex encoded. It's used only inside FRM.
Note, it's mangled with FIELDFLAG_TREAT_BIT_AS_CHAR.
This should not be harmful:
- BIT and ENUM/SET columns are handled by two separate code branches
when opening an FRM
- The flag is unset immediately after decoding TYPELIB, so the rest
of the code does not se an unexpected flag combination.
A 0xHHHH / b'..' literal keeps its hex_hybrid type through COALESCE/IF/CASE/...,
but was evaluated, sized and stored as a string, so it behaved differently from
a bare literal (10+COALESCE(0x31) gave 11, not 59).
Type_handler_hex_hybrid now implements the numeric side to match the bare
literal: val_int/val_real/val_decimal read bytes as a hybrid, Item_decimal_precision
returns the integer precision, Item_save_in_field stores like the bare literal.
The traditional type merge keeps the hybrid through a typeless NULL (as for BIT),
and Item_hybrid_func_fix_attributes attributes it binary/unsigned.
Problem:
SELECT on a table with a TIMESTAMP column and a virtual column
expression containing a field reference crashes with assertion
marked_for_read() in Field_timestamp0::get_timestamp.
Cause:
convert_item_for_comparison() physically evaluates the subject item
via Datetime dt(thd, subject, ...) to attempt a TIMESTAMP vs DATETIME
optimization. During vcol_fix_expr() at open_table() time,
used_tables_cache is not yet propagated up the item tree, causing
const_item() to return true incorrectly for a non-constant expression
containing a field reference. The evaluation then reads Field_timestamp0
before read bitmaps are initialized, violating marked_for_read().
Fix:
Add find_field_processor() to the Item base class, overridden in
Item_field to return true. Guard convert_item_for_comparison() with
a walk() check to skip physical evaluation if the subject subtree
contains any field reference.
Field::get_date and Item::get_date can return true and not
have their ltime pointer structure initialized.
This was the case with Field_vers_trx_id::get_date.
Type_handler_time_common::stored_field_cmp_to_item rather
than doing a my_time_compare on uninitialized values can
return a non-equality (!=0) value for when the fetching of
Field or Item fails and avoid the call to my_time_compare.
Added checks for other invalid results before being
passed to my_time_compare.
Add nonnull as an attribute on my_time_compare and the
unpack_time function it used. Also pack_time since related
and it obviously requires a non-null argument.
Reviewed by: Alexey Midenkov
When setup_copy_fields() needs to copy a non-aggregate function value,
it doesn't construct an Item_copy directly. Instead, it calls
Type_handler::create_item_copy, which is a kind of factory. The base
Type_handler::create_item_copy returns Item_copy_string. Some type
handlers override it, like timestamp and fixed binary. However, the
numeric type handlers (e.g., float, double, int, decimal) did not, so
they fell through to that base and got Item_copy_string.
A SELECT that aggregates will copy each non aggregate function value
into a temporary table through an Item_copy object, whose concrete
type is chosen by the create_item_copy method on the value's type
handler. For numeric types that method returned Item_copy_string,
which stores the value as text. A FLOAT keeps only FLT_DIG
significant digits as text, too few to reproduce its 24 bit mantissa,
so the copied value differed from the original. With one row per
group, CAST(c1 AS FLOAT) - MIN(CAST(c1 AS FLOAT)) returned a large
number instead of zero.
Add Item_copy_real with Item_copy_float and Item_copy_double variants
that keep the value as a double, the same way Item_cache_real does, and
let the float and double type handlers create them. This mirrors the
existing copy items for timestamp and fixed binary types.
Year::year_precision(): Make static, so that VYear::VYear
and VYear_op::VYear_op() will avoid invoking this function with
an unnecessary "this" pointer to an uninitialized object.
Item_singerow_subselect::fix_length_and_dec() incorrectly calculated
its Item::max_length when the underlying expression was Item_int.
The reason of the problem:
Item_int has an optimized max_length to make CONCAT(1) create a VARCHAR(1)
column rather than a VARCHAR(2) column. Its max_length does not include one
extra character for the sign in case the value is positive but the value
is not marked as Item::unsigned==true.
So copying max_length from the underlying Item_int (with value==9)
in cases like this:
SELECT CONCAT((SELECT 9 FROM t0));
was not correct.
Implementing a new virtual method
Type_handler::Item_type_std_attributes_generic(const Item *item)
- The default implementation just copies attributes from "item" as is.
- In case of Type_handler_int_result it evaluates max_length
using item->decimal_precision() rather than item->max_length.
This works correctly for both "optimized" items like Item_int and
non-"optimized" Items whose max_length includes +1 for the sign
in case of signed expressions.
all params are initialized to use Type_handler_null.
NULL params do not have a value to read from the packet buffer,
so their Item_param_set_param_func() should do nothing.
Type_handler_null inherits from Type_handler_string_result,
that is, Type_handler_null *must* override parent's
Item_param_set_param_func().
In commit fa7d4abf16
when the 16-bit type alias decimal_digits_t was introduced,
some incompatible use of the 32-bit uint data type was overlooked.
Only for converting Item_result_field::max_length (uint), we will use
explicit conversion.
The Count_handler (derived from Internal_error_handler) used in
Type_handler_datetime_common::convert_item_for_comparison()
suppressed all error codes. This caused a DBUG_ASSERT(thd->is_error())
in Field::sp_prepare_and_store_item() when the error happened
while evaluating a subselect in a scenario like this:
CREATE FUNCTION f() RETURNS INT DETERMINISTIC RETURN (SELECT a FROM t);
CREATE TABLE t (c TIMESTAMP);
SELECT * FROM t WHERE c=f();
Notice there is no such column 'a' in in the table 't'.
Fix:
Handle only DATETIME->TIMESTAMP conversion related errors in Count_handler:
- ER_TRUNCATED_WRONG_VALUE
- ER_DATETIME_FUNCTION_OVERFLOW
thus let other error kinds be processed in their usual way.
The reported scenario now returns this (expected) error:
ERROR 1054 (42S22): Unknown column 'a' in 'SELECT'
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
The functions queue_compare, qsort2_cmp, and qsort_cmp2
all had similar interfaces, and were used interchangable
and unsafely cast to one another.
This patch consolidates the functions all into the
qsort_cmp2 interface.
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
LEAST() and GREATEST() erroneously calcucalted the result as signed
for BIGINT UNSIGNED arguments.
Adding a new method for unsigned arguments:
Item_func_min_max::val_uint_native()
Search conditions were evaluated using val_int(), which was wrong.
Fixing the code to use val_bool() instead.
Details:
- Adding a new item_base_t::IS_COND flag which marks Items used
as <search condition> in WHERE, HAVING, JOIN ON, CASE WHEN clauses.
The flag is at the parse time.
These expressions must be evaluated using val_bool() rather than val_int().
Note, the optimizer creates more Items which are used as search conditions.
Most of these items are not marked with IS_COND yet. This is OK for now,
but eventually these Items can also be fixed to have the flag.
- Adding a method Item::is_cond() which tests if the Item has the IS_COND flag.
- Implementing Item_cache_bool. It evaluates the cached expression using
val_bool() rather than val_int().
Overriding Type_handler_bool::Item_get_cache() to create Item_cache_bool.
- Implementing Item::save_bool_in_field(). It uses val_bool() rather than
val_int() to evaluate the expression.
- Implementing Type_handler_bool::Item_save_in_field()
using Item::save_bool_in_field().
- Fixing all Item_bool_func descendants to implement a virtual val_bool()
rather than a virtual val_int().
- To find places where val_int() should be fixed to val_bool(), a few
DBUG_ASSERT(!is_cond()) where added into val_int() implementations
of selected (most frequent) classes:
Item_field
Item_str_func
Item_datefunc
Item_timefunc
Item_datetimefunc
Item_cache_bool
Item_bool_func
Item_func_hybrid_field_type
Item_basic_constant descendants
- Fixing all places where DBUG_ASSERT() happened during an "mtr" run
to use val_bool() instead of val_int().
The code erroneously called sec_since_epoch() for dates with zeros,
e.g. '2024-00-01'.
Fixi: adding a test that the date does not have zeros before
calling TIME_to_native().
A mixture of a multi-byte *TEXT column and a short binary column
produced a too large column.
For example, COALESCE(tinytext_utf8mb4, short_varbinary)
produced a BLOB column instead of an expected TINYBLOB.
- Adding a virtual method Type_all_attributes::character_octet_length(),
returning max_length by default.
- Overriding Item_field::character_octet_length() to extract
the octet length from the underlying Field.
- Overriding Item_ref::character_octet_length() to extract
the octet length from the references Item (e.g. as VIEW fields).
- Fixing Type_numeric_attributes::find_max_octet_length() to
take the octet length using the new method character_octet_length()
instead of accessing max_length directly.
Fixing applying the COLLATE clause to a parameter caused an error error:
COLLATION '...' is not valid for CHARACTER SET 'binary'
Fix:
- Changing the collation derivation for a non-prepared Item_param
to DERIVATION_IGNORABLE.
- Allowing to apply any COLLATE clause to expressions with DERIVATION_IGNORABLE.
This includes:
1. A non-prepared Item_param
2. An explicit NULL
3. Expressions derived from #1 and #2
For example:
SELECT ? COLLATE utf8mb_unicode_ci;
SELECT NULL COLLATE utf8mb_unicode_ci;
SELECT CONCAT(?) COLLATE utf8mb_unicode_ci;
SELECT CONCAT(NULL) COLLATE utf8mb_unicode_ci
- Additional change: preserving the collation of an expression when
the expression gets assigned to a PS parameter and evaluates to SQL NULL.
Before this change, the collation of the parameter was erroneously set
to &my_charset_binary.
- Additional change: removing the multiplication to mbmaxlen from the
fix_char_length_ulonglong() argument, because the multiplication already
happens inside fix_char_length_ulonglong().
This fixes a too large column size created for a COLLATE clause.
The optimization code replacing DATETIME comparison to TIMESTAMP comparison
in conditions like:
- WHERE timestamp_col=const_expr
- WHERE const_expr IN (SELECT timestamp_column FROM t1)
worked as follows:
- Install an internal condition handler (suppressing and counting warnings).
- Convert const_expr from its data type to TIMESTAMP
- Check the warning count collected by the internal condition handler:
* If any warnings happened during the constant conversion,
then continue with DATETIME comparison.
* Otherwise, go to the next stage of switching to TIMESTAMP comparison.
This scenario did not take into account that in some cases warnings
are escalated to errors. Errors were not caught by the internal handler,
so Type_handler_datetime_common::convert_item_for_comparison()
returned with an SQL error in the diagnostics area.
The calling code did not expect this.
Fixing the code to suppress and count both errors and warnings, to make sure
Type_handler_datetime_common::convert_item_for_comparison() returns without
adding any errors to DA if the conversion to TIMESTAMP fails and it decides
to go with DATETIME comparison.
Some fixes related to commit f838b2d799 and
Rows_log_event::do_apply_event() and Update_rows_log_event::do_exec_row()
for system-versioned tables were provided by Nikita Malyavin.
This was required by test versioning.rpl,trx_id,row.
