Extend current dbug_print* functions in such a way as to facilitate easy
high level debugging. Introduce a single overloaded print function
defined in dbp.h, DBUG_PRINT_FUNCTION, settable to your preference.
Allow a shell environment variable DBUG_PRINT_FUNCTION to override
this definition
When InnoDB computes an indexed virtual column (e.g. the long unique
hash), TABLE::update_virtual_field() walked its expression with
update_vcol_processor() and recomputed every vcol it referenced,
including STORED ones. innobase_allocate_row_for_vcol() saves and
restores Field_blob::value only for non-stored blob vcols, so
recomputing a STORED blob vcol in Field_blob::store_field() did
val_str(&value) and freed the buffer that record[0] still pointed to.
The row-format binlog then packed record[0] from freed memory.
The buffer is allocated only when the value is converted, so the bug
needs a character set conversion (e.g. BLOB to utf8mb3 TEXT).
The fix skips STORED vcols in update_vcol_processor(). Their values are
already in the record: InnoDB copies them as base columns of the
virtual column, and the SQL layer computes them before writing.
The test cases detect the bug only under ASAN or Valgrind.
Problem:
Item_insert_value::fix_fields() created a temporary field
without preserving vcol_info. As a result, VALUE()/VALUES()
on virtual columns lost dependency metadata and could trigger:
Assertion `marked_for_read()' failed during virtual column evaluation.
Cause:
The temporary Field_string copied field_index but did not copy virtual column metadata from the original field.
Fix:
Preserve vcol_info when creating temporary fields in Item_insert_value::fix_fields() so virtual column dependency
tracking remains intact.
Between two executions of a prepared statement, Item_sp::cleanup() freed
the stored function's memory root but left the arena's free list pointing
into it. That list is filled when the function call returns and the active
arena is restored.
On the next execution Item_sum_sp::clear() walked the stale list and
destroyed already freed items.
Free the items before the memory they live in. The sequence is now
in Item_sp::free_call_ctx(), so the three places that repeat it
cannot drift apart again.
Introduces new dbug_print() functions for the following types:
dbug_print(NESTED_JOIN)
dbug_print(JOIN)
dbug_print(TABLE_LIST)
dbug_print(JOIN_TAB)
dbug_print(List<Item>)
Typically pointers to instances of the given types are passed.
The dbug_print(List<Item>) is intended to show result row before
sending to client (e.g., invoke in debugger while tracing end_send)
but may work in other contexts.
These functions produce nicely formatted output, please use them
in conjunction with the formatted printfs available in GDB or LLDB:
(gdb) printf "%s", dbug_print(join_tab)
(lldb) p printf("%s", dbug_print(table))
Example output from dbug_print(JOIN) in LLDB. The last line with
value 1302 is the number of characters produced by the printf
command.
(lldb) p printf("%s",dbug_print(join))
JOIN [0x15801bfe8] table_count=3 const_tables=0
join_list 0x158018340 [1 element(s)]:
--- #0 ---
[0x15801af10] "(nest_last_join)" join=INNER (outer_join=0) nested_join=0x15801b618 (elements=2)
NESTED_JOIN [0x15801b618]
used_tables: 0x0
not_null_tables: 0x0
n_tables: 0
counter: 0
nest_type: 1 (JOIN_OP_NEST)
nj_map: 0x0
join_list:
join_list 0x15801b618 [2 element(s)]:
--- #0 ---
[0x15801a308] "t3" join=LEFT (outer_join=1) map=0x4
ON: `test`.`t2`.`a` = `test`.`t3`.`a`
--- #1 ---
[0x158019b30] "(nest_last_join)" join=INNER (outer_join=0) nested_join=0x15801a238 (elements=2)
NESTED_JOIN [0x15801a238]
used_tables: 0x0
not_null_tables: 0x0
n_tables: 0
counter: 0
nest_type: 0
nj_map: 0x0
join_list:
join_list 0x15801a238 [2 element(s)]:
--- #0 ---
[0x1580187c0] "t1" join=RIGHT (outer_join=2) map=0x1
ON: `test`.`t1`.`a` = `test`.`t2`.`a`
--- #1 ---
[0x158018f08] "t2" join=INNER (outer_join=0) map=0x2
(int) 1302
When a grouping select referring to a derived table has an outer
reference within it's select list, that outer reference isn't
properly fixed. It isn't added to
outer_context->select_lex->inner_refs_list for later resolution during
JOIN::prepare. As it is not fixed, the Item pointer arrays associated
with the temporary table filled during the join process are now incorrect
and we end up with an incorrect join->outer_ref_cond evaluation and
potentially a wrong result.
A 0xHHHH / b'..' literal keeps its hex_hybrid type through COALESCE/IF/CASE/...,
but was evaluated, sized and stored as a string, so it behaved differently from
a bare literal (10+COALESCE(0x31) gave 11, not 59).
Type_handler_hex_hybrid now implements the numeric side to match the bare
literal: val_int/val_real/val_decimal read bytes as a hybrid, Item_decimal_precision
returns the integer precision, Item_save_in_field stores like the bare literal.
The traditional type merge keeps the hybrid through a typeless NULL (as for BIT),
and Item_hybrid_func_fix_attributes attributes it binary/unsigned.
Under ONLY_FULL_GROUP_BY, a query that aggregates an outer column inside
a subquery is wrongly rejected. This is fixed in
Item_field::fix_outer_field by not appending the field to
select->join->non_agg_fields when thd->lex->in_sum_func is not null.
Furthermore, in Item_sum::check_sum_func, for all outer fields that are
not aggregated at their SELECT_LEX's nest level, we append these fields
to sel->join->non_agg_fields in order to ensure that
ER_WRONG_FIELD_WITH_GROUP is still raised for invalid aggregation.
When setup_copy_fields() needs to copy a non-aggregate function value,
it doesn't construct an Item_copy directly. Instead, it calls
Type_handler::create_item_copy, which is a kind of factory. The base
Type_handler::create_item_copy returns Item_copy_string. Some type
handlers override it, like timestamp and fixed binary. However, the
numeric type handlers (e.g., float, double, int, decimal) did not, so
they fell through to that base and got Item_copy_string.
A SELECT that aggregates will copy each non aggregate function value
into a temporary table through an Item_copy object, whose concrete
type is chosen by the create_item_copy method on the value's type
handler. For numeric types that method returned Item_copy_string,
which stores the value as text. A FLOAT keeps only FLT_DIG
significant digits as text, too few to reproduce its 24 bit mantissa,
so the copied value differed from the original. With one row per
group, CAST(c1 AS FLOAT) - MIN(CAST(c1 AS FLOAT)) returned a large
number instead of zero.
Add Item_copy_real with Item_copy_float and Item_copy_double variants
that keep the value as a double, the same way Item_cache_real does, and
let the float and double type handlers create them. This mirrors the
existing copy items for timestamp and fixed binary types.
REPLACE() tries to modify its first argument in-place, provided it's
not a constant (like in REPLACE("foo", "bar"))
When the first argument was Item_cache_str, it was not marked as
a constant, thus REPLACE modified it in-place, and result of the
previous row leaked into the next one.
(Edited by Sergei Petrunia)
Problem: when the optimizer attempts to push a condition in form
"merged_view.not_null_column IS NULL" into a derived table other
than the merged_view itself, it may crash.
The cause is this scenario:
1. Condition pushdown code uses walk() with excl_dep_on_grouping_fields()
(or excl_dep_on_table()) to determine whether an Item expression can
be pushed. Item_args::excl_dep_on_table() assumes that any item with
const_item()==1 can be pushed anywhere.
2. Item_func_isnull(not_null_expr) will report const_item()=1 as it
will always evaluate false. However it will have non-const expression
as its argument. As described in #1, this item will be considered pushable.
3. Transformation of item for pushdown is done in "transformer function"
Item_XXX::{derived|grouping}_field_transformer_for_where().
This function will be called for Item_func_isnull's argument.
If the argument is an Item_direct_view_ref, its transformer function
will call find_matching_field_pair() (or find_producing_item())
to locate the origin of the item inside the SELECT we're pushing the
condition into. It will get a NULL pointer (origin not found) and crash.
== The fix ==
Let Item_func_is_null(not_null_expr) report const_item()==true but
let its used_tables() still report those from not_null_expr.
Then, in Item_args, make excl_dep_on_grouping_fields() and
excl_dep_on_table() to check the used tables/columns for items
that report const_item()==true but have non-zero used_tables().
Add comments to
- find_producing_item()
- Item_field::derived_field_transformer_for_where()
- Item_field::grouping_field_transformer_for_where()
Also note that Item_default_value::grouping_field_transformer_for_where()
is not implemented.
SELECT 1 union select 2 UNION SELECT 1 from a JOIN a b ON
(SELECT 1 FROM dual WHERE AAA)
Crashes during fix_outer_field while resolving field item AAA
In our resolver, once we have determined that a field item isn't
local to our select, we call Item::fix_outer_field(), which
iterates outwards towards the top level select, looking for where
our Item_field might be resolvable.
In our example here, the item isn't resolvable and we expose
fragility in the loop, which i will detail here.
After we initialize the variable 'outer_context' (to a context
containing /* select#3 */ select 1 AS `1` from (a join a b on
((subquery#4))) ) we enter a loop
│ 5927 for (;
│ 5928 outer_context;
│ 5929 outer_context= outer_context->outer_context)
│ 5930 {
│ 5931 select= outer_context->select_lex;
│ 5932 Item_subselect *prev_subselect_item=
│ 5933 last_checked_context->select_lex->master_unit()->item;
│ 5934 last_checked_context= outer_context;
here 'last_checked_context' is the context inner to the current
'outer_context', and we initialize prev_subselect_item to the
Item enclosing the unit containing this inner select.
So for the first iteration of the loop,
select: select #3
last_checked_context: from select #4 to select #3.
prev_subselect_item: item enclosing select #4 (where
field item AAA is defined)
The rest of the loop calls find_field_in_tables() /
resolve_ref_in_select_and_group() in an attempt to
resolve this item with this 'outer_context'.
After the item fails resolution, we move to an outer context
select: select #4294967295 (fake_select_lex)
last_checked_context: from select #3 to the fake select lex
containing the union (i.e. outermost)
prev_subselect_item: null, there is no Item that contains this,
it is the outermost select.
We still need to execute the rest of the loop to determine whether
AAA is resolvable here, but executing
│ 5937 place= prev_subselect_item->parsing_place;
We are now following a null pointer. We introduce a test for this
null pointer, indicating that we are now evaluating the outermost
select and we are not to try accessing the enclosing subselect item.
Approved by: Oleksandr "Sanja" Byelkin (sanja@mariadb.com)
Rename `Item::clone_item()` to `clone_constant()`, and do
the same for any overloads in descendant items.
The function returns non-NULL only for items that represent
constant literals.
Rename cloning methods of class Item and its descendants
in the following way:
(from) (to)
do_build_clone -> deep_copy
build_clone -> deep_copy_with_checks
do_get_copy -> shallow_copy
get_copy -> shallow_copy_with_checks
to better reflect their functionality.
Also make Item::deep_copy() and shallow_copy() protected.
Outside users should call deep_copy_with_checks()
and shallow_copy_with_checks().
MDEV-29300 fix causes a wrong result by incorrectly removing a wrapper to
an item that needed to be wrapped for the correct result. Direct access
to the item causes an incorrect table reference to be used during
join evaluation. We revert that fix.
Our original problem query is this
SELECT (SELECT 0 GROUP BY c1 HAVING (SELECT c1)) FROM t1 group by c1;
JOIN::prepare on
/* select#2 */ select 0 group by t1.c1 having (subquery#3)
fixing t1.c1 in group by clause, calls fix_outer_field()
this item is resolved in an outer select (#1) and it is a grouping select,
so we wrap it in Item_outer_ref and set this item to unfixed for later
fixing in fix_inner_refs().
JOIN::prepare continues onto the having clause and fixes (subquery#3) which
calls initiates the prepare series of calls, leading to setup_fields on the
fields in this JOIN, one of which is an outer reference c1.
This is resolved to the item in the next most outer select in the group by
clause. This item has been wrapped with an unfixed Item_outer_ref.
It is found in resolve_ref_in_select_and_group() is it expected that
this item will have already been fixed, hence this call in
Item_field::fix_outer_field()
DBUG_ASSERT(*ref && (*ref)->fixed());
but as explained above, it isn't fixed and debug builds assert here.
Because this wrapper cannot be resolved here for reasons detailed in
fix_inner_refs, and we cannot remove this wrapper without potentially
returning an incorrect result, we have to relax this assertion.
Approved by: Oleksandr "Sanja" Byelkin (sanja@mariadb.com)
Problem:
When building a pushable condition that can be pushed from HAVING into
WHERE, the server tries to clone aggregate functions. This is not
necessary because aggregate functions can not be pushed into WHERE
anyway.
Fix:
This fix introduces a check within `Item::build_pushable_cond` to skip
cloning aggregate functions.
Also fixes assert failure in MDEV-38492, by adding a missing copy method
for `Item_aggregate_ref`.
VALUE() should only use table->insert_values when
table->insert_values contains row values.
table->insert_values gets row values for the ODKU clause
so if VALUE() is used before that it shouldn't use table->insert_values
- Removed duplicate words, like "the the" and "to to"
- Removed duplicate lines (one double sort line found in mysql.cc)
- Fixed some typos found while searching for duplicate words.
Command used to find duplicate words:
egrep -rI "\s([a-zA-Z]+)\s+\1\s" | grep -v param
Thanks to Artjoms Rimdjonoks for the command and pointing out the
spelling errors.
don't construct a "default value field" by moving
field's ptr/null_ptr. Field can have its null_ptr moved to
extra_null_bitmap for BEFORE triggers. Perhaps there can be other
reasons for null_ptr and ptr not to be at the right offset to each other.
Instead, use pointers from TABLE_SHARE::field, which always point
to default values. Except when there's no TABLE_SHARE::field,
which can happen for TEMPTABLE views, for example, but these views
are not updatable anyway.
Add an assert to Field::move_field_offset() to ensure it's only
used for appropriately set ptr/null_ptr pairs.
On handling SP statement `FOR IN lower_bound..func() DO` the instruction
sp_instr_set is allocated on sp_head's memory root, whereas an instance
of the class Item_func_sp pointed by the data member
sp_instr_set::sp_result_field
is allocated on runtime memory root. In result, on finishing the first
execution of a stored routine the memory allocated for the instance of
the class Item_func_sp is released whereas the pointer
sp_instr_set::sp_result_field
still references the deleted memory. Next time the same stored routine
is run dereferencing deallocated memory results in abnormal server
termination.
To fix the issue, allocate an instance of the class Item_func_sp on
sp_head memory root. Do this allocation only once, meaning the
Item_func_sp::cleanup
doesn't do deletion an instance of the class Item_func_sp and
nullifying the data member sp_instr_set::sp_result_field.
Lots of different cases, SELECT, SELECT DEFAULT(),
UPDATE t SET x=DEFAULT, prepares statements,
opening of a table for the I_S, prelocking (so TL_WRITE),
insert with subquery (so SQLCOM_SELECT), etc.
Don't check NEXTVAL privileges in fix_fields() anymore, it cannot
possibly handle all the cases correctly. Make a special method
Item_func_nextval::check_access() for that and invoke it from
* fix_fields on explicit SELECT NEXTVAL()
(but not if NEXTVAL() is used in a DEFAULT clause)
* when DEFAULT bareword in used in, say, UPDATE t SET x=DEFAULT
(but not if DEFAULT() itself is used in a DEFAULT clause)
* in CREATE TABLE
* in ALTER TABLE ALGORITHM=INPLACE (that doesn't go CREATE TABLE path)
* on INSERT
helpers
* Virtual_column_info::check_access() to walk the item tree and invoke
Item::check_access()
* TABLE::check_sequence_privileges() to iterate default expressions
and invoke Virtual_column_info::check_access()
also, single-table UPDATE in prepared statements now associates
value items with fields just as multi-update already did, fixes the
case of PREPARE s "UPDATE t SET x=?"; EXECUTE s USING DEFAULT.
CHECK constraint uses caching items and the value is cached even in
case of error. At the second execution the cached value is taken and
the error is not thrown. The fix does not cache value in case error
was thrown during value retrieval.
This issue arises when we have an outer reference resolved in another outer reference
where that outer reference is resolved in a outer select that is a grouping select.
Under these circumstances, the intermediate item is wrapped in an Item_outer_ref and
fixing is deferred later until fix_inner_refs. As this item wrapper isn't fixed
we fail this assertion. The fix here is to resolve the item at the lowest level
to the item inside the wrapper, which is fixed. This item can then get it's
own wrapper pointing to the ultimate resolution of this item.
Approved by Sanja Byelkin (sanja@mariadb.com) 2025-06-13
The problem is that copy function was used in field list but never
copied in this execution path.
So copy should be performed before returning result.
Protection against uninitialized copy usage added.
Item:print_for_table_def() uses QT_TO_SYSTEM_CHARSET to print
the DEFAULT expression into FRM file during CREATE TABLE.
Therefore, the expression is encoded in utf8 in FRM.
get_field_default_value() erroneously used field->charset() to
print the DEFAULT expression at SHOW CREATE TABLE time.
Fixing get_field_default_value() to use &my_charset_utf8mb4_general_ci instead.
This makes DEFAULT work in the way way with:
- virtual column expressions:
if (field->vcol_info)
{
StringBuffer<MAX_FIELD_WIDTH> str(&my_charset_utf8mb4_general_ci);
field->vcol_info->print(&str);
- check constraint expressions:
if (field->check_constraint)
{
StringBuffer<MAX_FIELD_WIDTH> str(&my_charset_utf8mb4_general_ci);
field->check_constraint->print(&str);
Additional cleanup:
Fixing system_charset_info to &my_charset_utf8mb4_general_ci in a few
places to make non-BMP characters work in DEFAULT, virtual column,
check constraint expressions.
MDEV-28127 did is_equal() which compared vcol expressions
literally. But another table vcol expression is not equal because of
different table name.
We implement another comparison method is_identical() which respects
different table name in vcol comparison. If any field item points to
table_A and compared field item points to table_B, such items are
treated as equal in (table_A, table_B) comparison. This is done by
cloning table_B expression and renaming any table_B entries to table_A
in it.
value.type_handler()->result_type()'
failed in virtual bool Item_param::get_date(THD*, MYSQL_TIME*, date_mode_t)
This is a cleanup for MDEV-25593. When binding from NULL, IGNORE or DEFAULT,
value.type_handler should be set to &type_handler_null,
to satisfy the DBUG_ASSERT in Item_param::get_date().
When binding to NULL, DEFAULT or IGNORE from an Item value, Item_param did not
change m_type_handler, so its value remained from the previous bind.
Thid led to DBUG_ASSERTs in Item_param::get_date() and
Timestamp_or_zero_datetime_native_null.
Fix:
Set Item_param::m_type_handler to &type_handler_null when
binding from an Item returning NULL.
This patch also fixes MDEV-35427.
