The first step for deprecating innodb_autoinc_lock_mode(see MDEV-27844) is:
- to switch statement binlog format to ROW if binlog format is MIXED and
the statement changes autoincremented fields
- issue warnings if innodb_autoinc_lock_mode == 2 and binlog format is
STATEMENT
The asserion failure was caused by this query
select /*id=1*/ from t1
where
col= ( select /*id=2*/ from ... where corr_cond1
union
select /*id=4*/ from ... where corr_cond2)
Here,
- select with id=2 was correlated due to corr_cond1.
- select with id=4 was initially correlated due to corr_cond2, but then
the optimizer optimized away the correlation, making the select with id=4
uncorrelated.
However, since select with id=2 remained correlated, the execution had to
re-compute the whole UNION. When it tried to execute select with id=4, it
hit an assertion (join buffer already free'd).
This is because select with id=4 has freed its execution structures after
it has been executed once. The select is uncorrelated, so it did not expect
it would need to be executed for the second time.
Fixed this by adding this logic in
st_select_lex::optimize_unflattened_subqueries():
If a member of a UNION is correlated, mark all its members as
correlated, so that they are prepared to be executed multiple times.
Changes:
1. Enabling IN/OUT/INOUT mode for sql_mode=DEFAULT,
adding tests for sql_mode=DEFAULT based by mostly
translating compat/oracle.sp-inout.test to SQL/PSM
with minor changes (e.g. testing trigger OLD.column and
NEW.column as IN/OUT parameters).
2. Removing duplicate grammar:
sp_pdparam and sp_fdparam implemented exactly the same syntax after
- the first patch for MDEV-10654 (for sql_mode=ORACLE)
- the change #1 from this patch (for sql_mode=DEFAULT)
Removing separate rules and adding a single "sp_param" rule instead,
which now covers both PRDEDURE and FUNCTION parameters
(and CURSOR parameters as well!).
3. Adding a helper rule sp_param_name_and_mode, which is a combination
of the parameter name and the IN/OUT/INOUT mode. It allows to simplify
the grammer a bit.
4. The first patch unintentionally allowed IN/OUT/INOUT mode
to be specified in CURSOR parameters.
This is good for the IN keyword - it is allowed in PL/SQL CURSORs.
This is not good the the OUT/INOUT keywords - they should not be allowed.
Adding a additional symantic post-check.
failed in Diagnostics_area::set_ok_status in my_ok from
mysql_sql_stmt_prepare
Analysis: Before PREPARE is executed, binlog_format is STATEMENT.
This PREPARE had SET STATEMENT which sets binlog_format to ROW. Now after
PREPARE is done we reset the binlog_format (back to STATEMENT). But we have
temporary table, it doesn't let changing binlog_format=ROW to
binlog_format=STATEMENT and gives error which goes unreported. This
unreported error eventually causes assertion failure.
Fix: Change return type for LEX::restore_set_statement_var() to bool and
make it return error state.
The bug occurs where the float token containing a dot with an 'e'
notation was dropped from the request completely.
This causes a manner of invalid SQL statements like:
select id 1.e, char 10.e(id 2.e), concat 3.e('a'12356.e,'b'1.e,'c'1.1234e)1.e, 12 1.e*2 1.e, 12 1.e/2 1.e, 12 1.e|2 1.e, 12 1.e^2 1.e, 12 1.e%2 1.e, 12 1.e&2 from test;
To be parsed correctly as if it was:
select id, char(id), concat('a','b','c'), 12*2, 12/2, 12|2, 12^2, 12%2, 12&2 from test.test;
This correct parsing occurs when e is followed by any of:
( ) . , | & % * ^ /
Syntax for CONVERT TABLE
ALTER TABLE tbl_name CONVERT TABLE tbl_name TO PARTITION partition_name partition_spec
Examples:
ALTER TABLE t1 CONVERT TABLE tp2 TO PARTITION p2 VALUES LESS THAN MAX_VALUE();
New ALTER_PARTITION_CONVERT_IN command for
fast_alter_partition_table() is done in alter_partition_convert_in()
function which basically does ha_rename_table().
Table structure and data check is basically the same as in EXCHANGE
PARTITION command. And these are done by
compare_table_with_partition() and check_table_data().
Atomic DDL is done by the scheme from MDEV-22166 (see the
corresponding commit message). The only differnce is that it also has
to drop source table frm and that is done by WFRM_DROP_CONVERTED_FROM.
Initial patch was done by Dmitry Shulga <dmitry.shulga@mariadb.com>
Syntax for CONVERT keyword
ALTER TABLE tbl_name
[alter_option [, alter_option] ...] |
[partition_options]
partition_option: {
...
| CONVERT PARTITION partition_name TO TABLE tbl_name
}
Examples:
ALTER TABLE t1 CONVERT PARTITION p2 TO TABLE tp2;
New ALTER_PARTITION_CONVERT_OUT command for
fast_alter_partition_table() is done in alter_partition_convert_out()
function which basically does ha_rename_table().
Partition to extract is marked with the same flag as dropped
partition: PART_TO_BE_DROPPED. Note that we cannot have multiple
partitioning commands in one ALTER.
For DDL logging basically the principle is the same as for other
fast_alter_partition_table() commands. The only difference is that it
integrates late Atomic DDL functions and introduces additional phase
of WFRM_BACKUP_ORIGINAL. That is required for binlog consistency
because otherwise we could not revert back after WFRM_INSTALL_SHADOW
is done. And before DDL log is complete if we crash or fail the
altered table will be already new but binlog will miss that ALTER
command. Note that this is different from all other atomic DDL in that
it rolls back until the ddl_log_complete() is done even if everything
was done fully before the crash.
Test cases added to:
parts.alter_table \
parts.partition_debug \
versioning.partition \
atomic.alter_partition
https://jira.mariadb.org/browse/MDEV-26221
my_sys DYNAMIC_ARRAY and DYNAMIC_STRING inconsistancy
The DYNAMIC_STRING uses size_t for sizes, but DYNAMIC_ARRAY used uint.
This patch adjusts DYNAMIC_ARRAY to use size_t like DYNAMIC_STRING.
As the MY_DIR member number_of_files is copied from a DYNAMIC_ARRAY,
this is changed to be size_t.
As MY_TMPDIR members 'cur' and 'max' are copied from a DYNAMIC_ARRAY,
these are also changed to be size_t.
The lists of plugins and stored procedures use DYNAMIC_ARRAY,
but their APIs assume a size of 'uint'; these are unchanged.
The changes to galera.galear_var_replicate_myisam_on
in commit d9b933bec6061758c5d7b34f55afcae32a85c110
are omitted due to conflicts
with commit 27d66d644cf2ebe9201e0362f2050036cce2908a.
Due to an integer overflow an invalid size of ref_pointer_array could be
allocated.
Using size_t allows this continue. Allocation failures are
handled gracefully if the value is too big.
Thanks to Zuming Jiang for the bug report and fuzzing MariaDB.
Reviewer: Sanja
This patch fixes parsing problems concerning derived tables that use table
value constructors (TVC) with LIMIT and ORDER BY clauses of the form
((VALUES ... LIMIT ...) ORDER BY ...) as dt
The fix has to be applied only to 10.3 as 10.4 that employs a different
grammar rules has no such problems. The test cases should be merged
upstream.
Approved by Oleksandr Byelkin <sanja@mariadb.com>
- Better, easier to read code (no used of 'random' constants).
- All defines are now unique, so it is easier to find bugs if
somethings goes wrong.
Other things:
- Created sub function of common code in Aggregator_distinct::setup() and
Item_func_group_concat::setup() that set item->marker
- More documentation
- Folded a few long lines.
- Allmost all changes in item.cc, sql_lex.cc and sql_window.cc are done
with 'replace'.
In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.
This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.
# Conflicts:
# sql/sql_cte.cc
# sql/sql_cte.h
# sql/sql_lex.cc
# sql/sql_lex.h
# sql/sql_view.cc
# sql/sql_yacc.yy
# sql/sql_yacc_ora.yy
