Parser changes made by Alexander Barkov <bar@mariadb.com>.
Part of the tests made by Iqbal Hassan <iqbal@hasprime.com>.
Initially marking with ORA_JOIN flag made also by
Iqbal Hassan <iqbal@hasprime.com>.
Main idea is that parser mark fields with (+) with a flag
(ORA_JOIN).
During Prepare the flag bring to the new created items if needed.
Later after preparing (fix_firlds()) WHERE confition the
relations betweel the tables analyzed and tables reordered
so to make JOIN/LEFT JOIN operators in chain equivalent to
the query with oracle outer join operator (+).
Then the flags of (+) removed.
The patch for SYS_REFCURSOR (MDEV-20034) overrode these methods:
- Item_func_case_searched::check_arguments()
- Item_func_if::check_arguments()
to validate WHEN-style arguments (e.g. args[0] in case of IF) for being
able to return a boolean result.
However, this unintentionally removed the test for the THEN-style arguments
that they are not expressions of the ROW data type.
This led to a crash inside Type_handler_hybrid_field_type::aggregate_for_result
on a DBUG_ASSERT that arguments are not of the ROW data type.
Fix:
The fix restores blocking ROW expressions in the not supported cases,
to avoid the DBUG_ASSERT and to raise an SQL error instead.
Blocking ROW_RESULT expressions is done per Item_func_case_expression
descendant individually, instead of blocking any ROW_RESULT arguments
at the Item_func_case_expression level.
The fix is done taking into account the upcoming patch for associative arrays
(MDEV-34319). It should be possible to pass associative array expressions into
some hybrid type functions, where ROW type expressions are not possible.
As a side effect, some lecagy ER_OPERAND_COLUMNS changed to
a newer ER_ILLEGAL_PARAMETER_DATA_TYPE_FOR_OPERATION
Changes in the top affected class Item_func_case_expression:
- item_func.h:
Overriding Item_func_case_expression::check_arguments() to return false,
without checking any arguments. Descendant validate arguments
in a various different ways. No needs to block all non-scalar data type at
this level, to prevent disallowing associative arrays.
Changes in descendants:
- item_cmpfunc.cc:
Adding a test in Item_func_case_simple::aggregate_switch_and_when_arguments()
preventing passing ROW_RESULT expression in predicant and WHEN in a
simple CASE:
CASE predicant WHEN when1 THEN .. WHEN when2 THEN .. END;
This is not supported yet. Should be preferrably fixed before MDEV-34319.
- item_cmpfunc.cc:
Calling args[0]->type_handler()->Item_hybrid_func_fix_attributes()
from Item_func_nullif::fix_length_and_dec().
This prevents a ROW expression to be passed to args[0] of NULLIF().
But will allow to pass associative arrays.
args[1] is still only checked to be comparable with args[0].
No needs to add additional tests for it.
- item_cmpfunc.h:
Adding a call for Item_hybrid_func_fix_attributes() in
Item_func_case_abbreviation2::cache_type_info().
This prevents calling the descendant functions with
a ROW expression in combination with an explicit NULL
in the THEN-style arguments (but will allow to pass associative arrays):
IFNULL(row_expression, NULL)
IFNULL(NULL, row_expression)
IF(switch, row_expression, NULL)
IF(switch, NULL, row_expression)
NVL2(switch, row_expression, NULL)
NVL2(switch, NULL, row_expression)
Adding a THD* argument into involved methods.
- item_cmpfunc.h:
Overriding Item_func_case_abbreviation2_switch::check_arguments() to
check that the first argument in IF() and NVL2() can return bool.
Removing Item_func_if::check_arguments(), as it become redundant.
- sql_type.cc:
Fixing sql_type.cc not to disallow items[0] with ROW_RESULT.
This makes it call Item_hybrid_func_fix_attributes() at the end,
which block ROW arguments into THEN-style arguments of hybrid functions.
But this will allow to pass Type_handler_assoc_array expressions.
- sql_type.cc:
Changing Type_handler_row::Item_hybrid_func_fix_attributes to raise the
ER_ILLEGAL_PARAMETER_DATA_TYPE_FOR_OPERATION error instead of the DBUG_ASSERT.
when a definer for SP/view is wrong - it shold be ER_MALFORMED_DEFINER,
not ER_NO_SUCH_USER
when one uses current_role as a definer or grantee but there's no
current role - it should be ER_INVALID_ROLE not ER_MALFORMED_DEFINER
when a non-existent user is specified - it should be ER_NO_SUCH_USER,
which should say "The user does not exist", not "Definer does not exist"
clarify ER_CANT_CHANGE_TX_CHARACTERISTICS to say what cannot be changed
This patch adds support for SYS_REFCURSOR (a weakly typed cursor)
for both sql_mode=ORACLE and sql_mode=DEFAULT.
Works as a regular stored routine variable, parameter and return value:
- can be passed as an IN parameter to stored functions and procedures
- can be passed as an INOUT and OUT parameter to stored procedures
- can be returned from a stored function
Note, strongly typed REF CURSOR will be added separately.
Note, to maintain dependencies easier, some parts of sql_class.h
and item.h were moved to new header files:
- select_results.h:
class select_result_sink
class select_result
class select_result_interceptor
- sp_cursor.h:
class sp_cursor_statistics
class sp_cursor
- sp_rcontext_handler.h
class Sp_rcontext_handler and its descendants
The implementation consists of the following parts:
- A new class sp_cursor_array deriving from Dynamic_array
- A new class Statement_rcontext which contains data shared
between sub-statements of a compound statement.
It has a member m_statement_cursors of the sp_cursor_array data type,
as well as open cursor counter. THD inherits from Statement_rcontext.
- A new data type handler Type_handler_sys_refcursor in plugins/type_cursor/
It is designed to store uint16 references -
positions of the cursor in THD::m_statement_cursors.
- Type_handler_sys_refcursor suppresses some derived numeric features.
When a SYS_REFCURSOR variable is used as an integer an error is raised.
- A new abstract class sp_instr_fetch_cursor. It's needed to share
the common code between "OPEN cur" (for static cursors) and
"OPER cur FOR stmt" (for SYS_REFCURSORs).
- New sp_instr classes:
* sp_instr_copen_by_ref - OPEN sys_ref_curor FOR stmt;
* sp_instr_cfetch_by_ref - FETCH sys_ref_cursor INTO targets;
* sp_instr_cclose_by_ref - CLOSE sys_ref_cursor;
* sp_instr_destruct_variable - to destruct SYS_REFCURSOR variables when
the execution goes out of the BEGIN..END block
where SYS_REFCURSOR variables are declared.
- New methods in LEX:
* sp_open_cursor_for_stmt - handles "OPEN sys_ref_cursor FOR stmt".
* sp_add_instr_fetch_cursor - "FETCH cur INTO targets" for both
static cursors and SYS_REFCURSORs.
* sp_close - handles "CLOSE cur" both for static cursors and SYS_REFCURSORs.
- Changes in cursor functions to handle both static cursors and SYS_REFCURSORs:
* Item_func_cursor_isopen
* Item_func_cursor_found
* Item_func_cursor_notfound
* Item_func_cursor_rowcount
- A new system variable @@max_open_cursors - to limit the number
of cursors (static and SYS_REFCURSORs) opened at the same time.
Its allowed range is [0-65536], with 50 by default.
- A new virtual method Type_handler::can_return_bool() telling
if calling item->val_bool() is allowed for Items of this data type,
or if otherwise the "Illegal parameter for operation" error should be raised
at fix_fields() time.
- New methods in Sp_rcontext_handler:
* get_cursor()
* get_cursor_by_ref()
- A new class Sp_rcontext_handler_statement to handle top level statement
wide cursors which are shared by all substatements.
- A new virtual method expr_event_handler() in classes Item and Field.
It's needed to close (and make available for a new OPEN)
unused THD::m_statement_cursors elements which do not have any references
any more. It can happen in various moments in time, e.g.
* after evaluation parameters of an SQL routine
* after assigning a cursor expression into a SYS_REFCURSOR variable
* when leaving a BEGIN..END block with SYS_REFCURSOR variables
* after setting OUT/INOUT routine actual parameters from formal
parameters.
is_package_public_routine() ignores errors that might've
happened during its execution and does not return an error status
to the caller. Thus there must be no errors during its execution.
Adding support for the "FM" format in function TO_CHAR(date_time, fmt).
"FM" in the format string disables padding of all components following it.
So now TO_CHAR() works as follows:
- By default string format components DAY (weekday name) and
MONTH (month name) are right-padded with spaces to the maximum
possible DAY and MONTH name lengths respectively,
according to the current locale specified in @@lc_time_names.
So for example, with lc_time_names='en_US' all month names are
right-padded with spaces up to 9 characters ('September' is the longest).
SET lc_time_names='en_US';
SELECT TO_CHAR('0001-02-03', 'MONTH'); -> 'February ' (padded to 9 chars)
NEW: When typed after FM, DAY and MONTH names are not right-padded
with trailing spaces any more:
SET lc_time_names='en_US';
SELECT TO_CHAR('0001-02-03', 'FMMONTH'); -> 'February' (not padded)
- By default numeric components YYYY, YYY, YY, Y, DD, H12, H24, MI, SS
are left-padded with leading digits '0' up to the maximum possible
number of digits in the component (e.g. 4 for YYYY):
SELECT TO_CHAR('0001-02-03', 'YYYY'); -> '0001' (padded to 4 chars)
NEW: When typed after FM, these numeric components are not left-padded
with leading zeros any more:
SELECT TO_CHAR('0001-02-03', 'FMYYYY'); -> '1' (not padded)
- If FM is specified multiple times in a format string,
every FM negates the previous padding state:
* an odd FM disables padding
* an even FM enables padding
Implementation details:
- Adding a helper class Date_time_format_oracle.
- Adding a helper method Date_time_format_oracle::append_lex_cstring()
- Moving the function append_val() to Date_time_format_oracle as a method.
- Moving the function make_date_time_oracle() to Date_time_format_oracle
as a method format().
- Adding helper methods month_name() and day_name() in class MY_LOCALE,
to return the corresponding components as LEX_CSTRINGs.
* Migrate `sql/share/errmsg-utf8.txt` to use suffix-based, `-Wformat`
-compatible `my_snprintf` format extensions introduced in MDEV-21978
* Update relevant tests caught by BuildBot as well
While GCC `-Wformat` (with `ATTRIBUTE_FORMAT`) can catch obsolete or
malformed format string literals, formats originating from other sources
(such as this translations file) (still) require manual review.
This commit also escapes the only (1) instance of existing strings
conflicted by the introduction of suffixes:
(Not all `printf`s goes to `my_snprintf`, thus I `grep`ped and
confirmed that this does indeed land on `my_snprintf` eventually.)
chi "不能%sSLAVE'%.*s'"
This commit also fixes the following: (You’re welcome.)
* Delete extraneous spaces after the `%` (they’re all Swahili)
* Update `extra/comp_err.c`
* Add the missing standard C/C++ specifiers `c`, `i`, `o`, `p` and `X`
(Especially `%i`: it otherwise was complaining about the new `%iE`)
* Removed the old and obsolete extension formats `%b`, `%M` and `%T`
It was not possible to use a package body variable as a
fetch target:
CREATE PACKAGE BODY pkg AS
vc INT := 0;
FUNCTION f1 RETURN INT AS
CURSOR cur IS SELECT 1 AS c FROM DUAL;
BEGIN
OPEN cur;
FETCH cur INTO vc; -- this returned "Undeclared variable: vc" error.
CLOSE cur;
RETURN vc;
END;
END;
FETCH assumed that all fetch targets reside of the same sp_rcontext
instance with the cursor. This patch fixes the problem.
Now a cursor and its fetch target can reside in different sp_rcontext
instances.
Details:
- Adding a helper class sp_rcontext_addr
(a combination of Sp_rcontext_handler pointer and an offset in the rcontext)
- Adding a new class sp_fetch_target deriving from sp_rcontext_addr.
Fetch targets in "FETCH cur INTO target1, target2 ..." are now collected
into this structure instead of sp_variable.
sp_variable cannot be used any more to store fetch targets,
because it does not have a pointer to Sp_rcontext_handler
(it only has the current rcontext offset).
- Removing members sp_instr_set members m_rcontext_handler and m_offset.
Deriving sp_instr_set from sp_rcontext_addr instead.
- Renaming sp_instr_cfetch member "List<sp_variable> m_varlist"
to "List<sp_fetch_target> m_fetch_target_list".
- Fixing LEX::sp_add_cfetch() to return the pointer to the
created sp_fetch_target instance (instead of returning bool).
This helps to make the grammar in sql_yacc.c simpler
- Renaming LEX::sp_add_cfetch() to LEX::sp_add_instr_cfetch(),
as `if(sp_add_cfetch())` changed its meaning to the opposite,
to avoid automatic wrong merge from earlier versions.
- Chaning the "List<sp_variable> *vars" parameter to sp_cursor::fetch
to have the data type "List<sp_fetch_target> *".
- Changing the data type of "List<sp_variable> &vars" in
sp_cursor::Select_fetch_into_spvars::send_data_to_variable_list()
to "List<sp_fetch_target> &".
- Adding THD helper methods get_rcontext() and get_variable().
- Moving the code from sql_yacc.yy into a new LEX method
LEX::make_fetch_target().
- Simplifying the grammar in sql_yacc.yy using the new LEX method.
Changing the data type of the bison rule sp_fetch_list from "void"
to "List<sp_fetch_target> *".
* rpl.rpl_system_versioning_partitions updated for MDEV-32188
* innodb.row_size_error_log_warnings_3 changed error for MDEV-33658
(checks are done in a different order)
Implement default values for parameters of stored routines
in both default and oracle mode.
- Default values for cursor parameters are *NOT* supported yet.
- An IN parameter with DEFAULT followed by an OUT param is not supported yet.
This combination will be enabled together with the arrow syntax:
sp1(v=>p2)
The default values can be either literals or expressions.
When it is an expression, it is only evaluated if the parameter
has not been supplied by the caller
(important if the expression has side effects).
table->move_fields has some limitations:
1. It cannot be used in cascade
2. It should always have a restoring pair
In this case, an error has occurred before the field ptr was restored, returning
from the function in that state. Even in case of an error, the table can be
reused afterwards and table->field[i]->ptr is not reset in between.
The solution is to restore the field pointers immanently whenever they've been
deviated.
Also add an assertion that ensures that table fields are restored after the use
in close_thread_tables.
Item:print_for_table_def() uses QT_TO_SYSTEM_CHARSET to print
the DEFAULT expression into FRM file during CREATE TABLE.
Therefore, the expression is encoded in utf8 in FRM.
get_field_default_value() erroneously used field->charset() to
print the DEFAULT expression at SHOW CREATE TABLE time.
Fixing get_field_default_value() to use &my_charset_utf8mb4_general_ci instead.
This makes DEFAULT work in the way way with:
- virtual column expressions:
if (field->vcol_info)
{
StringBuffer<MAX_FIELD_WIDTH> str(&my_charset_utf8mb4_general_ci);
field->vcol_info->print(&str);
- check constraint expressions:
if (field->check_constraint)
{
StringBuffer<MAX_FIELD_WIDTH> str(&my_charset_utf8mb4_general_ci);
field->check_constraint->print(&str);
Additional cleanup:
Fixing system_charset_info to &my_charset_utf8mb4_general_ci in a few
places to make non-BMP characters work in DEFAULT, virtual column,
check constraint expressions.
This patch was suggested by Sergei Golubchik.
It reverts the second patch from the PR:
commit fa5eeb4931
Fixed ALTER TABLE NOCOPY keyword failure
and adds NOCOPY_SYM into keyword_func_sp_var_and_label.
The price is one extra shift/recuce conflict in yy_oracle.yy.
This should to tolerable.
Adding support for the ROW data type in the stored function RETURNS clause:
- explicit ROW(..members...) for both sql_mode=DEFAULT and sql_mode=ORACLE
CREATE FUNCTION f1() RETURNS ROW(a INT, b VARCHAR(32)) ...
- anchored "ROW TYPE OF [db1.]table1" declarations for sql_mode=DEFAULT
CREATE FUNCTION f1() RETURNS ROW TYPE OF test.t1 ...
- anchored "[db1.]table1%ROWTYPE" declarations for sql_mode=ORACLE
CREATE FUNCTION f1() RETURN test.t1%ROWTYPE ...
Adding support for anchored scalar data types in RETURNS clause:
- "TYPE OF [db1.]table1.column1" for sql_mode=DEFAULT
CREATE FUNCTION f1() RETURNS TYPE OF test.t1.column1;
- "[db1.]table1.column1" for sql_mode=ORACLE
CREATE FUNCTION f1() RETURN test.t1.column1%TYPE;
Details:
- Adding a new sql_mode_t parameter to
sp_head::create()
sp_head::sp_head()
sp_package::create()
sp_package::sp_package()
to guarantee early initialization of sp_head::m_sql_mode.
Before this change, this member was not initialized at all during
CREATE FUNCTION/PROCEDURE/PACKAGE statements, and was not used.
Now it needs to be initialized to write properly the
mysql.proc.returns column, according to the create time sql_mode.
- Code refactoring to make the things simpler and functions smaller:
* Adding a new method
Field_row::row_create_fields(THD *thd, List<Spvar_definition> *list)
to make a Virtual_tmp_table with Fields for ROW members
from an explicit definition.
* Adding a new method
Field_row::row_create_fields(THD *thd, const Spvar_definition &def)
to make a Virtual_tmp_table with Fields for ROW members
from an explicit or a table anchored definition.
* Adding a new method
Item_args::add_array_of_item_field(THD *thd, const Virtual_tmp_table &vtable)
to create and array of Item_field corresponding to all Field instances
in a Virtual_tmp_table
* Removing Item_field_row::row_create_items(). It was decomposed
into the new methods described above.
* Moving the code from the loop body in sp_rcontext::init_var_items()
into a separate method Spvar_definition::make_item_field_row(),
to make the code clearer (smaller functions).
make_item_field_row() itself uses the new methods described above.
- Changing the data type of sp_head::m_return_field_def
from Column_definition to Spvar_definition.
So now it supports not only SQL column field types,
but also explicit ROW and anchored ROW data types,
as well as anchored column types.
- Adding a new Column_definition parameter to sp_head::create_result_field().
Before this patch, create_result_field() took the definition only
from m_return_field_def. Now it's also called with a local Column_definition
variable which contains the explicit definition resolved from an
anchored defition.
- Modifying sql_yacc.yy to support the new grammar.
Adding new helper methods:
* sf_return_fill_definition_row()
* sf_return_fill_definition_rowtype_of()
* sf_return_fill_definition_type_of()
- Fixing tests in:
* Virtual_tmp_table::setup_field_pointers() in sql_select.cc
* Send_field::normalize() in field.h
* store_column_type()
to prevent calling Type_handler_row::field_type(),
which is implemented a DBUG_ASSERT(0).
Before this patch the affected methods and functions were called only
for scalar data types. Now ROW is also possible.
- Adding a new virtual method Field::cols()
- Overriding methods:
Item_func_sp::cols()
Item_func_sp::element_index()
Item_func_sp::check_cols()
Item_func_sp::bring_value()
to support the ROW data type.
- Extending the rule sp_return_type to support
* explicit ROW and anchored ROW data types
* anchored scalar data types
- Overriding Field_row::sql_type() to print
the data type of an explicit ROW.
PURGE BINARY LOGS did not always purge binary logs. This commit fixes
some of the issues and adds notifications if a binary log cannot be
purged.
User visible changes:
- 'PURGE BINARY LOG TO log_name' and 'PURGE BINARY LOGS BEFORE date'
worked differently. 'TO' ignored 'slave_connections_needed_for_purge'
while 'BEFORE' did not. Now both versions ignores the
'slave_connections_needed_for_purge variable'.
- 'PURGE BINARY LOG..' commands now returns 'note' if a binary log cannot
be deleted like
Note 1375 Binary log 'master-bin.000004' is not purged because it is
the current active binlog
- Automatic binary log purges, based on date or size, will write a
note to the error log if a binary log matching the size or date
cannot yet be deleted.
- If 'slave_connections_needed_for_purge' is set from a config or
command line, it is set to 0 if Galera is enabled and 1 otherwise
(old default). This ensures that automatic binary log purge works
with Galera as before the addition of
'slave_connections_needed_for_purge'.
If the variable is changed to 0, a warning will be printed to the error
log.
Code changes:
- Added THD argument to several purge_logs related functions that needed
THD.
- Added 'interactive' options to purge_logs functions. This allowed
me to remove testing of sql_command == SQLCOM_PURGE.
- Changed purge_logs_before_date() to first check if log is applicable
before calling can_purge_logs(). This ensures we do not get a
notification for logs that does not match the remove criteria.
- MYSQL_BIN_LOG::can_purge_log() will write notifications to the user
or error log if a log cannot yet be removed.
- log_in_use() will return reason why a binary log cannot be removed.
Changes to keep code consistent:
- Moved checking of binlog_format for Galera to be after Galera is
initialized (The old check never worked). If Galera is enabled
we now change the binlog_format to ROW, with a warning, instead of
aborting the server. If this change happens a warning will be printed to
the error log.
- Print a warning if Galera or FLASHBACK changes the binlog_format
to ROW. Before it was done silently.
Reviewed by: Sergei Golubchik <serg@mariadb.com>,
Kristian Nielsen <knielsen@knielsen-hq.org>
