This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 12.3.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
A field of an element of an associative array of RECORDs, passed to
an OUT or INOUT parameter of a stored routine, crashed the server:
CREATE PROCEDURE p1(a OUT INT) AS BEGIN a:= 7; END;
DECLARE
TYPE rec_t IS RECORD (i INT, s VARCHAR(10));
TYPE tab_t IS TABLE OF rec_t INDEX BY INT;
t tab_t:= tab_t(1 => rec_t(0, 'x'));
BEGIN
p1(t(1).i);
END;
There were three problems:
- Item_splocal_assoc_array_element_field inherited set_value() of
Item_splocal_row_field_by_name, which treated the associative array
variable as a ROW variable (assertion failure in
sp_rcontext::virtual_tmp_table_for_row()). Added
Item_splocal_assoc_array_element_field::set_value(), which stores the
value into the field of the element.
- Before the call, sp_head::bind_input_param() destructed the whole
variable for an OUT argument, even if the argument was only a part of
the variable (a ROW field, an assoc array element or its field), so
the other elements of the array were lost: after p1(a0(2)), the array
a0(1), a0(2), a0(3) had only one element left. Added
Item_splocal::is_part_of_variable().
- sp_head::bind_output_param() evaluated the actual argument in the
context of the callee. For arguments like t(i) or t(i).field this
resolved the variables t and i in a wrong context. Now the argument
is evaluated in the context of the caller.
The second problem also affected a SYS_REFCURSOR field of a ROW variable
passed as an OUT argument: the cursors in the other fields of the ROW
variable were closed. Now they are kept.
Known limitations, not changed by this fix:
- The actual OUT argument is evaluated when the value is copied back,
not when the routine is called. If an earlier OUT argument of the same
call changes the variable used as the key, as in CALL p(i, a0(i)), the
new value of the key is used.
- For an OUT argument which is a part of a variable (a ROW field, an
assoc array element or its field), only the destruct of the whole
variable is skipped, the part itself is not destructed. A
SYS_REFCURSOR field of a ROW stays open during the call and is
released when the new value is copied back.
Tests: sp-assoc-array (OUT and INOUT elements and fields of elements,
NULL key, element deleted by the callee, ROW and RECORD fields) and
type_sys_refcursor-type_row-debug (SYS_REFCURSOR fields of a ROW
variable).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
With lower_case_table_names 0 the server can have databases Db1 and db1,
each with a function f1. A single statement naming both databases, like
SELECT Db1.f1(), db1.f1(), reported that db1.f1 does not exist.
The set of routines a statement uses compared its entries without regard
to case. Only one routine was loaded but the reference to the other
found nothing. The set now compares its entries exactly, as the routine
cache and the lock manager already do.
Each nested BEGIN adds one sp_pcontext. Two walks over the finished
tree recursed once per nesting level and overran the thread stack.
~sp_pcontext() freed its children recursively. sp_head now threads a
single linked list through the contexts and frees them iteratively, so
teardown depth is constant and a context no longer frees its children.
retrieve_field_definitions() descends the children to build the
run-time frame. It emits them in run-time offset order, so it stays
recursive, but it now checks the stack and returns an error instead
of crashing.
The "is it my own thread" check compared the login user name
(Security_context::user) and ignored the host, so u1@localhost could see
and kill threads of u1@'127.0.0.1'.
Compare the authenticated account instead:
* all comparisons are done in sctx->is_priv_user() now
* change user_matches() to priv_user_matches(), which uses is_priv_user()
* use it in KILL, KILL USER, SHOW PROCESSLIST, I_S.PROCESSLIST,
COM_PROCESS_INFO and SHOW EXPLAIN/ANALYZE FOR.
* all the remaining places use is_priv_user() directly instead
of doing strcmp: SHOW GRANTS, SHOW CREATE PROCEDURE, I_S.VIEWS,
the DEFINER clause, optimizer trace and change_security_context().
Assisted-by: Claude:claude-5-opus
On parsing the statement 'DECLARE var1,...,varN TYPE OF cur DEFAULT value'
an instance of the class Item_splocal that created on handling the clause
'DEFAULT value' is shared between two classes sp_instr_cursor_copy_struct and
sp_instr_set. In result, in case metadata changed and cursor has to be re-parsed,
items associated with the SP instruction sp_instr_cursor_copy_struct
are deallocated and one of deallocated items is the one representing the value
the `DEFAULT value` clause. On the other hand, the data member
sp_instr_set::m_value still references the deallocated item. It leads to server
abnormal termination on second run of a stored routine containing the statement
'DECLARE ... TYPE OF ... DEFAULT value'
To fix the issue pass owning of an instance of the class Item_splocal to
the class sp_instr_cursor_copy_struct and deallocated it explicitly on
closing the stored routine.
On re-parsing of a failed cursor statement inside the stored routine,
the free_list of sp_lex_cursor could point to items placed on a dedicated
memory root that is created during re-parsing of the failed statement.
In result, when sp_head object is destroyed the mem_root created for
re-parsing the cursor's statement is de-allocated but the free_list
pointer of sp_lex_cursor still point to objects previously allocated
on this memory root.
To fix the issue, delay deallocation of mem_roots created for re-parsing
of SP instructions till the moment the sp_head be destroyed.
Adding support for cursors on prepared statements.
- SQL Standard way:
DECLARE c CURSOR FOR stmt;
PREPARE stmt FROM 'SELECT ?';
OPEN c USING 1;
- Oracle-style way with SYS_REFCURSOR variables:
DECLARE
c SYS_REFCURSOR;
BEGIN
OPEN c FOR 'SELECT ?' USING 1;
* rename to follow the standard vprintf name convention
* remove `(LEX_CSTRING*) &lex_string` casts
* remove some redundant casts too
* LEX_CSTRING -> Lex_ident_db in path
* Hash_set<Sroutine_hash_entry>
* move assert out of the loop
* some comments and whitespace changes
Definition
==========
Implement the SET PATH statement.
```
SET PATH 'schema1, schema2, schema3';
```
Function
--------
In MariaDB, the SQL-path will also be used to lookup packages in
package routine invocations, so the following script:
SET PATH 'sys';
SELECT UTL_ENCODE.BASE64_DECODE('data');
invokes SYS.UTL_ENCODE.BASE64_DECODE('data'), i.e. the function
BASE64_DECODE() in the packge UTL_ENCODE in the database SYS
(if such package and package routine exist).
MariaDB implementation details
------------------------------
The ticket's requirement stated the grammar should support delimited
identifiers with backticks or double quotes within the string quote,
ie: the single tick. A simple parser was implemented to handle this
requirement in Sql_path::from_text.
Only 16 schemas are supported per session - we do not want too many
schemas in the path anyway since it will impact performance. This also
means that the path cache is a simple static array for easier memory
management.
TABLE_LIST parsed from procedure code is transferred into tables to
lock for INSERT. The procedure code is CREATE VIEW so its TABLE_LIST
is parsed as TL_IGNORE, but same view exists and when existing view is
opened mysql_make_view() uses same TABLE_LIST that was initialized
from CREATE VIEW and then added as part of prelocking context. So
existing view is opened and its table is assigned TL_IGNORE from
prelocking context. Finally, INSERT has TABLE_LIST duplication: the
one that was parsed from INSERT; the another one came from procedure
prelocking, its lock_type came from the procedure code and the real
table was found via existing view.
The sequence of execution:
1. Procedure p is compiled as part of open_and_process_routine(), its
code is parsed and create_or_alter_view_finalize() initializes v
TABLE_LIST as TL_IGNORE;
2. Procedure p prelocking adds v to prelocking_ctx with TL_IGNORE;
3. DML prelocking adds v from prelocking_ctx;
4. View is opened, mysql_make_view() assigns t lock_type from v;
5. open_and_lock_tables() attempts to lock t with TL_IGNORE.
The fix skips TL_IGNORE at 2. when table list parsed by procedure is
added for prelocking:
if (my_hash_insert(&m_sptabs, (uchar *)tab))
return FALSE;
m_sptabs designation was defined as strictly for prelocking:
/**
Multi-set representing optimized list of tables to be locked by this
routine. Does not include tables which are used by invoked routines.
@note
For prelocking-free SPs this multiset is constructed too.
We do so because the same instance of sp_head may be called both
in prelocked mode and in non-prelocked mode.
*/
HASH m_sptabs;
The fix was proposed by Sergei Golubchik <serg@mariadb.org>.
This patch adds support for associative arrays in stored procedures
for sql_mode=ORACLE.
The syntax follows Oracle's PL/SQL syntax for associative arrays -
TYPE assoc_array_t IS TABLE OF VARCHAR2(100) INDEX BY INTEGER;
or
TYPE assoc_array_t IS TABLE OF record_t INDEX BY VARCHAR2(100);
where record_t is a record type.
The following functions were added for associative arrays:
- COUNT - Retrieve the number of elements within the arra
- EXISTS - Check whether given key exists in the array
- FIRST - Retrieve the first key in the array
- LAST - Retrieve the last key in the array
- PRIOR - Retrieve the key before the given key
- NEXT - Retrieve the key after the given key
- DELETE - Remove the element with the given key or remove all elements
if no key is given
The arrays/elements can be initialized with the following methods:
- Constructor
i.e. array:= assoc_array_t('key1'=>1, 'key2'=>2, 'key3'=>3)
- Assignment
i.e. array(key):= record_t(1, 2)
- SELECT INTO
i.e. SELECT x INTO array(key)
TODOs:
- Nested tables are not supported yet.
i.e. TYPE assoc_array_t IS TABLE OF other_assoc_array_t INDEX BY INTEGER;
- Associative arrays comparisons are not supported yet.
main/statistics_json.result is updated for f8ba5ced55 (MDEV-36099)
The test uses 'delete from t1' in many places and then populates
the table again. The natural order of rows in a MyISAM table is well
defined and the test was implicitly relying on that.
before f8ba5ced55 delete was deleting rows one by one, using
ha_myisam::delete_row() because the connection was stuck in rbr mode.
This caused rows to be shown in the reverse insertion order (because of
the delete link list).
MDEV-36099 fixes this bug and the server now correctly uses
ha_myisam::delete_all_rows(). This makes rows to be shown in the
insertion order as expected.
This patch adds support for SYS_REFCURSOR (a weakly typed cursor)
for both sql_mode=ORACLE and sql_mode=DEFAULT.
Works as a regular stored routine variable, parameter and return value:
- can be passed as an IN parameter to stored functions and procedures
- can be passed as an INOUT and OUT parameter to stored procedures
- can be returned from a stored function
Note, strongly typed REF CURSOR will be added separately.
Note, to maintain dependencies easier, some parts of sql_class.h
and item.h were moved to new header files:
- select_results.h:
class select_result_sink
class select_result
class select_result_interceptor
- sp_cursor.h:
class sp_cursor_statistics
class sp_cursor
- sp_rcontext_handler.h
class Sp_rcontext_handler and its descendants
The implementation consists of the following parts:
- A new class sp_cursor_array deriving from Dynamic_array
- A new class Statement_rcontext which contains data shared
between sub-statements of a compound statement.
It has a member m_statement_cursors of the sp_cursor_array data type,
as well as open cursor counter. THD inherits from Statement_rcontext.
- A new data type handler Type_handler_sys_refcursor in plugins/type_cursor/
It is designed to store uint16 references -
positions of the cursor in THD::m_statement_cursors.
- Type_handler_sys_refcursor suppresses some derived numeric features.
When a SYS_REFCURSOR variable is used as an integer an error is raised.
- A new abstract class sp_instr_fetch_cursor. It's needed to share
the common code between "OPEN cur" (for static cursors) and
"OPER cur FOR stmt" (for SYS_REFCURSORs).
- New sp_instr classes:
* sp_instr_copen_by_ref - OPEN sys_ref_curor FOR stmt;
* sp_instr_cfetch_by_ref - FETCH sys_ref_cursor INTO targets;
* sp_instr_cclose_by_ref - CLOSE sys_ref_cursor;
* sp_instr_destruct_variable - to destruct SYS_REFCURSOR variables when
the execution goes out of the BEGIN..END block
where SYS_REFCURSOR variables are declared.
- New methods in LEX:
* sp_open_cursor_for_stmt - handles "OPEN sys_ref_cursor FOR stmt".
* sp_add_instr_fetch_cursor - "FETCH cur INTO targets" for both
static cursors and SYS_REFCURSORs.
* sp_close - handles "CLOSE cur" both for static cursors and SYS_REFCURSORs.
- Changes in cursor functions to handle both static cursors and SYS_REFCURSORs:
* Item_func_cursor_isopen
* Item_func_cursor_found
* Item_func_cursor_notfound
* Item_func_cursor_rowcount
- A new system variable @@max_open_cursors - to limit the number
of cursors (static and SYS_REFCURSORs) opened at the same time.
Its allowed range is [0-65536], with 50 by default.
- A new virtual method Type_handler::can_return_bool() telling
if calling item->val_bool() is allowed for Items of this data type,
or if otherwise the "Illegal parameter for operation" error should be raised
at fix_fields() time.
- New methods in Sp_rcontext_handler:
* get_cursor()
* get_cursor_by_ref()
- A new class Sp_rcontext_handler_statement to handle top level statement
wide cursors which are shared by all substatements.
- A new virtual method expr_event_handler() in classes Item and Field.
It's needed to close (and make available for a new OPEN)
unused THD::m_statement_cursors elements which do not have any references
any more. It can happen in various moments in time, e.g.
* after evaluation parameters of an SQL routine
* after assigning a cursor expression into a SYS_REFCURSOR variable
* when leaving a BEGIN..END block with SYS_REFCURSOR variables
* after setting OUT/INOUT routine actual parameters from formal
parameters.
sp_head::execute_procedure() and sp_head::execute_function() did not
check that Item_param could be passed as an actual parameter to a ROW type
formal parameter of a stored routine. Example:
CREATE PROCEDURE p0(OUT a ROW(a INT,b INT)) ...;
PREPARE s0 'CALL p0(?)';
EXECUTE p0 USING @a;
In case of passing a user variable as an OUT parameter it led to
a crash after executing routine instructions, when copying formal
OUT parameters to the bound actual parameters.
Fix:
Check cases when Item_param is being bound to a ROW type formal parameter.
Raise an error if so. The new check is done for all parameter modes:
IN, OUT, INOUT, for a consistent error message.
The new check is done before executing the routine instructions.
It was not possible to use a package body variable as a
fetch target:
CREATE PACKAGE BODY pkg AS
vc INT := 0;
FUNCTION f1 RETURN INT AS
CURSOR cur IS SELECT 1 AS c FROM DUAL;
BEGIN
OPEN cur;
FETCH cur INTO vc; -- this returned "Undeclared variable: vc" error.
CLOSE cur;
RETURN vc;
END;
END;
FETCH assumed that all fetch targets reside of the same sp_rcontext
instance with the cursor. This patch fixes the problem.
Now a cursor and its fetch target can reside in different sp_rcontext
instances.
Details:
- Adding a helper class sp_rcontext_addr
(a combination of Sp_rcontext_handler pointer and an offset in the rcontext)
- Adding a new class sp_fetch_target deriving from sp_rcontext_addr.
Fetch targets in "FETCH cur INTO target1, target2 ..." are now collected
into this structure instead of sp_variable.
sp_variable cannot be used any more to store fetch targets,
because it does not have a pointer to Sp_rcontext_handler
(it only has the current rcontext offset).
- Removing members sp_instr_set members m_rcontext_handler and m_offset.
Deriving sp_instr_set from sp_rcontext_addr instead.
- Renaming sp_instr_cfetch member "List<sp_variable> m_varlist"
to "List<sp_fetch_target> m_fetch_target_list".
- Fixing LEX::sp_add_cfetch() to return the pointer to the
created sp_fetch_target instance (instead of returning bool).
This helps to make the grammar in sql_yacc.c simpler
- Renaming LEX::sp_add_cfetch() to LEX::sp_add_instr_cfetch(),
as `if(sp_add_cfetch())` changed its meaning to the opposite,
to avoid automatic wrong merge from earlier versions.
- Chaning the "List<sp_variable> *vars" parameter to sp_cursor::fetch
to have the data type "List<sp_fetch_target> *".
- Changing the data type of "List<sp_variable> &vars" in
sp_cursor::Select_fetch_into_spvars::send_data_to_variable_list()
to "List<sp_fetch_target> &".
- Adding THD helper methods get_rcontext() and get_variable().
- Moving the code from sql_yacc.yy into a new LEX method
LEX::make_fetch_target().
- Simplifying the grammar in sql_yacc.yy using the new LEX method.
Changing the data type of the bison rule sp_fetch_list from "void"
to "List<sp_fetch_target> *".
