Allowing prepared statements in stored functions when
a stored function is used in an assignment right hand.
Both DEFAULT clause of a variable initialization and
the right side of the SET statement are supported:
CREATE PROCEDURE p1()
BEGIN
-- case 1: DEFAULT clause
DECLARE spvar1 INT DEFAULT f1_with_ps(); -- OK
-- case 2: SP variable assignment statement
DECLARE spvar2 INT;
SET spvar2= f1_with_ps(); -- OK
END;
- Only assignments to SP variables works for now:
* SET spvar= func_with_ps(); -- OK
* SET @uvar= func_with_ps(); -- Error
- Only bare function calls are supported for now. Using a function in
an expression does not make it PS-safe yet:
SET v= f1()+0;
- The parser now does not reject PS statements in stored functions.
PS applicability in stored functions is now detected at run time.
Note, PS statements in triggers are still prohibited by the parser.
- Functions with PS do not acquire MDL locks on tables, and no MDL is
taken on the routines themselves either. They work like procedures in
terms of table opening and routine locking: a concurrent DROP FUNCTION
can complete while such a function is executing.
- Functions with PS are not replicated as a single `SELECT f1()` call.
They are replicated per-statement, like procedures.
Helper changes:
- Changing the return result for LEX::sp_variable_declarations_init()
from void to bool to catch errors in the caller properly.
Misc:
- This patch incorporates fixes for the following bugs found during debugging:
MDEV-40224,MDEV-40225,MDEV-40226,MDEV-40227,MDEV-40240,
MDEV-40285,MDEV-40288,MDEV-40315,MDEV-40318,MDEV-40890,MDEV-40900,
MDEV-40901,MDEV-40913,MDEV-40914,MDEV-41013,MDEV-41015,MDEV-41019
Assisted-by: Claude - reviews and minor clean-ups
A segfault in mysql_lock_merge waa the result of a null argument
when called from open_and_lock_internal_tables. As the argument
was null there was no lock to merge and thd->lock was already in place,
so no futher action is required in open_and_lock_internal_tables.
After passing the previous correctly on open_table of the
sequence used in the table, we get to the checking the thd->open_table
list where tdc is flushed. This puts the open_table into an error
condition to reopen all tables. When we are just opening a sequence,
this condition isn't one we want to fail on.
On the LOAD DATA path the base columns are filled directly from the
input file and fill_record_n_invoke_before_triggers() is then called
with an empty field list (there is no SET clause). After the BEFORE
INSERT trigger changed a base column, the stored generated columns were
not recomputed, so they kept the value derived from the pre-trigger
input (e.g. g=24 instead of 40 for g=v*2 with v set to 20 by the
trigger). A regular INSERT was unaffected.
The recompute was guarded by "fields.elements". That condition is a
leftover from the original computed-columns implementation
(f7a75b999b), where fill_record_n_invoke_before_triggers() had no
TABLE* argument and had to reverse-derive the table from the first
item of the field list:
if (fields.elements)
{
fld= (Item_field*)f++;
item_field= fld->field_for_view_update();
table= item_field->field->table;
...
}
With an empty field list there was no way to obtain the table, so the
recompute was silently skipped. Since bc4a456758 (MDEV-452) the
function receives TABLE* explicitly, which made the whole derivation
dead code (as the in-place DBUG_ASSERT(table == item_field->field->table)
confirmed). Recompute the virtual fields unconditionally on
table->vfield, the same way the Field** overload of
fill_record_n_invoke_before_triggers() already does.
Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS
PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
PARTITION partition_name VALUES LESS THAN (value)
[, PARTITION partition_name VALUES LESS THAN (value) ... ]
)
where
- col_name is the name of one column of type DATE or DATETIME or
TIMESTAMP
- at least one partition is supplied, and the highest partition cannot
have MAXVALUE range
- INTERVAL interval is a positive time interval. it can be mariadb
format or oracle NUMTODSINTERVAL/NUMTOYMINTERVAL format. Like
versioning, the smallest unit is second, i.e. no subsecond like
microsecond.
- DATE column cannot have interval with values less than a day
- Subpartitions are allowed, but restricted to existing subpartition
types, i.e. [LINEAR] (KEY|HASH)
When performing one of the following DML statements on such a table,
it will first add partitions by the specified interval until the
partition covers the current time:
- INSERT
- INSERT ... SELECT
- LOAD
- UPDATE
- REPLACE
- REPLACE ... SELECT
Partition addition will not cause an implicit commit like DDL normally
does.
The partitions are named pN.
Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.
Note that TIMESTAMP is not allowed as a type for PARTITION BY RANGE
COLUMNS otherwise.
Including the following fixes:
1.
MDEV-40088 Disallow subqueries in INTERVAL clause in range interval auto partitioning
This is consistent with system time (versioning) partitioning. Also
consistent is that both allow expressions otherwise.
2.
MDEV-40048 Allow trigger and LOCK TABLES to work with range interval auto partitioning
When a range interval auto partitioned table is the target of a
trigger, the triggering statement is not necessarily one that would
cause the auto-creation of new partitions, so we need to account for
that.
Also added support for LOCK TABLES ... WRITE.
Improved tests coverage by adapting tests from versioning.partition.
3.
MDEV-40467 Improve Y2038/2106 problem error message
When nearing the end times TIMESTAMP_MAX_VALUE, auto-partition
creation may encounter overflow when calculating new partition range.
This patch improves the message of such an overflow
if a user tries to access a table or a database they have no priivleges
on, the error is alwaus "access denied", independently from whether
the object exists or not. Do the same for columns.
Analysis:
When OLD_VALUE() is used on a view, field resolution creates an
Item_direct_view_ref instead of an Item_ref where real_item() points to the
expression. But Item_old_field::fix_fields() continues assuming that a Field
object was available, but view references do not have a field pointer,
which resulted in a crash.
Fix:
Store the real referenced item in Item_old_field::expr and use it when
the OLD_VALUE() reference does not have a Field object. This allows
OLD_VALUE() to work with view fields and avoids dereferencing a NULL
field pointer. Fix relevant methods accordingly.
- Change thd->backup_commit_lock from MDL_request* to MDL_ticket*,
and convert related call sites in handler.cc, log.cc, sql_class.cc,
and xa.cc to use MDL_ACQUIRE_LOCK.
- Convert reload_acl_and_cache() in sql_reload.cc, purge_master_logs()
and reset_master() in sql_repl.cc to MDL_ACQUIRE_LOCK, holding the
ticket in a local MDL_ticket* (these functions used a local
MDL_request originally and should not touch thd->backup_commit_lock).
- Convert acquire_lock() in partition_info.cc to MDL_ACQUIRE_LOCK.
- Add MDL_REQUEST_LIST_ADD() helper for enqueuing lock requests into
MDL_request_list, and convert call sites in sql_base.cc and sp.cc.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Add new acquire_lock() overloads on MDL_context that combine MDL_request
initialization and lock acquisition into a single call, returning
MDL_ticket* directly. Convert applicable call sites across the codebase
to use the new ACQUIRE_LOCK macro.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
The patch introduces the OLD_VALUE() expression to reference the value
of a column before it was updated. The parser is extended to support
RETURNING and OLD_VALUE(), and RETURNING expressions are stored in a
separate returning_list in SELECT_LEX with independent wildcard tracking.
RETURNING is rejected for multi-table UPDATE.
During setup of RETURNING fields, THD::is_setting_returning is used
when resolving fields, particularly for updates through views.
When resolving view fields, Item_direct_view_ref may point to the
view's item_list, losing the information about whether the value
should be old or new. The original item in returning_list still
contains the correct is_old_value_reference flag, which is copied
back to the resolved item.
OLD_VALUE() is implemented by extending Item_field with a new
Item_old_field class. Item_field::set_field() initializes
Field::ptr_old to the corresponding location in record[1],
which stores the old row during UPDATE execution.
When sending result rows, Item_old_field::send() temporarily switches
the field pointer from record[0] (current row) to ptr_old so
OLD_VALUE() returns the value before the update.
The UPDATE execution path is modified to send a result set when
RETURNING is present instead of an OK packet.
This patch implements support for implicit query block (QB) names in
optimizer hints, allowing hints to reference query blocks and tables
within derived tables, views and CTEs without requiring explicit
QB_NAME hints.
Examples.
-- Addressing a table inside a derived table using implicit QB name
select /*+ no_index(t1@dt) */ *
from (select * from t1 where a > 10) as DT;
-- this is an equivalent to:
select /*+ no_index(t1@dt) */ * from
(select /*+ qb_name(dt)*/ * from t1 where a > 10) as DT;
-- Addressing a query block corresponding to the derived table
select /*+ no_bnl(@dt) */ *
from (select * from t1, t2 where t.1.a > t2.a) as DT;
-- View
create view v1 as select * from t1 where a > 10 and b > 100;
-- referencing a table inside a view by implicit QB name:
select /*+ index_merge(t1@v1 idx_a, idx_b) */ *
from v1, t2 where v1.a = t2.a;
-- equivalent to:
create view v1 as select /*+ qb_name(qb_v1) */ *
from t1 where a > 10 and b > 100;
select /*+ index_merge(t1@qb_v1 idx_a, idx_b) */ *
from v1, t2 where v1.a = t2.a;
-- CTE
with aless100 as (select a from t1 where b <100)
select /*+ index(t1@aless100) */ * from aless100;
-- equivalent to:
with aless100 as (select /*+ qb_name(aless100) */ a from t1 where b <100)
select /*+ index(t1@aless100) */ * from aless100;
Limitations:
- Only SELECT statements support implicit QB names. DML operations
(UPDATE, DELETE, INSERT) only support explicit QB names
1. `TABLE_LIST::init_derived` is called twice during derived tables
processing: first time from `mysql_derived_init` and the second
time from `mysql_derived_prepare`. Both times there is a check
for whether an optimizer hint or switch setting allows a derived
table to be merged.
However, it is not necessary to restrict merging during the
initialization call, it is enough to apply hints/switch setting
during the preparation phase.
2. `open_normal_and_derived_tables()` runs processing of all phases
of derived tables handling with a single call. But for future
commits it is required to separate DT initialization from other
phases. This commit implements the separation.
Problem:
========
A single InnoDB purge worker thread can process undo logs from different
tables within the same batch. But get_purge_table(), open_purge_table()
incorrectly assumes that a 1:1 relationship between a purge worker thread
and a table within a single batch. Based on this wrong assumtion,
InnoDB attempts to reuse TABLE objects cached in thd->open_tables for
virtual column computation.
1) Purge worker opens Table A and caches the TABLE pointer in thd->open_tables.
2) Same purge worker moves to Table B in the same batch, get_purge_table()
retrieves the cached pointer for Table A instead of opening Table B.
3) Because innobase::open() is ignored for Table B, the virtual column
template is never initialized.
4) virtual column computation for Table B aborts the server
Solution:
========
- Introduced purge_table class which has the following
purge_table: Stores either TABLE* (for tables with indexed virtual
columns) or MDL_ticket* (for tables without) in a single union
using LSB as a flag.
For tables with indexed virtual columns: opens TABLE*, accesses
MDL_ticket* via TABLE->mdl_ticket
For tables without indexed virtual columns: stores only MDL_ticket*.
trx_purge_attach_undo_recs(): Coordinator opens both dict_table_t*
and TABLE* with proper MDL protection. Workers access cached
table pointers from purge_node_t->tables without opening
their own handles
purge_sys.coordinator_thd: Distinguish coordinator from workers
in cleanup logic. Skip innobase_reset_background_thd() for
coordinator thread to prevent premature table closure during
batch processing. Workers still call cleanup to release their
thread-local resources
trx_purge_close_tables():
Rewrite for purge coordinator thread
1) Close all dict_table_t* objects first
2) Call close_thread_tables() once for all TABLE* objects
3) Release MDL tickets last, after tables are closed
Added table->lock_mutex protection when reading (or) writing
vc_templ->mysql_table and mysql_table_query_id. Clear cached
TABLE* pointers before closing tables to prevent stale pointer
access
Declared open_purge_table() and close_thread_tables() in trx0purge.cc
Declared reset_thd() in row0purge.cc and dict0stats_bg.cc.
Removed innobase_reset_background_thd()
Problem:
Server crashes when a procedure with default parameter is executed and
default value for the parameter is evaluated after the procedure's
first execution. For example:
CREATE PROCEDURE p(x INT DEFAULT (SELECT 1)) ...
CALL p(1); -- first exec, marks mem_root read-only
CALL p(); -- tries to allocate on locked when executing DEFAULT
Fix:
Temporarily clear the read-only flag on the statement memory root to
allow the allocation. Also adds a debug hook to verify allocation
happens not more than once per prepared statement and per internal
table added to the list.
Problem:
========
A single InnoDB purge worker thread can process undo logs from different
tables within the same batch. But get_purge_table(), open_purge_table()
incorrectly assumes that a 1:1 relationship between a purge worker thread
and a table within a single batch. Based on this wrong assumtion,
InnoDB attempts to reuse TABLE objects cached in thd->open_tables for
virtual column computation.
1) Purge worker opens Table A and caches the TABLE pointer in thd->open_tables.
2) Same purge worker moves to Table B in the same batch, get_purge_table()
retrieves the cached pointer for Table A instead of opening Table B.
3) Because innobase::open() is ignored for Table B, the virtual column
template is never initialized.
4) virtual column computation for Table B aborts the server
Solution:
========
- Introduced purge_table class which has the following
purge_table: Stores either TABLE* (for tables with indexed virtual
columns) or MDL_ticket* (for tables without) in a single union
using LSB as a flag.
For tables with indexed virtual columns: opens TABLE*, accesses
MDL_ticket* via TABLE->mdl_ticket
For tables without indexed virtual columns: stores only MDL_ticket*.
trx_purge_attach_undo_recs(): Coordinator opens both dict_table_t*
and TABLE* with proper MDL protection. Workers access cached
table pointers from purge_node_t->tables without opening
their own handles
purge_sys.coordinator_thd: Distinguish coordinator from workers
in cleanup logic. Skip innobase_reset_background_thd() for
coordinator thread to prevent premature table closure during
batch processing. Workers still call cleanup to release their
thread-local resources
trx_purge_close_tables():
Rewrite for purge coordinator thread
1) Close all dict_table_t* objects first
2) Call close_thread_tables() once for all TABLE* objects
3) Release MDL tickets last, after tables are closed
Added table->lock_mutex protection when reading (or) writing
vc_templ->mysql_table and mysql_table_query_id. Clear cached
TABLE* pointers before closing tables to prevent stale pointer
access
Declared open_purge_table() and close_thread_tables() in trx0purge.cc
Declared reset_thd() in row0purge.cc and dict0stats_bg.cc.
Removed innobase_reset_background_thd()
Problem:
When a prepared statement uses `DEFAULT` with sequence, on its
second or subsequent execution, server tries to allocate TABLE_LIST
object (used for adding sequence table to `internal_tables` list) on
statement's memory root. This fails with an assert because statement's
memory root is marked read-only after the first execution.
CREATE SEQUENCE s;
CREATE TABLE t (a INT DEFAULT(NEXTVAL(s)));
PREPARE stmt FROM "UPDATE t SET a = ?";
EXECUTE stmt USING 3; -- first execution, marks mem_root read-only
EXECUTE stmt USING DEFAULT; -- tries to allocate on second execution
Fix:
Temporarily clear the read-only flag on the statement memory root to
allow the allocation. Also adds a debug hook to verify allocation
happens not more than once per prepared statement and per internal
table added to the list.
The data in MEMORY tables is temporary and is lost when the server
restarts. Thus, when these tables are rediscovered empty,
it’s well-intentioned to record TRUNCATE statements to
the binary log, complete with an increment to the `@@gtid_binlog_pos`.
However, those unaware of this mechanism may not expect this increment
and its sign of replication divergence, namely in `@@gtid_strict_mode`.
This commit adds a binary log warning
when such a table is found `implicit_emptied`.
Since general uses of memory tables may already expect this effect,
to avoid unnecessary verbosity, this warning only emits from
`@@read_only` servers with CHANGE MASTER configured. `@@read_only`
indicates that the server should not receive changes; for a slave,
it means it only expects replication to binlog changes with GTID.
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
The data in MEMORY tables is temporary and is lost when the server
restarts. Thus, it is well-intentioned to record TRUNCATE statements
to the binary log when these tables are rediscovered empty. However,
as entries with no explicit user query associated (especially the lack
of SHUTDOWN on crashes), those unaware of this mechanism can find them
unexpected, not to mention their significance downstream in replication.
This commit adds a comment to these automatically generated
TRUNCATE entries to briefly self-describe their source and purpose.
As a part of the generated query, this comment is visible together
with the TRUNCATE itself in SHOW BINLOG EVENTS and `mariadb-binlog`,
while maintaining seamlessness in replication.
There are no other changes in behaviour or storage engine API.
Reviewed-by: Andrei Elkin <andrei.elkin@mariadb.com>
Followup commit for 4802bfe4f9.
After returning error from check_vers_constants() the stack
immediately returns error status up to mysql_admin_table() where it
does close_thread_tables(). The latter expects MDL acquired.
The bug discloses the generic problem of fallback mechanism throwing
an error when the upper frame does close_thread_tables().
* rename to follow the standard vprintf name convention
* remove `(LEX_CSTRING*) &lex_string` casts
* remove some redundant casts too
* LEX_CSTRING -> Lex_ident_db in path
* Hash_set<Sroutine_hash_entry>
* move assert out of the loop
* some comments and whitespace changes
don't create a permanent stack of paths on the heap and in THD.
Do it on the stack like for sql_mode, sctx, abort_on_warning, and
other THD properties that often need to be changed temporarily.
Fix Sql_path constructor.
Definition
==========
Implement the SET PATH statement.
```
SET PATH 'schema1, schema2, schema3';
```
Function
--------
In MariaDB, the SQL-path will also be used to lookup packages in
package routine invocations, so the following script:
SET PATH 'sys';
SELECT UTL_ENCODE.BASE64_DECODE('data');
invokes SYS.UTL_ENCODE.BASE64_DECODE('data'), i.e. the function
BASE64_DECODE() in the packge UTL_ENCODE in the database SYS
(if such package and package routine exist).
MariaDB implementation details
------------------------------
The ticket's requirement stated the grammar should support delimited
identifiers with backticks or double quotes within the string quote,
ie: the single tick. A simple parser was implemented to handle this
requirement in Sql_path::from_text.
Only 16 schemas are supported per session - we do not want too many
schemas in the path anyway since it will impact performance. This also
means that the path cache is a simple static array for easier memory
management.
Issue:
On galera write node INSERT statements does not acquire MDL locks on it's all child
tables and thereby wsrep certification keys are also added for limited tables, but
on applier nodes it does acquire MDL locks for all child tables. This can result
into MDL BF-BF conflict on applier node when transactions referring to parent and
child tables are executed concurrently. For example:
Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4
Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4
On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.
On applier nodes Write_rows event is implicitly a REPLACE, deleting all conflicting
rows which can cause cascading FK actions and locks on foreign key children tables.
Solution:
For Galera applier nodes the Write_rows event is considered pure INSERT
which will never cause cascading FK actions and locks on foreign key children tables.
In find_field_in_view(), we call field_it.create_item() which
creates item on a statement mem_root.
Then we set its name. Make sure the name is allocated on a statement
mem_root, too.
in SIMULTANEOUS_ASSIGNMENT there is no need to switch value items
to new nullable copies of table Field's - they must refer to old
values in the row, which can never be null anyway.
skipping this redundant step simplifies moving field to record[1]
and back in fill_record()
