Attach the children of a MERGE table once per statement, and keep the
value of pos_in_table_list for a MERGE table on subsequent executions
of a prepared statement.
A segfault in mysql_lock_merge waa the result of a null argument
when called from open_and_lock_internal_tables. As the argument
was null there was no lock to merge and thd->lock was already in place,
so no futher action is required in open_and_lock_internal_tables.
After passing the previous correctly on open_table of the
sequence used in the table, we get to the checking the thd->open_table
list where tdc is flushed. This puts the open_table into an error
condition to reopen all tables. When we are just opening a sequence,
this condition isn't one we want to fail on.
On the LOAD DATA path the base columns are filled directly from the
input file and fill_record_n_invoke_before_triggers() is then called
with an empty field list (there is no SET clause). After the BEFORE
INSERT trigger changed a base column, the stored generated columns were
not recomputed, so they kept the value derived from the pre-trigger
input (e.g. g=24 instead of 40 for g=v*2 with v set to 20 by the
trigger). A regular INSERT was unaffected.
The recompute was guarded by "fields.elements". That condition is a
leftover from the original computed-columns implementation
(f7a75b999b), where fill_record_n_invoke_before_triggers() had no
TABLE* argument and had to reverse-derive the table from the first
item of the field list:
if (fields.elements)
{
fld= (Item_field*)f++;
item_field= fld->field_for_view_update();
table= item_field->field->table;
...
}
With an empty field list there was no way to obtain the table, so the
recompute was silently skipped. Since bc4a456758 (MDEV-452) the
function receives TABLE* explicitly, which made the whole derivation
dead code (as the in-place DBUG_ASSERT(table == item_field->field->table)
confirmed). Recompute the virtual fields unconditionally on
table->vfield, the same way the Field** overload of
fill_record_n_invoke_before_triggers() already does.
if a user tries to access a table or a database they have no priivleges
on, the error is alwaus "access denied", independently from whether
the object exists or not. Do the same for columns.
Problem:
========
A single InnoDB purge worker thread can process undo logs from different
tables within the same batch. But get_purge_table(), open_purge_table()
incorrectly assumes that a 1:1 relationship between a purge worker thread
and a table within a single batch. Based on this wrong assumtion,
InnoDB attempts to reuse TABLE objects cached in thd->open_tables for
virtual column computation.
1) Purge worker opens Table A and caches the TABLE pointer in thd->open_tables.
2) Same purge worker moves to Table B in the same batch, get_purge_table()
retrieves the cached pointer for Table A instead of opening Table B.
3) Because innobase::open() is ignored for Table B, the virtual column
template is never initialized.
4) virtual column computation for Table B aborts the server
Solution:
========
- Introduced purge_table class which has the following
purge_table: Stores either TABLE* (for tables with indexed virtual
columns) or MDL_ticket* (for tables without) in a single union
using LSB as a flag.
For tables with indexed virtual columns: opens TABLE*, accesses
MDL_ticket* via TABLE->mdl_ticket
For tables without indexed virtual columns: stores only MDL_ticket*.
trx_purge_attach_undo_recs(): Coordinator opens both dict_table_t*
and TABLE* with proper MDL protection. Workers access cached
table pointers from purge_node_t->tables without opening
their own handles
purge_sys.coordinator_thd: Distinguish coordinator from workers
in cleanup logic. Skip innobase_reset_background_thd() for
coordinator thread to prevent premature table closure during
batch processing. Workers still call cleanup to release their
thread-local resources
trx_purge_close_tables():
Rewrite for purge coordinator thread
1) Close all dict_table_t* objects first
2) Call close_thread_tables() once for all TABLE* objects
3) Release MDL tickets last, after tables are closed
Added table->lock_mutex protection when reading (or) writing
vc_templ->mysql_table and mysql_table_query_id. Clear cached
TABLE* pointers before closing tables to prevent stale pointer
access
Declared open_purge_table() and close_thread_tables() in trx0purge.cc
Declared reset_thd() in row0purge.cc and dict0stats_bg.cc.
Removed innobase_reset_background_thd()
Problem:
Server crashes when a procedure with default parameter is executed and
default value for the parameter is evaluated after the procedure's
first execution. For example:
CREATE PROCEDURE p(x INT DEFAULT (SELECT 1)) ...
CALL p(1); -- first exec, marks mem_root read-only
CALL p(); -- tries to allocate on locked when executing DEFAULT
Fix:
Temporarily clear the read-only flag on the statement memory root to
allow the allocation. Also adds a debug hook to verify allocation
happens not more than once per prepared statement and per internal
table added to the list.
Problem:
========
A single InnoDB purge worker thread can process undo logs from different
tables within the same batch. But get_purge_table(), open_purge_table()
incorrectly assumes that a 1:1 relationship between a purge worker thread
and a table within a single batch. Based on this wrong assumtion,
InnoDB attempts to reuse TABLE objects cached in thd->open_tables for
virtual column computation.
1) Purge worker opens Table A and caches the TABLE pointer in thd->open_tables.
2) Same purge worker moves to Table B in the same batch, get_purge_table()
retrieves the cached pointer for Table A instead of opening Table B.
3) Because innobase::open() is ignored for Table B, the virtual column
template is never initialized.
4) virtual column computation for Table B aborts the server
Solution:
========
- Introduced purge_table class which has the following
purge_table: Stores either TABLE* (for tables with indexed virtual
columns) or MDL_ticket* (for tables without) in a single union
using LSB as a flag.
For tables with indexed virtual columns: opens TABLE*, accesses
MDL_ticket* via TABLE->mdl_ticket
For tables without indexed virtual columns: stores only MDL_ticket*.
trx_purge_attach_undo_recs(): Coordinator opens both dict_table_t*
and TABLE* with proper MDL protection. Workers access cached
table pointers from purge_node_t->tables without opening
their own handles
purge_sys.coordinator_thd: Distinguish coordinator from workers
in cleanup logic. Skip innobase_reset_background_thd() for
coordinator thread to prevent premature table closure during
batch processing. Workers still call cleanup to release their
thread-local resources
trx_purge_close_tables():
Rewrite for purge coordinator thread
1) Close all dict_table_t* objects first
2) Call close_thread_tables() once for all TABLE* objects
3) Release MDL tickets last, after tables are closed
Added table->lock_mutex protection when reading (or) writing
vc_templ->mysql_table and mysql_table_query_id. Clear cached
TABLE* pointers before closing tables to prevent stale pointer
access
Declared open_purge_table() and close_thread_tables() in trx0purge.cc
Declared reset_thd() in row0purge.cc and dict0stats_bg.cc.
Removed innobase_reset_background_thd()
Problem:
When a prepared statement uses `DEFAULT` with sequence, on its
second or subsequent execution, server tries to allocate TABLE_LIST
object (used for adding sequence table to `internal_tables` list) on
statement's memory root. This fails with an assert because statement's
memory root is marked read-only after the first execution.
CREATE SEQUENCE s;
CREATE TABLE t (a INT DEFAULT(NEXTVAL(s)));
PREPARE stmt FROM "UPDATE t SET a = ?";
EXECUTE stmt USING 3; -- first execution, marks mem_root read-only
EXECUTE stmt USING DEFAULT; -- tries to allocate on second execution
Fix:
Temporarily clear the read-only flag on the statement memory root to
allow the allocation. Also adds a debug hook to verify allocation
happens not more than once per prepared statement and per internal
table added to the list.
The data in MEMORY tables is temporary and is lost when the server
restarts. Thus, when these tables are rediscovered empty,
it’s well-intentioned to record TRUNCATE statements to
the binary log, complete with an increment to the `@@gtid_binlog_pos`.
However, those unaware of this mechanism may not expect this increment
and its sign of replication divergence, namely in `@@gtid_strict_mode`.
This commit adds a binary log warning
when such a table is found `implicit_emptied`.
Since general uses of memory tables may already expect this effect,
to avoid unnecessary verbosity, this warning only emits from
`@@read_only` servers with CHANGE MASTER configured. `@@read_only`
indicates that the server should not receive changes; for a slave,
it means it only expects replication to binlog changes with GTID.
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
The data in MEMORY tables is temporary and is lost when the server
restarts. Thus, it is well-intentioned to record TRUNCATE statements
to the binary log when these tables are rediscovered empty. However,
as entries with no explicit user query associated (especially the lack
of SHUTDOWN on crashes), those unaware of this mechanism can find them
unexpected, not to mention their significance downstream in replication.
This commit adds a comment to these automatically generated
TRUNCATE entries to briefly self-describe their source and purpose.
As a part of the generated query, this comment is visible together
with the TRUNCATE itself in SHOW BINLOG EVENTS and `mariadb-binlog`,
while maintaining seamlessness in replication.
There are no other changes in behaviour or storage engine API.
Reviewed-by: Andrei Elkin <andrei.elkin@mariadb.com>
Followup commit for 4802bfe4f9.
After returning error from check_vers_constants() the stack
immediately returns error status up to mysql_admin_table() where it
does close_thread_tables(). The latter expects MDL acquired.
The bug discloses the generic problem of fallback mechanism throwing
an error when the upper frame does close_thread_tables().
Issue:
On galera write node INSERT statements does not acquire MDL locks on it's all child
tables and thereby wsrep certification keys are also added for limited tables, but
on applier nodes it does acquire MDL locks for all child tables. This can result
into MDL BF-BF conflict on applier node when transactions referring to parent and
child tables are executed concurrently. For example:
Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4
Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4
On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.
On applier nodes Write_rows event is implicitly a REPLACE, deleting all conflicting
rows which can cause cascading FK actions and locks on foreign key children tables.
Solution:
For Galera applier nodes the Write_rows event is considered pure INSERT
which will never cause cascading FK actions and locks on foreign key children tables.
In find_field_in_view(), we call field_it.create_item() which
creates item on a statement mem_root.
Then we set its name. Make sure the name is allocated on a statement
mem_root, too.
in SIMULTANEOUS_ASSIGNMENT there is no need to switch value items
to new nullable copies of table Field's - they must refer to old
values in the row, which can never be null anyway.
skipping this redundant step simplifies moving field to record[1]
and back in fill_record()
SELECT privilege
Analysis:
When setup_fields() is called, the want_privilege is set to NO_ACL, so
correct priveleges are not checked.
Fix:
Since RETURNING requires SELECT_ACL privelige, when we are setting up
the returning fields for the given query, set want_privilege to SELECT_ACL.
Reset to original value of want_privilege once done.
table->pos_in_table_list is problematic, it can become
dangled when an SP is reparsed. Let's avoid it and directly
prefer TABLEs where a sequence is ready for inserts.
followup for 516f68af9e
table->pos_in_table_list is problematic, it can become
dangled when an SP is reparsed. Let's avoid it and directly
prefer TABLEs where a sequence is ready for inserts.
followup for 516f68af9e
don't reload stored routines in the middle of the execution
of a routine. we don't want different iterations of a loop
to see diffefent definitions
For this: remember Cversion in THD on the first sp cache lookup,
after that only compare versions with this value not with Cversion.
Instead of adding another TABLE_LIST to
Item_func_nextval->table_list->next_local, update instead
Item_func_nextval->table_list->table with the correct table.
This removes all checking of table_list->table and table_list->next_local
when using sequences.
if a bunch of tables are prelocked, when a table is actually
needed in the routine, open_tables picks one table out of the
prelocked list with a smallest "distance". Distance is simply
a difference between the actual table lock and the requested
table lock. Say, if the prelocked set contains both t1 write-locked
and t1 read-locked, than an UPDATE will prefer write-locked t1
and SELECT will prefer read-locked. if there's only write-locked
table in the set, both UPDATE and SELECT will use it.
this doesn't distingush between UPDATE and INSERT, but INSERT
marks tables with tables->for_insert_data=1, which causes
prelocking to invoke add_internal_tables() and prepare sequences
for execution.
in this bug there were two prelocked t1's, one for INSERT (with
for_insert_data=1) and one for UPDATE.
INSERT picks the second (they both are write-locked, so the distance is
the same), its sequence is not prepared and crashes.
Let's add for_insert_data as the lowest bit into the distance.
Issue:
The Commit: 0584846 'Add TL_FIRST_WRITE in SQL layer for determining R/W'
limits INSERT statements on write nodes to acquire MDL locks on it's all child
tables and thereby wsrep certification keys added, but on applier nodes it does
acquire MDL locks for all child tables. This can result into MDL BF-BF conflict
on applier node when transactions referring to parent and child tables are
executed concurrently. For example:
Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4
Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4
On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.
The Commit: 0584846 change limits MDL lock to be taken on all child nodes for
read-only FK checks (INSERT t1). But this doesn't works on applier nodes
because Write_rows_log_event event logged for INSERT is also used to record
update (check Write_rows_log_event::get_trg_event_map()), and
therefore MDL locks is taken for all the child tables on applier node
for update and insert event.
Solution:
Additional keys for the referenced/foreign table needs to be added
to avoid potential MDL conflicts with concurrent update and DDLs.
close_thread_tables() would not flush pending row events to the binlog cache
in certain conditions if LOCK TABLES was active. This could result in the
row events being binlogged without STMT_END_F flag, and eventually leave the
THD in an invalid state that triggered assertions later.
Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
