table->pos_in_table_list is problematic, it can become
dangled when an SP is reparsed. Let's avoid it and directly
prefer TABLEs where a sequence is ready for inserts.
followup for 516f68af9e
don't reload stored routines in the middle of the execution
of a routine. we don't want different iterations of a loop
to see diffefent definitions
For this: remember Cversion in THD on the first sp cache lookup,
after that only compare versions with this value not with Cversion.
Instead of adding another TABLE_LIST to
Item_func_nextval->table_list->next_local, update instead
Item_func_nextval->table_list->table with the correct table.
This removes all checking of table_list->table and table_list->next_local
when using sequences.
if a bunch of tables are prelocked, when a table is actually
needed in the routine, open_tables picks one table out of the
prelocked list with a smallest "distance". Distance is simply
a difference between the actual table lock and the requested
table lock. Say, if the prelocked set contains both t1 write-locked
and t1 read-locked, than an UPDATE will prefer write-locked t1
and SELECT will prefer read-locked. if there's only write-locked
table in the set, both UPDATE and SELECT will use it.
this doesn't distingush between UPDATE and INSERT, but INSERT
marks tables with tables->for_insert_data=1, which causes
prelocking to invoke add_internal_tables() and prepare sequences
for execution.
in this bug there were two prelocked t1's, one for INSERT (with
for_insert_data=1) and one for UPDATE.
INSERT picks the second (they both are write-locked, so the distance is
the same), its sequence is not prepared and crashes.
Let's add for_insert_data as the lowest bit into the distance.
Issue:
The Commit: 0584846 'Add TL_FIRST_WRITE in SQL layer for determining R/W'
limits INSERT statements on write nodes to acquire MDL locks on it's all child
tables and thereby wsrep certification keys added, but on applier nodes it does
acquire MDL locks for all child tables. This can result into MDL BF-BF conflict
on applier node when transactions referring to parent and child tables are
executed concurrently. For example:
Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4
Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4
On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.
The Commit: 0584846 change limits MDL lock to be taken on all child nodes for
read-only FK checks (INSERT t1). But this doesn't works on applier nodes
because Write_rows_log_event event logged for INSERT is also used to record
update (check Write_rows_log_event::get_trg_event_map()), and
therefore MDL locks is taken for all the child tables on applier node
for update and insert event.
Solution:
Additional keys for the referenced/foreign table needs to be added
to avoid potential MDL conflicts with concurrent update and DDLs.
close_thread_tables() would not flush pending row events to the binlog cache
in certain conditions if LOCK TABLES was active. This could result in the
row events being binlogged without STMT_END_F flag, and eventually leave the
THD in an invalid state that triggered assertions later.
Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
The issue was that unpack_vcol_info_from_frm() wrongly linked the used
sequence tables into tables->internal_tables when more than one sequence
table was used.
Other things:
- Fixed internal_table_exists() to take db into account.
(This is making the code easier to read. As we where comparing
pointers the old code also worked).
TABLE_LIST parsed from procedure code is transferred into tables to
lock for INSERT. The procedure code is CREATE VIEW so its TABLE_LIST
is parsed as TL_IGNORE, but same view exists and when existing view is
opened mysql_make_view() uses same TABLE_LIST that was initialized
from CREATE VIEW and then added as part of prelocking context. So
existing view is opened and its table is assigned TL_IGNORE from
prelocking context. Finally, INSERT has TABLE_LIST duplication: the
one that was parsed from INSERT; the another one came from procedure
prelocking, its lock_type came from the procedure code and the real
table was found via existing view.
The sequence of execution:
1. Procedure p is compiled as part of open_and_process_routine(), its
code is parsed and create_or_alter_view_finalize() initializes v
TABLE_LIST as TL_IGNORE;
2. Procedure p prelocking adds v to prelocking_ctx with TL_IGNORE;
3. DML prelocking adds v from prelocking_ctx;
4. View is opened, mysql_make_view() assigns t lock_type from v;
5. open_and_lock_tables() attempts to lock t with TL_IGNORE.
The fix skips TL_IGNORE at 2. when table list parsed by procedure is
added for prelocking:
if (my_hash_insert(&m_sptabs, (uchar *)tab))
return FALSE;
m_sptabs designation was defined as strictly for prelocking:
/**
Multi-set representing optimized list of tables to be locked by this
routine. Does not include tables which are used by invoked routines.
@note
For prelocking-free SPs this multiset is constructed too.
We do so because the same instance of sp_head may be called both
in prelocked mode and in non-prelocked mode.
*/
HASH m_sptabs;
The fix was proposed by Sergei Golubchik <serg@mariadb.org>.
Part of an umbrella task MDEV-33281 for implementing optimizer hints.
This commit introduces hints affecting the use of indexes:
- JOIN_INDEX, NO_JOIN_INDEX
- GROUP_INDEX, NO_GROUP_INDEX
- ORDER_INDEX, NO_ORDER_INDEX
- INDEX, NO_INDEX
Syntax of index hints:
hint_name([@query_block_name] tbl_name [index_name [, index_name] ...])
hint_name(tbl_name@query_block_name [index_name [, index_name] ...])
JOIN_INDEX, NO_JOIN_INDEX: Forces the server to use or ignore the specified
index or indexes for any access method, such as ref, range, index_merge,
and so on. Equivalent to FORCE INDEX FOR JOIN, IGNORE INDEX FOR JOIN.
GROUP_INDEX, NO_GROUP_INDEX: Enable or disable the specified index or indexes
for index scans for GROUP BY operations. Equivalent to the index hints
FORCE INDEX FOR GROUP BY, IGNORE INDEX FOR GROUP BY.
ORDER_INDEX, NO_ORDER_INDEX: Causes the server to use or to ignore
the specified index or indexes for sorting rows. Equivalent to
FORCE INDEX FOR ORDER BY, IGNORE INDEX FOR ORDER BY.
INDEX, NO_INDEX: Acts as the combination of JOIN_INDEX, GROUP_INDEX
and ORDER_INDEX, forcing the server to use the specified index or indexes
for any and all scopes, or as the combination of NO_JOIN_INDEX, NO_GROUP_INDEX
and NO_ORDER_INDEX, which causes the server to ignore the specified index
or indexes for any and all scopes. Equivalent to FORCE INDEX, IGNORE INDEX.
Two kinds of index hints were introduced during implementation:
the global kind for [NO_]INDEX hint, and the non-global kind for all others.
Possible conflicts which will generate warnings:
- for a table level hint
- a hint of the same type or the opposite kind has already been specified
for the same table
- for a index level hint
- the same type of hint has already been specified for the same
table or for the same index, OR
- the opposite kind of hint has already been specified for the
same index
- For a multi index hint like JOIN_INDEX(t1 i1, i2, i3), it conflicts
with a previous hint if any of the JOIN_INDEX(t1 i1), JOIN_INDEX(t1 i2),
JOIN_INDEX(t1 i3) conflicts with a previous hint
Parser changes made by Alexander Barkov <bar@mariadb.com>.
Part of the tests made by Iqbal Hassan <iqbal@hasprime.com>.
Initially marking with ORA_JOIN flag made also by
Iqbal Hassan <iqbal@hasprime.com>.
Main idea is that parser mark fields with (+) with a flag
(ORA_JOIN).
During Prepare the flag bring to the new created items if needed.
Later after preparing (fix_firlds()) WHERE confition the
relations betweel the tables analyzed and tables reordered
so to make JOIN/LEFT JOIN operators in chain equivalent to
the query with oracle outer join operator (+).
Then the flags of (+) removed.
main/statistics_json.result is updated for f8ba5ced55 (MDEV-36099)
The test uses 'delete from t1' in many places and then populates
the table again. The natural order of rows in a MyISAM table is well
defined and the test was implicitly relying on that.
before f8ba5ced55 delete was deleting rows one by one, using
ha_myisam::delete_row() because the connection was stuck in rbr mode.
This caused rows to be shown in the reverse insertion order (because of
the delete link list).
MDEV-36099 fixes this bug and the server now correctly uses
ha_myisam::delete_all_rows(). This makes rows to be shown in the
insertion order as expected.
Only do trigger prelocking for tables that are doing to be
modified (with a write lock). A table can cause prelocking
if its DEFAULT value is used (because DEFAULT can be NEXTVAL),
even if the table itself is only used for reads. Don't process
triggers for such a table
XAER_RMFAIL means the admin statement was not allowed, it's not a
per-table message, so must fail the whole statement. And must
not rollback, obviously, it's not allowed after prepare.
Also, remove duplicate XAER_RMFAIL in open_tables(),
check_has_uncommitted_xa() already issues it.
Issue:
Mariadb acquires additional MDL locks on UPDATE/INSERT/DELETE statements
on table with foreign keys. For example, table t1 references t2, an
UPDATE to t1 will MDL lock t2 in addition to t1.
A replica may deliver an ALTER t1 and UPDATE t2 concurrently for
applying. Then the UPDATE may acquire MDL lock for t1, followed by a
conflict when the ALTER attempts to MDL lock on t1. Causing a BF-BF
conflict.
Solution:
Additional keys for the referenced/foreign table needs to be added
to avoid potential MDL conflicts with concurrent update and DDLs.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
When processing queries like
INSERT INTO t1 (..) SELECT .. FROM t1, t2 ...,
there is a single query block (i.e., a single SELECT_LEX) for both INSERT and
SELECT parts. During hints resolution, when hints are attached to particular
TABLE_LIST's, the search is performed by table name across the whole
query block.
So, if a table mentioned in an optimizer hint is present in the INSERT part,
the hint is attached to the that table. This is obviously wrong as
optimizer hints are supposed to only affect the SELECT part of
an INSERT..SELECT clause.
This commit disables possible attaching hints to tables in the INSERT part
and fixes some other bugs related to INSERT..SELECT statements processing
- add a comment that opt_hints_global->check_unresolved() is never called
- improve comments
- rename everything with "resolved_children" to "fully_resolved_children"
- Opt_hints_table::adjust_key_hints() now returns value
- less "reach-back-to-parent" logic
- rename Hint "adjustment" and "resolution" (yes, both terms were used)
to "fixing". "Resolution" is already used for parse-tree objects
It places a limit N (a timeout value in milliseconds) on how long
a statement is permitted to execute before the server terminates it.
Syntax:
SELECT /*+ MAX_EXECUTION_TIME(milliseconds) */ ...
Only top-level SELECT statements support the hint.
- Using Lex_ident_sys to scan identifiers, like the SQL parser does.
This fixes handling of double-quote-delimited and backtick-delimited identifiers,
as well as handling of non-ASCII identifiers.
Unescaping and converting from the client character set to the system
character set is now done using Lex_ident_cli_st and Lex_ident_sys,
like it's done in the SQL tokenizer/parser.
Adding helper methods to_ident_cli() and to_ident_sys()
in Optimizer_hint_parser::Token.
- Fixing the hint parser to report a syntax error when an empty identifiers:
SELECT /*+ BKA(``) */ * FROM t1;
- Moving a part of the code from opt_hints_parser.h to opt_hints_parser.cc
Moving these method definitions:
- Optimizer_hint_tokenizer::find_keyword()
- Optimizer_hint_tokenizer::get_token()
to avoid huge pieces of the code in the header file.
- A Lex_ident_cli_st cleanup
Fixing a few Lex_ident_cli_st methods to return Lex_ident_cli_st &
instead of void, to use them easier in the caller code.
- Fixing the hint parser to display the correct line number
Adding a new data type Lex_comment_st
(a combination of LEX_CSTRING and a line number)
Using it in sql_yacc.yy
- Getting rid of redundant dependencies on sql_hints_parser.h
Moving void LEX::resolve_optimizer_hints() from sql_lex.h to sql_lex.cc
Adding a class Optimizer_hint_parser_output, deriving from
Optimizer_hint_parser::Hint_list. Fixing the hint parser to
return a pointer to an allocated instance of Optimizer_hint_parser_output
rather than an instance of Optimizer_hint_parser::Hint_list.
This allows to use a forward declaration of Optimizer_hint_parser_output
in sql_lex.h and thus avoid dependencies on sql_hints_parser.h.
This commit introduces:
- the infrastructure for optimizer hints;
- hints for join buffering: BNL(), NO_BNL(), BKA(), NO_BKA();
- NO_ICP() hint for disabling index condition pushdown;
- MRR(), MO_MRR() hint for multi-range reads control;
- NO_RANGE_OPTIMIZATION() for disabling range optimization;
- QB_NAME() for assigning names for query blocks.
Valgrind is single threaded and only changes threads as part of
system calls or waits.
Some busy loops were identified and fixed where the server assumes
that some other thread will change the state, which will not happen
with valgrind.
Based on patch by Monty. Original patch introduced VALGRIND_YIELD,
which emits pthread_yield() only in valgrind builds. However it was
agreed that it is a good idea to emit yield() unconditionally, such
that other affected schedulers (like SCHED_FIFO) benefit from this
change. Also avoid pthread_yield() in favour of standard
std::this_thread::yield().
