and MDL".
Concurrent execution of a multi-DELETE statement and ALTER
TABLE statement which affected one of the tables used in
the multi-DELETE sometimes led to deadlock.
Similar deadlocks might have occured when one performed
INSERT/UPDATE/DELETE on a view and concurrently executed
ALTER TABLE for the view's underlying table, or when one
concurrently executed TRUNCATE TABLE for InnoDB table and
ALTER TABLE for the same table.
These deadlocks were caused by a discrepancy between types of
metadata and thr_lock.cc locks acquired by those statements.
What happened was that multi-DELETE/TRUNCATE/DML-through-the-
view statement in the first connection acquired SR lock on a
table, then ALTER TABLE would come in in the second connection
and acquire SNW metadata lock and TL_WRITE_ALLOW_READ
thr_lock.c lock and then would start waiting for the first
connection during lock upgrade. After that the statement in
the first connection would try to acquire TL_WRITE lock on
table and would start waiting for the second connection,
creating a deadlock.
This patch solves this problem by ensuring that we acquire
SW metadata lock in all cases in which we acquiring write
thr_lock.c lock. This guarantees that deadlocks like the
one described above won't occur since all lock conflicts
in such situation are resolved within MDL subsystem.
This patch also adds assert which should guarantee that
such situations won't arise in future.
failed on HANDLER + I_S
This assert was triggered when an I_S query tried to acquire a
metadata lock on a table which was already locked by a HANDLER
statement in the same connection.
First the HANDLER took a MDL_SHARED lock. Afterwards, the I_S query
requested a MDL_SHARED_HIGH_PRIO lock. The existing MDL_SHARED ticket
is found in find_ticket() since it satisfies
ticket->has_stronger_or_equal_type(mdl_request->type) as MDL_SHARED
and MDL_SHARED_HIGH_PRIO have equal strengths, just different priority.
However, two asserts later check lock type strengths using relational
operators (>= and <=) rather than MDL_ticket::has_stronger_or_equal_type().
These asserts are triggered since MDL_SHARED >= MDL_SHARED_HIGH_PRIORITY
is false (mapped to 1 and 2 respectively).
This patch updates the asserts to use MDL_ticket::has_stronger_or_equal_type()
rather than relational operators to check lock type strength.
Test case added to include/handler.inc.
Change the error code for ER_WARN_I_S_SKIPPED_TABLE, to not
upset the tests that rely on ER_SLAVE_CONVERSION_ERROR error
code = 1667.
Fix a merge bug with binlogging of CREATE TABLE (temporary tables).
HANDLER OPEN
The problem was a too restrictive assert in the code for
HANDLER ... OPEN and HANDLER ... READ that checked table->next
to verify that we didn't open views or merge tables.
This pointer is also used to link temporary tables together
(see thd->temporary_tables). In this case TABLE::next can be
set even if we're trying to open a single table.
This patch adjust the two asserts to also check for the presence
of temporary tables.
Test case added to handler_myisam.test.
Cherry-pick a fix Bug#37148 from next-mr, to preserve
file ids of the added files, and ensure that all the necessary
changes have been pulled.
Since initially Bug#37148 was null-merged into 6.0,
the changeset that is now being cherry-picked was likewise
null merged into next-4284.
Now that Bug#37148 has been reapplied to 6.0, try to make
it work with next-4284. This is also necessary to be able
to pull other changes from 5.1-rep into next-4284.
To resolve the merge issues use this changeset applied
to 6.0:
revid:jperkin@sun.com-20091216103628-ylhqf7s6yegui2t9
revno: 3776.1.1
committer: He Zhenxing <zhenxing.he@sun.com>
branch nick: 6.0-codebase-bugfixing
timestamp: Thu 2009-12-17 17:02:50 +0800
message:
Fix merge problem with Bug#37148
This was a deadlock between ALTER TABLE and another DML statement
(or LOCK TABLES ... READ). ALTER TABLE would wait trying to upgrade
its lock to MDL_EXCLUSIVE and the DML statement would wait trying
to acquire a TL_READ_NO_INSERT table level lock.
This could happen if one connection first acquired a MDL_SHARED_READ
lock on a table. In another connection ALTER TABLE is then started.
ALTER TABLE eventually blocks trying to upgrade to MDL_EXCLUSIVE,
but while holding a TL_WRITE_ALLOW_READ table level lock.
If the first connection then tries to acquire TL_READ_NO_INSERT,
it will block and we have a deadlock since neither connection can
proceed.
This patch fixes the problem by allowing TL_READ_NO_INSERT
locks to be granted if another connection holds TL_WRITE_ALLOW_READ
on the same table. This will allow the DML statement to proceed
such that it eventually can release its MDL lock which in turn
makes ALTER TABLE able to proceed.
Note that TL_READ_NO_INSERT was already partially compatible with
TL_WRITE_ALLOW_READ as the latter would be granted if the former
lock was held. This patch just makes the opposite true as well.
Also note that since ALTER TABLE takes an upgradable MDL lock,
there will be no starvation of ALTER TABLE statements by
statements acquiring TL_READ or TL_READ_NO_INSERT.
Test case added to lock_sync.test.
failed in open_ltable()
The problem was too restrictive asserts that enforced that
open_ltable() was called without any active HANDLERs, LOCK TABLES
or global read locks.
However, this can happen in several cases when opening system
tables. The assert would, for example, be triggered when drop
function was called from a connection with active HANDLERs as
this would cause open_ltable() to be called for mysql.proc.
The assert could also be triggered when using table-based
general log (mysql.general_log).
This patch removes the asserts since they will be triggered in
several legitimate cases and because the asserts are no longer
relevant due to changes in how locks are released.
The patch also fixes set_needs_thr_lock_abort() that before
ignored its parameter and always set the member variable to TRUE.
Test case added to mdl_sync.test.
Thanks to Dmitry Lenev for help with this bug!
Post merge fixes. In sys_vars tests suite, whenever we use LOCK TABLE WRITE,
adjust to lock the subject table indirecty via a view, to ensure that
there the desired lock conflict in thr_lock.c is still taking place.
If locked directly in LOCK TABLES list, thr_lock.c does not get
involved.
Adjust the thread wait state names in information_schema.processlist.
m_tickets.front() == m_trans_sentinel'".
Debug build of server crashed due to assert failure in MDL
subsystem when one tried to execute multi-table REPAIR or
OPTIMIZE in autocommit=0 mode.
The assert failure occured when multi-table REPAIR or OPTIMIZE
started processing of second table from its table list and
tried to acquire upgradable metadata lock on this table.
The cause of the assert failure were MDL locks left over from
processing of previous table. It turned out that in autocommit=0
mode close_thread_tables() which happens at the end of table
processing doesn't release metadata locks.
This fix solves problem by releasing locks explicitly using
MDL_context::release_trans_locks() call.
fulltext search and row op.
The search for fulltext indexes is searching for some special
predicate layouts. While doing so it's not checking for the number
of columns of the expressions it tries to calculate.
And since row expressions can't return a single scalar value there
was a crash.
Fixed by checking if the expressions are scalar (in addition to
being constant) before calling Item::val_xxx() methods.
The test case rpl_binlog_corruption fails on windows because when
adding a line to the binary log index file it gets terminated
with a CR+LF (which btw, is the normal case in windows, but not on
Unixes - LF). This causes mismatch between the relay log names,
causing mysqld to report that it cannot find the log file.
We fix this by creating the instrumented index file through
mysql, ie, using SELECT ... INTO DUMPFILE ..., as opposed on
relying on ultimatly OS commands like: -- echo "..." >
index. These changes go into the file and make the procedure
platform independent:
include/setup_fake_relay_log.inc
Side note: when using SELECT ... INTO DUMPFILE ..., one needs to
check if mysqld is running with secure_file_priv. If it is, we do
it in two steps: 1. create the file on the allowed location;
2. move it to the datadir. If it is not, then we just create the
file directly on the datadir (so previous step 2. is not needed).
Fix Bug#50555 "handler commands crash server in my_hash_first()"
as a post-merge fix (the new handler tests are not passing
otherwise).
- in hash.c, don't call calc_hash if ! my_hash_inited().
- add tests and results for the test case for Bug#50555
Original revision:
------------------------------------------------------------
revision-id: li-bing.song@sun.com-20100130124925-o6sfex42b6noyc6x
parent: joro@sun.com-20100129145427-0n79l9hnk0q43ajk
committer: <Li-Bing.Song@sun.com>
branch nick: mysql-5.1-bugteam
timestamp: Sat 2010-01-30 20:49:25 +0800
message:
Bug #48321 CURRENT_USER() incorrectly replicated for DROP/RENAME USER;
REVOKE/GRANT; ALTER EVENT.
The following statements support the CURRENT_USER() where a user is needed.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENT
but, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, All above statements are rewritten when they are binlogged.
The CURRENT_USER() is expanded to the real user's name and host.
------------------------------------------------------------
Linux x86_64 debug
Two test cases fail because the suppression for the unsafe
warning needs to be updated (BUG@39934 refactored this part and
these changes are only in mysql-next-mr - this is why we notice
them now when merging in next-mr). This is the case for
rpl_nondeterministic_functions and rpl_misc_functions test
cases. rpl_stm_binlog_direct test case is not needed in version >
5.1. The rpl_heartbeat_basic test case fails because patch for
BUG@50397 removed the CHANGE MASTER in the slave that would set
it's period to 1/10 of the master. This would cause the test
assertion to fail.
The fixes for the issues described above are:
- rpl_misc_functions - updated suppression message
- rpl_nondeterministic_functions - updated suppression message
- rpl_stm_binlog_direct - removed the test case (it is not
needed in versions > 5.1)
- rpl_heartbeat_basic - deployed instruction:
CHANGE MASTER TO MASTER_HEARTBEAT_PERIOD=0.1;
Conflicts:
- mysql-test/r/mysqld--help-win.result
- sql/sys_vars.cc
Original revsion (in next-mr-bugfixing):
------------------------------------------------------------
revno: 2971 [merge]
revision-id: alfranio.correia@sun.com-20100121210527-rbuheu5rnsmcakh1
committer: Alfranio Correia <alfranio.correia@sun.com>
branch nick: mysql-next-mr-bugfixing
timestamp: Thu 2010-01-21 21:05:27 +0000
message:
BUG#46364 MyISAM transbuffer problems (NTM problem)
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
So, in this patch we introduce the option
- "--binlog-direct-non-transactional-updates" that can be used to bypass
the current behavior in order to write directly to binary log statements
that change non-transactional tables.
Besides, it is used to enable the WL#2687 which is disabled by default.
------------------------------------------------------------
revno: 2970.1.1
revision-id: alfranio.correia@sun.com-20100121131034-183r4qdyld7an5a0
parent: alik@sun.com-20100121083914-r9rz2myto3tkdya0
committer: Alfranio Correia <alfranio.correia@sun.com>
branch nick: mysql-next-mr-bugfixing
timestamp: Thu 2010-01-21 13:10:34 +0000
message:
BUG#46364 MyISAM transbuffer problems (NTM problem)
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
So, in this patch we introduce the option
- "--binlog-direct-non-transactional-updates" that can be used to bypass
the current behavior in order to write directly to binary log statements
that change non-transactional tables.
Besides, it is used to enable the WL#2687 which is disabled by default.
which implemented new type-of-operation-aware metadata
locks and added a wait-for graph based deadlock detector
to the MDL subsystem (this patch fixed bug #46272 "MySQL
5.4.4, new MDL: unnecessary deadlock" and bug #37346
"innodb does not detect deadlock between update and alter
table").
These hangs were caused by missing include of
wait_condition.inc. This fix simply adds them.
Add a wait-for graph based deadlock detector to the
MDL subsystem.
Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and
bug #37346 "innodb does not detect deadlock between update and
alter table".
The first bug manifested itself as an unwarranted abort of a
transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER
statement, when this transaction tried to repeat use of a
table, which it has already used in a similar fashion before
ALTER started.
The second bug showed up as a deadlock between table-level
locks and InnoDB row locks, which was "detected" only after
innodb_lock_wait_timeout timeout.
A transaction would start using the table and modify a few
rows.
Then ALTER TABLE would come in, and start copying rows
into a temporary table. Eventually it would stumble on
the modified records and get blocked on a row lock.
The first transaction would try to do more updates, and get
blocked on thr_lock.c lock.
This situation of circular wait would only get resolved
by a timeout.
Both these bugs stemmed from inadequate solutions to the
problem of deadlocks occurring between different
locking subsystems.
In the first case we tried to avoid deadlocks between metadata
locking and table-level locking subsystems, when upgrading shared
metadata lock to exclusive one.
Transactions holding the shared lock on the table and waiting for
some table-level lock used to be aborted too aggressively.
We also allowed ALTER TABLE to start in presence of transactions
that modify the subject table. ALTER TABLE acquires
TL_WRITE_ALLOW_READ lock at start, and that block all writes
against the table (naturally, we don't want any writes to be lost
when switching the old and the new table). TL_WRITE_ALLOW_READ
lock, in turn, would block the started transaction on thr_lock.c
lock, should they do more updates. This, again, lead to the need
to abort such transactions.
The second bug occurred simply because we didn't have any
mechanism to detect deadlocks between the table-level locks
in thr_lock.c and row-level locks in InnoDB, other than
innodb_lock_wait_timeout.
This patch solves both these problems by moving lock conflicts
which are causing these deadlocks into the metadata locking
subsystem, thus making it possible to avoid or detect such
deadlocks inside MDL.
To do this we introduce new type-of-operation-aware metadata
locks, which allow MDL subsystem to know not only the fact that
transaction has used or is going to use some object but also what
kind of operation it has carried out or going to carry out on the
object.
This, along with the addition of a special kind of upgradable
metadata lock, allows ALTER TABLE to wait until all
transactions which has updated the table to go away.
This solves the second issue.
Another special type of upgradable metadata lock is acquired
by LOCK TABLE WRITE. This second lock type allows to solve the
first issue, since abortion of table-level locks in event of
DDL under LOCK TABLES becomes also unnecessary.
Below follows the list of incompatible changes introduced by
this patch:
- From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those
statements that acquire TL_WRITE_ALLOW_READ lock)
wait for all transactions which has *updated* the table to
complete.
- From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE
(i.e. all statements which acquire TL_WRITE table-level lock) wait
for all transaction which *updated or read* from the table
to complete.
As a consequence, innodb_table_locks=0 option no longer applies
to LOCK TABLES ... WRITE.
- DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort
statements or transactions which use tables being dropped or
renamed, and instead wait for these transactions to complete.
- Since LOCK TABLES WRITE now takes a special metadata lock,
not compatible with with reads or writes against the subject table
and transaction-wide, thr_lock.c deadlock avoidance algorithm
that used to ensure absence of deadlocks between LOCK TABLES
WRITE and other statements is no longer sufficient, even for
MyISAM. The wait-for graph based deadlock detector of MDL
subsystem may sometimes be necessary and is involved. This may
lead to ER_LOCK_DEADLOCK error produced for multi-statement
transactions even if these only use MyISAM:
session 1: session 2:
begin;
update t1 ... lock table t2 write, t1 write;
-- gets a lock on t2, blocks on t1
update t2 ...
(ER_LOCK_DEADLOCK)
- Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE
was abandoned.
LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same
priority as the usual LOCK TABLE ... WRITE.
SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in
the wait queue.
- We do not take upgradable metadata locks on implicitly
locked tables. So if one has, say, a view v1 that uses
table t1, and issues:
LOCK TABLE v1 WRITE;
FLUSH TABLE t1; -- (or just 'FLUSH TABLES'),
an error is produced.
In order to be able to perform DDL on a table under LOCK TABLES,
the table must be locked explicitly in the LOCK TABLES list.
