----------------------------------------------------------
revno: 2617.69.2
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-azalea-bugfixing
timestamp: Mon 2009-08-03 19:26:04 +0400
message:
A fix and a test case for Bug#45035 "Altering table under LOCK TABLES
results in "Error 1213 Deadlock found...".
If a user had a table locked with LOCK TABLES
for READ and for WRITE in the same connection, ALTER TABLE
could fail.
Root cause analysis:
If a connection issues
LOCK TABLE t1 write, t1 a read, t1 b read;
the new LOCK TABLES code in 6.0 (part of WL 3726) will create
the following list of TABLE_LIST objects
(thd->locked_tables_list->m_locked_tables):
{"t1" "b" tl_read_no_insert}, {"t1" "a" tl_read_no_insert},
{"t1" "t1" tl_write }
Later on, when we try to ALTER table t1, mysql_alter_table()
closes all TABLE instances and releases its thr_lock locks,
keeping only an exclusive metadata lock on t1.
But when ALTER is finished, Locked_table_list::reopen_tables()
tries to restore the original list of open and locked tables.
Before this patch, it used to do so one by one:
Open t1 b, get TL_READ_NO_INSERT lock,
Open t1 a, get TL_READ_NO_INSERT lock
Open t1, try to get TL_WRITE lock, deadlock.
The cause of the deadlock is that thr_lock.c doesn't
resolve the situation when the read list only consists
of locks taken by the same thread, followed by this very
thread trying to take a WRITE lock. Indeed, since
thr_lock_multi always gets a sorted list of locks,
WRITE locks always precede READ locks in the list
to lock.
Don't try to fix thr_lock.c deficiency, keep this
code simple.
Instead, try to take all thr_lock locks at once
in ::reopen_tables().
------------------------------------------------------------
revno: 2617.43.3
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 40188-6.0
timestamp: Thu 2009-05-07 13:15:54 +0200
message:
Sort results as the file list of the database directory is not
sorted (MY_DONT_SORT).
(This is a follow-up fix for WL#4284).
The problem is not actually related to 2byte collation IDs.
The same crash happens if you change the collation ID in
mysql-test/str_data/Index.xml to a value smaller than 256.
Crash happened in SQL parser, because the "ident_map" and "state_map"
arrays were not initialized in loadable utf8 collations.
Fix: adding proper initialization of the "ident_map" and "state_map"
members for loadable utf8 collations.
2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1
- initial changeset that introduced the fix for
Bug#989 and follow up fixes for all test suite failures
introduced in the initial changeset.
------------------------------------------------------------
revno: 2617.31.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Fri 2009-03-06 19:17:00 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking
Currently the MySQL server does not keep metadata locks on
schema objects for the duration of a transaction, thus failing
to guarantee the integrity of the schema objects being used
during the transaction and to protect then from concurrent
DDL operations. This also poses a problem for replication as
a DDL operation might be replicated even thought there are
active transactions using the object being modified.
The solution is to defer the release of metadata locks until
a active transaction is either committed or rolled back. This
prevents other statements from modifying the table for the
entire duration of the transaction. This provides commitment
ordering for guaranteeing serializability across multiple
transactions.
- Incompatible change:
If MySQL's metadata locking system encounters a lock conflict,
the usual schema is to use the try and back-off technique to
avoid deadlocks -- this schema consists in releasing all locks
and trying to acquire them all in one go.
But in a transactional context this algorithm can't be utilized
as its not possible to release locks acquired during the course
of the transaction without breaking the transaction commitments.
To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be
returned if a lock conflict is encountered during a transaction.
Let's consider an example:
A transaction has two statements that modify table t1, then table
t2, and then commits. The first statement of the transaction will
acquire a shared metadata lock on table t1, and it will be kept
utill COMMIT to ensure serializability.
At the moment when the second statement attempts to acquire a
shared metadata lock on t2, a concurrent ALTER or DROP statement
might have locked t2 exclusively. The prescription of the current
locking protocol is that the acquirer of the shared lock backs off
-- gives up all his current locks and retries. This implies that
the entire multi-statement transaction has to be rolled back.
- Incompatible change:
FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ
LOCK won't cause locked tables to be implicitly unlocked anymore.
Before this patch, semisync assumed transactions running in parallel
can not be larger than max_connections, but this is not true when
the event scheduler is executing events, and cause semisync run out
of preallocated transaction nodes.
Fix the problem by allocating transaction nodes dynamically.
This patch also fixed a possible deadlock when running UNINSTALL
PLUGIN rpl_semi_sync_master and updating in parallel. Fixed by
releasing the internal Delegate lock before unlock the plugins.
------------------------------------------------------------
revno: 2617.23.18
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Mon 2009-03-02 18:18:26 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking
This is a prerequisite patch:
These changes are intended to split lock requests from granted
locks and to allow the memory and lifetime of granted locks to
be managed within the MDL subsystem. Furthermore, tickets can
now be shared and therefore are used to satisfy multiple lock
requests, but only shared locks can be recursive.
The problem is that the MDL subsystem morphs lock requests into
granted locks locks but does not manage the memory and lifetime
of lock requests, and hence, does not manage the memory of
granted locks either. This can be problematic because it puts the
burden of tracking references on the users of the subsystem and
it can't be easily done in transactional contexts where the locks
have to be kept around for the duration of a transaction.
Another issue is that recursive locks (when the context trying to
acquire a lock already holds a lock on the same object) requires
that each time the lock is granted, a unique lock request/granted
lock structure structure must be kept around until the lock is
released. This can lead to memory leaks in transactional contexts
as locks taken during the transaction should only be released at
the end of the transaction. This also leads to unnecessary wake
ups (broadcasts) in the MDL subsystem if the context still holds
a equivalent of the lock being released.
These issues are exacerbated due to the fact that WL#4284 low-level
design says that the implementation should "2) Store metadata locks
in transaction memory root, rather than statement memory root" but
this is not possible because a memory root, as implemented in mysys,
requires all objects allocated from it to be freed all at once.
This patch combines review input and significant code contributions
from Konstantin Osipov (kostja) and Dmitri Lenev (dlenev).
------------------------------------------------------------
revno: 3035.4.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 39897-6.0
timestamp: Thu 2009-01-15 12:17:57 -0200
message:
Bug#39897: lock_multi fails in pushbuild: timeout waiting for processlist
The problem is that relying on the "Table lock" thread state in
its current position to detect that a thread is waiting on a lock
is race prone. The "Table lock" state change happens before the
thread actually tries to grab a lock on a table.
The solution is to move the "Table lock" state so that its set
only when a thread is actually going to wait for a lock. The state
change happens after the thread fails to grab the lock (because it
is owned by other thread) and proceeds to wait on a condition.
This is considered part of work related to WL#4284 "Transactional
DDL locking"
Warning: this patch contains an incompatible change.
When waiting on a lock in thr_lock.c, the server used to display "Locked"
processlist state. After this patch, the state is "Table lock".
The new state was actually intended to be display since year 2002,
when Monty added it. But up until removal of thd->locked boolean
member, this state was ignored by SHOW PROCESSLIST code.
------------------------------------------------------------
revno: 2630.13.17
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Sun 2008-07-27 10:14:46 -0300
message:
Post-merge fixes:
Remove dependency on binlog, require not embedded as test uses
the event scheduler and disable abort on error for syntax only
available on servers built with debugging support.
This is a patch in scope of WL#4284 "Transactional DDL locking"
------------------------------------------------------------
revno: 2630.13.16
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: WL#4284
timestamp: Sat 2008-07-26 13:38:20 -0300
message:
WL#4284: Transactional DDL locking
SQL statements' effect on transactions.
Currently the MySQL server and its storage engines are not
capable of rolling back operations that define or modify data
structures (also known as DDL statements) or operations that
alter any of the system tables (the mysql database). Allowing
these group of statements to participate in transactions
is unfeasible at this time (since rollback has no effect
whatsoever on them) and goes against the design of our metadata
locking subsystem.
The solution is to issue implicit commits before and after
those statements execution. This effectively confines each of
those statements to its own special transaction and ensures
that metadata locks taken during this special transaction
are not leaked into posterior statements/transactions.
When a .CSV file for table in the CSV engine contains
\X characters as part of unquoted fields, e.g.
2,naraya\nan
\n is not interpreted as a new line (it is however interpreted as a
newline in a quoted field).
The old algorithm copied the entire value for a unquoted field without
parsing the \X characters.
The new algorithm adds the capability to handle \X characters in the
unquoted fields of a .CSV file.
----------------------------------------------------------
revno: 2630.4.38
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-4144
timestamp: Wed 2008-06-25 22:07:06 +0400
message:
WL#4144 - Lock MERGE engine children.
Committing a version of the patch merged with WL#3726
on behalf of Ingo.
Step #1: Move locking from parent to children.
MERGE children are now left in the query list of tables
after inserted there in open_tables(). So they are locked
by lock_tables() as all other tables are.
The MERGE parent does not store locks any more. It appears
in a MYSQL_LOCK with zero lock data. This is kind of a "dummy"
lock.
All other lock handling is also done directly on the children.
To protect against parent or child modifications during LOCK
TABLES, the children are detached after every statement and
attached before every statement, even under LOCK TABLES.
The children table list is removed from the query list of tables
on every detach and on close of the parent.
Step #2: Move MERGE specific functionality from SQL layer
into table handler.
Functionality moved from SQL layer (mainly sql_base.cc)
to the table handler (ha_myisammrg.cc).
Unnecessary code is removed from the SQL layer.
Step #3: Moved all MERGE specific members from TABLE
to ha_myisammrg.
Moved members from TABLE to ha_myisammrg.
Renamed some mebers.
Fixed comments.
Step #4: Valgrind and coverage testing
Valgrind did not uncover new problems.
Added purecov comments.
Added a new test for DATA/INDEX DIRECTORY options.
Changed handling of ::reset() for non-attached children.
Fixed the merge-big test.
Step #5: Fixed crashes detected during review
Changed detection when to attach/detach.
Added new tests.
Backport also the fix for Bug#44040 "MySQL allows creating a
MERGE table upon VIEWs but crashes when using it"
----------------------------------------------------------
revno: 2630.4.35
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-3726
timestamp: Wed 2008-06-25 16:44:00 +0400
message:
Fix a MyISAM-specific bug in the new implementation of
LOCK TABLES (WL#3726).
If more than one instance of a MyISAM table are open in the
same connection, all of them must share the same status_param.
Otherwise, unlock of a table may lead to lost records.
See also comments in thr_lock.c.
----------------------------------------------------------
revno: 2630.10.1
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-lock-tables-tidyup
timestamp: Wed 2008-06-11 15:49:58 +0400
message:
WL#3726, review fixes.
Now that we have metadata locks, we don't need to keep a crippled
TABLE instance in the table cache to indicate that a table is locked.
Remove all code that used this technique. Instead, rely on metadata
locks and use the standard open_table() and close_thread_table()
to manipulate with the table cache tables.
Removes a list of functions that have become unused (see the comment
for sql_base.cc for details).
Under LOCK TABLES, keep a TABLE_LIST instance for each table
that may be temporarily closed. For that, implement an own class for
LOCK TABLES mode, Locked_tables_list.
This is a pre-requisite patch for WL#4144.
This is not exactly a backport: there is no new
online ALTER table in Celosia, so the old alter table
code was changed to work with the new table cache API.
Constant expressions in WHERE/HAVING/ON clauses aren't cached and evaluated
for each row. This causes slowdown of query execution especially if constant
UDF/SP function are used.
Now WHERE/HAVING/ON expressions are analyzed in the top-bottom direction with
help of the compile function. When analyzer meets a constant item it
sets a flag for the tree transformer to cache the item and doesn't allow tree
walker to go deeper. Thus, the topmost item of a constant expression if
cached. This is done after all other optimizations were applied to
WHERE/HAVING/ON expressions
A helper function called cache_const_exprs is added to the JOIN class.
It calls compile method with caching analyzer and transformer on WHERE,
HAVING, ON expressions if they're present.
The cache_const_expr_analyzer and cache_const_expr_transformer functions are
added to the Item class. The first one check if the item can be cached and
the second caches it if so.
A new Item_cache_datetime class is derived from the Item_cache class.
It caches both int and string values of the underlying item independently to
avoid DATETIME aware int-to-string conversion. Thus it completely relies on
the ability of the underlying item to correctly convert DATETIME value from
int to string and vice versa.
------------------------------------------------------------
revno: 2630.9.3
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w3
timestamp: Wed 2008-06-11 08:33:36 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
Changed close_cached_tables() not to flush all unused TABLE
instances when flushing individual table.
Renamed expel_table_from_cache() to tdc_remove_table() and
added enum parameter to be able more explicitly specify type
of removal, rewrote its code to be more efficient.
******
Backport of:
------------------------------------------------------------
revno: 2630.9.4
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w3
timestamp: Wed 2008-06-11 15:53:53 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After-review fixes in progress.
Minor changes in order to improve code readability
and simplify debugging.
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/r/show_check.result
Text conflict in mysql-test/r/sp-code.result
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/t/disabled.def
Text conflict in mysql-test/t/show_check.test
Text conflict in mysys/my_delete.c
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/log.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/repl_failsafe.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_parse.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_yacc.yy
Text conflict in storage/myisam/ha_myisam.cc
Corrected results for
stm_auto_increment_bug33029.reject 2009-12-01
20:01:49.000000000 +0300
<andrei> @@ -42,9 +42,6 @@
<andrei> RETURN i;
<andrei> END//
<andrei> CALL p1();
<andrei> -Warnings:
<andrei> -Note 1592 Statement may not be safe to log in statement
format.
<andrei> -Note 1592 Statement may not be safe to log in statement
format.
There should be indeed no Note present because there is in fact autoincrement
top-level query in sp() that triggers inserting in yet another auto-inc table.
(todo: alert DaoGang to improve the test).
----------------------------------------------------------------
2630.4.9 Dmitry Lenev 2008-05-26
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress. Adjusted test case
according to review.
------------------------------------------------------------
revno: 2630.4.7
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sun 2008-05-25 11:19:02 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Fixed silly mistake in test case which caused sporadic
kill.test failures.
Backport of:
------------------------------------------------------------
revno: 2630.4.1
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Fri 2008-05-23 17:54:03 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
------------------------------------------------------------
This is the first patch in series. It transforms the metadata
locking subsystem to use a dedicated module (mdl.h,cc). No
significant changes in the locking protocol.
The import passes the test suite with the exception of
deprecated/removed 6.0 features, and MERGE tables. The latter
are subject to a fix by WL#4144.
Unfortunately, the original changeset comments got lost in a merge,
thus this import has its own (largely insufficient) comments.
This patch fixes Bug#25144 "replication / binlog with view breaks".
Warning: this patch introduces an incompatible change:
Under LOCK TABLES, it's no longer possible to FLUSH a table that
was not locked for WRITE.
Under LOCK TABLES, it's no longer possible to DROP a table or
VIEW that was not locked for WRITE.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.2
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 14:03:45 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.3
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 14:08:51 +0400
message:
WL#3726 "DDL locking for all metadata objects"
Fixed failing Windows builds by adding mdl.cc to the lists
of files needed to build server/libmysqld on Windows.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.4
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 21:57:58 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Fix for assert failures in kill.test which occured when one
tried to kill ALTER TABLE statement on merge table while it
was waiting in wait_while_table_is_used() for other connections
to close this table.
These assert failures stemmed from the fact that cleanup code
in this case assumed that temporary table representing new
version of table was open with adding to THD::temporary_tables
list while code which were opening this temporary table wasn't
always fulfilling this.
This patch changes code that opens new version of table to
always do this linking in. It also streamlines cleanup process
for cases when error occurs while we have new version of table
open.
******
WL#3726 "DDL locking for all metadata objects"
Add libmysqld/mdl.cc to .bzrignore.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.6
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sun 2008-05-25 00:33:22 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Addition to the fix of assert failures in kill.test caused by
changes for this worklog.
Make sure we close the new table only once.
The semisync plugin library names on Unix like systems were prefixed with
'lib', which did not follow the conventions.
Fix the problem by removing the 'lib' prefix on Unix systems.
table .frm file
Problem:
========
Myisampack --join did not create the destination table .frm file.
The user had to copy one of the source table .frm file as destination .frm
file for mysql server to recognize. This is just 'user-friendliness' issue.
How it was solved
=================
After successful join and compression we copy the frm file from the first
source table.
Functionality added
===================
myisampack --join=/path/t3 /path/t1 /path/t2 creates
/path/t3.frm (which is bascially copied from first table's frm /path/t1)
Tests
=====
Modified myisampack.test to test two scenario's
1. Positive myisampack --join test
In this case after the join operation is done,we test if the destination
table is accessible from the server
2. Positive myisampack --join test with an existing .frm file.
We test the above case with an existing .frm file for the destination
table. It should return success even in this case.
3. Positive myisampack --join test with no .frm file for source tables
We test the join operation with no .frm files for source tables. It should
complete the join operation without any warnings and error messages
4. Negative myisampack --join test
We test myisampack --join with existing .MYI,.MDI,.frm files for the
destination table. It should fail with exit status 2 in this case.
