corrupts a MERGE table
Bug 26867 - LOCK TABLES + REPAIR + merge table result in
memory/cpu hogging
Bug 26377 - Deadlock with MERGE and FLUSH TABLE
Bug 25038 - Waiting TRUNCATE
Bug 25700 - merge base tables get corrupted by
optimize/analyze/repair table
Bug 30275 - Merge tables: flush tables or unlock tables
causes server to crash
Bug 19627 - temporary merge table locking
Bug 27660 - Falcon: merge table possible
Bug 30273 - merge tables: Can't lock file (errno: 155)
The problems were:
Bug 26379 - Combination of FLUSH TABLE and REPAIR TABLE
corrupts a MERGE table
1. A thread trying to lock a MERGE table performs busy waiting while
REPAIR TABLE or a similar table administration task is ongoing on
one or more of its MyISAM tables.
2. A thread trying to lock a MERGE table performs busy waiting until all
threads that did REPAIR TABLE or similar table administration tasks
on one or more of its MyISAM tables in LOCK TABLES segments do UNLOCK
TABLES. The difference against problem #1 is that the busy waiting
takes place *after* the administration task. It is terminated by
UNLOCK TABLES only.
3. Two FLUSH TABLES within a LOCK TABLES segment can invalidate the
lock. This does *not* require a MERGE table. The first FLUSH TABLES
can be replaced by any statement that requires other threads to
reopen the table. In 5.0 and 5.1 a single FLUSH TABLES can provoke
the problem.
Bug 26867 - LOCK TABLES + REPAIR + merge table result in
memory/cpu hogging
Trying DML on a MERGE table, which has a child locked and
repaired by another thread, made an infinite loop in the server.
Bug 26377 - Deadlock with MERGE and FLUSH TABLE
Locking a MERGE table and its children in parent-child order
and flushing the child deadlocked the server.
Bug 25038 - Waiting TRUNCATE
Truncating a MERGE child, while the MERGE table was in use,
let the truncate fail instead of waiting for the table to
become free.
Bug 25700 - merge base tables get corrupted by
optimize/analyze/repair table
Repairing a child of an open MERGE table corrupted the child.
It was necessary to FLUSH the child first.
Bug 30275 - Merge tables: flush tables or unlock tables
causes server to crash
Flushing and optimizing locked MERGE children crashed the server.
Bug 19627 - temporary merge table locking
Use of a temporary MERGE table with non-temporary children
could corrupt the children.
Temporary tables are never locked. So we do now prohibit
non-temporary chidlren of a temporary MERGE table.
Bug 27660 - Falcon: merge table possible
It was possible to create a MERGE table with non-MyISAM children.
Bug 30273 - merge tables: Can't lock file (errno: 155)
This was a Windows-only bug. Table administration statements
sometimes failed with "Can't lock file (errno: 155)".
These bugs are fixed by a new implementation of MERGE table open.
When opening a MERGE table in open_tables() we do now add the
child tables to the list of tables to be opened by open_tables()
(the "query_list"). The children are not opened in the handler at
this stage.
After opening the parent, open_tables() opens each child from the
now extended query_list. When the last child is opened, we remove
the children from the query_list again and attach the children to
the parent. This behaves similar to the old open. However it does
not open the MyISAM tables directly, but grabs them from the already
open children.
When closing a MERGE table in close_thread_table() we detach the
children only. Closing of the children is done implicitly because
they are in thd->open_tables.
For more detail see the comment at the top of ha_myisammrg.cc.
Changed from open_ltable() to open_and_lock_tables() in all places
that can be relevant for MERGE tables. The latter can handle tables
added to the list on the fly. When open_ltable() was used in a loop
over a list of tables, the list must be temporarily terminated
after every table for open_and_lock_tables().
table_list->required_type is set to FRMTYPE_TABLE to avoid open of
special tables. Handling of derived tables is suppressed.
These details are handled by the new function
open_n_lock_single_table(), which has nearly the same signature as
open_ltable() and can replace it in most cases.
In reopen_tables() some of the tables open by a thread can be
closed and reopened. When a MERGE child is affected, the parent
must be closed and reopened too. Closing of the parent is forced
before the first child is closed. Reopen happens in the order of
thd->open_tables. MERGE parents do not attach their children
automatically at open. This is done after all tables are reopened.
So all children are open when attaching them.
Special lock handling like mysql_lock_abort() or mysql_lock_remove()
needs to be suppressed for MERGE children or forwarded to the parent.
This depends on the situation. In loops over all open tables one
suppresses child lock handling. When a single table is touched,
forwarding is done.
Behavioral changes:
===================
This patch changes the behavior of temporary MERGE tables.
Temporary MERGE must have temporary children.
The old behavior was wrong. A temporary table is not locked. Hence
even non-temporary children were not locked. See
Bug 19627 - temporary merge table locking.
You cannot change the union list of a non-temporary MERGE table
when LOCK TABLES is in effect. The following does *not* work:
CREATE TABLE m1 ... ENGINE=MRG_MYISAM ...;
LOCK TABLES t1 WRITE, t2 WRITE, m1 WRITE;
ALTER TABLE m1 ... UNION=(t1,t2) ...;
However, you can do this with a temporary MERGE table.
You cannot create a MERGE table with CREATE ... SELECT, neither
as a temporary MERGE table, nor as a non-temporary MERGE table.
CREATE TABLE m1 ... ENGINE=MRG_MYISAM ... SELECT ...;
Gives error message: table is not BASE TABLE.
If a stored function that contains a drop temporary table statement
is invoked by a create temporary table of the same name may cause
a server crash. The problem is that when dropping a table no check
is done to ensure that table is not being used by some outer query
(or outer statement), potentially leaving the outer query with a
reference to a stale (freed) table.
The solution is when dropping a temporary table, always check if
the table is being used by some outer statement as a temporary
table can be dropped inside stored procedures.
The check is performed by looking at the TABLE::query_id value for
temporary tables. To simplify this check and to solve a bug related
to handling of temporary tables in prelocked mode, this patch changes
the way in which this member is used to track the fact that table is
used/unused. Now we ensure that TABLE::query_id is zero for unused
temporary tables (which means that all temporary tables which were
used by a statement should be marked as free for reuse after it's
execution has been completed).
The mysql_change_user command fails to properly update the database pointer
when no database is selected, leading to "use after free" errors. The same
happens on the user privilege pointer in the thread security context.
The solution is to properly reset and update the database name. Also update
the user_priv pointer so that it doesn't point to freed memory.
check_user()/check_connection()/check_for_max_user_connections().
This is a pre-requisite patch for the fix for Bug#12713 "Error in a stored
function called from a SELECT doesn't cause ROLLBACK of statem"
Implement review comments.
The SET PASSWORD statement is non-transactional (no explicit transaction
boundaries) in nature and hence is forbidden inside stored functions and
triggers, but it weren't being effectively forbidden.
The implemented fix is to issue a implicit commit with every SET PASSWORD
statement, effectively prohibiting these statements in stored functions
and triggers.
The general log write function (general_log_print) uses printf style
arguments which need to be pre-processed, meaning that the all arguments
are copied to a single buffer and the problem is that the buffer size is
constant (1022 characters) but queries can be much larger then this.
The solution is to introduce a new log write function that accepts a
buffer and it's length as arguments. The function is to be used when
a formatted output is not required, which is the case for almost all
query write-to-log calls.
This is a incompatible change with respect to the log format of prepared
statements.
The problem was that the RETURNS column in the mysql.proc was of
CHAR(64). That was not enough for storing long-named datatypes.
The fix is to change CHAR(64) to LONGBLOB, and to throw warnings
at the time a stored routine is created if some data is truncated
during writing into mysql.proc.
The root cause of the issue was that the CREATE FUNCTION grammar,
for User Defined Functions, was using the sp_name rule.
The sp_name rule is intended for fully qualified stored procedure names,
like either ident.ident, or just ident but with a default database
implicitly selected.
A UDF does not have a fully qualified name, only a name (ident), and should
not use the sp_name grammar fragment during parsing.
The fix is to re-organize the CREATE FUNCTION grammar, to better separate:
- creating UDF (no definer, can have AGGREGATE, simple ident)
- creating Stored Functions (definer, no AGGREGATE, fully qualified name)
With the test case provided, another issue was exposed which is also fixed:
the DROP FUNCTION statement was using sp_name and also failing when no database
is implicitly selected, when droping UDF functions.
The fix is also to change the grammar so that DROP FUNCTION works with
both the ident.ident syntax (to drop a stored function), or just the ident
syntax (to drop either a UDF or a Stored Function, in the current database)
insert_id after succ. mysql_change_user() call.
See also WL 4066.
This bug reveals two problems:
- the problem on the client side which was described originally;
- the problem in protocol / the server side: connection context
on client and server should be like after mysql_real_connect()
and be consistent. The server however just resets character
set variables to the global defaults.
The fix seems to be as follows:
- extend the protocol so that the client be able to send
character set information in COM_CHANGE_USER command;
- change the server so that it understands client character set
in the command;
- change the client:
- reset character set to the default value (which has been
read from the configuration);
- send character set in COM_CHANGE_USER command.
The problem was that aborted_threads variable was updated
twice when a client connection had been aborted.
The fix is to refactor a code to have aborted_threads updated
only in one place.
UPGRADE)
Bug 17565 (RENAME DATABASE destroys events)
Bug#28360 (RENAME DATABASE destroys routines)
Removed the
RENAME DATABASE db1 TO db2
statement.
Implemented the
ALTER DATABASE db UPGRADE DATA DIRECTORY NAME
statement, which has the same function.
When dumping database from a 4.x server, the mysqldump client
inserted a delimiter sign inside special commentaries of the form:
/*!... CREATE DATABASE IF NOT EXISTS ... ;*/
During restoration that dump file was splitten by delimiter signs on
the client side, and the rest of some commentary strings was prepended
to following statements.
The 4x_server_emul test case option has been added for use with the
DBUG_EXECUTE_IF debugging macro. This option affects debug server
builds only to emulate particular behavior of a 4.x server for
the mysqldump client testing. Non-debugging builds are not affected.
Bug#21422 GRANT/REVOKE possible inside stored function, probably in a trigger
Bug#17244 GRANT gives strange error message when used in a stored function
GRANT/REVOKE statements are non-transactional (no explicit transaction
boundaries) in nature and hence are forbidden inside stored functions and
triggers, but they weren't being effectively forbidden. Furthermore, the
absence of implict commits makes changes made by GRANT/REVOKE statements to
not be rolled back.
The implemented fix is to issue a implicit commit with every GRANT/REVOKE
statement, effectively prohibiting these statements in stored functions
and triggers. The implicit commit also fixes the replication bug, and looks
like being in concert with the behavior of DDL and administrative statements.
Since this is a incompatible change, the following sentence should be
added to the Manual in the very end of the 3rd paragraph, subclause
13.4.3 "Statements That Cause an Implicit Commit": "Beginning with
MySQL 5.0.??, the GRANT and REVOKE statements cause an implicit commit."
Patch contributed by Vladimir Shebordaev
The SELECT query with more than 31 nested dependent SELECT queries returned
wrong result.
New error message has been added: ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT.
It will be reported as: "Too high level of nesting for select".
Problem: thd->thread_specific_used flag is not set executing a statement
containig connection_id() function using PS protocol, that leads to
improper binlog event creation.
Fix: set the flag in the Item_func_connection_id::fix_fields().
