when used with --tab
1) New syntax: added CHARACTER SET clause to the
SELECT ... INTO OUTFILE (to complement the same clause in
LOAD DATA INFILE).
mysqldump is updated to use this in --tab mode.
2) ESCAPED BY/ENCLOSED BY field parameters are documented as
accepting CHAR argument, however SELECT .. INTO OUTFILE
silently ignored rests of multisymbol arguments.
For the symmetrical behavior with LOAD DATA INFILE the
server has been modified to fail with the same error:
ERROR 42000: Field separator argument is not what is
expected; check the manual
3) Current LOAD DATA INFILE recognizes field/line separators
"as is" without converting from client charset to data
file charset. So, it is supposed, that input file of
LOAD DATA INFILE consists of data in one charset and
separators in other charset. For the compatibility with
that [buggy] behaviour SELECT INTO OUTFILE implementation
has been saved "as is" too, but the new warning message
has been added:
Non-ASCII separator arguments are not fully supported
This message warns on field/line separators that contain
non-ASCII symbols.
If using statement based replication (SBR), repeatedly calling
statements which are unsafe for SBR will cause a warning message
to be written to the error for each statement. This might lead
to filling up the error log and there is no way to disable this
behavior.
The solution is to only log these message (about statements unsafe
for statement based replication) if the log_warnings option is set.
For example:
SET GLOBAL LOG_WARNINGS = 0;
INSERT INTO t1 VALUES(UUID());
SET GLOBAL LOG_WARNINGS = 1;
INSERT INTO t1 VALUES(UUID());
In this case the message will be printed only once:
[Warning] Statement may not be safe to log in statement format.
Statement: INSERT INTO t1 VALUES(UUID())
procedures causes crashes!
The problem of that bugreport was mostly fixed by the
patch for bug 38691.
However, attached test case focused on another crash or
valgrind warning problem: SHOW PROCESSLIST query accesses
freed memory of SP instruction that run in a parallel
connection.
Changes of thd->query/thd->query_length in dangerous
places have been guarded with the per-thread
LOCK_thd_data mutex (the THD::LOCK_delete mutex has been
renamed to THD::LOCK_thd_data).
Suppress warnings if binlog_format=STATEMENT and the current
database is filtered out using --binlog-[do|ignore]-db. This
was a regression in my previous patch.
This is a post-push fix addressing review requests and
problems with extra warnings.
Problem 1: The sub-statement where an unsafe warning was detected was
printed as part of the warning. This was ok for statements that
were unsafe due to, e.g., calls to UUID(), but did not make
sense for statements that were unsafe because there was more than
one autoincrement column (unsafeness in this case comes from the
combination of several sub-statements).
Fix 1: Instead of printing the sub-statement, print an explanation
of why the statement is unsafe.
Problem 2:
When a recursive construct (i.e., stored proceure, stored
function, trigger, view, prepared statement) contained several
sub-statements, and at least one of them was unsafe, there would be
one unsafeness warning per sub-statement - even for safe
sub-statements.
Fix 2:
Ensure that each type of warning is printed at most once, by
remembering throughout the execution of the statement which types
of warnings have been printed.
NOTE: This undoes changes by BUG#42829 in sql_class.cc:binlog_query().
I will revert the change in a post-push fix (the binlog filter should
be checked in sql_base.cc:decide_logging_format()).
General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.
Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
- binlog_format=STATEMENT for a row injection
- binlog_format=STATEMENT for an unsafe statement
- binlog_format=STATEMENT for an engine only supporting row logging
- binlog_format=ROW for an engine only supporting statement logging
- statement is unsafe and engine does not support row logging
- row injection in a table that does not support statement logging
- statement modifies one table that does not support row logging and
one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()
Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().
Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.
Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.
format." warnings
Despite the fact that a statement would be filtered out from binlog, a
warning would still be thrown if it was issued with the LIMIT.
This patch addresses this issue by checking the filtering rules before
printing out the warning.
such as quit and shutdown
Logging to slow log can produce an undetermined value for
Rows_examined in special cases. In debug mode this manifests
itself as any of the various marker values used to mark
uninitialized memory on various platforms.
If logging happens on a THD object that hasn't performed any
row reads (on this or any previous connections), the
THD::examined_row_count may be uninitialized. This patch adds
initialization for this attribute.
No automated test cases are added, as for this to be
meaningful, we need to ensure that we're using a THD
fulfilling the above conditions. This is hard to do in the
mysql-test-run framework. The patch has been verified
manually, however, by restarting mysqld and running the test
included with the bug report.
variable. The problem was that THD::connect_utime could be
used without being initialized when the main thread is used
to handle connections (--thread-handling=no-threads).
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
MySQL crashes if a user without proper privileges attempts to create a procedure.
The crash happens because more than one error state is pushed onto the Diagnostic
area. In this particular case the user is denied to implicitly create a new user
account with the implicitly granted privileges ALTER- and EXECUTE ROUTINE.
The new account is needed if the original user account contained a host mask.
A user account with a host mask is a distinct user account in this context.
An alternative would be to first get the most permissive user account which
include the current user connection and then assign privileges to that
account. This behavior change is considered out of scope for this bug patch.
The implicit assignment of privileges when a user creates a stored routine is a
considered to be a feature for user convenience and as such it is not
a critical operation. Any failure to complete this operation is thus considered
non-fatal (an error becomes a warning).
The patch back ports a stack implementation of the internal error handler interface.
This enables the use of multiple error handlers so that it is possible to intercept
and cancel errors thrown by lower layers. This is needed as a error handler already
is used in the call stack emitting the errors which needs to be converted.
Problem: storing "SELECT ... INTO @var ..." results in variables we used val_xxx()
methods which returned results of the current row.
So, in some cases (e.g. SELECT DISTINCT, GROUP BY or HAVING) we got data
from the first row of a new group (where we evaluate a clause) instead of
data from the last row of the previous group.
Fix: use val_xxx_result() counterparts to get proper results.
Mysql server crashes because unsafe statements warning is wrongly elevated to error,
which is set the error status of Diagnostics_area of the thread in THD::binlog_query().
Yet the caller believes that binary logging shouldn't touch the status, so it will
set the status also later by my_ok(), my_error() or my_message() seperately
according to the execution result of the statement or transaction.
But the status of Diagnostics_area of the thread is allowed to set only once.
Fixed to clear the error wrongly set by binary logging, but keep the warning message.
due to name_const substitution
Problem:
"In general, statements executed within a stored procedure
are written to the binary log using the same rules that
would apply were the statements to be executed in standalone
fashion. Some special care is taken when logging procedure
statements because statement execution within procedures
is not quite the same as in non-procedure context".
For example, each reference to a local variable in SP's
statements is replaced by NAME_CONST(var_name, var_value).
Queries like
"CREATE TABLE ... SELECT FUNC(local_var ..."
are logged as
"CREATE TABLE ... SELECT FUNC(NAME_CONST("local_var", var_value) ..."
that leads to differrent field names and
might result in "Incorrect column name" if var_value is long enough.
Fix: in 5.x we'll issue a warning in such a case.
In 6.0 we should get rid of NAME_CONST().
Note: this issue and change should be described in the documentation
("Binary Logging of Stored Programs").
Fine-tuning. Broke out comparison into method by
suggestion of Davi. Clarified comments. Reverting
test-case which I find too brittle; proper test
case in 5.1+.
This is a back port from 5.1 to 5.0.
Fix for BUG 20023: mysql_change_user() resets the value
of SQL_BIG_SELECTS.
The bug was that SQL_BIG_SELECTS was not properly set
in COM_CHANGE_USER.
The fix is to update SQL_BIG_SELECTS properly.
When binlog_format is STATEMENT and the statement is unsafe before,
the unsafe warning/error message was issued without checking
whether the SQL_LOG_BIN was turned on or not.
Fixed with adding a sql_log_bin_toplevel flag in THD to check
whether SQL_LOG_BIN is ON in current session whatever the current is in sp or not.
Problem: storing "SELECT ... INTO @var ..." results in variables we used val_xxx()
methods which returned results of the current row.
So, in some cases (e.g. SELECT DISTINCT, GROUP BY or HAVING) we got data
from the first row of a new group (where we evaluate a clause) instead of
data from the last row of the previous group.
Fix: use val_xxx_result() counterparts to get proper results.
date_format functions
String::realloc() did not check whether the existing string data fits in
the newly allocated buffer for cases when reallocating a String object
with external buffer (i.e.alloced == FALSE). This could lead to memory
overruns in some cases.
The problem is that a unfiltered user query was being passed as
the format string parameter of sql_print_warning which later
performs printf-like formatting, leading to crashes if the user
query contains formatting instructions (ie: %s). Also, it was
using THD::query as the source of the user query, but this
variable is not meaningful in some situations -- in a delayed
insert, it points to the table name.
The solution is to pass the user query as a parameter for the
format string and use the function parameter query_arg as the
source of the user query.
functions
String::realloc() did not check whether the existing string data fits in the newly
allocated buffer for cases when reallocating a String object with external buffer
(i.e.alloced == FALSE). This could lead to memory overruns in some cases.
The function thd_security_context allocates memory on an unprotected MEM_ROOT if the
message length becomes longer than requested and the initial buffer memory needs to
be reallocated.
This patch fixes the design error by copying parts of the reallocated buffer
to the destination buffer. This works because the destination buffer isn't
owned by the String object and thus isn't freed when a new buffer is allocated.
Any new memory allocated by the String object is reclaimed when the object
is destroyed at the end of the function call.
