* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
when used with --tab
1) New syntax: added CHARACTER SET clause to the
SELECT ... INTO OUTFILE (to complement the same clause in
LOAD DATA INFILE).
mysqldump is updated to use this in --tab mode.
2) ESCAPED BY/ENCLOSED BY field parameters are documented as
accepting CHAR argument, however SELECT .. INTO OUTFILE
silently ignored rests of multisymbol arguments.
For the symmetrical behavior with LOAD DATA INFILE the
server has been modified to fail with the same error:
ERROR 42000: Field separator argument is not what is
expected; check the manual
3) Current LOAD DATA INFILE recognizes field/line separators
"as is" without converting from client charset to data
file charset. So, it is supposed, that input file of
LOAD DATA INFILE consists of data in one charset and
separators in other charset. For the compatibility with
that [buggy] behaviour SELECT INTO OUTFILE implementation
has been saved "as is" too, but the new warning message
has been added:
Non-ASCII separator arguments are not fully supported
This message warns on field/line separators that contain
non-ASCII symbols.
client/mysqldump.c:
mysqldump has been updated to call SELECT ... INTO OUTFILE
statement with a charset from the --default-charset command
line parameter.
mysql-test/r/mysqldump.result:
Added test case for bug #30946.
mysql-test/r/outfile_loaddata.result:
Added test case for bug #30946.
mysql-test/t/mysqldump.test:
Added test case for bug #30946.
mysql-test/t/outfile_loaddata.test:
Added test case for bug #30946.
sql/field.cc:
String conversion code has been moved from check_string_copy_error()
to convert_to_printable() for reuse.
sql/share/errmsg.txt:
New WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED message has been added.
sql/sql_class.cc:
The select_export::prepare() method has been modified to:
1) raise the ER_WRONG_FIELD_TERMINATORS error on multisymbol
ENCLOSED BY/ESCAPED BY field arguments like LOAD DATA INFILE;
2) warn with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
message on non-ASCII field or line separators.
The select_export::send_data() merhod has been modified to
convert item data to output charset (see new SELECT INTO OUTFILE
syntax). By default the BINARY charset is used for backward
compatibility.
sql/sql_class.h:
The select_export::write_cs field added to keep output
charset.
sql/sql_load.cc:
mysql_load has been modified to warn on non-ASCII field or
line separators with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
message.
sql/sql_string.cc:
New global function has been added: convert_to_printable()
(common code has been moved from check_string_copy_error()).
sql/sql_string.h:
New String::is_ascii() method and new global convert_to_printable()
function have been added.
sql/sql_yacc.yy:
New syntax: added CHARACTER SET clause to the
SELECT ... INTO OUTFILE (to complement the same clause in
LOAD DATA INFILE). By default the BINARY charset is used for
backward compatibility.
when used with --tab
1) New syntax: added CHARACTER SET clause to the
SELECT ... INTO OUTFILE (to complement the same clause in
LOAD DATA INFILE).
mysqldump is updated to use this in --tab mode.
2) ESCAPED BY/ENCLOSED BY field parameters are documented as
accepting CHAR argument, however SELECT .. INTO OUTFILE
silently ignored rests of multisymbol arguments.
For the symmetrical behavior with LOAD DATA INFILE the
server has been modified to fail with the same error:
ERROR 42000: Field separator argument is not what is
expected; check the manual
3) Current LOAD DATA INFILE recognizes field/line separators
"as is" without converting from client charset to data
file charset. So, it is supposed, that input file of
LOAD DATA INFILE consists of data in one charset and
separators in other charset. For the compatibility with
that [buggy] behaviour SELECT INTO OUTFILE implementation
has been saved "as is" too, but the new warning message
has been added:
Non-ASCII separator arguments are not fully supported
This message warns on field/line separators that contain
non-ASCII symbols.
NOTE: This undoes changes by BUG#42829 in sql_class.cc:binlog_query().
I will revert the change in a post-push fix (the binlog filter should
be checked in sql_base.cc:decide_logging_format()).
NOTE: This undoes changes by BUG#42829 in sql_class.cc:binlog_query().
I will revert the change in a post-push fix (the binlog filter should
be checked in sql_base.cc:decide_logging_format()).
General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.
Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
- binlog_format=STATEMENT for a row injection
- binlog_format=STATEMENT for an unsafe statement
- binlog_format=STATEMENT for an engine only supporting row logging
- binlog_format=ROW for an engine only supporting statement logging
- statement is unsafe and engine does not support row logging
- row injection in a table that does not support statement logging
- statement modifies one table that does not support row logging and
one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()
Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().
Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.
Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.
mysql-test/extra/rpl_tests/create_recursive_construct.inc:
Added auxiliary file used by binlog_unsafe.test to create and
execute recursive constructs
(functions/procedures/triggers/views/prepared statements).
mysql-test/extra/rpl_tests/rpl_foreign_key.test:
removed unnecessary set @@session.binlog_format
mysql-test/extra/rpl_tests/rpl_insert_delayed.test:
Filter out table id from table map events in binlog listing.
Got rid of $binlog_format_statement.
mysql-test/extra/rpl_tests/rpl_ndb_apply_status.test:
disable warnings around call to unsafe procedure
mysql-test/include/rpl_udf.inc:
Disabled warnings for code that generates warnings
for some binlog formats. That would otherwise cause
inconsistencies in the result file.
mysql-test/r/mysqldump.result:
Views are now unsafe if they contain a LIMIT clause.
That fixed BUG#45831. Due to BUG#45832, a warning is
printed for the CREATE VIEW statement.
mysql-test/r/sp_trans.result:
Unsafe statements in stored procedures did not give a warning if
binlog_format=statement. This is BUG#45824. Now they do, so this
result file gets a new warning.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
Error message changed.
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
INSERT DELAYED didn't generate a warning when binlog_format=STATEMENT.
That was BUG#45825. Now there is a warning, so result file needs to be
updated.
mysql-test/suite/binlog/r/binlog_stm_ps.result:
Changed error message.
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file:
- error message changed
- added test for most combinations of unsafe constructs invoked
from recursive constructs
- INSERT DELAYED now gives a warning (because BUG#45826 is fixed)
- INSERT..SELECT..LIMIT now gives a warning from inside recursive
constructs (because BUG#45785 was fixed)
- When a recursive construct (e.g., stored proc or function)
contains more than one statement, at least one of which is
unsafe, then all statements in the recursive construct give
warnings. This is a new bug introduced by this changeset.
It will be addressed in a post-push fix.
mysql-test/suite/binlog/t/binlog_innodb.test:
Changed error code for innodb updates with READ COMMITTED or
READ UNCOMMITTED transaction isolation level and
binlog_format=statement.
mysql-test/suite/binlog/t/binlog_multi_engine.test:
The error code has changed for statements where more than one
engine is involved and one of them is self-logging.
mysql-test/suite/binlog/t/binlog_unsafe-master.opt:
Since binlog_unsafe now tests unsafe-ness of UDF's, we need an extra
flag in the .opt file.
mysql-test/suite/binlog/t/binlog_unsafe.test:
- Clarified comment.
- Rewrote first part of test. Now it tests not only unsafe variables
and functions, but also unsafe-ness due to INSERT..SELECT..LIMIT,
INSERT DELAYED, insert into two autoinc columns, use of UDF's, and
access to log tables in the mysql database.
Also, in addition to functions, procedures, triggers, and prepared
statements, it now also tests views; and it constructs recursive
calls in two levels by combining these recursive constructs.
Part of the logic is in extra/rpl_tests/create_recursive_construct.inc.
- added tests for all special system variables that should not be unsafe.
- added specific tests for BUG#45785 and BUG#45825
mysql-test/suite/rpl/r/rpl_events.result:
updated result file
mysql-test/suite/rpl/r/rpl_extraColmaster_innodb.result:
updated result file
mysql-test/suite/rpl/r/rpl_extraColmaster_myisam.result:
updated result file
mysql-test/suite/rpl/r/rpl_foreign_key_innodb.result:
updated result file
mysql-test/suite/rpl/r/rpl_idempotency.result:
updated result file
mysql-test/suite/rpl/r/rpl_mix_found_rows.result:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). This file equals
the second half of the old rpl_found_rows.result, with the following
modifications:
- minor formatting changes
- additional initialization
mysql-test/suite/rpl/r/rpl_mix_insert_delayed.result:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/r/rpl_rbr_to_sbr.result:
updated result file
mysql-test/suite/rpl/r/rpl_row_idempotency.result:
Moved the second half of rpl_idempotency.test, which only
executed in row mode, to rpl_row_idempotency.test. This is
the new result file.
mysql-test/suite/rpl/r/rpl_row_insert_delayed.result:
Got rid of unnecessary explicit setting of binlog format.
mysql-test/suite/rpl/r/rpl_stm_found_rows.result:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). Changes in
this file:
- minor formatting changes
- warning is now issued for unsafe statements inside procedures
(since BUG#45824 is fixed)
- second half of file is moved to rpl_mix_found_rows.result
mysql-test/suite/rpl/r/rpl_stm_insert_delayed.result:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
error message changed
mysql-test/suite/rpl/r/rpl_temporary_errors.result:
updated result file
mysql-test/suite/rpl/r/rpl_udf.result:
Remove explicit set of binlog format (and triplicate test execution)
and rely on test system executing the test in all binlog formats.
mysql-test/suite/rpl/t/rpl_bug31076.test:
Test is only valid in mixed or row mode since it generates row events.
mysql-test/suite/rpl/t/rpl_events.test:
Removed explicit set of binlog_format and removed duplicate testing.
Instead, we rely on the test system to try all binlog formats.
mysql-test/suite/rpl/t/rpl_extraColmaster_innodb.test:
Removed triplicate testing and instead relying on test system.
Test is only relevant for row format since statement-based replication
cannot handle extra columns on master.
mysql-test/suite/rpl/t/rpl_extraColmaster_myisam.test:
Removed triplicate testing and instead relying on test system.
Test is only relevant for row format since statement-based replication
cannot handle extra columns on master.
mysql-test/suite/rpl/t/rpl_idempotency-slave.opt:
Removed .opt file to avoid server restarts.
mysql-test/suite/rpl/t/rpl_idempotency.test:
- Moved out row-only tests to a new test file, rpl_row_idempotency.test.
rpl_idempotency now only contains tests that execute in all
binlog_formats.
- While I was here, also removed .opt file to avoid server restarts.
The slave_exec_mode is now set inside the test instead.
mysql-test/suite/rpl/t/rpl_mix_found_rows.test:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). This file
contains the second half of the original rpl_found_rows.test with the
follwing changes:
- initialization
- removed SET_BINLOG_FORMAT and added have_binlog_format_mixed.inc
- minor formatting changes
mysql-test/suite/rpl/t/rpl_mix_insert_delayed.test:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/t/rpl_rbr_to_sbr.test:
Test cannot execute in statement mode, since we no longer
switch to row format when binlog_format=statement.
Enforced mixed mode throughout the test.
mysql-test/suite/rpl/t/rpl_row_idempotency.test:
Moved the second half of rpl_idempotency.test, which only
executed in row mode, to this new file. We now rely on the
test system to set binlog format.
mysql-test/suite/rpl/t/rpl_row_insert_delayed.test:
- Got rid of unnecessary explicit setting of binlog format.
- extra/rpl_tests/rpl_insert_delayed.test does not need the
$binlog_format_statement variable any more, so that was
removed.
mysql-test/suite/rpl/t/rpl_slave_skip.test:
The test switches binlog_format internally and master generates both
row and statement events. Hence, the slave must be able to log in both
statement and row format. Hence test was changed to only execute in
mixed mode.
mysql-test/suite/rpl/t/rpl_stm_found_rows.test:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). Changes in
this file:
- minor formatting changes
- added have_binlog_format_statement and removed SET BINLOG_FORMAT.
- second half of file is moved to rpl_mix_found_rows.test
- added cleanup code
mysql-test/suite/rpl/t/rpl_stm_insert_delayed.test:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/t/rpl_switch_stm_row_mixed.test:
The test switches binlog_format internally and master generates both
row and statement events. Hence, the slave must be able to log in both
statement and row format. Hence test was changed to only execute in
mixed mode on slave.
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
Removed explicit set of binlog format. Instead, the test now only
executes in row mode.
mysql-test/suite/rpl/t/rpl_udf.test:
Remove explicit set of binlog format (and triplicate test execution)
and rely on test system executing the test in all binlog formats.
mysql-test/suite/rpl_ndb/combinations:
Added combinations file for rpl_ndb.
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
new result file
mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result:
updated result file
mysql-test/suite/rpl_ndb/t/rpl_ndb_2innodb.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_2myisam.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_basic.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors-master.opt:
new option file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors-slave.opt:
new option file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
New test case to verify all errors and warnings generated by
decide_logging_format.
mysql-test/suite/rpl_ndb/t/rpl_ndb_blob.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_blob2.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_circular.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_circular_simplex.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
While I was here, also made the test clean up after itself.
mysql-test/suite/rpl_ndb/t/rpl_ndb_commit_afterflush.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_ctype_ucs2_def.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_delete_nowhere.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_do_db.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_do_table.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_func003.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_innodb_trans.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_insert_ignore.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_mixed_engines_transactions.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_multi_update3.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_rep_ignore.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_row_001.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_sp003.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_sp006.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_trig004.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/t/partition_innodb_stmt.test:
Changed error code for innodb updates with READ COMMITTED or
READ UNCOMMITTED transaction isolation level and
binlog_format=statement.
sql/event_db_repository.cc:
Use member function to read current_stmt_binlog_row_based.
sql/events.cc:
Use member function to read current_stmt_binlog_row_based.
sql/ha_ndbcluster_binlog.cc:
reset_current_stmt_binlog_row_based() is not a no-op for the ndb_binlog
thread any more. Instead, the ndb_binlog thread now forces row mode both
initially and just after calling mysql_parse. (mysql_parse() is the only
place where reset_current_stmt_binlog_row_based() may be called from
the ndb_binlog thread, so these are the only two places that need to
change.)
sql/ha_partition.cc:
Use member function to read current_stmt_binlog_row_based.
sql/handler.cc:
Use member function to read current_stmt_binlog_row_based.
sql/item_create.cc:
Added DBUG_ENTER to some functions, to be able to trace when
set_stmt_unsafe is called.
sql/log.cc:
Use member function to read current_stmt_binlog_row_based.
sql/log_event.cc:
- Moved logic for changing to row format out of do_apply_event (and into
decide_logging_format).
- Added @todo comment for post-push cleanup.
sql/log_event_old.cc:
Move logic for changing to row format out of do_apply_event (and into
decide_logging_format).
sql/mysql_priv.h:
Make decide_logging_format() a member of the THD class, for two reasons:
- It is natural from an object-oriented perspective.
- decide_logging_format() needs to access private members of THD
(specifically, the new binlog_warning_flags field).
sql/rpl_injector.cc:
Removed call to set_current_stmt_binlog_row_based().
From now on, only decide_logging_fromat is allowed to modify
current_stmt_binlog_row_based. This call is from the ndb_binlog
thread, mostly executing code in ha_ndbcluster_binlog.cc.
This call can be safely removed, because:
- current_stmt_binlog_row_based is initialized for the ndb_binlog
thread's THD object when the THD object is created. So we're
not going to read uninitialized memory.
- The behavior of ndb_binlog thread does not use the state of the
current_stmt_binlog_row_based. It is conceivable that the
ndb_binlog thread would rely on the current_stmt_binlog_format
in two situations:
(1) when it calls mysql_parse;
(2) when it calls THD::binlog_query.
In case (1), it always clears THD::options&OPTION_BIN_LOG (because
run_query() in ha_ndbcluster_binlog.cc is only called with
disable_binlogging = TRUE).
In case (2), it always uses qtype=STMT_QUERY_TYPE.
sql/set_var.cc:
Added @todo comment for post-push cleanup.
sql/share/errmsg.txt:
Added new error messages and clarified ER_BINLOG_UNSAFE_STATEMENT.
sql/sp.cc:
Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
Got rid of MYSQL_QUERY_TYPE: it was equivalent to STMT_QUERY_TYPE.
sql/sp_head.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sp_head.h:
Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
sql/sql_acl.cc:
Got rid of MYSQL_QUERY_TYPE: it was equivalent to STMT_QUERY_TYPE.
sql/sql_base.cc:
- Made decide_logging_format take care of all logic for deciding the
logging format, and for determining the related warnings and errors.
See comment above decide_logging_format for details.
- Made decide_logging_format a member function of THD, since it needs
to access private members of THD and since its purpose is to update
the state of a THD object.
- Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
sql/sql_class.cc:
- Moved logic for determining unsafe warnings away from THD::binlog_query
(and into decide_logging_format()). Now, it works like this:
1. decide_logging_format detects that the current statement shall
produce a warning, if it ever makes it to the binlog
2. decide_logging_format sets a flag of THD::binlog_warning_flags.
3. THD::binlog_query reads the flag. If the flag is set, it generates
a warning.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_class.h:
- Added THD::binlog_warning_flags (see sql_class.cc for explanation).
- Made decide_logging_format() and reset_for_next_command() member
functions of THD (instead of standalone functions). This was needed
for two reasons: (1) the functions need to access the private member
THD::binlog_warning_flags; (2) the purpose of these functions is to
update the staet of a THD object, so from an object-oriented point
of view they should be member functions.
- Encapsulated current_stmt_binlog_row_based, so it is now private and
can only be accessed from a member function. Also changed the
data type to an enumeration instead of a bool.
- Removed MYSQL_QUERY_TYPE, because it was equivalent to
STMT_QUERY_TYPE anyways.
- When reset_current_stmt_binlog_row_based was called from the
ndb_binlog thread, it would behave as a no-op. This special
case has been removed, and the behavior of
reset_current_stmt_binlog_row_based does not depend on which thread
calls it any more. The special case did not serve any purpose,
since the ndb binlog thread did not take the
current_stmt_binlog_row_based flag into account anyways.
sql/sql_delete.cc:
- Moved logic for setting row format for DELETE..LIMIT away from
mysql_prepare_delete.
(Instead, we mark the statement as unsafe at parse time (sql_yacc.yy)
and rely on decide_logging_format() (sql_class.cc) to set row format.)
This is part of the fix for BUG#45831.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_insert.cc:
- Removed unnecessary calls to thd->lex->set_stmt_unsafe() and
thd->set_current_stmt_binlog_row_based_if_mixed() from
handle_delayed_insert(). The calls are unnecessary because they
have already been made; they were made in the constructor of
the `di' object.
- Since decide_logging_format() is now a member function of THD, code
that calls decide_logging_format() had to be updated.
- Added DBUG_ENTER call, to be able to trace when set_stmt_unsafe is
called.
- Moved call to set_stmt_unsafe() for INSERT..SELECT..LIMIT away from
mysql_insert_select_prepare() (and into decide_logging_format).
This is part of the fix for BUG#45831.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_lex.h:
- Added the flag BINLOG_STMT_FLAG_ROW_INJECTION to enum_binlog_stmt_flag.
This was necessary so that a statement can identify itself as a row
injection.
- Added appropriate setter and getter functions for the new flag.
- Added or clarified some comments.
- Added DBUG_ENTER()
sql/sql_load.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_parse.cc:
- Made mysql_reset_thd_for_next_command() clear thd->binlog_warning_flags.
- Since thd->binlog_warning_flags is private, it must be set in a
member function of THD. Hence, moved the body of
mysql_reset_thd_for_next_command() to the new member function
THD::reset_thd_for_next_command(), and made
mysql_reset_thd_for_next_command() call
THD::reset_thd_for_next_command().
- Removed confusing comment.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_repl.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_table.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_udf.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_update.cc:
Moved logic for setting row format for UPDATE..LIMIT away from
mysql_prepare_update.
(Instead, we mark the statement as unsafe at parse time (sql_yacc.yy)
and rely on decide_logging_format() (sql_class.cc) to set row format.)
This is part of the fix for BUG#45831.
sql/sql_yacc.yy:
Made INSERT DELAYED, INSERT..SELECT..LIMIT, UPDATE..LIMIT, and
DELETE..LIMIT mark themselves as unsafe at parse time (instead
of at execution time).
This is part of the fixes BUG#45831 and BUG#45825.
storage/example/ha_example.cc:
Made exampledb accept inserts. This was needed by the new test case
rpl_ndb_binlog_format_errors, because it needs an engine that
is statement-only (and accepts inserts).
storage/example/ha_example.h:
Made exampledb a statement-only engine instead of a row-only engine.
No existing test relied exampledb's row-only capabilities. The new
test case rpl_ndb_binlog_format_errors needs an engine that is
statement-only.
storage/innobase/handler/ha_innodb.cc:
- Changed error error code and message given by innodb when
binlog_format=STATEMENT and transaction isolation level is
READ COMMITTED or READ UNCOMMITTED.
- While I was here, also simplified the condition for
checking when to give the error.
General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.
Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
- binlog_format=STATEMENT for a row injection
- binlog_format=STATEMENT for an unsafe statement
- binlog_format=STATEMENT for an engine only supporting row logging
- binlog_format=ROW for an engine only supporting statement logging
- statement is unsafe and engine does not support row logging
- row injection in a table that does not support statement logging
- statement modifies one table that does not support row logging and
one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()
Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().
Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.
Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
sql/log_event.cc:
Changed constructors of Query_log_event and Execute_load_log_event to accept the error code argument instead of figuring it out by itself
sql/log_event.h:
Changed constructors of Query_log_event and Execute_load_log_event to accept the error code argument
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
mysql-test/r/information_schema.result:
Fixed a result file.
mysql-test/r/innodb-autoinc.result:
Fixed a result file.
mysql-test/t/connect.test:
Fixed a problem with merge, needed to close
connections and use the default
If secure-file-priv was set on slave, it became unable to execute
LOAD DATA INFILE statements sent from master using mixed or
statement-based replication.
This patch fixes the issue by ignoring this security restriction
and checking if the files are created and read by the slave in the
--slave-load-tmpdir while executing the SQL Thread.
If secure-file-priv was set on slave, it became unable to execute
LOAD DATA INFILE statements sent from master using mixed or
statement-based replication.
This patch fixes the issue by ignoring this security restriction
and checking if the files are created and read by the slave in the
--slave-load-tmpdir while executing the SQL Thread.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Added braces around expressions with &&, ||, & and |
- Added empty line before ; for empty while and for loops
- Added () around if with assignments
- Removed const before function returning simple type
Changed BUILD scripts to not build with NDB
BUILD/SETUP.sh:
By default, don't build ndb with --max in Maria tree.
NDB is not kept up to date anyway in 5.1
client/mysql.cc:
Added braces around && to get rid of compiler warnings
sql/event_db_repository.cc:
Added braces around && to get rid of compiler warnings
sql/events.cc:
Added braces around && to get rid of compiler warnings
sql/field.cc:
Added braces around && to get rid of compiler warnings
Fixed for loops
sql/field.h:
Added braces around & to get rid of compiler warnings
sql/field_conv.cc:
Added braces around && to get rid of compiler warnings
Fixed bug when copying between DATETIME fields and strict dates are used
Removed not needeed else
sql/gstream.cc:
Added braces around && to get rid of compiler warnings
sql/ha_ndbcluster.cc:
Added braces around && to get rid of compiler warnings
Added {} to get rid of compiler warnings
sql/handler.cc:
Added braces around && to get rid of compiler warnings
sql/item.cc:
Added braces around && to get rid of compiler warnings
sql/item_cmpfunc.cc:
Added braces around && to get rid of compiler warnings
Removed some not needed space
sql/item_func.cc:
Added braces around && to get rid of compiler warnings
sql/item_strfunc.cc:
Added braces around && to get rid of compiler warnings
sql/item_subselect.cc:
Added braces around && to get rid of compiler warnings
sql/item_sum.cc:
Added braces around && to get rid of compiler warnings
sql/item_timefunc.cc:
Added braces around && to get rid of compiler warnings
sql/item_xmlfunc.cc:
Added empty line before ; for empty while and for loops
sql/log.cc:
Added braces around && to get rid of compiler warnings
sql/log_event.cc:
Added braces around && to get rid of compiler warnings
Removed not needed else
sql/log_event_old.cc:
Added braces around && to get rid of compiler warnings
sql/opt_range.cc:
Added braces around && to get rid of compiler warnings
sql/opt_sum.cc:
Added braces around && to get rid of compiler warnings
sql/set_var.cc:
Added empty line before ; for empty while and for loops
Added () around if with assignments
sql/slave.cc:
Added braces around && to get rid of compiler warnings
Added empty line before ; for empty while and for loops
sql/spatial.h:
Added braces around && to get rid of compiler warnings
sql/sql_acl.cc:
Added braces around && to get rid of compiler warnings
sql/sql_analyse.cc:
Added empty line before ; for empty while and for loops
sql/sql_base.cc:
Added braces around && to get rid of compiler warnings
sql/sql_connect.cc:
Added braces around && to get rid of compiler warnings
sql/sql_db.cc:
Added braces around && to get rid of compiler warnings
sql/sql_delete.cc:
Added braces around && to get rid of compiler warnings
sql/sql_help.cc:
Added empty line before ; for empty while and for loops
sql/sql_insert.cc:
Added braces around && to get rid of compiler warnings
Added () around if with assignments
sql/sql_lex.cc:
Cast char array references to uchar; Fixed wrong array referencing when using characters > ASCII 128 in SQL statments
Added empty line before ; for empty while and for loops
Trivial indent fixes
Added braces around && to get rid of compiler warnings
sql/sql_load.cc:
Added braces around && to get rid of compiler warnings
sql/sql_parse.cc:
Added braces around && to get rid of compiler warnings
sql/sql_partition.cc:
Added braces around && to get rid of compiler warnings
sql/sql_plugin.cc:
Fixed bug in detecing if option variable should be readonly
Added empty line before ; for empty while and for loops
sql/sql_prepare.cc:
Added braces around && to get rid of compiler warnings
sql/sql_select.cc:
Added braces around && to get rid of compiler warnings
Added () around if with assignments
Added empty line before ; for empty while and for loops
sql/sql_show.cc:
Added braces around && to get rid of compiler warnings
sql/sql_table.cc:
Added braces around && to get rid of compiler warnings
sql/sql_trigger.cc:
Added braces around && to get rid of compiler warnings
sql/sql_update.cc:
Added braces around && to get rid of compiler warnings
sql/sql_yacc.yy:
Added braces around && to get rid of compiler warnings
sql/table.cc:
Added braces around && to get rid of compiler warnings
sql/table.h:
Added braces around && to get rid of compiler warnings
sql/time.cc:
Added braces around && to get rid of compiler warnings
sql/tztime.cc:
Added braces around && to get rid of compiler warnings
sql/uniques.cc:
Added braces around && to get rid of compiler warnings
storage/federated/ha_federated.cc:
Fixed bug in testing of variable to ha_info() (Not critical)
storage/heap/ha_heap.cc:
Added braces around && to get rid of compiler warnings
storage/maria/ha_maria.cc:
Fixed bug: Mark that maria_log_dir_path is readonly
Added braces around && to get rid of compiler warnings
storage/ndb/include/ndbapi/NdbEventOperation.hpp:
Removed const before function returning simple type
storage/ndb/include/ndbapi/NdbOperation.hpp:
Removed const before function returning simple type
storage/ndb/src/ndbapi/Ndb.cpp:
Added empty line before ; for empty while and for loops
storage/ndb/src/ndbapi/NdbEventOperation.cpp:
Removed const before function returning simple type
storage/ndb/src/ndbapi/NdbEventOperationImpl.cpp:
Removed const before function returning simple type
storage/ndb/src/ndbapi/NdbEventOperationImpl.hpp:
Removed const before function returning simple type
storage/ndb/src/ndbapi/NdbRecAttr.cpp:
Added empty line before ; for empty while and for loops
storage/ndb/src/ndbapi/TransporterFacade.hpp:
Added braces around && to get rid of compiler warnings
NO_BACKSLASH_ESCAPES was not heeded in LOAD DATA INFILE
and SELECT INTO OUTFILE. It is now.
mysql-test/r/loaddata.result:
Show that SQL-mode NO_BACKSLASH_ESCAPES is heeded in
INFILE/OUTFILE, and that dump/restore cycles work!
mysql-test/t/loaddata.test:
Show that SQL-mode NO_BACKSLASH_ESCAPES is heeded in
INFILE/OUTFILE, and that dump/restore cycles work!
sql/sql_class.cc:
Add function to enquire whether ESCAPED BY was given.
When doing SELECT...OUTFILE, use ESCAPED BY if specifically
given; otherwise use sensible default value depending on
SQL-mode features NO_BACKSLASH_ESCAPES.
sql/sql_class.h:
Add function to enquire whether ESCAPED BY was given.
sql/sql_load.cc:
When doing LOAD DATA INFILE, use ESCAPED BY if specifically
given; otherwise use sensible default value depending on
SQL-mode features NO_BACKSLASH_ESCAPES.
into mysql.com:/home/my/mysql-maria
mysql-test/r/maria.result:
Auto merged
mysql-test/suite/ndb/r/ndb_auto_increment.result:
Auto merged
mysql-test/t/maria.test:
Auto merged
mysys/hash.c:
Auto merged
mysys/thr_lock.c:
Auto merged
sql/field.cc:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/ha_ndbcluster.h:
Auto merged
sql/ha_partition.cc:
Auto merged
sql/ha_partition.h:
Auto merged
sql/handler.cc:
Auto merged
sql/handler.h:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/mysqld.cc:
Auto merged
sql/protocol.cc:
Auto merged
sql/sql_load.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/csv/ha_tina.cc:
Auto merged
storage/federated/ha_federated.cc:
Auto merged
storage/maria/Makefile.am:
Auto merged
storage/maria/ma_check.c:
Auto merged
storage/maria/ma_control_file.c:
Auto merged
storage/maria/ma_delete_all.c:
Auto merged
storage/maria/ma_dynrec.c:
Auto merged
storage/maria/ma_init.c:
Auto merged
storage/maria/ma_key_recover.c:
Auto merged
storage/maria/ma_open.c:
Auto merged
storage/maria/ma_page.c:
Auto merged
storage/maria/ma_range.c:
Auto merged
storage/maria/ma_recovery.c:
Auto merged
storage/maria/ma_test1.c:
Auto merged
storage/maria/maria_read_log.c:
Auto merged
storage/maria/unittest/ma_test_all-t:
Auto merged
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Auto merged
storage/maria/unittest/ma_test_recovery.pl:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
storage/myisam/myisamdef.h:
Auto merged
include/my_base.h:
Manual merge where error code are kept same as in 5.1
mysys/my_handler.c:
No changes
sql/item.cc:
Manual merge
sql/sql_class.cc:
Manual merge
sql/sql_insert.cc:
Manual merge
storage/maria/ha_maria.cc:
Manual merge
storage/maria/ma_blockrec.c:
Manual merge
storage/maria/ma_delete.c:
Manual merge
storage/maria/ma_write.c:
Manual merge
Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation
Fixed wrong call to strmake
Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert
Allow storing year 2155 in year field
When running with purify/valgrind avoid copying structures over themself
Added hook 'trnnam_end_trans_hook' that is called when transaction ends
Added trn->used_tables that is used to an entry for all tables used by transaction
Fixed that ndb doesn't crash on duplicate key error when start_bulk_insert/end_bulk_insert are not called
include/maria.h:
Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation
include/my_tree.h:
Added macro 'reset_free_element()' to be able to ignore calls to the external free function.
Is used to optimize end-bulk-insert in case of failures, in which case we don't want write the remaining keys in the tree
mysql-test/install_test_db.sh:
Upgrade to new mysql_install_db options
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria.result:
New tests
mysql-test/suite/ndb/r/ndb_auto_increment.result:
Fixed error message now when bulk insert is not always called
mysql-test/suite/ndb/t/ndb_auto_increment.test:
Fixed error message now when bulk insert is not always called
mysql-test/t/maria-mvcc.test:
Added testing of versioning of count(*)
mysql-test/t/maria-page-checksum.test:
Added comment
mysql-test/t/maria.test:
More tests
mysys/hash.c:
Code style change
sql/field.cc:
Allow storing year 2155 in year field
sql/ha_ndbcluster.cc:
Added new argument to end_bulk_insert() to signal if the bulk insert should ignored
sql/ha_ndbcluster.h:
Added new argument to end_bulk_insert() to signal if the bulk insert should ignored
sql/ha_partition.cc:
Added new argument to end_bulk_insert() to signal if the bulk insert should ignored
sql/ha_partition.h:
Added new argument to end_bulk_insert() to signal if the bulk insert should ignored
sql/handler.cc:
Don't call get_dup_key() if there is no table object. This can happen if the handler generates a duplicate key error on commit
sql/handler.h:
Added new argument to end_bulk_insert() to signal if the bulk insert should ignored (ie, the table will be deleted)
sql/item.cc:
Style fix
Removed compiler warning
sql/log_event.cc:
Added new argument to ha_end_bulk_insert()
sql/log_event_old.cc:
Added new argument to ha_end_bulk_insert()
sql/mysqld.cc:
Removed compiler warning
sql/protocol.cc:
Added DBUG
sql/sql_class.cc:
Added DBUG
Fixed wrong call to strmake
sql/sql_insert.cc:
Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert involves a lot of if's)
Added new argument to ha_end_bulk_insert()
sql/sql_load.cc:
Added new argument to ha_end_bulk_insert()
sql/sql_parse.cc:
Style fixes
Avoid goto in common senario
sql/sql_select.cc:
When running with purify/valgrind avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings
sql/sql_select.h:
Avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings
sql/sql_table.cc:
Call HA_EXTRA_PREPARE_FOR_DROP if table created by ALTER TABLE is going to be dropped
Added new argument to ha_end_bulk_insert()
storage/archive/ha_archive.cc:
Added new argument to end_bulk_insert()
storage/archive/ha_archive.h:
Added new argument to end_bulk_insert()
storage/federated/ha_federated.cc:
Added new argument to end_bulk_insert()
storage/federated/ha_federated.h:
Added new argument to end_bulk_insert()
storage/maria/Makefile.am:
Added ma_state.c and ma_state.h
storage/maria/ha_maria.cc:
Versioning of count(*) and checksum
- share->state.state is now assumed to be correct, not handler->state
- Call _ma_setup_live_state() in external lock to get count(*)/checksum versioning. In case of
not versioned and not concurrent insertable table, file->s->state.state contains the correct state information
Other things:
- file->s -> share
- Added DBUG_ASSERT() for unlikely case
- Optimized end_bulk_insert() to not write anything if table is going to be deleted (as in failed alter table)
- Indentation changes in external_lock becasue of removed 'goto' caused a big conflict even if very little was changed
storage/maria/ha_maria.h:
New argument to end_bulk_insert()
storage/maria/ma_blockrec.c:
Update for versioning of count(*) and checksum
Keep share->state.state.data_file_length up to date (not info->state->data_file_length)
Moved _ma_block_xxxx_status() and maria_versioning() functions to ma_state.c
storage/maria/ma_check.c:
Update and use share->state.state instead of info->state
info->s to share
Update info->state at end of repair
Call _ma_reset_state() to update share->state_history at end of repair
storage/maria/ma_checkpoint.c:
Call _ma_remove_not_visible_states() on checkpoint to clean up not visible state history from tables
storage/maria/ma_close.c:
Remember state history for running transaction even if table is closed
storage/maria/ma_commit.c:
Ensure we always call trnman_commit_trn() even if other calls fails. If we don't do that, the translog and state structures will not be freed
storage/maria/ma_delete.c:
Versioning of count(*) and checksum:
- Always update info->state->checksum and info->state->records
storage/maria/ma_delete_all.c:
Versioning of count(*) and checksum:
- Ensure that share->state.state is updated, as here is where we store the primary information
storage/maria/ma_dynrec.c:
Use lock_key_trees instead of concurrent_insert to check if trees should be locked.
This allows us to lock trees both for concurrent_insert and for index versioning.
storage/maria/ma_extra.c:
Versioning of count(*) and checksum:
- Use share->state.state instead of info->state
- share->concurrent_insert -> share->non_transactional_concurrent_insert
- Don't update share->state.state from info->state if transactional table
Optimization:
- Don't flush io_cache or bitmap if we are using FLUSH_IGNORE_CHANGED
storage/maria/ma_info.c:
Get most state information from current state
storage/maria/ma_init.c:
Add hash table and free function to store states for closed tables
Install hook for transaction commit/rollback to update history state
storage/maria/ma_key_recover.c:
Versioning of count(*) and checksum:
- Use share->state.state instead of info->state
storage/maria/ma_locking.c:
Versioning of count(*) and checksum:
- Call virtual functions (if exists) to restore/update status
- Move _ma_xxx_status() functions to ma_state.c
info->s -> share
storage/maria/ma_open.c:
Versioning of count(*) and checksum:
- For not transactional tables, set info->state to point to new allocated state structure.
- Initialize new info->state_start variable that points to state at start of transaction
- Copy old history states from hash table (maria_stored_states) first time the table is opened
- Split flag share->concurrent_insert to non_transactional_concurrent_insert & lock_key_tree
- For now, only enable versioning of tables without keys (to be fixed in soon!)
- Added new virtual function to restore status in maria_lock_database)
More DBUG
storage/maria/ma_page.c:
Versioning of count(*) and checksum:
- Use share->state.state instead of info->state
- Modify share->state.state.key_file_length under share->intern_lock
storage/maria/ma_range.c:
Versioning of count(*) and checksum:
- Lock trees based on share->lock_key_trees
info->s -> share
storage/maria/ma_recovery.c:
Versioning of count(*) and checksum:
- Use share->state.state instead of info->state
- Update state information on close and when reenabling logging
storage/maria/ma_rkey.c:
Versioning of count(*) and checksum:
- Lock trees based on share->lock_key_trees
storage/maria/ma_rnext.c:
Versioning of count(*) and checksum:
- Lock trees based on share->lock_key_trees
storage/maria/ma_rnext_same.c:
Versioning of count(*) and checksum:
- Lock trees based on share->lock_key_trees
- Only skip rows based on file length if non_transactional_concurrent_insert is set
storage/maria/ma_rprev.c:
Versioning of count(*) and checksum:
- Lock trees based on share->lock_key_trees
storage/maria/ma_rsame.c:
Versioning of count(*) and checksum:
- Lock trees based on share->lock_key_trees
storage/maria/ma_sort.c:
Use share->state.state instead of info->state
Fixed indentation
storage/maria/ma_static.c:
Added maria_stored_state
storage/maria/ma_update.c:
Versioning of count(*) and checksum:
- Always update info->state->checksum and info->state->records
- Remove optimization for index file update as it doesn't work for transactional tables
storage/maria/ma_write.c:
Versioning of count(*) and checksum:
- Always update info->state->checksum and info->state->records
storage/maria/maria_def.h:
Move MARIA_STATUS_INFO to ma_state.h
Changes to MARIA_SHARE:
- Added state_history to store count(*)/checksum states
- Added in_trans as counter if table is used by running transactions
- Split concurrent_insert into lock_key_trees and on_transactional_concurrent_insert.
- Added virtual function lock_restore_status
Changes to MARIA_HA:
- save_state -> state_save
- Added state_start to store state at start of transaction
storage/maria/maria_pack.c:
Versioning of count(*) and checksum:
- Use share->state.state instead of info->state
Indentation fixes
storage/maria/trnman.c:
Added hook 'trnnam_end_trans_hook' that is called when transaction ends
Added trn->used_tables that is used to an entry for all tables used by transaction
More DBUG
Changed return type of trnman_end_trn() to my_bool
Added trnman_get_min_trid() to get minimum trid in use.
Added trnman_exists_active_transactions() to check if there exist a running transaction started between two commit id
storage/maria/trnman.h:
Added 'used_tables'
Moved all pointers into same groups to get better memory alignment
storage/maria/trnman_public.h:
Added prototypes for new functions and variables
Chagned return type of trnman_end_trn() to my_bool
storage/myisam/ha_myisam.cc:
Added argument to end_bulk_insert() if operation should be aborted
storage/myisam/ha_myisam.h:
Added argument to end_bulk_insert() if operation should be aborted
storage/maria/ma_state.c:
Functions to handle state of count(*) and checksum
storage/maria/ma_state.h:
Structures and declarations to handle state of count(*) and checksum
into mysql.com:/home/my/mysql-new
BitKeeper/etc/ignore:
auto-union
BUILD/SETUP.sh:
Auto merged
CMakeLists.txt:
Auto merged
client/get_password.c:
Auto merged
client/mysqldump.c:
Auto merged
client/mysqltest.c:
Auto merged
cmd-line-utils/readline/bind.c:
Auto merged
cmd-line-utils/readline/display.c:
Auto merged
cmd-line-utils/readline/histexpand.c:
Auto merged
cmd-line-utils/readline/history.c:
Auto merged
cmd-line-utils/readline/readline.c:
Auto merged
cmd-line-utils/readline/text.c:
Auto merged
dbug/user.r:
Auto merged
extra/yassl/src/handshake.cpp:
Auto merged
include/config-win.h:
Auto merged
include/m_string.h:
Auto merged
include/my_global.h:
Auto merged
include/my_pthread.h:
Auto merged
include/mysql/plugin.h:
Auto merged
include/mysql_com.h:
Auto merged
include/thr_alarm.h:
Auto merged
libmysql/CMakeLists.txt:
Auto merged
libmysql/Makefile.shared:
Auto merged
libmysql/dll.c:
Auto merged
libmysql/get_password.c:
Auto merged
libmysql/libmysql.c:
Auto merged
libmysqld/Makefile.am:
Auto merged
mysql-test/lib/mtr_cases.pl:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/r/alter_table.result:
Auto merged
mysql-test/r/change_user.result:
Auto merged
mysql-test/r/create.result:
Auto merged
mysql-test/r/innodb.result:
Auto merged
mysql-test/r/merge.result:
Auto merged
mysql-test/r/mix2_myisam.result:
Auto merged
mysql-test/r/mysqldump.result:
Auto merged
mysql-test/r/query_cache.result:
Auto merged
mysql-test/r/subselect.result:
Auto merged
mysql-test/valgrind.supp:
Auto merged
mysql-test/r/view.result:
Auto merged
mysql-test/suite/rpl/r/rpl_events.result:
Auto merged
mysql-test/suite/rpl/t/rpl_switch_stm_row_mixed.test:
Auto merged
mysql-test/t/create.test:
Auto merged
mysql-test/t/mysqldump.test:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
mysql-test/t/view.test:
Auto merged
mysys/mf_iocache.c:
Auto merged
mysys/mf_tempfile.c:
Auto merged
mysys/my_atomic.c:
Auto merged
mysys/my_bit.c:
Auto merged
mysys/my_bitmap.c:
Auto merged
mysys/my_compress.c:
Auto merged
mysys/my_create.c:
Auto merged
mysys/my_delete.c:
Auto merged
mysys/my_error.c:
Auto merged
mysys/my_init.c:
Auto merged
mysys/my_open.c:
Auto merged
mysys/my_realloc.c:
Auto merged
mysys/my_rename.c:
Auto merged
mysys/my_symlink.c:
Auto merged
mysys/my_sync.c:
Auto merged
mysys/my_thr_init.c:
Auto merged
mysys/thr_alarm.c:
Auto merged
mysys/thr_lock.c:
Auto merged
scripts/make_binary_distribution.sh:
Auto merged
server-tools/instance-manager/mysql_connection.cc:
Auto merged
sql/CMakeLists.txt:
Auto merged
sql/Makefile.am:
Auto merged
sql/events.cc:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/filesort.cc:
Auto merged
sql/gen_lex_hash.cc:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/ha_partition.h:
Auto merged
sql/handler.h:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_strfunc.cc:
Auto merged
sql/item_strfunc.h:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/lock.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/net_serv.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/partition_info.cc:
Auto merged
sql/rpl_injector.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/slave.cc:
Auto merged
sql/slave.h:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_cache.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_delete.cc:
Auto merged
sql/sql_load.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_prepare.cc:
Auto merged
sql/sql_repl.cc:
Auto merged
sql/sql_test.cc:
Auto merged
sql/sql_union.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/table.cc:
Auto merged
sql/table.h:
Auto merged
sql/unireg.cc:
Auto merged
sql/share/errmsg.txt:
Auto merged
storage/csv/ha_tina.cc:
Auto merged
storage/csv/ha_tina.h:
Auto merged
storage/myisam/CMakeLists.txt:
Auto merged
storage/myisam/ft_boolean_search.c:
Auto merged
storage/myisam/ft_eval.c:
Auto merged
storage/myisam/ft_nlq_search.c:
Auto merged
storage/myisam/ft_parser.c:
Auto merged
storage/myisam/ft_static.c:
Auto merged
storage/myisam/ft_stopwords.c:
Auto merged
storage/myisam/ft_test1.c:
Auto merged
storage/myisam/ft_update.c:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
storage/myisam/mi_check.c:
Auto merged
storage/myisam/mi_create.c:
Auto merged
storage/myisam/mi_delete.c:
Auto merged
storage/myisam/mi_delete_all.c:
Auto merged
storage/myisam/mi_dynrec.c:
Auto merged
storage/myisam/mi_key.c:
Auto merged
storage/myisam/mi_packrec.c:
Auto merged
storage/myisam/mi_range.c:
Auto merged
storage/myisam/mi_search.c:
Auto merged
storage/myisam/mi_test1.c:
Auto merged
storage/myisam/mi_test2.c:
Auto merged
storage/myisam/mi_test3.c:
Auto merged
storage/myisam/mi_unique.c:
Auto merged
storage/myisam/mi_write.c:
Auto merged
storage/myisam/myisamchk.c:
Auto merged
storage/myisam/myisamdef.h:
Auto merged
storage/myisam/myisampack.c:
Auto merged
storage/myisam/sort.c:
Auto merged
storage/myisam/sp_test.c:
Auto merged
support-files/mysql.spec.sh:
Auto merged
tests/mysql_client_test.c:
Auto merged
configure.in:
Manual merge
dbug/dbug.c:
Restore to original state in Maria tree
The big diff comes from a wrong pull from 5.0 -> 5.1 after backporting dbug to 5.0 from 5.1
include/Makefile.am:
Manual merge
include/my_atomic.h:
Ignore changes
include/my_base.h:
Manual merge
include/my_dbug.h:
Use orginal my_dbug.h from maria tree
include/my_handler.h:
Manual merge
include/my_sys.h:
Manual merge
include/myisam.h:
Manual merge
mysql-test/lib/mtr_report.pl:
Manual merge
mysql-test/r/myisam.result:
Manual merge
mysql-test/suite/binlog/r/binlog_unsafe.result:
Manual merge
mysql-test/suite/binlog/t/binlog_unsafe.test:
Manual merge
mysql-test/suite/rpl/r/rpl_row_flsh_tbls.result:
Manual merge
mysql-test/suite/rpl/r/rpl_switch_stm_row_mixed.result:
No changes
mysql-test/suite/rpl/t/rpl_row_flsh_tbls.test:
Manual merge
mysql-test/t/change_user.test:
Manual merge
mysql-test/t/disabled.def:
Manual merge
mysql-test/t/merge.test:
No changes
mysql-test/t/myisam.test:
Manual merge
mysys/Makefile.am:
Manual merge
mysys/array.c:
Manual merge
mysys/mf_keycache.c:
Manual merge
mysys/my_getsystime.c:
Manual merge
mysys/my_handler.c:
Manual merge
mysys/my_pread.c:
Manual merge
mysys/safemalloc.c:
Manual merge
sql/ha_partition.cc:
Manual merge
sql/handler.cc:
Manual merge
sql/lex.h:
Manual merge
sql/mysql_priv.h:
Manual merge
sql/mysqld.cc:
Manual merge
sql/set_var.h:
Manual merge
sql/sql_class.cc:
Manual merge
sql/sql_insert.cc:
Manual merge
sql/sql_parse.cc:
Manual merge
sql/sql_select.cc:
Manual merge
sql/sql_show.cc:
Manual merge
sql/sql_table.cc:
Manual merge
storage/myisam/mi_checksum.c:
No changes
storage/myisam/mi_extra.c:
Manual merge
storage/myisam/mi_open.c:
Manual merge
storage/myisammrg/ha_myisammrg.cc:
Manual merge
strings/strmake.c:
No changes
The problem was that LOAD DATA code (sql_load.cc) didn't take into
account that there may be items, representing references to other
columns. This is a usual case in views. The crash happened because
Item_direct_view_ref was casted to Item_user_var_as_out_param,
which is not a base class.
The fix is to
1) Handle references properly;
2) Ensure that an item is treated as a user variable only when
it is a user variable indeed;
3) Report an error if LOAD DATA is used to load data into
non-updatable column.
mysql-test/r/loaddata.result:
Update result file.
mysql-test/t/loaddata.test:
Add a test case form Bug#35469: server crash with
LOAD DATA INFILE to a VIEW.
sql/share/errmsg.txt:
Introduce a new error.
sql/sql_load.cc:
Handle reference-items properly.
mysql-test/std_data/bug35649.data:
Add a data file for the test case.
The problem was that LOAD DATA code (sql_load.cc) didn't take into
account that there may be items, representing references to other
columns. This is a usual case in views. The crash happened because
Item_direct_view_ref was casted to Item_user_var_as_out_param,
which is not a base class.
The fix is to
1) Handle references properly;
2) Ensure that an item is treated as a user variable only when
it is a user variable indeed;
3) Report an error if LOAD DATA is used to load data into
non-updatable column.
into magare.gmz:/home/kgeorge/mysql/work/B26461-5.1-opt
CMakeLists.txt:
Auto merged
include/config-win.h:
Auto merged
include/my_global.h:
Auto merged
sql/procedure.h:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_acl.h:
Auto merged
sql/sql_analyse.cc:
Auto merged
sql/sql_analyse.h:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_delete.cc:
Auto merged
sql/sql_load.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/handler.h:
merged bug 26461 to 5.1-opt
sql/mysql_priv.h:
merged bug 26461 to 5.1-opt
sql/sql_base.cc:
merged bug 26461 to 5.1-opt
sql/sql_prepare.cc:
merged bug 26461 to 5.1-opt
The bool data type was redefined to BOOL (4 bytes on windows).
Removed the #define and fixed some of the warnings that were uncovered
by this.
Note that the fix also disables 2 warnings :
4800 : 'type' : forcing value to bool 'true' or 'false' (performance warning)
4805: 'operation' : unsafe mix of type 'type' and type 'type' in operation
These warnings will be handled in a separate bug, as they are performance related or bogus.
Fixed to int the return type of functions that return more than
2 distinct values.
CMakeLists.txt:
Bug #26461: disable the C4800 and C4805 warnings temporarily
include/config-win.h:
Bug #26461:
- no need for this define for Windows.
- windows C++ compilers have a bool type
include/my_global.h:
Bug #26461: removed bool_defined (no longer needed)
sql/handler.h:
Bug #26461: bool functions must return boolean values
sql/mysql_priv.h:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/procedure.h:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_acl.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_acl.h:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_analyse.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_analyse.h:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_base.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_db.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_delete.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_load.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_parse.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_prepare.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
sql/sql_update.cc:
Bug #26461: fixed return type of functions that return more than
2 distinct values.
The bool data type was redefined to BOOL (4 bytes on windows).
Removed the #define and fixed some of the warnings that were uncovered
by this.
Note that the fix also disables 2 warnings :
4800 : 'type' : forcing value to bool 'true' or 'false' (performance warning)
4805: 'operation' : unsafe mix of type 'type' and type 'type' in operation
These warnings will be handled in a separate bug, as they are performance related or bogus.
Fixed to int the return type of functions that return more than
2 distinct values.
a SELECT doesn't cause ROLLBACK of statem".
The idea of the fix is to ensure that we always commit the current
statement at the end of dispatch_command(). In order to not issue
redundant disc syncs, an optimization of the two-phase commit
protocol is implemented to bypass the two phase commit if
the transaction is read-only.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Update test results.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Update test results.
mysql-test/suite/rpl_ndb/t/disabled.def:
Disable the tests, for which this changeset reveals a bug:
the injector thread does not always add 'statement commit' to the
rows injected in circular replication set up.
To be investigated separately.
sql/ha_ndbcluster_binlog.cc:
Add close_thread_tables() to run_query: this ensures
that all tables are closed and there is no pending statement transaction.
sql/handler.cc:
Implement optimisation of read-only transactions.
If a transaction consists only of DML statements that do not change
data, we do not perform a two-phase commit for it
(run one phase commit only).
sql/handler.h:
Implement optimisation of read-only transactions.
If a transaction consists only of DML statements that do not change
data, we do not perform a two-phase commit for it
(run one phase commit only).
sql/log.cc:
Mark the binlog transaction read-write whenever it's started.
We never read from binlog, so it's safe and least intrusive to add
this mark up here.
sql/log_event.cc:
Update to the new layout of thd->transaction.
sql/rpl_injector.cc:
Always commit statement transaction before committing the global one.
sql/sp.cc:
Ad comments.
sql/sp_head.cc:
Add comments.
sql/sql_base.cc:
Commit transaction at the end of the statement. Always.
sql/sql_class.cc:
Update thd_ha_data to return the right pointer in the new layout.
Fix select_dumpvar::send_data to properly return operation status.
A test case from commit.inc would lead to an assertion failure in the
diagnostics area (double assignment). Not test otherwise by the test suite.
sql/sql_class.h:
Implement a new layout of storage engine transaction info in which
it is easy to access all members related to the handlerton only
based on ht->slot.
sql/sql_cursor.cc:
Update to the new layout of thd->transaction.
sql/sql_delete.cc:
Remove wrong and now redundant calls to ha_autocommit_or_rollback.
The transaction is committed in one place, at the end of the statement.
Remove calls to mysql_unlock_tables, since some engines count locks
and commit statement transaction in unlock_tables(), which essentially
equates mysql_unlock_tables to ha_autocommit_or_rollback.
Previously it was necessary to unlock tables soon because we wanted
to avoid sending of 'ok' packet to the client under locked tables.
This is no longer necessary, since OK packet is also sent from one place
at the end of transaction.
sql/sql_do.cc:
Add DO always clears the error, we must rollback the current
statement before this happens. Otherwise the statement will be committed,
and not rolled back in the end.
sql/sql_insert.cc:
Remove wrong and now redundant calls to ha_autocommit_or_rollback.
The transaction is committed in one place, at the end of the statement.
Remove calls to mysql_unlock_tables, since some engines count locks
and commit statement transaction in unlock_tables(), which essentially
equates mysql_unlock_tables to ha_autocommit_or_rollback.
Previously it was necessary to unlock tables soon because we wanted
to avoid sending of 'ok' packet to the client under locked tables.
This is no longer necessary, since OK packet is also sent from one place
at the end of transaction.
sql/sql_load.cc:
Remove wrong and now redundant calls to ha_autocommit_or_rollback.
The transaction is committed in one place, at the end of the statement.
Remove calls to mysql_unlock_tables, since some engines count locks
and commit statement transaction in unlock_tables(), which essentially
equates mysql_unlock_tables to ha_autocommit_or_rollback.
Previously it was necessary to unlock tables soon because we wanted
to avoid sending of 'ok' packet to the client under locked tables.
This is no longer necessary, since OK packet is also sent from one place
at the end of transaction.
sql/sql_parse.cc:
Implement optimisation of read-only transactions: bypass 2-phase
commit for them.
Always commit statement transaction before commiting the global one.
Fix an unrelated crash in check_table_access, when called from
information_schema.
sql/sql_partition.cc:
Partitions commit at the end of a DDL operation.
Make sure that send_ok() is done only if the commit has succeeded.
sql/sql_table.cc:
Use ha_autocommit_or_rollback and end_active_trans everywhere.
Add end_trans to mysql_admin_table, so that it leaves no pending
transaction.
sql/sql_udf.cc:
Remvove a redundant call to close_thread_tables()
sql/sql_update.cc:
Remove wrong and now redundant calls to ha_autocommit_or_rollback.
The transaction is committed in one place, at the end of the statement.
Remove calls to mysql_unlock_tables, since some engines count locks
and commit statement transaction in unlock_tables(), which essentially
equates mysql_unlock_tables to ha_autocommit_or_rollback.
Previously it was necessary to unlock tables soon because we wanted
to avoid sending of 'ok' packet to the client under locked tables.
This is no longer necessary, since OK packet is also sent from one place
at the end of transaction.
mysql-test/include/commit.inc:
New BitKeeper file ``mysql-test/include/commit.inc''
mysql-test/r/commit_1innodb.result:
New BitKeeper file ``mysql-test/r/commit_1innodb.result''
mysql-test/t/commit_1innodb.test:
New BitKeeper file ``mysql-test/t/commit_1innodb.test''
