variable
The User_var_log_event was not serializing the unsigned
flag. This would cause the slave to always assume signed values.
We fix this by extending the User_var_log_event to also contain
information on the unsigned_flag, meaning that it gets into the
binlog as well, therefore the slave will get this information as
well. Events without information on unsigned flag (old events)
are treated as they were before (always signed: unsigned_flag=
FALSE).
The information on the unsigned_flag, is shipped in an extra byte
appended to the end of the User_var_log_event and added by this
patch. This extra byte holds values for general purpose
User_var_log_event flags which are now packed in the binlog as
well. One of these flags contains information about whether the
value is signed or unsigned (currently this extra byte is only
used to hold data on the unsigned flag, in the future we can use
it to pack extra flags if there is the need to).
In auto-commit mode, updating both trx and non-trx tables (i.e. issuing a mixed
statement) causes the following sequence of events:
1 - "Flush trx changes" (MYSQL_BIN_LOG::write) - T1:
1.1 - mutex_lock (&LOCK_log)
1.2 - mutex_lock (&LOCK_prep_xids)
1.3 - increase prepared_xids
1.4 - mutex_unlock (&LOCK_prep_xids)
1.5 - mutex_unlock (&LOCK_log)
2 - "Flush non-trx changes" (MYSQL_BIN_LOG::write) - T1:
2.1 - mutex_lock (&LOCK_log)
2.2 - mutex_unlock (&LOCK_log)
3. "unlog" - T1
3.1 - mutex_lock (&LOCK_prep_xids)
3.2 - decrease prepared xids
3.3 - pthread_cond_signal(&COND_prep_xids);
3.4 - mutex_unlock (&LOCK_prep_xids)
The "FLUSH logs" command produces the following sequence of events:
1 - "FLUSH logs" command (MYSQL_BIN_LOG::new_file_impl) - user thread:
1.1 - mutex_lock (&LOCK_log)
1.2 - mutex_lock (&LOCK_prep_xids)
1.3 - while (prepared_xids) pthread_cond_wait(..., &LOCK_prep_xids);
1.4 - mutex_unlock (&LOCK_prep_xids)
1.5 - mutex_unlock (&LOCK_log)
A deadlock will arise if T1 flushes the trx changes and thus increases
prepared_xids but before it is able to continue the execution and flush the
non-trx changes, an user thread calls the "FLUSH logs" command and wait that
the prepared_xids is decreased and gets to zero. However, T1 cannot proceed
with the call to "Flush non-trx changes" because it will block in the mutex
"LOCK_log" and by consequence cannot complete the execution and call the
unlog to decrease the prepared_xids.
To fix the problem, we ensure that the non-trx changes are always flushed
before the trx changes.
Note that if you call "Flush non-trx changes" and a concurrent "FLUSH logs" is
issued, the "Flush non-trx changes" may block, but a deadlock will never happen
because the prepared_xids will eventually get to zero. Bottom line, there will
not be any transaction able to increase the prepared_xids because they will
block in the mutex "LOCK_log" (MYSQL_BIN_LOG::write) and those that increased
the prepared_xids will eventually commit and decrease the prepared_xids.
When @@session.binlog_format is modified inside a transaction,
it can cause slave to go out of sync.
To fix the problem, make the session variable 'binlog_format'
read-only inside a transaction.
mysql-test/suite/binlog/r/binlog_format_switch_inside_trans.result:
Test result for bug#47863.
mysql-test/suite/binlog/t/binlog_format_switch_inside_trans.test:
Added test file to verify if the session variable 'binlog_format'
is read-only inside a transaction and in sub-statements.
sql/set_var.cc:
Added code to make the session variable 'binlog_format'
read-only inside a transaction.
Problem: The test was written before BUG#45827 was fixed.
The test contained code that assumed the wrong behavior,
pre-BUG#45827. Then, the fix for BUG#45827 was merged
from 5.1-rep+2 to 5.1-rep+3. Since the test case assumed
the wrong behavior, it failed. This should have been fixed
by making the test assume the correct behavior, but was
fixed by updating the result file to assert failure.
Fix 1: fix the test to assume correct behavior
(post-BUG#45827), update result file.
Fix 2: make test fail with 'die' instead of 'exit' when
wrong behavior is detected. Thus, the test cannot be
silenced with a wrong result file in case the behavior
will change again.
mysql-test/extra/rpl_tests/create_recursive_construct.inc:
Replaced 'exit' by 'die' to avoid similar
mysql-test/suite/binlog/r/binlog_unsafe.result:
Updated result file.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Since BUG#45827 is now fixed, we need to update the test case
in two places where it expects the wrong result because of
BUG#45827.
Support for flushing individual logs, so that the user can
selectively flush a subset of the server logs.
Flush of individual logs is done according to the
following syntax:
FLUSH <log_category> LOGS;
The syntax is extended so that the user is able to flush a
subset of logs:
FLUSH [log_category LOGS,];
where log_category is one of:
SLOW
ERROR
BINARY
ENGINE
GENERAL
RELAY.
mysql-test/suite/rpl/r/rpl_flush_logs.result:
Test result for WL#5142.
mysql-test/suite/rpl/t/rpl_flush_logs.test:
Added the test file to verify if the 'flush individual log'
statement works fine.
sql/log.cc:
Added the two functions to flush slow and general log.
sql/sql_parse.cc:
Added code to flush specified logs against the option.
sql/sql_yacc.yy:
Added code to parse the 'flush * log' statement syntax and
set its option to Lex->type.
beyond unsigned long.
BUG#44779: binlog.binlog_max_extension may be causing failure on
next test in PB
NOTE1: this is the backport to next-mr.
NOTE2: already includes patch for BUG#44779.
Binlog file extensions would turn into negative numbers once the
variable used to hold the value reached maximum for signed
long. Consequently, incrementing value to the next (negative) number
would lead to .000000 extension, causing the server to fail.
This patch addresses this issue by not allowing negative extensions
and by returning an error on find_uniq_filename, when the limit is
reached. Additionally, warnings are printed to the error log when the
limit is approaching. FLUSH LOGS will also report warnings to the
user, if the extension number has reached the limit. The limit has been
set to 0x7FFFFFFF as the maximum.
mysql-test/suite/binlog/t/binlog_max_extension.test:
Test case added that checks the maximum available number for
binlog extensions.
sql/log.cc:
Changes to find_uniq_filename and test_if_number.
sql/log.h:
Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.
The mentioned on the bug report set of bugs fixes have not be pushed to the main trees.
Fixed with extracting commits done to 6.0-rpl tree and applying them to the main 5.1.
Notes.
1. part of changes - the mtr's specific - were packported to the main 5.0 tree for mtr v1
as http://lists.mysql.com/commits/46562
However, there is no that fix anymore in the mtr v2. (This fact was mailed to mtr maintaining
people).
2. Bug@36929 crash in kill_zombie_dump_threads-> THD::awake() with replication tests
is not backported because the base code of the patch is libevent and that was removed
from the main trees due to its instability.
client/mysqlbinlog.cc:
fixes for BUG#35546
mysql-test/suite/rpl/r/rpl_bug41902.result:
the new tests result file is added.
mysql-test/suite/rpl/t/rpl_bug41902-slave.opt:
conf file for bug41902 testing is added.
mysql-test/suite/rpl/t/rpl_bug41902.test:
regression tests for Bug #41902 is added.
sql/log.cc:
collection of changes due to Bug #48463.
sql/log.h:
collection of changes due to Bug #48463.
sql/rpl_rli.h:
collection of changes due to Bug #48463.
sql/slave.cc:
collection of changes due to Bug #48463.
sql/sql_repl.cc:
collection of changes due to Bug #48463.
After fix of bug46322, logging to table is turned off, each test
that need this should turn on it in it's opt file.
Add suppression to unsafe statement warnings for binlog_unsafe.test.
mysql-test/suite/binlog/t/binlog_unsafe-master.opt:
After fix of bug46322, logging to table is turned off, each test that need this should turn on it in it's opt file.
mysql-test/suite/binlog/t/binlog_unsafe.test:
suppress Unsafe statement warnings
One statement that have more than one different tables to update with
autoinc columns just was marked as unsafe in mixed mode, so the unsafe
warning can't be produced in statement mode.
To fix the problem, mark the statement as unsafe in statement mode too.
mysql-test/extra/rpl_tests/rpl_insert_id.test:
The test case is updated due to the patch of bug#45827.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
The test result is updated due to the patch of bug#45827.
mysql-test/suite/binlog/r/binlog_unsafe.result:
Test result for bug#45827.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Added test to verify if stmt that have more than one
different tables to update with autoinc columns will
produce unsafe warning
mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result:
The test result is updated due to the patch of bug#45827.
mysql-test/suite/rpl/t/rpl_trigger.test:
The test case is updated due to the patch of bug#45827.
mysql-test/suite/rpl/t/rpl_variables_stm.test:
The test case is updated due to the patch of bug#45827.
sql/sql_base.cc:
Reomved the 'set_current_stmt_binlog_row_based_if_mixed' function
for producing unsafe warnings by executing 'decide_logging_format'
function later in statement mode
Create a set of test cases to see if some DDL statements implicitly commit
a transaction on the NDB and are written directly to the binary log without
going through either the Statement- or Transactional-Cache.
mysql-test/extra/binlog_tests/binlog_failure_mixing_engines.test:
Merged the test case into rpl_mixing_engines.test.
mysql-test/extra/rpl_tests/rpl_mixing_engines.inc:
Incorporated some tests from binlog_failure_mixing_engines.test and
fixed after BUG#47323.
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
Incorporated some tests from binlog_failure_mixing_engines.test and
fixed after BUG#47323.
mysql-test/suite/binlog/r/binlog_mixed_failure_mixing_engines.result:
Merged the test case into rpl_mixing_engines.test.
mysql-test/suite/binlog/r/binlog_row_failure_mixing_engines.result:
Merged the test case into rpl_mixing_engines.test.
mysql-test/suite/binlog/t/binlog_mixed_failure_mixing_engines.test:
Merged the test case into rpl_mixing_engines.test.
mysql-test/suite/binlog/t/binlog_row_failure_mixing_engines.test:
Merged the test case into rpl_mixing_engines.test.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
Incorporated some tests from binlog_failure_mixing_engines.test and
changed after BUG#47323.
mysql-test/suite/rpl/r/rpl_row_mixing_engines.result:
Incorporated some tests from binlog_failure_mixing_engines.test and
changed after BUG#47323.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
Incorporated some tests from binlog_failure_mixing_engines.test and
changed after BUG#47323.
mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/r/rpl_mysql_upgrade.result:
Suppressed warning messages due to unsafe statements.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/t/rpl_mixed_implicit_commit_binlog.test:
Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/t/rpl_mysql_upgrade.test:
Suppressed warning messages due to unsafe statements.
mysql-test/suite/rpl/t/rpl_row_implicit_commit_binlog.test:
Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/t/rpl_stm_implicit_commit_binlog.test:
Re-enabled some parts of the test after BUG#46572
Non-transactional updates that take place inside a transaction present problems
for logging because they are visible to other clients before the transaction
is committed, and they are not rolled back even if the transaction is rolled
back. It is not always possible to log correctly in statement format when both
transactional and non-transactional tables are used in the same transaction.
In the current patch, we ensure that such scenario is completely safe under the
ROW and MIXED modes.
Detailed revision comments:
r6101 | jyang | 2009-10-23 11:45:50 +0300 (Fri, 23 Oct 2009) | 7 lines
branches/zip: Update test result with the WARN_LEVEL_ERROR
to WARN_LEVEL_WARN change. This is the same result as
submitted in rb://172 review, which approved by Sunny Bains
and Marko.
Detailed revision comments:
r6100 | jyang | 2009-10-22 06:51:07 +0300 (Thu, 22 Oct 2009) | 6 lines
branches/zip: As a request from mysql, WARN_LEVEL_ERROR cannot
be used for push_warning_* call any more. Switch to
WARN_LEVEL_WARN. Bug #47233.
rb://172 approved by Sunny Bains and Marko.
Detailed revision comments:
r6095 | vasil | 2009-10-19 16:04:59 +0300 (Mon, 19 Oct 2009) | 7 lines
branches/zip:
Fix Bug#47808 innodb_information_schema.test fails when run under valgrind
by using the wait_until_rows_count macro that loops until the number of
rows becomes 14 instead of sleep 0.1, which is obviously very fragile.
The 'rpl_get_master_version_and_clock' test verifies if the slave I/O
thread tries to reconnect to master when it tries to get the values of
the UNIX_TIMESTAMP, SERVER_ID from master under network disconnection.
So the master server is restarted for making the transient network
disconnection. Restarting master server can bring two problems as following:
1. The time out error is encountered sporadically. The slave I/O thread tries
to reconnect master ten times, which is set in my.cnf. So in the test
framework sporadically the slave I/O thread really stoped when it can't
reconnect to master in the ten times successfully before the master starts,
then the time out error will be encountered while waiting for the slave to
start.
2. These warnings and errors are produced in server log file when
the slave I/O thread tries to get the values of the UNIX_TIMESTAMP,
SERVER_ID from master under the transient network disconnection.
To fix problem 1, increase the master retry count to sixty times,
so that the slave I/O thread has enough time to reconnect master
successfully.
To fix problem 2, suppress these warnings and errors by mtr suppression,
because they are expected.
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock-slave.opt:
Added the *.opt file for increasing master retry count to
sixty times.
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test:
Added mtr suppression for suppressing warnings and errors
in server log file.
binlog, replication aborts
In SBR or MBR, the schema name is not being written to the binlog
when executing a LOAD DATA statement. This becomes a problem when
the current database (lets call it db1) is different from the
table's schema (lets call it db2). For instance, take the
following statements:
use db1;
load data local infile 'infile.txt' into table db2.t
Should this statement be logged without t's schema (db2), when
replaying it, one can get db1.t populated instead of db2.t (if
db1.t exists). On the other hand, if there is no db1.t at all,
replication will stop.
We fix this by always logging the table (in load file) with fully
qualified name when its schema is different from the current
database or when no default database was selected.
CMakeLists.txt:
Add plugin/semisync subdirectory
mysql-test/mysql-test-run.pl:
Check for semisync dll for Windows
mysql-test/suite/rpl/r/rpl_semi_sync.result:
Update result file
mysql-test/suite/rpl/t/rpl_semi_sync.test:
Test semi-sync on Windows
plugin/semisync/semisync_master.cc:
Define gettimeofday for Windows
Before the patch, slaves only appear in the output of SHOW SLAVE HOSTS
when report-host option is set. If an expected slave does not appear in
the list, nobody knows whether the slave does not connect or has started
without the "report-host" option. The output also contains a strange
field "Rpl_recovery_rank" which has never been implemented and the manual
of MySQL5.4 declares that the field has been removed from MySQL5.4.
This patch is done with these,
According to the manual of MySQL5.4, "Rpl_recovery_rank" is removed.
Slaves will register themselves to master no matter if report_host option is set
or not. When slaves are registering themselves, their Server_ids, report_host
and other information are together sent to master. Sever_ids are never null
and is unique in one replication group. Slaves always can be identified with
different Server_ids no matter if report_host exists.
file
Issuing 'FLUSH LOGS' does not close and reopen indexfile.
Instead a SEEK_SET is performed.
This patch makes index file to be closed and reopened whenever a
rotation happens (FLUSH LOGS is issued or binary log exceeds
maximum configured size).
mysql-test/suite/binlog/r/binlog_delete_and_flush_index.result:
Result file.
mysql-test/suite/binlog/t/binlog_delete_and_flush_index.test:
Test case.
sql/log.cc:
Added LOG_CLOSE_INDEX flag when calling MYSQL_BIN_LOG::close
from within MYSQL_BIN_LOG::new_file_impl (which should just be
called whenever a rotation is to happen - FLUSH LOGS issued or
binlog size exceeds the maximum configured).
Post-push fix.
Problem: In a previous patch for BUG#39934, rpl_idempotency.test
was split in two tests. The mtr suppressions in the original test
did not make it into the new test. This caused pushbuild warnings.
Fix: copy the mtr suppressions from rpl_idempotency.test to
rpl_row_idempotency.test
mysql-test/suite/rpl/r/rpl_row_idempotency.result:
updated result file
mysql-test/suite/rpl/t/rpl_row_idempotency.test:
copied the warnings from rpl_idempotency.test to
rpl_row_idempotency.test
The rpl_ndb/combinations file was introduced as part of the fix.
The file contained an error: ndb suites shall not run with
binlog_format=mixed. Removed that combination.
mysql-test/suite/rpl_ndb/combinations:
removed binlog_format=statement combination since ndb does not
support statement format.
mysql-test/suite/rpl_ndb/t/rpl_ndb_mixed_tables.test:
Added have_binlog_format_mixed_or_row.inc.
post-push fixes. the test case tried to use a udf that had not been loaded. removed the udf.
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
updated result file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
myfunc_int() was supposed to be a udf. but the udf was not loaded.
since the test already uses another plugin, and we don't support
multiple plugin directories, i removed the udf function. it was not
strictly needed.
Post-push fix.
Problem: After the original bugfix, if a statement is unsafe,
binlog_format=mixed, and engine is statement-only, a warning was
generated and the statement executed. However, it is a fundamental
principle of binlogging that binlog_format=mixed should guarantee
correct logging, no compromise. So correct behavior is to generate
an error and don't execute the statement.
Fix: Generate error instead of warning.
Since issue_unsafe_warnings can only generate one error message,
this allows us to simplify the code a bit too:
decide_logging_format does not have to save the error code for
issue_unsafe_warnings
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_ps.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
updated result file
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file
mysql-test/suite/rpl/r/rpl_stm_found_rows.result:
updated result file
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
updated result file
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
updated result file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
updated test:
- ER_BINLOG_UNSAFE_AND_STMT_ENGINE is now an error.
- added test for multiple types of unsafety
sql/share/errmsg.txt:
- Reformulated ER_BINLOG_UNSAFE_AND_STMT_ENGINE to reflect that it
is now an error, not a warning.
- Added "Reason for unsafeness" to ER_BINLOG_UNSAFE_STATEMENT and
ER_BINLOG_UNSAFE_AND_STMT_ENGINE.
sql/sql_class.cc:
In decide_logging_format:
- generate an error immediately in case 3, instead of scheduling a
warning to be generated later. also updated comments accordingly
- in case 7, there is only one unsafe warning error code now, so we
don't need to store it in binlog_unsafe_warning_flags
(see changes in sql_lex.h)
- fixed compilation warning in DBUG_PRINT
In issue_binlog_warning:
- moved array of error codes to sql_lex.h (so that they are
accessible also from decide_logging_format)
- simplified code after the first set of bits in
binlog_unsafe_warning_flags was removed
sql/sql_class.h:
- got rid of enum_binlog_stmt_warning. It's not needed anymore
since we only have one type of unsafe warning (one of them
turned into an error)
- updated comments accordingly
sql/sql_lex.cc:
added initialization of the array of error codes that has been
moved from THD::issue_unsafe_warnings to LEX.
sql/sql_lex.h:
Moved array of error codes from THD::issue_unsafe_warnings to LEX.
