Bug #49984 Test 'rpl_loaddata_symlink' fails with "Could not find target log"
Sometimes the symbolic link is available on Windows if
you have some Unix (emulation) layer installed like
Cygwin, MKS or other. But symbolic link is not always
available. It depends on versions, file volume and
system environment of Windows. And the symbolic link
is not typically used on Windows, at least not in the
degree and style they are used on Unix. We can not
change the test case to do without symbolic link,
because the test case is used to test that if the
symbolic link works fine.
To fix the problem, skip the test on windows.
mysql-test/suite/rpl/t/rpl_loaddata_symlink.test:
Added code to skip the test on windows.
When the $diff_statement variable for diff_master_slave.inc was
put in multiple lines, the rear part of the statement would be
missing when being executed on Windows systems.
Fixed the problem by always putting the value for $diff_statement
in one line.
mysql-test/suite/rpl/t/rpl_killed_ddl.test:
putting the statement for $diff_statement variable in one line.
The bug is caused by a race condition between the
INSERT DELAYED thread and the client thread's FLUSH TABLE. The
FLUSH TABLE does not guarantee (as is (wrongly) suggested in the
test case) that the INSERT DELAYED is ever executed. The
execution of the test case will thus not be deterministic.
The fix has been to do a deterministic verification that both
threads are complete by checking the content of the table.
This test case tests a circular replication of four hosts.
A--->B--->C--->D--->A
The replicate is slow and needs more time to replicate all data in the circle.
The time it spends to replicate, sometimes, is longer than the time that
wait_condition.inc spends to wait that all data has been replicated. This
cause sporadical failure of this test case.
This patch uses sync_slave_with_master to ensure that all data can be replicated
successfully in the circle.
'LOAD DATA CONCURRENT [LOCAL] INFILE ...' statment only is binlogged as
'LOAD DATA [LOCAL] INFILE ...' in SBR and MBR. As a result, if replication is on,
queries on slaves will be blocked by the replication SQL thread.
This patch write code to write 'CONCURRENT' into the log event if 'CONCURRENT' option
is in the original statement in SBR and MBR.
This fix changes the character set used within the
IBMDB2I handler to hash table names to information
about open tables. Previously, tables with names
that differed only in letter case would hash to the
same data structure. This caused incorrect behavior
or errors when two such tables were in use simultaneously.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_49329.result:
Bug#49329 example (and other) engines use wrong collation for open tables hash
Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_49329.test:
Bug#49329 example (and other) engines use wrong collation for open tables hash
Test case for the bug fix.
storage/ibmdb2i/ha_ibmdb2i.cc:
Bug#49329 example (and other) engines use wrong collation for open tables hash
change the character set used within the IBMDB2I
handler to hash table names to information about
open tables.
{PROCEDURE|FUNCTION} FROM ...'
The master would hit an assertion when binary log was
active. This was due to the fact that the thread's diagnostics
area was being cleared before writing to the binlog,
independently of mysql_routine_grant returning an error or
not. When mysql_routine_grant was to return an error, the return
value and the diagnostics area contents would
mismatch. Consequently, neither my_ok would be called nor an
error would be signaled in the diagnostics area, eventually
triggering the assertion in net_end_statement.
We fix this by not clearing the diagnostics area at binlogging
time.
escaped field names
When in mixed or statement mode, the master logs LOAD DATA
queries by resorting to an Execute_load_query_log_event. This
event does not contain the original query, but a rewritten
version of it, which includes the table field names. However, the
rewrite does not escape the field names. If these names match a
reserved keyword, then the slave will stop with a syntax error
when executing the event.
We fix this by escaping the fields names as it happens already
for the table name.
mysql-test/extra/rpl_tests/rpl_loaddata.test:
Added test case for the reported bug.
mysql-test/r/mysqlbinlog.result:
Changed result to support escaped field name.
mysql-test/suite/binlog/r/binlog_killed_simulate.result:
Changed result to support escaped field name.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Changed result to support escaped field name.
mysql-test/suite/binlog/r/binlog_stm_blackhole.result:
Changed result to support escaped field name.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Changed result to support escaped field name.
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
Changed result to support escaped field name.
mysql-test/suite/rpl/r/rpl_loaddata.result:
Added result for new test.
Changed show slave status positions which are now different because of
extra escape character in field names.
mysql-test/suite/rpl/r/rpl_loaddata_fatal.result:
Changed show slave status positions which are now different because of
extra escape character.
mysql-test/suite/rpl/r/rpl_loaddata_map.result:
Changed result to support escaped field name.
mysql-test/suite/rpl/r/rpl_stm_log.result:
Changed result to support escaped field name.
mysql-test/t/mysqlbinlog.test:
Changed positions which is now different because of extra escape
character in field names.
sql/sql_load.cc:
Appended escape characters before and after field names.
This patch fixes three bugs as follows. First, aborting the server while purging
binary logs might generate orphan files due to how the purge operation was
implemented:
(purge routine - sql/log.cc - MYSQL_BIN_LOG::purge_logs)
1 - register the files to be removed in a temporary buffer.
2 - update the log-bin.index.
3 - flush the log-bin.index.
4 - erase the files whose names where register in the temporary buffer
in step 1.
Thus a failure while executing step 4 would generate an orphan file. Second,
a similar issue might happen while creating a new binary as follows:
(create routine - sql/log.cc - MYSQL_BIN_LOG::open)
1 - open the new log-bin.
2 - update the log-bin.index.
Thus a failure while executing step 1 would generate an orphan file.
To fix these issues, we record the files to be purged or created before really
removing or adding them. So if a failure happens such records can be used to
automatically remove dangling files. The new steps might be outlined as follows:
(purge routine - sql/log.cc - MYSQL_BIN_LOG::purge_logs)
1 - register the files to be removed in the log-bin.~rec~ placed in
the data directory.
2 - update the log-bin.index.
3 - flush the log-bin.index.
4 - delete the log-bin.~rec~.
(create routine - sql/log.cc - MYSQL_BIN_LOG::open)
1 - register the file to be created in the log-bin.~rec~
placed in the data directory.
2 - open the new log-bin.
3 - update the log-bin.index.
4 - delete the log-bin.~rec~.
(recovery routine - sql/log.cc - MYSQL_BIN_LOG::open_index_file)
1 - open the log-bin.index.
2 - open the log-bin.~rec~.
3 - for each file in log-bin.~rec~.
3.1 Check if the file is in the log-bin.index and if so ignore it.
3.2 Otherwise, delete it.
The third issue can be described as follows. The purge operation was allowing
to remove a file in use thus leading to the loss of data and possible
inconsistencies between the master and slave. Roughly, the routine was only
taking into account the dump threads and so if a slave was not connect the
file might be delete even though it was in use.
Fixes several test cases.
applied revisions: r6181, r6182, r6183, r6184
r6183 - changes are made only to tests in innodb suite which is only
innodb-consistent-master.opt
Detailed revision comments:
r6181 | vasil | 2009-11-17 12:21:41 +0200 (Tue, 17 Nov 2009) | 33 lines
branches/zip:
At the end of innodb-index.test: restore the environment as it was before
the test was started to silence this warning:
MTR's internal check of the test case 'main.innodb-index' failed.
This means that the test case does not preserve the state that existed
before the test case was executed. Most likely the test case did not
do a proper clean-up.
This is the diff of the states of the servers before and after the
test case was executed:
mysqltest: Logging to '/tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.log'.
mysqltest: Results saved in '/tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.result'.
mysqltest: Connecting to server localhost:13000 (socket /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...
mysqltest: ... Connected.
mysqltest: Start processing test commands from './include/check-testcase.test' ...
mysqltest: ... Done processing test commands.
--- /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.result 2009-11-17 13:10:40.000000000 +0300
+++ /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.reject 2009-11-17 13:10:54.000000000 +0300
@@ -84,7 +84,7 @@
INNODB_DOUBLEWRITE ON
INNODB_FAST_SHUTDOWN 1
INNODB_FILE_FORMAT Antelope
-INNODB_FILE_FORMAT_CHECK Antelope
+INNODB_FILE_FORMAT_CHECK Barracuda
INNODB_FILE_PER_TABLE OFF
INNODB_FLUSH_LOG_AT_TRX_COMMIT 1
INNODB_FLUSH_METHOD
mysqltest: Result content mismatch
not ok
r6182 | marko | 2009-11-17 13:49:15 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: Set svn:eol-style on mysql-test files.
r6183 | marko | 2009-11-17 13:51:16 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: Prepend loose_ to plugin-only mysql-test options.
r6184 | marko | 2009-11-17 13:52:01 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: innodb-index.test: Restore innodb_file_format_check.
1. add testcase for BUG#46676
2. Allow CREATE INDEX to be interrupted
3. ha_innobase::change_active_index(): When the history is
missing, report it to the client, not to the error log
4. ChangeLog entries
appplied revisions:r6169, r6170, r6175, r6177, r6179
Detailed revision comments:
r6169 | calvin | 2009-11-12 14:40:43 +0200 (Thu, 12 Nov 2009) | 6 lines
branches/zip: add test case for bug#46676
This crash is reproducible with InnoDB plugin 1.0.4 + MySQL 5.1.37.
But no longer reproducible after MySQL 5.1.38 (with plugin 1.0.5).
Add test case to catch future regression.
r6170 | marko | 2009-11-12 15:49:08 +0200 (Thu, 12 Nov 2009) | 4 lines
branches/zip: Allow CREATE INDEX to be interrupted. (Issue #354)
rb://183 approved by Heikki Tuuri
r6175 | vasil | 2009-11-16 20:07:39 +0200 (Mon, 16 Nov 2009) | 4 lines
branches/zip:
Wrap line at 78th char in the ChangeLog
r6177 | calvin | 2009-11-16 20:20:38 +0200 (Mon, 16 Nov 2009) | 2 lines
branches/zip: add an entry to ChangeLog for r6065
r6179 | marko | 2009-11-17 10:19:34 +0200 (Tue, 17 Nov 2009) | 2 lines
branches/zip: ha_innobase::change_active_index(): When the history is
missing, report it to the client, not to the error log.
applied revisions: r6157
Detailed revision comments:
r6157 | jyang | 2009-11-11 14:27:09 +0200 (Wed, 11 Nov 2009) | 10 lines
branches/zip: Fix an issue that a local variable defined
in innodb_file_format_check_validate() is being referenced
across function in innodb_file_format_check_update().
In addition, fix "set global innodb_file_format_check =
DEFAULT" call.
Bug #47167: "set global innodb_file_format_check" cannot
set value by User-Defined Variable."
rb://169 approved by Sunny Bains and Marko.
The 'slave_patternload_file' is assigned to the real path of the load data file
when initializing the object of Relay_log_info. But the path of the load data
file is not formatted to real path when executing event from relay log. So the
error will be encountered if the path of the load data file is a symbolic link.
Actually the global 'opt_secure_file_priv' is not formatted to real path when
loading data from file. So the same thing will happen too.
To fix these errors, the path of the load data file should be formatted to
real path when executing event from relay log. And the 'opt_secure_file_priv'
should be formatted to real path when loading data infile.
mysql-test/suite/rpl/r/rpl_loaddata_symlink.result:
Test result for bug#43913.
mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh:
Added the test file to create a link from $MYSQLTEST_VARDIR/std_data
to $MYSQLTEST_VARDIR/std_data_master_link
mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh:
Added the test file to create a link from $MYSQLTEST_VARDIR/std_data
to $MYSQLTEST_VARDIR/std_data_slave_link
mysql-test/suite/rpl/t/rpl_loaddata_symlink.test:
Added the test file to verify if loading data infile will work fine
if the path of the load data file is a symbolic link.
sql/rpl_rli.cc:
Added call 'my_realpath' function for avoiding sometimes the 'fn_format'
function can't format real path rightly.
<tmp_tbl> with RBL
When binlogging the statement, the server always handle the existing
object as a table, even though it is a view. However a view is
handled differently in other parts of the code thus leading the
statement to crash in RBL if the view exists.
This happens because the underlying tables for the view are not opened
when we try to call store_create_info() on the view in order to build
a CREATE TABLE statement.
This patch will only address the crash problem, other binlogging
problems related to CREATE TABLE IF NOT EXISTS LIKE when the existing
object is a view will be solved by BUG 47442.
In RBR, All statements operating on temporary tables should not be binlogged.
Despite this fact, after executing 'TRUNCATE... ' on a temporary table,
the command is still logged, even if in row-based mode. Consequently, this raises
problems in the slave as the table may not exist, resulting in an
execution failure. Ultimately, this causes the slave to report
an error and abort.
After this patch, 'TRUNCATE ...' statement on a temporary table will not be
binlogged in RBR.
Problem: Some system functions that could return different values on
master and slave were not marked unsafe. In particular:
GET_LOCK
IS_FREE_LOCK
IS_USED_LOCK
MASTER_POS_WAIT
RELEASE_LOCK
SLEEP
SYSDATE
VERSION
Fix: Mark these functions unsafe.
mysql-test/extra/rpl_tests/rpl_stm_000001.test:
- The test does not work in mixed mode any more, since it tries to
simulate an error in the sql thread in a query that uses get_lock.
Since get_lock now causes the query to be logged in row format,
the error didn't happen. Hence, we now force statement mode.
- Warnings must be disabled when the unsafe query is issued.
- Replaced some save_master_pos+connection slave+sync_with_master
by sync_slave_with_master.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_row.result:
updated result file
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file
mysql-test/suite/binlog/t/binlog_killed.test:
binlog_killed only works in statement format now, since
it switches to row mode in mixed mode.
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test:
suppress warnings for unsafe statements
mysql-test/suite/binlog/t/binlog_stm_row.test:
- Suppress warnings in test that causes warnings.
- The test sets binlog format explicitly, so no need to execute it
twice.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Added test for all unsafe system functions. This test also includes
system functions that were unsafe prior to BUG#47995.
mysql-test/suite/rpl/r/rpl_err_ignoredtable.result:
updated result file
mysql-test/suite/rpl/r/rpl_get_lock.result:
updated result file
mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result:
new result file
mysql-test/suite/rpl/r/rpl_stm_000001.result:
updated result file
mysql-test/suite/rpl/r/rpl_trigger.result:
updated result file
mysql-test/suite/rpl/t/rpl_err_ignoredtable.test:
- suppress warnings for unsafe statement
- replaced save_master_pos+connection slave+sync_with_master
with sync_slave_with_master
mysql-test/suite/rpl/t/rpl_get_lock.test:
update test case that causes new warnings
mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test:
Added new test case for nondeterministic functions.
mysql-test/suite/rpl/t/rpl_trigger.test:
update test case that causes new warnings
sql/item_create.cc:
Marked some system functions unsafe.
sql/item_strfunc.cc:
Clarified comment related to this bug.
sql/sql_yacc.yy:
Marked sysdate unsafe.
Detailed revision comments:
r6101 | jyang | 2009-10-23 11:45:50 +0300 (Fri, 23 Oct 2009) | 7 lines
branches/zip: Update test result with the WARN_LEVEL_ERROR
to WARN_LEVEL_WARN change. This is the same result as
submitted in rb://172 review, which approved by Sunny Bains
and Marko.
Detailed revision comments:
r6100 | jyang | 2009-10-22 06:51:07 +0300 (Thu, 22 Oct 2009) | 6 lines
branches/zip: As a request from mysql, WARN_LEVEL_ERROR cannot
be used for push_warning_* call any more. Switch to
WARN_LEVEL_WARN. Bug #47233.
rb://172 approved by Sunny Bains and Marko.
Detailed revision comments:
r6095 | vasil | 2009-10-19 16:04:59 +0300 (Mon, 19 Oct 2009) | 7 lines
branches/zip:
Fix Bug#47808 innodb_information_schema.test fails when run under valgrind
by using the wait_until_rows_count macro that loops until the number of
rows becomes 14 instead of sleep 0.1, which is obviously very fragile.
The 'rpl_get_master_version_and_clock' test verifies if the slave I/O
thread tries to reconnect to master when it tries to get the values of
the UNIX_TIMESTAMP, SERVER_ID from master under network disconnection.
So the master server is restarted for making the transient network
disconnection. Restarting master server can bring two problems as following:
1. The time out error is encountered sporadically. The slave I/O thread tries
to reconnect master ten times, which is set in my.cnf. So in the test
framework sporadically the slave I/O thread really stoped when it can't
reconnect to master in the ten times successfully before the master starts,
then the time out error will be encountered while waiting for the slave to
start.
2. These warnings and errors are produced in server log file when
the slave I/O thread tries to get the values of the UNIX_TIMESTAMP,
SERVER_ID from master under the transient network disconnection.
To fix problem 1, increase the master retry count to sixty times,
so that the slave I/O thread has enough time to reconnect master
successfully.
To fix problem 2, suppress these warnings and errors by mtr suppression,
because they are expected.
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock-slave.opt:
Added the *.opt file for increasing master retry count to
sixty times.
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test:
Added mtr suppression for suppressing warnings and errors
in server log file.
binlog, replication aborts
In SBR or MBR, the schema name is not being written to the binlog
when executing a LOAD DATA statement. This becomes a problem when
the current database (lets call it db1) is different from the
table's schema (lets call it db2). For instance, take the
following statements:
use db1;
load data local infile 'infile.txt' into table db2.t
Should this statement be logged without t's schema (db2), when
replaying it, one can get db1.t populated instead of db2.t (if
db1.t exists). On the other hand, if there is no db1.t at all,
replication will stop.
We fix this by always logging the table (in load file) with fully
qualified name when its schema is different from the current
database or when no default database was selected.
Backporting BUG#43789 to mysql-5.1-bugteam
The replication was generating corrupted data, warning messages on Valgrind
and aborting on debug mode while replicating a "null" to "not null" field.
Specifically the unpack_row routine, was considering the slave's table
definition and trying to retrieve a field value, where there was nothing to be
retrieved, ignoring the fact that the value was defined as "null" by the master.
To fix the problem, we proceed as follows:
1 - If it is not STRICT sql_mode, implicit default values are used, regardless
if it is multi-row or single-row statement.
2 - However, if it is STRICT mode, then a we do what follows:
2.1 If it is a transactional engine, we do a rollback on the first NULL that is
to be set into a NOT NULL column and return an error.
2.2 If it is a non-transactional engine and it is the first row to be inserted
with multi-row, we also return the error. Otherwise, we proceed with the
execution, use implicit default values and print out warning messages.
Unfortunately, the current patch cannot mimic the behavior showed by the master
for updates on multi-tables and multi-row inserts. This happens because such
statements are unfolded in different row events. For instance, considering the
following updates and strict mode:
(master)
create table t1 (a int);
create table t2 (a int not null);
insert into t1 values (1);
insert into t2 values (2);
update t1, t2 SET t1.a=10, t2.a=NULL;
t1 would have (10) and t2 would have (0) as this would be handled as a
multi-row update. On the other hand, if we had the following updates:
(master)
create table t1 (a int);
create table t2 (a int);
(slave)
create table t1 (a int);
create table t2 (a int not null);
(master)
insert into t1 values (1);
insert into t2 values (2);
update t1, t2 SET t1.a=10, t2.a=NULL;
On the master t1 would have (10) and t2 would have (NULL). On
the slave, t1 would have (10) but the update on t1 would fail.
Backporting BUG#38173 to mysql-5.1-bugteam
The reason of the bug was incompatibile with the master side behaviour.
INSERT query on the master is allowed to insert into a table without specifying
values of DEFAULT-less fields if sql_mode is not strict.
Fixed with checking sql_mode by the sql thread to decide how to react.
Non-strict sql_mode should allow Write_rows event to complete.
todo: warnings can be shown via show slave status, still this is a
separate rather general issue how to show warnings for the slave threads.
file
Issuing 'FLUSH LOGS' does not close and reopen indexfile.
Instead a SEEK_SET is performed.
This patch makes index file to be closed and reopened whenever a
rotation happens (FLUSH LOGS is issued or binary log exceeds
maximum configured size).
mysql-test/suite/binlog/r/binlog_delete_and_flush_index.result:
Result file.
mysql-test/suite/binlog/t/binlog_delete_and_flush_index.test:
Test case.
sql/log.cc:
Added LOG_CLOSE_INDEX flag when calling MYSQL_BIN_LOG::close
from within MYSQL_BIN_LOG::new_file_impl (which should just be
called whenever a rotation is to happen - FLUSH LOGS issued or
binlog size exceeds the maximum configured).
