Fix a bunch of issues found with locking, ordering, and non-thread-safe stuff
in Relay_log_info.
Now able to do a simple benchmark, showing 4.5 times speedup for applying a
binlog with 10000 REPLACE statements.
Fix some bugs around waiting for worker threads to end during SQL slave stop.
Free Log_event after parallel execution (still needs to be made thread-safe by
using rpl_group_info rather than rli).
Wait for all worker threads to finish when stopping the SQL thread.
(Only a basic wait; this still needs to be fixed to include timeout
logic as in sql_slave_killed()).
Hook in the wait-for-prior-commit logic (not really tested yet).
Clean up some resource maintenance around rpl_group_info (may still be some
smaller issues there though).
Add a ToDo list at the top of rpl_parallel.cc
Now whenever we reach the GTID point requested from the slave (when using GTID
position to connect), we send a fake Gtid_list event. This event is used by
the slave to know the current old-style position for MASTER_POS_WAIT(), and
later the similar binlog position for MASTER_GTID_WAIT().
Without this fake event, if the slave is already fully up-to-date with the
master, there may be no events sent at the given position for an indeterminate
time.
If the mysql.gtid_slave_pos table is not available, we cannot load nor update
the current GTID position persistently. This can happen eg. after an upgrade,
before mysql_upgrade_db is run, or if the table is InnoDB and the server is
restarted without the InnoDB storage engine enabled.
Before, replication always failed to start if the table was unavailable. With
this patch, we try to continue with old-style replication, after suitable
complaints in the error log. In strict mode, or if slave is configured to use
GTID, slave still refuses to start.
There was some old code that cleared the position in CHANGE MASTER,
it was forgotten to be removed.
In addition, add code that saves/restores the old-style position
when we nuke the old relay logs as part of GTID slave start.
Normally we will not use these, but it could be useful in case
the GTID connect fails and user wants to go back to the old-style
coordinates.
Fix problems related to reconnect. When we need to reconnect (ie. explict
stop/start of just the IO thread by user, or automatic reconnect due to
loosing network connection with the master), it is a bit complex to correctly
resume at the right point without causing duplicate or missing events in the
relay log. The previous code had multiple problems in this regard.
With this patch, the problem is solved as follows. The IO thread keeps track
(in memory) of which GTID was last queued to the relay log. If it needs to
reconnect, it resumes at that GTID position. It also counts number of events
received within the last, possibly partial, event group, and skips the same
number of events after a reconnect, so that events already enqueued before the
reconnect are not duplicated.
(There is no need to keep any persistent state; whenever we restart slave
threads after both of them being stopped (such as after server restart), we
erase the relay logs and start over from the last GTID applied by SQL thread.
But while the SQL thread is running, this patch is needed to get correct relay
log).
There were several cases where the slave GTID position was not loaded
correctly before being used. This caused various failures such as
corrupting the position at slave start and empty values of
@@gtid_slave_pos and @@gtid_current_pos.
Fixed by adding more checks for loaded position, and by always loading
the position at server startup.
When @@GLOBAL.gtid_strict_mode=1, then certain operations result
in error that would otherwise result in out-of-order binlog files
between servers.
GTID sequence numbers are now allocated independently per domain;
this results in less/no holes in GTID sequences, increasing the
likelyhood that diverging binlogs will be caught by the slave when
GTID strict mode is enabled.
Change of user interface to be more logical and more in line with expectations
to work similar to old-style replication.
User can now explicitly choose in CHANGE MASTER whether binlog position is
taken into account (master_gtid_pos=current_pos) or not (master_gtid_pos=
slave_pos) when slave connects to master.
@@gtid_pos is replaced by three separate variables @@gtid_slave_pos (can
be set by user, replicated GTIDs only), @@gtid_binlog_pos (read only), and
@@gtid_current_pos (a combination of the two, most recent GTID within each
domain). mysql.rpl_slave_state is renamed to mysql.gtid_slave_pos to match.
This fixes MDEV-4474.
Implement START SLAVE UNTIL master_gtid_pos = "<GTID position>".
Add test cases, including a test showing how to use this to promote
a new master among a set of slaves.
- Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
- Concurrent LOAD DATA commands from two master connections could use the same file name
Other bug fixes:
- Enlarge buffer for connection names with 'special characters' one can't store in filenames
Optimization:
- Don't do 'lower case' of connection names. We can use cmp_connection_name, where we already have the connection name in lower case.
mysql-test/suite/multi_source/load_data.result:
Test case for MDEV-4352
mysql-test/suite/multi_source/load_data.test:
Test case for MDEV-4352
sql/log_event.cc:
Fixed: MDEV-4352
- Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
- Concurrent LOAD DATA commands from two master connections could use the same file name
The fix was to add the connection name (if one exists) to all slave temporary files used by LOAD DATA
sql/rpl_mi.cc:
Enlarge buffer for connection names with 'special characters' one can't store in filenames
Use mi->cmp_connection_name for connection file names.
sql/rpl_rli.cc:
Use mi->cmp_connection_name for connection file names.
sql/slave.cc:
Removed not needed empty line
sql/sql_const.h:
Added MAX_FILENAME_MBWIDTH to be able to calculate buffer length for connection_names stored in file names
sql/sql_repl.cc:
Use mi->cmp_connection_name for connection file names.
sql/keycaches.cc:
Added free_all_rpl_filters() to be able to free all filters at cleanup
sql/keycaches.h:
Added prototype
sql/rpl_rli.cc:
Fixed compiler warning
sql/slave.cc:
Free all rpl_filters at cleanup
sql/sp.cc:
Fixed compiler warning when not all struct elements was initialized
sql/sql_acl.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits_summary.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_ews_global_by_event_name.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_instances.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_summary.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_performance_timers.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_consumers.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_instruments.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_timers.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_sync_instances.cc:
Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_threads.cc:
Fixed compiler warning when not all struct elements was initialized
storage/xtradb/os/os0file.c:
Fixed compiler warning when not all struct elements was initialized
Users can set different repplication filter rules for each replication connection, in my.cnf or command line.
But the rules set online will not record in master.info, it means if users restart MySQL, these rules will lose.
So if users wantn't their replication filter rules lose, they should write the rules in my.cnf.
Users can set rules by 2 ways:
1. Online SET command, "SET connection_name.replication_filter_settings = rules;".
2. In my.cnf, "connection_name.replication_filter_settings = rules".
If no connection_name in my.cnf, this rule will apply for ALL replication connection.
If no connetion_name in SET statement, this rull will apply for default_connection_name.
Merge of 10.0-mdev26 feature tree into 10.0-base.
Global transaction ID is prepended to each event group in the binlog.
Slave connect can request to start from GTID position instead of specifying
file name/offset of master binlog. This facilitates easy switch to a new
master.
Slave GTID state is stored in a table mysql.rpl_slave_state, which can be
InnoDB to get crash-safe slave state.
GTID includes a replication domain ID, allowing to keep track of distinct
positions for each of multiple masters.
Fixed MDEV-4033 Unable to use slave's temporary directory /tmp - Can't create/write to file '/tmp/SQL_LOAD-' (Errcode: 17 "File exists")
- Cache value of check_temp_dir() to avoid race condition
- Set $rpl_server_count to avoid error in show_rpl_debug_info.inc
mysql-test/suite/multi_source/info_logs.test:
Set $rpl_server_count
mysql-test/suite/multi_source/multisource.test:
Set $rpl_server_count
mysql-test/suite/multi_source/relaylog_events.test:
Set $rpl_server_count
mysql-test/suite/multi_source/reset_slave.test:
Set $rpl_server_count
mysql-test/suite/multi_source/simple.test:
Set $rpl_server_count
mysql-test/suite/multi_source/skip_counter.test:
Set $rpl_server_count
mysql-test/suite/multi_source/status_vars.test:
Set $rpl_server_count
sql/slave.cc:
Cache value of check_temp_dir() to avoid race condition
Move combining slave and gtid binlog state into a separate function.
Make SHOW ALL SLAVES STATUS use the same function, so it shows the
same value used by slave connect.
Add a test case.
update 5.1 to replicate from 10.0 and
to show the server version (as of 10.0) correctly
sql-common/client.c:
mdev:4088
sql/slave.cc:
use the version number, not just the first character of the version string
(we want 10 > 4 not "10" < "4").
Problem - When the slave was disconnected from the master, under certain
conditions, upon reconnect, it will report that it received a
packet larger the slave_max_allowed_packet which causes the
replication to stop.
Analysis -The reason of this failure is that on reconnect
the slave sets the max_allowed_packet from the master's mi->mysql
object which keeps the max_allowed_packet as 1MB. This causes the
slave to report such error on recieving packet bigger than 1MB.
START SLAVE on the slave fixes the problem since it restarts
slave threads which initializes the max_allowed_packet to
slave_max_allowed_packet.
Fix - The problem is fixed by some code refactoring and introduction of a new
function which updates the max_allowed_packet for the THD object of the
slave thread and the mysql->options max_allowed_packet.
Problem - When the slave was disconnected from the master, under certain
conditions, upon reconnect, it will report that it received a
packet larger the slave_max_allowed_packet which causes the
replication to stop.
Analysis -The reason of this failure is that on reconnect
the slave sets the max_allowed_packet from the master's mi->mysql
object which keeps the max_allowed_packet as 1MB. This causes the
slave to report such error on recieving packet bigger than 1MB.
START SLAVE on the slave fixes the problem since it restarts
slave threads which initializes the max_allowed_packet to
slave_max_allowed_packet.
Fix - The problem is fixed by some code refactoring and introduction of a new
function which updates the max_allowed_packet for the THD object of the
slave thread and the mysql->options max_allowed_packet.
Fix MDEV-4278: Slave does not check that master understands GTID.
Now the slave will abort with a suitable error if an attempt is made to connect
with GTID to a master that does not support GTID.
