mirror of
https://github.com/MariaDB/server.git
synced 2025-07-29 05:21:33 +03:00
MDEV-4991: GTID binlog indexing
Improve the performance of slave connect using B+-Tree indexes on each binlog file. The index allows fast lookup of a GTID position to the corresponding offset in the binlog file, as well as lookup of a position to find the corresponding GTID position. This eliminates a costly sequential scan of the starting binlog file to find the GTID starting position when a slave connects. This is especially costly if the binlog file is not cached in memory (IO cost), or if it is encrypted or a lot of slaves connect simultaneously (CPU cost). The size of the index files is generally less than 1% of the binlog data, so not expected to be an issue. Most of the work writing the index is done as a background task, in the binlog background thread. This minimises the performance impact on transaction commit. A simple global mutex is used to protect index reads and (background) index writes; this is fine as slave connect is a relatively infrequent operation. Here are the user-visible options and status variables. The feature is on by default and is expected to need no tuning or configuration for most users. binlog_gtid_index On by default. Can be used to disable the indexes for testing purposes. binlog_gtid_index_page_size (default 4096) Page size to use for the binlog GTID index. This is the size of the nodes in the B+-tree used internally in the index. A very small page-size (64 is the minimum) will be less efficient, but can be used to stress the BTree-code during testing. binlog_gtid_index_span_min (default 65536) Control sparseness of the binlog GTID index. If set to N, at most one index record will be added for every N bytes of binlog file written. This can be used to reduce the number of records in the index, at the cost only of having to scan a few more events in the binlog file before finding the target position Two status variables are available to monitor the use of the GTID indexes: Binlog_gtid_index_hit Binlog_gtid_index_miss The "hit" status increments for each successful lookup in a GTID index. The "miss" increments when a lookup is not possible. This indicates that the index file is missing (eg. binlog written by old server version without GTID index support), or corrupt. Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This commit is contained in:
@ -8,12 +8,12 @@ wait/synch/mutex/sql/Ack_receiver::mutex YES YES
|
||||
wait/synch/mutex/sql/Cversion_lock YES YES
|
||||
wait/synch/mutex/sql/Delayed_insert::mutex YES YES
|
||||
wait/synch/mutex/sql/Event_scheduler::LOCK_scheduler_state YES YES
|
||||
wait/synch/mutex/sql/Gtid_index_writer::gtid_index_mutex YES YES
|
||||
wait/synch/mutex/sql/gtid_waiting::LOCK_gtid_waiting YES YES
|
||||
wait/synch/mutex/sql/hash_filo::lock YES YES
|
||||
wait/synch/mutex/sql/HA_DATA_PARTITION::LOCK_auto_inc YES YES
|
||||
wait/synch/mutex/sql/LOCK_active_mi YES YES
|
||||
wait/synch/mutex/sql/LOCK_after_binlog_sync YES YES
|
||||
wait/synch/mutex/sql/LOCK_audit_mask YES YES
|
||||
select * from performance_schema.setup_instruments
|
||||
where name like 'Wait/Synch/Rwlock/sql/%'
|
||||
and name not in (
|
||||
|
@ -23,6 +23,7 @@ from performance_schema.file_summary_by_instance
|
||||
where file_name like "%master-%" order by file_name;
|
||||
FILE_NAME EVENT_NAME COUNT_READ COUNT_WRITE SUM_NUMBER_OF_BYTES_READ SUM_NUMBER_OF_BYTES_WRITE
|
||||
master-bin.000001 wait/io/file/sql/binlog MANY MANY MANY MANY
|
||||
master-bin.000001.idx wait/io/file/sql/gtid_index NONE MANY NONE MANY
|
||||
master-bin.index wait/io/file/sql/binlog_index MANY MANY MANY MANY
|
||||
select * from performance_schema.file_summary_by_instance
|
||||
where file_name like "%slave-%" order by file_name;
|
||||
@ -112,6 +113,7 @@ where file_name like "%slave-%"
|
||||
order by file_name;
|
||||
FILE_NAME EVENT_NAME COUNT_READ COUNT_WRITE SUM_NUMBER_OF_BYTES_READ SUM_NUMBER_OF_BYTES_WRITE
|
||||
slave-bin.000001 wait/io/file/sql/binlog MANY MANY MANY MANY
|
||||
slave-bin.000001.idx wait/io/file/sql/gtid_index NONE MANY NONE MANY
|
||||
slave-bin.index wait/io/file/sql/binlog_index MANY MANY MANY MANY
|
||||
slave-relay-bin.000001 wait/io/file/sql/relaylog MANY MANY MANY MANY
|
||||
slave-relay-bin.000002 wait/io/file/sql/relaylog MANY MANY MANY MANY
|
||||
|
Reference in New Issue
Block a user