database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Code Activity

MDEV-41214 PURGE BINARY LOGS does not delete non-active binlog files

Browse Source 
PURGE BINARY LOGS could end with a warning instead of deleting files, as
the test below shows when run on the base of this commit. The warning
names a wrong reason. It follows an automatic purge that was refused by
the slave guard, slave_connections_needed_for_purge: the automatic purge
does not delete a binlog while it might still be needed by a slave, not
necessarily a connected one, that is while fewer than the variable's
number of slaves are connected.
Since MDEV-34504 the manual PURGE is meant to ignore the guard.
Yet it still protects a binlog that a slave is reading.

Technically this was possible because the automatic and the manual purge
share a code path, can_purge_log(), which cached the refusal and replayed
it, under the reason of the first arm below, to any later purge:

  if (is_active(name) ||
      (!is_relay_log && waiting_for_slave_to_change_binlog &&
       purge_sending_new_binlog_file == sending_new_binlog_file &&
       !strcmp(name, purge_binlog_name)))
    reason= "it is the current active binlog";

So "active" had to be read as "active or busy". The fix removes this
duality too: the two arms are now separate and have their own reasons.

The main issue is fixed by narrowing the busy arm with an additional
`!interactive &&` conjunct, so that a manual PURGE never consults the
cached refusal.

Running the test below on an unfixed 11.4 server (source tree at commit
42038e1145) fails with "Result length mismatch" and, in part 2, this
diff against the recorded result:

   PURGE BINARY LOGS TO 'master-bin.000006';
  +Warnings:
  +Note	1375	Binary log 'master-bin.000003' is not purged because it is the current active binlog
   SHOW WARNINGS;
   Level	Code	Message
  +Note	1375	Binary log 'master-bin.000003' is not purged because it is the current active binlog
   show binary logs;
   Log_name	File_size
  +master-bin.000003	#
  +master-bin.000004	#
  +master-bin.000005	#
   master-bin.000006	#
   master-bin.000007	#
   master-bin.000008	#

Test binlog.binlog_purge_stale_refusal_cache: the test sets
slave_connections_needed_for_purge=1, binlog_expire_logs_seconds=0 and
max_binlog_total_size=0 first and restores them at the end. It starts
with RESET MASTER so that the binlog names, 000003 and so on, do not depend
on the tests run before it in the same mtr environment. Part 1 is a control with
nothing cached; part 2 makes an automatic purge refuse 000003 (the files
must be older than binlog_expire_logs_seconds, hence --real_sleep 2 before
the rotation) and then runs PURGE BINARY LOGS TO 'master-bin.000006',
which must remove 000003..000005.

As a workaround, a PURGE could work if the following settings are made
before it, in this order:

  SET GLOBAL binlog_expire_logs_seconds=0, slave_connections_needed_for_purge=0;

The second setting invalidates the cached refusal. The first one keeps
the purge that this setting triggers from refusing and caching again.
Note that both settings change the server's policy, not only this one
PURGE: the time-based expiry and the slave guard are off until they are
restored.
This commit is contained in:
Andrei Elkin
2026-10-08 14:48:13 +03:00
committed by Kristian Nielsen
parent f7eeefbed7
commit b2064bbc17
3 changed files with 171 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

72
mysql-test/suite/binlog/r/binlog_purge_stale_refusal_cache.result Normal file
View File

@@ -0,0 +1,72 @@
SET @@global.binlog_expire_logs_seconds= 0;
SET @@global.max_binlog_total_size= 0;
SET @@global.slave_connections_needed_for_purge= 1;
SELECT @@global.slave_connections_needed_for_purge;
@@global.slave_connections_needed_for_purge
1
SELECT @@global.binlog_expire_logs_seconds, @@global.max_binlog_total_size;
@@global.binlog_expire_logs_seconds @@global.max_binlog_total_size
0 0
# No replica has ever connected:
SELECT COUNT(*) AS dump_threads FROM information_schema.PROCESSLIST
WHERE COMMAND = 'Binlog Dump';
dump_threads
0
RESET MASTER;
#
# Part 1 (control). Nothing has attempted a purge yet, so no refusal is
# cached, and the interactive purge correctly ignores the slave's guard.
#
FLUSH BINARY LOGS;
FLUSH BINARY LOGS;
FLUSH BINARY LOGS;
show binary logs;
Log_name File_size
master-bin.000001 #
master-bin.000002 #
master-bin.000003 #
master-bin.000004 #
PURGE BINARY LOGS TO 'master-bin.000003';
SHOW WARNINGS;
Level Code Message
show binary logs;
Log_name File_size
master-bin.000003 #
master-bin.000004 #
#
# Part 2. Let one automatic purge be refused, then try PURGE that must
# be effective on the fixed server.
#
FLUSH BINARY LOGS;
FLUSH BINARY LOGS;
FLUSH BINARY LOGS;
show binary logs;
Log_name File_size
master-bin.000003 #
master-bin.000004 #
master-bin.000005 #
master-bin.000006 #
master-bin.000007 #
# Let the files age past binlog_expire_logs_seconds: purge_logs_before_date
# skips can_purge_log() for any file whose mtime is not older than that.
SET @@global.binlog_expire_logs_seconds= 1;
# Rotating attempts an automatic purge, which the guard refuses. That
# refusal is what populates the cache.
FLUSH BINARY LOGS;
# The automatic refusal states the correct reason in the error log:
FOUND 1 /Binary log .master-bin.000003. is not purged because less than .slave_connections_needed_for_purge. slaves have processed it/ in mysqld.1.err
# Now the interactive purge. It should behave exactly as in part 1.
PURGE BINARY LOGS TO 'master-bin.000006';
# On affected versions this reports master-bin.000003 as the current active
# binlog, while master-bin.000008 is active:
SHOW WARNINGS;
Level Code Message
show binary logs;
Log_name File_size
master-bin.000006 #
master-bin.000007 #
master-bin.000008 #
SET @@global.binlog_expire_logs_seconds = $save_binlog_expire_logs_seconds;
SET @@global.max_binlog_total_size= $save_max_binlog_total_size;
SET @@global.slave_connections_needed_for_purge= $save_slave_connections_needed_for_purge;
RESET MASTER;

90
mysql-test/suite/binlog/t/binlog_purge_stale_refusal_cache.test Normal file
View File

@@ -0,0 +1,90 @@
# MDEV-41214 PURGE BINARY LOGS does not delete non-active binlog files
#
# PURGE BINARY LOGS could end with a warning instead of deleting files, as
# this test shows when run on the base of the fix. The warning names a wrong
# reason. It follows an automatic purge that was refused by the slave guard,
# slave_connections_needed_for_purge: the automatic purge does not delete a
# binlog while it might still be needed by a slave, not necessarily a
# connected one, that is while fewer than the variable's number of slaves
# are connected.
#
# Part 1 is a control: the same statement shape, with nothing cached, succeeds.
#
# The test sets slave_connections_needed_for_purge=1,
# binlog_expire_logs_seconds=0 and max_binlog_total_size=0, and restores the
# old values at the end. With slave_connections_needed_for_purge=0 the guard
# is off and the bug does not show.
#
--source include/have_log_bin.inc
--let $save_slave_connections_needed_for_purge= `SELECT @@global.slave_connections_needed_for_purge`
--let $save_binlog_expire_logs_seconds= `SELECT @@global.binlog_expire_logs_seconds`
--let $save_max_binlog_total_size= `SELECT @@global.max_binlog_total_size`
SET @@global.binlog_expire_logs_seconds= 0;
SET @@global.max_binlog_total_size= 0;
SET @@global.slave_connections_needed_for_purge= 1;
SELECT @@global.slave_connections_needed_for_purge;
SELECT @@global.binlog_expire_logs_seconds, @@global.max_binlog_total_size;
--echo # No replica has ever connected:
SELECT COUNT(*) AS dump_threads FROM information_schema.PROCESSLIST
WHERE COMMAND = 'Binlog Dump';
RESET MASTER;
--echo #
--echo # Part 1 (control). Nothing has attempted a purge yet, so no refusal is
--echo # cached, and the interactive purge correctly ignores the slave's guard.
--echo #
FLUSH BINARY LOGS;
FLUSH BINARY LOGS;
FLUSH BINARY LOGS;
--source include/show_binary_logs.inc
PURGE BINARY LOGS TO 'master-bin.000003';
SHOW WARNINGS;
--source include/show_binary_logs.inc
--let $first= query_get_value(SHOW BINARY LOGS, Log_name, 1)
if ($first != master-bin.000003)
{
--echo # Control failed: expected master-bin.000003 first, got $first
--die Interactive purge did not work even with no cached refusal
}
--echo #
--echo # Part 2. Let one automatic purge be refused, then try PURGE that must
--echo # be effective on the fixed server.
--echo #
FLUSH BINARY LOGS;
FLUSH BINARY LOGS;
FLUSH BINARY LOGS;
--source include/show_binary_logs.inc
--echo # Let the files age past binlog_expire_logs_seconds: purge_logs_before_date
--echo # skips can_purge_log() for any file whose mtime is not older than that.
--real_sleep 2
SET @@global.binlog_expire_logs_seconds= 1;
--echo # Rotating attempts an automatic purge, which the guard refuses. That
--echo # refusal is what populates the cache.
FLUSH BINARY LOGS;
--let $first= query_get_value(SHOW BINARY LOGS, Log_name, 1)
if ($first != master-bin.000003)
{
--echo # Setup failed: expected the automatic purge to delete nothing.
--die The automatic purge was not refused, so nothing was cached
}
--echo # The automatic refusal states the correct reason in the error log:
--let SEARCH_FILE= $MYSQLTEST_VARDIR/log/mysqld.1.err
--let SEARCH_PATTERN= Binary log .master-bin.000003. is not purged because less than .slave_connections_needed_for_purge. slaves have processed it
--source include/search_pattern_in_file.inc
--echo # Now the interactive purge. It should behave exactly as in part 1.
PURGE BINARY LOGS TO 'master-bin.000006';
--echo # On affected versions this reports master-bin.000003 as the current active
--echo # binlog, while master-bin.000008 is active:
SHOW WARNINGS;
--source include/show_binary_logs.inc
--evalp SET @@global.binlog_expire_logs_seconds = $save_binlog_expire_logs_seconds
--evalp SET @@global.max_binlog_total_size= $save_max_binlog_total_size
--evalp SET @@global.slave_connections_needed_for_purge= $save_slave_connections_needed_for_purge
RESET MASTER;

13
sql/log.cc
View File

@@ -6594,14 +6594,19 @@ MYSQL_BIN_LOG::can_purge_log(const char *log_file_name_arg,
const char *reason;
DBUG_ASSERT(is_relay_log || !opt_binlog_engine_hton);
if (is_active(log_file_name_arg) ||
(!is_relay_log && waiting_for_slave_to_change_binlog &&
purge_sending_new_binlog_file == sending_new_binlog_file &&
!strcmp(log_file_name_arg, purge_binlog_name)))
if (is_active(log_file_name_arg))
{
reason= "it is the current active binlog";
goto error;
}
if (!interactive && !is_relay_log && waiting_for_slave_to_change_binlog &&
purge_sending_new_binlog_file == sending_new_binlog_file &&
!strcmp(log_file_name_arg, purge_binlog_name))
{
reason= "it is in use by a slave thread or less than "
"'slave_connections_needed_for_purge' slaves have processed it";
goto error;
}
DBUG_ASSERT(!is_relay_log || binlog_xid_count_list.is_empty());
if (!is_relay_log)