database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-08-08 11:22:35 +03:00
Code Activity

MDEV-31404 Implement binlog_space_limit

Browse Source 
binlog_space_limit is a variable in Percona server used to limit the total
size of all binary logs.

This implementation is based on code from Percona server 5.7.

In MariaDB we decided to call the variable max-binlog-total-size to be
similar to max-binlog-size. This makes it easier to find in the output
from 'mariadbd --help --verbose'). MariaDB will also support
binlog_space_limit for compatibility with Percona.

Some internal notes to explain implementation notes:

- When running MariaDB does not delete binary logs that are either
  used by slaves or have active xid that are not yet committed.

Some implementation notes:

- max-binlog-total-size is by default 0 (no limit).
- max-binlog-total-size can be changed without server restart.
- Binlog file sizes are checked on startup, or if
  max-binlog-total-size is set to a value > 0, not for every log write.
  The total size of all binary logs is cached and dynamically updated
  when updating the binary log on binary log rotation.
- max-binlog-total-size is checked against existing log files during
  serverstart, binlog rotation, FLUSH LOGS, when writing to binary log
  or when max-binlog-total-size changes value.
- Option --slave-connections-needed-for-purge with 1 as default added.
  This allows one to ensure that we do not delete binary logs if there
  is less than 'slave-connections-needed-for-purge' connected.
  Without this option max-binlog-total-size would potentially delete
  binlogs needed by slaves on server startup or when a slave disconnects
  as there are then no connected slaves to protect active binlogs.
- PURGE BINARY LOGS TO ... will be executed as if
  slave-connectitons-needed-for-purge would be zero. In other words
  it will do the purge even if there is no slaves connected. If there
  are connected slaves working on the logs, these will be protected.
- If binary log is on and max-binlog-total_size <> 0 then the status
  variable 'Binlog_disk_use' shows the current size of all old binary
  logs + the state of the current one.
- Removed test of strcmp(log_file_name, log_info.log_file_name) in
  purge_logs_before_date() as this is tested in can_purge_logs()
- To avoid expensive calls of log_in_use() we cache the result for the
  last log that is in use by a slave. Future calls to can_purge_logs()
  for this binary log will be quickly detected and false will be returned
  until a slave starts working on a new log.
- Note that after a binary log rotation caused by max_binlog_size,
  the last log will not be purged directly as it is still in use
  internally. The next binary log write will purge binlogs if needed.

Reviewer:Kristian Nielsen <knielsen@knielsen-hq.org>
This commit is contained in:
Monty
2023-12-03 21:42:44 +02:00
committed by Sergei Golubchik
parent 9933a8cc88
commit 18dfcfdecf
23 changed files with 906 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
mysql-test/suite/sys_vars/r/max_binlog_total_size_basic.result Normal file
View File

@@ -0,0 +1,41 @@
select @@global.max_binlog_total_size;
@@global.max_binlog_total_size
0
select @@session.max_binlog_total_size;
ERROR HY000: Variable 'max_binlog_total_size' is a GLOBAL variable
show global variables like 'max_binlog_total_size';
Variable_name Value
max_binlog_total_size 0
show session variables like 'max_binlog_total_size';
Variable_name Value
max_binlog_total_size 0
select * from information_schema.global_variables where variable_name='max_binlog_total_size';
VARIABLE_NAME VARIABLE_VALUE
MAX_BINLOG_TOTAL_SIZE 0
select * from information_schema.session_variables where variable_name='max_binlog_total_size';
VARIABLE_NAME VARIABLE_VALUE
MAX_BINLOG_TOTAL_SIZE 0
set global max_binlog_total_size=1;
select @@global.max_binlog_total_size, @@global.binlog_space_limit;
@@global.max_binlog_total_size @@global.binlog_space_limit
1 1
set global max_binlog_total_size=1;
select @@global.max_binlog_total_size;
@@global.max_binlog_total_size
1
set global binlog_space_limit=2;
select @@global.max_binlog_total_size, @@global.binlog_space_limit;
@@global.max_binlog_total_size @@global.binlog_space_limit
2 2
set session max_binlog_total_size=1;
ERROR HY000: Variable 'max_binlog_total_size' is a GLOBAL variable and should be set with SET GLOBAL
set global max_binlog_total_size=default;
CREATE USER user1@localhost;
connect con2,localhost,user1,,;
set global max_binlog_total_size=1;
ERROR 42000: Access denied; you need (at least one of) the BINLOG ADMIN privilege(s) for this operation
set global binlog_space_limit=1;
ERROR 42000: Access denied; you need (at least one of) the BINLOG ADMIN privilege(s) for this operation
disconnect con2;
connection default;
DROP USER user1@localhost;

30
mysql-test/suite/sys_vars/r/sysvars_server_embedded.result
View File

@@ -492,6 +492,16 @@ NUMERIC_BLOCK_SIZE NULL
ENUM_VALUE_LIST NO_LOG,MINIMAL,FULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME BINLOG_SPACE_LIMIT
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
VARIABLE_COMMENT Alias for max_binlog_total_size. Compatibility with Percona server.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 18446744073709551615
NUMERIC_BLOCK_SIZE 1
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME BINLOG_STMT_CACHE_SIZE
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
@@ -1902,6 +1912,16 @@ NUMERIC_BLOCK_SIZE 4096
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME MAX_BINLOG_TOTAL_SIZE
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
VARIABLE_COMMENT Maximum space to use for all binary logs. Extra logs are deleted on server start, log rotation, FLUSH LOGS or when writing to binlog. Default is 0, which means no size restrictions. See also slave_connections_needed_for_purge
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 18446744073709551615
NUMERIC_BLOCK_SIZE 1
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME MAX_CONNECTIONS
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
@@ -3392,6 +3412,16 @@ NUMERIC_BLOCK_SIZE NULL
ENUM_VALUE_LIST OFF,ON
READ_ONLY NO
COMMAND_LINE_ARGUMENT OPTIONAL
VARIABLE_NAME SLAVE_CONNECTIONS_NEEDED_FOR_PURGE
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE INT UNSIGNED
VARIABLE_COMMENT Minimum number of connected slaves required for automatic binary log purge with max_binlog_total_size, binlog_expire_logs_seconds or binlog_expire_logs_days.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 4294967295
NUMERIC_BLOCK_SIZE 1
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME SLAVE_MAX_ALLOWED_PACKET
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED

30
mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result
View File

@@ -542,6 +542,16 @@ NUMERIC_BLOCK_SIZE NULL
ENUM_VALUE_LIST NO_LOG,MINIMAL,FULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME BINLOG_SPACE_LIMIT
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
VARIABLE_COMMENT Alias for max_binlog_total_size. Compatibility with Percona server.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 18446744073709551615
NUMERIC_BLOCK_SIZE 1
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME BINLOG_STMT_CACHE_SIZE
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
@@ -2102,6 +2112,16 @@ NUMERIC_BLOCK_SIZE 4096
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME MAX_BINLOG_TOTAL_SIZE
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
VARIABLE_COMMENT Maximum space to use for all binary logs. Extra logs are deleted on server start, log rotation, FLUSH LOGS or when writing to binlog. Default is 0, which means no size restrictions. See also slave_connections_needed_for_purge
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 18446744073709551615
NUMERIC_BLOCK_SIZE 1
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME MAX_CONNECTIONS
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
@@ -3962,6 +3982,16 @@ NUMERIC_BLOCK_SIZE NULL
ENUM_VALUE_LIST OFF,ON
READ_ONLY NO
COMMAND_LINE_ARGUMENT OPTIONAL
VARIABLE_NAME SLAVE_CONNECTIONS_NEEDED_FOR_PURGE
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE INT UNSIGNED
VARIABLE_COMMENT Minimum number of connected slaves required for automatic binary log purge with max_binlog_total_size, binlog_expire_logs_seconds or binlog_expire_logs_days.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 4294967295
NUMERIC_BLOCK_SIZE 1
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME SLAVE_DDL_EXEC_MODE
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE ENUM