database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-08-08 11:22:35 +03:00
Code Activity

MDEV-23399: Performance regression with write workloads

Browse Source 
The buffer pool refactoring in MDEV-15053 and MDEV-22871 shifted
the performance bottleneck to the page flushing.

The configuration parameters will be changed as follows:

innodb_lru_flush_size=32 (new: how many pages to flush on LRU eviction)
innodb_lru_scan_depth=1536 (old: 1024)
innodb_max_dirty_pages_pct=90 (old: 75)
innodb_max_dirty_pages_pct_lwm=75 (old: 0)

Note: The parameter innodb_lru_scan_depth will only affect LRU
eviction of buffer pool pages when a new page is being allocated. The
page cleaner thread will no longer evict any pages. It used to
guarantee that some pages will remain free in the buffer pool. Now, we
perform that eviction 'on demand' in buf_LRU_get_free_block().
The parameter innodb_lru_scan_depth(srv_LRU_scan_depth) is used as follows:
 * When the buffer pool is being shrunk in buf_pool_t::withdraw_blocks()
 * As a buf_pool.free limit in buf_LRU_list_batch() for terminating
   the flushing that is initiated e.g., by buf_LRU_get_free_block()
The parameter also used to serve as an initial limit for unzip_LRU
eviction (evicting uncompressed page frames while retaining
ROW_FORMAT=COMPRESSED pages), but now we will use a hard-coded limit
of 100 or unlimited for invoking buf_LRU_scan_and_free_block().

The status variables will be changed as follows:

innodb_buffer_pool_pages_flushed: This includes also the count of
innodb_buffer_pool_pages_LRU_flushed and should work reliably,
updated one by one in buf_flush_page() to give more real-time
statistics. The function buf_flush_stats(), which we are removing,
was not called in every code path. For both counters, we will use
regular variables that are incremented in a critical section of
buf_pool.mutex. Note that show_innodb_vars() directly links to the
variables, and reads of the counters will *not* be protected by
buf_pool.mutex, so you cannot get a consistent snapshot of both variables.

The following INFORMATION_SCHEMA.INNODB_METRICS counters will be
removed, because the page cleaner no longer deals with writing or
evicting least recently used pages, and because the single-page writes
have been removed:
* buffer_LRU_batch_flush_avg_time_slot
* buffer_LRU_batch_flush_avg_time_thread
* buffer_LRU_batch_flush_avg_time_est
* buffer_LRU_batch_flush_avg_pass
* buffer_LRU_single_flush_scanned
* buffer_LRU_single_flush_num_scan
* buffer_LRU_single_flush_scanned_per_call

When moving to a single buffer pool instance in MDEV-15058, we missed
some opportunity to simplify the buf_flush_page_cleaner thread. It was
unnecessarily using a mutex and some complex data structures, even
though we always have a single page cleaner thread.

Furthermore, the buf_flush_page_cleaner thread had separate 'recovery'
and 'shutdown' modes where it was waiting to be triggered by some
other thread, adding unnecessary latency and potential for hangs in
relatively rarely executed startup or shutdown code.

The page cleaner was also running two kinds of batches in an
interleaved fashion: "LRU flush" (writing out some least recently used
pages and evicting them on write completion) and the normal batches
that aim to increase the MIN(oldest_modification) in the buffer pool,
to help the log checkpoint advance.

The buf_pool.flush_list flushing was being blocked by
buf_block_t::lock for no good reason. Furthermore, if the FIL_PAGE_LSN
of a page is ahead of log_sys.get_flushed_lsn(), that is, what has
been persistently written to the redo log, we would trigger a log
flush and then resume the page flushing. This would unnecessarily
limit the performance of the page cleaner thread and trigger the
infamous messages "InnoDB: page_cleaner: 1000ms intended loop took 4450ms.
The settings might not be optimal" that were suppressed in
commit d1ab89037a unless log_warnings>2.

Our revised algorithm will make log_sys.get_flushed_lsn() advance at
the start of buf_flush_lists(), and then execute a 'best effort' to
write out all pages. The flush batches will skip pages that were modified
since the log was written, or are are currently exclusively locked.
The MDEV-13670 message "page_cleaner: 1000ms intended loop took" message
will be removed, because by design, the buf_flush_page_cleaner() should
not be blocked during a batch for extended periods of time.

We will remove the single-page flushing altogether. Related to this,
the debug parameter innodb_doublewrite_batch_size will be removed,
because all of the doublewrite buffer will be used for flushing
batches. If a page needs to be evicted from the buffer pool and all
100 least recently used pages in the buffer pool have unflushed
changes, buf_LRU_get_free_block() will execute buf_flush_lists() to
write out and evict innodb_lru_flush_size pages. At most one thread
will execute buf_flush_lists() in buf_LRU_get_free_block(); other
threads will wait for that LRU flushing batch to finish.

To improve concurrency, we will replace the InnoDB ib_mutex_t and
os_event_t native mutexes and condition variables in this area of code.
Most notably, this means that the buffer pool mutex (buf_pool.mutex)
is no longer instrumented via any InnoDB interfaces. It will continue
to be instrumented via PERFORMANCE_SCHEMA.

For now, both buf_pool.flush_list_mutex and buf_pool.mutex will be
declared with MY_MUTEX_INIT_FAST (PTHREAD_MUTEX_ADAPTIVE_NP). The critical
sections of buf_pool.flush_list_mutex should be shorter than those for
buf_pool.mutex, because in the worst case, they cover a linear scan of
buf_pool.flush_list, while the worst case of a critical section of
buf_pool.mutex covers a linear scan of the potentially much longer
buf_pool.LRU list.

mysql_mutex_is_owner(), safe_mutex_is_owner(): New predicate, usable
with SAFE_MUTEX. Some InnoDB debug assertions need this predicate
instead of mysql_mutex_assert_owner() or mysql_mutex_assert_not_owner().

buf_pool_t::n_flush_LRU, buf_pool_t::n_flush_list:
Replaces buf_pool_t::init_flush[] and buf_pool_t::n_flush[].
The number of active flush operations.

buf_pool_t::mutex, buf_pool_t::flush_list_mutex: Use mysql_mutex_t
instead of ib_mutex_t, to have native mutexes with PERFORMANCE_SCHEMA
and SAFE_MUTEX instrumentation.

buf_pool_t::done_flush_LRU: Condition variable for !n_flush_LRU.

buf_pool_t::done_flush_list: Condition variable for !n_flush_list.

buf_pool_t::do_flush_list: Condition variable to wake up the
buf_flush_page_cleaner when a log checkpoint needs to be written
or the server is being shut down. Replaces buf_flush_event.
We will keep using timed waits (the page cleaner thread will wake
_at least_ once per second), because the calculations for
innodb_adaptive_flushing depend on fixed time intervals.

buf_dblwr: Allocate statically, and move all code to member functions.
Use a native mutex and condition variable. Remove code to deal with
single-page flushing.

buf_dblwr_check_block(): Make the check debug-only. We were spending
a significant amount of execution time in page_simple_validate_new().

flush_counters_t::unzip_LRU_evicted: Remove.

IORequest: Make more members const. FIXME: m_fil_node should be removed.

buf_flush_sync_lsn: Protect by std::atomic, not page_cleaner.mutex
(which we are removing).

page_cleaner_slot_t, page_cleaner_t: Remove many redundant members.

pc_request_flush_slot(): Replaces pc_request() and pc_flush_slot().

recv_writer_thread: Remove. Recovery works just fine without it, if we
simply invoke buf_flush_sync() at the end of each batch in
recv_sys_t::apply().

recv_recovery_from_checkpoint_finish(): Remove. We can simply call
recv_sys.debug_free() directly.

srv_started_redo: Replaces srv_start_state.

SRV_SHUTDOWN_FLUSH_PHASE: Remove. logs_empty_and_mark_files_at_shutdown()
can communicate with the normal page cleaner loop via the new function
flush_buffer_pool().

buf_flush_remove(): Assert that the calling thread is holding
buf_pool.flush_list_mutex. This removes unnecessary mutex operations
from buf_flush_remove_pages() and buf_flush_dirty_pages(),
which replace buf_LRU_flush_or_remove_pages().

buf_flush_lists(): Renamed from buf_flush_batch(), with simplified
interface. Return the number of flushed pages. Clarified comments and
renamed min_n to max_n. Identify LRU batch by lsn=0. Merge all the functions
buf_flush_start(), buf_flush_batch(), buf_flush_end() directly to this
function, which was their only caller, and remove 2 unnecessary
buf_pool.mutex release/re-acquisition that we used to perform around
the buf_flush_batch() call. At the start, if not all log has been
durably written, wait for a background task to do it, or start a new
task to do it. This allows the log write to run concurrently with our
page flushing batch. Any pages that were skipped due to too recent
FIL_PAGE_LSN or due to them being latched by a writer should be flushed
during the next batch, unless there are further modifications to those
pages. It is possible that a page that we must flush due to small
oldest_modification also carries a recent FIL_PAGE_LSN or is being
constantly modified. In the worst case, all writers would then end up
waiting in log_free_check() to allow the flushing and the checkpoint
to complete.

buf_do_flush_list_batch(): Clarify comments, and rename min_n to max_n.
Cache the last looked up tablespace. If neighbor flushing is not applicable,
invoke buf_flush_page() directly, avoiding a page lookup in between.

buf_flush_space(): Auxiliary function to look up a tablespace for
page flushing.

buf_flush_page(): Defer the computation of space->full_crc32(). Never
call log_write_up_to(), but instead skip persistent pages whose latest
modification (FIL_PAGE_LSN) is newer than the redo log. Also skip
pages on which we cannot acquire a shared latch without waiting.

buf_flush_try_neighbors(): Do not bother checking buf_fix_count
because buf_flush_page() will no longer wait for the page latch.
Take the tablespace as a parameter, and only execute this function
when innodb_flush_neighbors>0. Avoid repeated calls of page_id_t::fold().

buf_flush_relocate_on_flush_list(): Declare as cold, and push down
a condition from the callers.

buf_flush_check_neighbor(): Take id.fold() as a parameter.

buf_flush_sync(): Ensure that the buf_pool.flush_list is empty,
because the flushing batch will skip pages whose modifications have
not yet been written to the log or were latched for modification.

buf_free_from_unzip_LRU_list_batch(): Remove redundant local variables.

buf_flush_LRU_list_batch(): Let the caller buf_do_LRU_batch() initialize
the counters, and report n->evicted.
Cache the last looked up tablespace. If neighbor flushing is not applicable,
invoke buf_flush_page() directly, avoiding a page lookup in between.

buf_do_LRU_batch(): Return the number of pages flushed.

buf_LRU_free_page(): Only release and re-acquire buf_pool.mutex if
adaptive hash index entries are pointing to the block.

buf_LRU_get_free_block(): Do not wake up the page cleaner, because it
will no longer perform any useful work for us, and we do not want it
to compete for I/O while buf_flush_lists(innodb_lru_flush_size, 0)
writes out and evicts at most innodb_lru_flush_size pages. (The
function buf_do_LRU_batch() may complete after writing fewer pages if
more than innodb_lru_scan_depth pages end up in buf_pool.free list.)
Eliminate some mutex release-acquire cycles, and wait for the LRU
flush batch to complete before rescanning.

buf_LRU_check_size_of_non_data_objects(): Simplify the code.

buf_page_write_complete(): Remove the parameter evict, and always
evict pages that were part of an LRU flush.

buf_page_create(): Take a pre-allocated page as a parameter.

buf_pool_t::free_block(): Free a pre-allocated block.

recv_sys_t::recover_low(), recv_sys_t::apply(): Preallocate the block
while not holding recv_sys.mutex. During page allocation, we may
initiate a page flush, which in turn may initiate a log flush, which
would require acquiring log_sys.mutex, which should always be acquired
before recv_sys.mutex in order to avoid deadlocks. Therefore, we must
not be holding recv_sys.mutex while allocating a buffer pool block.

BtrBulk::logFreeCheck(): Skip a redundant condition.

row_undo_step(): Do not invoke srv_inc_activity_count() for every row
that is being rolled back. It should suffice to invoke the function in
trx_flush_log_if_needed() during trx_t::commit_in_memory() when the
rollback completes.

sync_check_enable(): Remove. We will enable innodb_sync_debug from the
very beginning.

Reviewed by: Vladislav Vaintroub
This commit is contained in:
Marko Mäkelä
2020-10-15 12:10:42 +03:00
parent 46b1f50098
commit 7cffb5f6e8
73 changed files with 2014 additions and 3826 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
extra/mariabackup/xtrabackup.cc
View File

@@ -3426,9 +3426,7 @@ xb_data_files_close()
{
ut_ad(!os_thread_count);
fil_close_all_files();
if (buf_dblwr) {
buf_dblwr_free();
}
buf_dblwr.close();
}
/***********************************************************************
@@ -4017,7 +4015,6 @@ fail:
}
srv_thread_pool_init();
sync_check_init();
ut_d(sync_check_enable());
/* Reset the system variables in the recovery module. */
trx_pool_init();
recv_sys.create();
@@ -5385,7 +5382,6 @@ static bool xtrabackup_prepare_func(char** argv)
}
sync_check_init();
ut_d(sync_check_enable());
recv_sys.create();
log_sys.create();
recv_sys.recovery_on = true;

8
include/my_pthread.h
View File

@@ -427,12 +427,10 @@ void safe_mutex_free_deadlock_data(safe_mutex_t *mp);
#define MYF_NO_DEADLOCK_DETECTION 2
#ifdef SAFE_MUTEX
#define safe_mutex_assert_owner(mp) \
DBUG_ASSERT((mp)->count > 0 && \
#define safe_mutex_is_owner(mp) ((mp)->count > 0 && \
pthread_equal(pthread_self(), (mp)->thread))
#define safe_mutex_assert_not_owner(mp) \
DBUG_ASSERT(! (mp)->count || \
! pthread_equal(pthread_self(), (mp)->thread))
#define safe_mutex_assert_owner(mp) DBUG_ASSERT(safe_mutex_is_owner(mp))
#define safe_mutex_assert_not_owner(mp) DBUG_ASSERT(!safe_mutex_is_owner(mp))
#define safe_mutex_setflags(mp, F) do { (mp)->create_flags|= (F); } while (0)
#define my_cond_timedwait(A,B,C) safe_cond_timedwait((A),(B),(C),__FILE__,__LINE__)
#define my_cond_wait(A,B) safe_cond_wait((A), (B), __FILE__, __LINE__)

2
include/mysql/psi/mysql_thread.h
View File

@@ -1,4 +1,5 @@
/* Copyright (c) 2008, 2013, Oracle and/or its affiliates.
Copyright (c) 2020, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License, version 2.0,
@@ -262,6 +263,7 @@ typedef struct st_mysql_cond mysql_cond_t;
*/
#ifndef DISABLE_MYSQL_THREAD_H
#define mysql_mutex_is_owner(M) safe_mutex_is_owner(&(M)->m_mutex)
/**
@def mysql_mutex_assert_owner(M)
Wrapper, to use safe_mutex_assert_owner with instrumented mutexes.

3
mysql-test/suite/encryption/t/innodb_encryption_discard_import.opt
View File

@@ -3,6 +3,5 @@
--innodb-encryption-rotate-key-age=15
--innodb-encryption-threads=4
--innodb-tablespaces-encryption
--innodb-max-dirty-pages-pct_lwm=0
--innodb-max-dirty-pages-pct=0.001

20
mysql-test/suite/innodb/r/ibuf_not_empty.result
View File

@@ -1,4 +1,3 @@
SET GLOBAL innodb_purge_rseg_truncate_frequency=1;
CREATE TABLE t1(
a INT AUTO_INCREMENT PRIMARY KEY,
b CHAR(1),
@@ -6,26 +5,11 @@ c INT,
INDEX(b))
ENGINE=InnoDB STATS_PERSISTENT=0;
SET GLOBAL innodb_change_buffering_debug = 1;
BEGIN;
INSERT INTO t1 VALUES(0,'x',1);
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
COMMIT;
InnoDB 0 transactions not purged
INSERT INTO t1 SELECT 0,'x',1 FROM seq_1_to_1024;
# restart: --innodb-force-recovery=6 --innodb-change-buffer-dump
check table t1;
Table Op Msg_type Msg_text
test.t1 check Warning InnoDB: Index 'b' contains #### entries, should be 4096.
test.t1 check Warning InnoDB: Index 'b' contains 990 entries, should be 1024.
test.t1 check error Corrupt
# restart
SET GLOBAL innodb_fast_shutdown=0;

2
mysql-test/suite/innodb/r/innodb-change-buffer-recovery.result
View File

@@ -50,5 +50,5 @@ Table Op Msg_type Msg_text
test.t1 check status OK
SHOW ENGINE INNODB STATUS;
Type Name Status
InnoDB insert 79, delete mark 1
InnoDB
DROP TABLE t1;

7
mysql-test/suite/innodb/r/innodb_skip_innodb_is_tables.result
View File

@@ -90,14 +90,10 @@ buffer_flush_neighbor_pages buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL N
buffer_flush_n_to_flush_requested buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of pages requested for flushing.
buffer_flush_n_to_flush_by_age buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of pages target by LSN Age for flushing.
buffer_flush_adaptive_avg_time_slot buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for adaptive flushing recently per slot.
buffer_LRU_batch_flush_avg_time_slot buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for LRU batch flushing recently per slot.
buffer_flush_adaptive_avg_time_thread buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for adaptive flushing recently per thread.
buffer_LRU_batch_flush_avg_time_thread buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for LRU batch flushing recently per thread.
buffer_flush_adaptive_avg_time_est buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Estimated time (ms) spent for adaptive flushing recently.
buffer_LRU_batch_flush_avg_time_est buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Estimated time (ms) spent for LRU batch flushing recently.
buffer_flush_avg_time buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for flushing recently.
buffer_flush_adaptive_avg_pass buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of adaptive flushes passed during the recent Avg period.
buffer_LRU_batch_flush_avg_pass buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of LRU batch flushes passed during the recent Avg period.
buffer_flush_avg_pass buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of flushes passed during the recent Avg period.
buffer_LRU_get_free_loops buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Total loops in LRU get free.
buffer_LRU_get_free_waits buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Total sleep waits in LRU get free.
@@ -124,9 +120,6 @@ buffer_LRU_batch_flush_pages buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL
buffer_LRU_batch_evict_total_pages buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_owner Total pages evicted as part of LRU batches
buffer_LRU_batches_evict buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_member Number of LRU batches
buffer_LRU_batch_evict_pages buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_member Pages queued as an LRU batch
buffer_LRU_single_flush_scanned buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_owner Total pages scanned as part of single page LRU flush
buffer_LRU_single_flush_num_scan buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_member Number of times single page LRU flush is called
buffer_LRU_single_flush_scanned_per_call buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_member Page scanned per single LRU flush
buffer_LRU_single_flush_failure_count Buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of times attempt to flush a single page from LRU failed
buffer_LRU_get_free_search Buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of searches performed for a clean page
buffer_LRU_search_scanned buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_owner Total pages scanned as part of LRU search

7
mysql-test/suite/innodb/r/monitor.result
View File

@@ -56,14 +56,10 @@ buffer_flush_neighbor_pages disabled
buffer_flush_n_to_flush_requested disabled
buffer_flush_n_to_flush_by_age disabled
buffer_flush_adaptive_avg_time_slot disabled
buffer_LRU_batch_flush_avg_time_slot disabled
buffer_flush_adaptive_avg_time_thread disabled
buffer_LRU_batch_flush_avg_time_thread disabled
buffer_flush_adaptive_avg_time_est disabled
buffer_LRU_batch_flush_avg_time_est disabled
buffer_flush_avg_time disabled
buffer_flush_adaptive_avg_pass disabled
buffer_LRU_batch_flush_avg_pass disabled
buffer_flush_avg_pass disabled
buffer_LRU_get_free_loops disabled
buffer_LRU_get_free_waits disabled
@@ -90,9 +86,6 @@ buffer_LRU_batch_flush_pages disabled
buffer_LRU_batch_evict_total_pages disabled
buffer_LRU_batches_evict disabled
buffer_LRU_batch_evict_pages disabled
buffer_LRU_single_flush_scanned disabled
buffer_LRU_single_flush_num_scan disabled
buffer_LRU_single_flush_scanned_per_call disabled
buffer_LRU_single_flush_failure_count disabled
buffer_LRU_get_free_search disabled
buffer_LRU_search_scanned disabled

4
mysql-test/suite/innodb/r/purge_secondary.result
View File

@@ -148,10 +148,6 @@ SELECT (variable_value > 0) FROM information_schema.global_status
WHERE LOWER(variable_name) LIKE 'INNODB_BUFFER_POOL_PAGES_FLUSHED';
(variable_value > 0)
1
SELECT NAME, SUBSYSTEM FROM INFORMATION_SCHEMA.INNODB_METRICS
WHERE NAME="buffer_LRU_batch_evict_total_pages" AND COUNT > 0;
NAME SUBSYSTEM
buffer_LRU_batch_evict_total_pages buffer
# Note: The OTHER_INDEX_SIZE does not cover any SPATIAL INDEX.
# To test that all indexes were emptied, replace DROP TABLE
# with the following, and examine the root pages in t1.ibd:

4
mysql-test/suite/innodb/t/ibuf_not_empty.combinations
View File

@@ -1,5 +1,9 @@
[strict_crc32]
--innodb-checksum-algorithm=strict_crc32
--innodb-page-size=4k
--innodb-force-recovery=2
[strict_full_crc32]
--innodb-checksum-algorithm=strict_full_crc32
--innodb-page-size=4k
--innodb-force-recovery=2

24
mysql-test/suite/innodb/t/ibuf_not_empty.test
View File

@@ -3,10 +3,8 @@
--source include/have_debug.inc
# Embedded server tests do not support restarting
--source include/not_embedded.inc
# The test is not big enough to use change buffering with larger page size.
--source include/have_innodb_max_16k.inc
--source include/have_sequence.inc
SET GLOBAL innodb_purge_rseg_truncate_frequency=1;
--disable_query_log
call mtr.add_suppression("InnoDB: Failed to find tablespace for table `test`\\.`t1` in the cache\\. Attempting to load the tablespace with space id");
call mtr.add_suppression("InnoDB: Allocated tablespace ID \\d+ for test.t1, old maximum was");
@@ -30,27 +28,10 @@ SET GLOBAL innodb_change_buffering_debug = 1;
# Create enough rows for the table, so that the change buffer will be
# used for modifying the secondary index page. There must be multiple
# index pages, because changes to the root page are never buffered.
BEGIN;
INSERT INTO t1 VALUES(0,'x',1);
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
INSERT INTO t1 SELECT 0,b,c FROM t1;
COMMIT;
INSERT INTO t1 SELECT 0,'x',1 FROM seq_1_to_1024;
let MYSQLD_DATADIR=`select @@datadir`;
let PAGE_SIZE=`select @@innodb_page_size`;
# Ensure that purge will not access the truncated .ibd file
--source include/wait_all_purged.inc
--source include/shutdown_mysqld.inc
# Corrupt the change buffer bitmap, to claim that pages are clean
@@ -87,7 +68,6 @@ EOF
--let $restart_parameters= --innodb-force-recovery=6 --innodb-change-buffer-dump
--source include/start_mysqld.inc
--replace_regex /contains \d+ entries/contains #### entries/
check table t1;
--source include/shutdown_mysqld.inc

2
mysql-test/suite/innodb/t/innodb-change-buffer-recovery.test
View File

@@ -76,6 +76,6 @@ SET GLOBAL innodb_fast_shutdown=0;
--let $restart_parameters=
--source include/restart_mysqld.inc
CHECK TABLE t1;
replace_regex /.*operations:.* (insert.*), delete \d.*discarded .*/\1/;
replace_regex /.*operations:.* insert [1-9][0-9]*, delete mark [1-9][0-9]*, delete \d.*discarded .*//;
SHOW ENGINE INNODB STATUS;
DROP TABLE t1;

3
mysql-test/suite/innodb/t/purge_secondary.test
View File

@@ -131,9 +131,6 @@ ALTER TABLE t1 FORCE, ALGORITHM=INPLACE;
SELECT (variable_value > 0) FROM information_schema.global_status
WHERE LOWER(variable_name) LIKE 'INNODB_BUFFER_POOL_PAGES_FLUSHED';
SELECT NAME, SUBSYSTEM FROM INFORMATION_SCHEMA.INNODB_METRICS
WHERE NAME="buffer_LRU_batch_evict_total_pages" AND COUNT > 0;
--echo # Note: The OTHER_INDEX_SIZE does not cover any SPATIAL INDEX.
--echo # To test that all indexes were emptied, replace DROP TABLE
--echo # with the following, and examine the root pages in t1.ibd:

1
mysql-test/suite/perfschema/t/show_sanity.test
View File

@@ -422,7 +422,6 @@ insert into test.sanity values
("JUNK: GLOBAL-ONLY", "I_S.SESSION_VARIABLES", "INNODB_DISABLE_RESIZE_BUFFER_POOL_DEBUG"),
("JUNK: GLOBAL-ONLY", "I_S.SESSION_VARIABLES", "INNODB_DISABLE_SORT_FILE_CACHE"),
("JUNK: GLOBAL-ONLY", "I_S.SESSION_VARIABLES", "INNODB_DOUBLEWRITE"),
("JUNK: GLOBAL-ONLY", "I_S.SESSION_VARIABLES", "INNODB_DOUBLEWRITE_BATCH_SIZE"),
("JUNK: GLOBAL-ONLY", "I_S.SESSION_VARIABLES", "INNODB_FAST_SHUTDOWN"),
("JUNK: GLOBAL-ONLY", "I_S.SESSION_VARIABLES", "INNODB_FILE_PER_TABLE"),
("JUNK: GLOBAL-ONLY", "I_S.SESSION_VARIABLES", "INNODB_FILL_FACTOR"),

24
mysql-test/suite/sys_vars/r/innodb_doublewrite_batch_size_basic.result
View File

@@ -1,24 +0,0 @@
select @@global.innodb_doublewrite_batch_size between 1 and 127;
@@global.innodb_doublewrite_batch_size between 1 and 127
1
select @@global.innodb_doublewrite_batch_size;
@@global.innodb_doublewrite_batch_size
120
select @@session.innodb_doublewrite_batch_size;
ERROR HY000: Variable 'innodb_doublewrite_batch_size' is a GLOBAL variable
show global variables like 'innodb_doublewrite_batch_size';
Variable_name Value
innodb_doublewrite_batch_size 120
show session variables like 'innodb_doublewrite_batch_size';
Variable_name Value
innodb_doublewrite_batch_size 120
select * from information_schema.global_variables where variable_name='innodb_doublewrite_batch_size';
VARIABLE_NAME VARIABLE_VALUE
INNODB_DOUBLEWRITE_BATCH_SIZE 120
select * from information_schema.session_variables where variable_name='innodb_doublewrite_batch_size';
VARIABLE_NAME VARIABLE_VALUE
INNODB_DOUBLEWRITE_BATCH_SIZE 120
set global innodb_doublewrite_batch_size=1;
ERROR HY000: Variable 'innodb_doublewrite_batch_size' is a read only variable
set @@session.innodb_doublewrite_batch_size='some';
ERROR HY000: Variable 'innodb_doublewrite_batch_size' is a read only variable

8
mysql-test/suite/sys_vars/r/innodb_max_dirty_pages_pct_basic.result
View File

@@ -7,7 +7,7 @@ SELECT @global_start_value;
SET @global_start_max_dirty_lwm_value = @@global.innodb_max_dirty_pages_pct_lwm;
SELECT @global_start_max_dirty_lwm_value;
@global_start_max_dirty_lwm_value
0
75
SET @@global.innodb_max_dirty_pages_pct_lwm = 0;
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
@@global.innodb_max_dirty_pages_pct_lwm
@@ -17,13 +17,13 @@ SET @@global.innodb_max_dirty_pages_pct = 0;
SET @@global.innodb_max_dirty_pages_pct = DEFAULT;
SELECT @@global.innodb_max_dirty_pages_pct;
@@global.innodb_max_dirty_pages_pct
75.000000
90.000000
'#---------------------FN_DYNVARS_046_02-------------------------#'
SET innodb_max_dirty_pages_pct = 1;
ERROR HY000: Variable 'innodb_max_dirty_pages_pct' is a GLOBAL variable and should be set with SET GLOBAL
SELECT @@innodb_max_dirty_pages_pct;
@@innodb_max_dirty_pages_pct
75.000000
90.000000
SELECT local.innodb_max_dirty_pages_pct;
ERROR 42S02: Unknown table 'local' in field list
SET global innodb_max_dirty_pages_pct = 0;
@@ -171,5 +171,5 @@ SELECT @@global.innodb_max_dirty_pages_pct;
SET @@global.innodb_max_dirty_pages_pct_lwm = @global_start_max_dirty_lwm_value;
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
@@global.innodb_max_dirty_pages_pct_lwm
0.000000
75.000000
SET @@global.innodb_max_dirty_pages_pct=@save_innodb_max_dirty_pages_pct;

34
mysql-test/suite/sys_vars/r/innodb_max_dirty_pages_pct_func.result
View File

@@ -1,26 +1,28 @@
SET @innodb_max_dirty_pages_pct_lwm = @@global.innodb_max_dirty_pages_pct_lwm;
SET @innodb_max_dirty_pages_pct = @@global.innodb_max_dirty_pages_pct;
'#--------------------FN_DYNVARS_044_02-------------------------#'
SET @@global.innodb_max_dirty_pages_pct_lwm = 0;
SET @@global.innodb_max_dirty_pages_pct = 80;
'connect (con1,localhost,root,,,,)'
SET @@global.innodb_max_dirty_pages_pct_lwm = 80;
connect con1,localhost,root,,,,;
'connection con1'
connection con1;
SELECT @@global.innodb_max_dirty_pages_pct;
@@global.innodb_max_dirty_pages_pct
80.000000
SET @@global.innodb_max_dirty_pages_pct = 70;
'connect (con2,localhost,root,,,,)'
Warnings:
Warning 1210 innodb_max_dirty_pages_pct cannot be set lower than innodb_max_dirty_pages_pct_lwm.
Warning 1210 Lowering innodb_max_dirty_page_pct_lwm to 70.000000
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
@@global.innodb_max_dirty_pages_pct_lwm
70.000000
connect con2,localhost,root,,,,;
'connection con2'
connection con2;
SELECT @@global.innodb_max_dirty_pages_pct;
@@global.innodb_max_dirty_pages_pct
70.000000
'connection default'
connection default;
'disconnect con2'
disconnect con2;
'disconnect con1'
disconnect con1;
SET @@global.innodb_max_dirty_pages_pct = @innodb_max_dirty_pages_pct;
'#--------------------FN_DYNVARS_044_02-------------------------#'
@@ -85,6 +87,22 @@ b CHAR(200)
) ENGINE = INNODB;
'---Check when innodb_max_dirty_pages_pct is 10---'
SET @@global.innodb_max_dirty_pages_pct = 10;
Warnings:
Warning 1210 innodb_max_dirty_pages_pct cannot be set lower than innodb_max_dirty_pages_pct_lwm.
Warning 1210 Lowering innodb_max_dirty_page_pct_lwm to 10.000000
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
@@global.innodb_max_dirty_pages_pct_lwm
10.000000
SET GLOBAL innodb_max_dirty_pages_pct_lwm = 15;
Warnings:
Warning 1210 innodb_max_dirty_pages_pct_lwm cannot be set higher than innodb_max_dirty_pages_pct.
Warning 1210 Setting innodb_max_dirty_page_pct_lwm to 10.000000
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
@@global.innodb_max_dirty_pages_pct_lwm
10.000000
SELECT @@global.innodb_max_dirty_pages_pct;
@@global.innodb_max_dirty_pages_pct
10.000000
FLUSH STATUS;
CALL add_until(10);
FLUSH TABLES;
@@ -98,4 +116,6 @@ DROP PROCEDURE add_until;
DROP PROCEDURE check_pct;
DROP FUNCTION dirty_pct;
DROP TABLE t1;
SET GLOBAL innodb_max_dirty_pages_pct_lwm = 0;
SET @@global.innodb_max_dirty_pages_pct = @innodb_max_dirty_pages_pct;
SET @@global.innodb_max_dirty_pages_pct_lwm = @innodb_max_dirty_pages_pct_lwm;

8
mysql-test/suite/sys_vars/r/innodb_max_dirty_pages_pct_lwm_basic.result
View File

@@ -3,7 +3,7 @@ set @@global.innodb_max_dirty_pages_pct=75;
SET @pct_lwm_start_value = @@global.innodb_max_dirty_pages_pct_lwm;
SELECT @pct_lwm_start_value;
@pct_lwm_start_value
0
75
SET @pct_start_value = @@global.innodb_max_dirty_pages_pct;
SELECT @pct_start_value;
@pct_start_value
@@ -13,13 +13,13 @@ SET @@global.innodb_max_dirty_pages_pct_lwm = 0;
SET @@global.innodb_max_dirty_pages_pct_lwm = DEFAULT;
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
@@global.innodb_max_dirty_pages_pct_lwm
0.000000
75.000000
'#---------------------FN_DYNVARS_046_02-------------------------#'
SET innodb_max_dirty_pages_pct_lwm = 1;
ERROR HY000: Variable 'innodb_max_dirty_pages_pct_lwm' is a GLOBAL variable and should be set with SET GLOBAL
SELECT @@innodb_max_dirty_pages_pct_lwm;
@@innodb_max_dirty_pages_pct_lwm
0.000000
75.000000
SELECT local.innodb_max_dirty_pages_pct_lwm;
ERROR 42S02: Unknown table 'local' in field list
SET global innodb_max_dirty_pages_pct_lwm = 0;
@@ -130,5 +130,5 @@ SELECT @@global.innodb_max_dirty_pages_pct;
SET @@global.innodb_max_dirty_pages_pct_lwm = @pct_lwm_start_value;
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
@@global.innodb_max_dirty_pages_pct_lwm
0.000000
75.000000
SET @@global.innodb_max_dirty_pages_pct=@save_innodb_max_dirty_pages_pct;

60
mysql-test/suite/sys_vars/r/sysvars_innodb,32bit.rdiff
View File

@@ -85,16 +85,7 @@
VARIABLE_COMMENT Percentage of empty space on a data page that can be reserved to make the page compressible.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 75
@@ -661,7 +661,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 120
VARIABLE_SCOPE GLOBAL
-VARIABLE_TYPE BIGINT UNSIGNED
+VARIABLE_TYPE INT UNSIGNED
VARIABLE_COMMENT Number of pages reserved in doublewrite buffer for batch flushing
NUMERIC_MIN_VALUE 1
NUMERIC_MAX_VALUE 127
@@ -757,7 +757,7 @@
@@ -745,7 +745,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 600
VARIABLE_SCOPE GLOBAL
@@ -103,7 +94,7 @@
VARIABLE_COMMENT Maximum number of seconds that semaphore times out in InnoDB.
NUMERIC_MIN_VALUE 1
NUMERIC_MAX_VALUE 4294967295
@@ -805,7 +805,7 @@
@@ -793,7 +793,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 0
VARIABLE_SCOPE GLOBAL
@@ -112,7 +103,7 @@
VARIABLE_COMMENT Make the first page of the given tablespace dirty.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 4294967295
@@ -817,7 +817,7 @@
@@ -805,7 +805,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 30
VARIABLE_SCOPE GLOBAL
@@ -121,7 +112,7 @@
VARIABLE_COMMENT Number of iterations over which the background flushing is averaged.
NUMERIC_MIN_VALUE 1
NUMERIC_MAX_VALUE 1000
@@ -841,7 +841,7 @@
@@ -829,7 +829,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 1
VARIABLE_SCOPE GLOBAL
@@ -130,7 +121,7 @@
VARIABLE_COMMENT Controls the durability/speed trade-off for commits. Set to 0 (write and flush redo log to disk only once per second), 1 (flush to disk at each commit), 2 (write to log at commit but flush to disk only once per second) or 3 (flush to disk at prepare and at commit, slower and usually redundant). 1 and 3 guarantees that after a crash, committed transactions will not be lost and will be consistent with the binlog and other transactional engines. 2 can get inconsistent and lose transactions if there is a power failure or kernel crash but not if mysqld crashes. 0 has no guarantees in case of crash. 0 and 2 can be faster than 1 or 3.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 3
@@ -865,7 +865,7 @@
@@ -853,7 +853,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 1
VARIABLE_SCOPE GLOBAL
@@ -139,7 +130,7 @@
VARIABLE_COMMENT Set to 0 (don't flush neighbors from buffer pool), 1 (flush contiguous neighbors from buffer pool) or 2 (flush neighbors from buffer pool), when flushing a block
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 2
@@ -913,7 +913,7 @@
@@ -901,7 +901,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 0
VARIABLE_SCOPE GLOBAL
@@ -148,7 +139,7 @@
VARIABLE_COMMENT Helps to save your data in case the disk image of the database becomes corrupt. Value 5 can return bogus data, and 6 can permanently corrupt data.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 6
@@ -937,7 +937,7 @@
@@ -925,7 +925,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 8000000
VARIABLE_SCOPE GLOBAL
@@ -157,7 +148,7 @@
VARIABLE_COMMENT InnoDB Fulltext search cache size in bytes
NUMERIC_MIN_VALUE 1600000
NUMERIC_MAX_VALUE 80000000
@@ -973,7 +973,7 @@
@@ -961,7 +961,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 84
VARIABLE_SCOPE GLOBAL
@@ -166,7 +157,7 @@
VARIABLE_COMMENT InnoDB Fulltext search maximum token size in characters
NUMERIC_MIN_VALUE 10
NUMERIC_MAX_VALUE 84
@@ -985,7 +985,7 @@
@@ -973,7 +973,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 3
VARIABLE_SCOPE GLOBAL
@@ -175,7 +166,7 @@
VARIABLE_COMMENT InnoDB Fulltext search minimum token size in characters
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 16
@@ -997,7 +997,7 @@
@@ -985,7 +985,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 2000
VARIABLE_SCOPE GLOBAL
@@ -184,7 +175,7 @@
VARIABLE_COMMENT InnoDB Fulltext search number of words to optimize for each optimize table call
NUMERIC_MIN_VALUE 1000
NUMERIC_MAX_VALUE 10000
@@ -1009,10 +1009,10 @@
@@ -997,10 +997,10 @@
SESSION_VALUE NULL
DEFAULT_VALUE 2000000000
VARIABLE_SCOPE GLOBAL
@@ -197,7 +188,7 @@
NUMERIC_BLOCK_SIZE 0
ENUM_VALUE_LIST NULL
READ_ONLY NO
@@ -1033,7 +1033,7 @@
@@ -1021,7 +1021,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 2
VARIABLE_SCOPE GLOBAL
@@ -206,7 +197,7 @@
VARIABLE_COMMENT InnoDB Fulltext search parallel sort degree, will round up to nearest power of 2 number
NUMERIC_MIN_VALUE 1
NUMERIC_MAX_VALUE 16
@@ -1045,7 +1045,7 @@
@@ -1033,7 +1033,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 640000000
VARIABLE_SCOPE GLOBAL
@@ -215,7 +206,7 @@
VARIABLE_COMMENT Total memory allocated for InnoDB Fulltext Search cache
NUMERIC_MIN_VALUE 32000000
NUMERIC_MAX_VALUE 1600000000
@@ -1069,7 +1069,7 @@
@@ -1057,7 +1057,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 100
VARIABLE_SCOPE GLOBAL
@@ -224,7 +215,7 @@
VARIABLE_COMMENT Up to what percentage of dirty pages should be flushed when innodb finds it has spare resources to do so.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 100
@@ -1105,22 +1105,22 @@
@@ -1093,22 +1093,22 @@
SESSION_VALUE NULL
DEFAULT_VALUE 200
VARIABLE_SCOPE GLOBAL
@@ -252,7 +243,7 @@
NUMERIC_BLOCK_SIZE 0
ENUM_VALUE_LIST NULL
READ_ONLY NO
@@ -1165,7 +1165,7 @@
@@ -1153,7 +1153,7 @@
SESSION_VALUE 50
DEFAULT_VALUE 50
VARIABLE_SCOPE SESSION
@@ -261,7 +252,7 @@
VARIABLE_COMMENT Timeout in seconds an InnoDB transaction may wait for a lock before being rolled back. Values above 100000000 disable the timeout.
NUMERIC_MIN_VALUE 0
NUMERIC_MAX_VALUE 1073741824
@@ -1177,10 +1177,10 @@
@@ -1165,10 +1165,10 @@
SESSION_VALUE NULL
DEFAULT_VALUE 16777216
VARIABLE_SCOPE GLOBAL
@@ -274,7 +265,7 @@
NUMERIC_BLOCK_SIZE 1024
ENUM_VALUE_LIST NULL
READ_ONLY YES
@@ -1225,7 +1225,7 @@
@@ -1213,7 +1213,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 1
VARIABLE_SCOPE GLOBAL
@@ -283,7 +274,7 @@
VARIABLE_COMMENT Deprecated parameter with no effect.
NUMERIC_MIN_VALUE 1
NUMERIC_MAX_VALUE 100
@@ -1273,7 +1273,7 @@
@@ -1261,7 +1261,7 @@
SESSION_VALUE NULL
DEFAULT_VALUE 8192
VARIABLE_SCOPE GLOBAL
@@ -292,6 +283,19 @@
VARIABLE_COMMENT Redo log write ahead unit size to avoid read-on-write, it should match the OS cache block IO size
NUMERIC_MIN_VALUE 512
NUMERIC_MAX_VALUE 16384
@@ -1273,10 +1273,10 @@
SESSION_VALUE NULL
DEFAULT_VALUE 100
VARIABLE_SCOPE GLOBAL
-VARIABLE_TYPE BIGINT UNSIGNED
+VARIABLE_TYPE INT UNSIGNED
VARIABLE_COMMENT How many pages to flush on LRU eviction
NUMERIC_MIN_VALUE 1
-NUMERIC_MAX_VALUE 18446744073709551615
+NUMERIC_MAX_VALUE 4294967295
NUMERIC_BLOCK_SIZE 0
ENUM_VALUE_LIST NULL
READ_ONLY NO
@@ -1285,10 +1285,10 @@
SESSION_VALUE NULL
DEFAULT_VALUE 1024

30
mysql-test/suite/sys_vars/r/sysvars_innodb.result
View File

@@ -657,18 +657,6 @@ NUMERIC_BLOCK_SIZE NULL
ENUM_VALUE_LIST OFF,ON
READ_ONLY YES
COMMAND_LINE_ARGUMENT NONE
VARIABLE_NAME INNODB_DOUBLEWRITE_BATCH_SIZE
SESSION_VALUE NULL
DEFAULT_VALUE 120
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
VARIABLE_COMMENT Number of pages reserved in doublewrite buffer for batch flushing
NUMERIC_MIN_VALUE 1
NUMERIC_MAX_VALUE 127
NUMERIC_BLOCK_SIZE 0
ENUM_VALUE_LIST NULL
READ_ONLY YES
COMMAND_LINE_ARGUMENT OPTIONAL
VARIABLE_NAME INNODB_ENCRYPTION_ROTATE_KEY_AGE
SESSION_VALUE NULL
DEFAULT_VALUE 1
@@ -1281,9 +1269,21 @@ NUMERIC_BLOCK_SIZE 512
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME INNODB_LRU_FLUSH_SIZE
SESSION_VALUE NULL
DEFAULT_VALUE 32
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
VARIABLE_COMMENT How many pages to flush on LRU eviction
NUMERIC_MIN_VALUE 1
NUMERIC_MAX_VALUE 18446744073709551615
NUMERIC_BLOCK_SIZE 0
ENUM_VALUE_LIST NULL
READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME INNODB_LRU_SCAN_DEPTH
SESSION_VALUE NULL
DEFAULT_VALUE 1024
DEFAULT_VALUE 1536
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE BIGINT UNSIGNED
VARIABLE_COMMENT How deep to scan LRU to keep it clean
@@ -1307,7 +1307,7 @@ READ_ONLY NO
COMMAND_LINE_ARGUMENT OPTIONAL
VARIABLE_NAME INNODB_MAX_DIRTY_PAGES_PCT
SESSION_VALUE NULL
DEFAULT_VALUE 75.000000
DEFAULT_VALUE 90.000000
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE DOUBLE
VARIABLE_COMMENT Percentage of dirty pages allowed in bufferpool.
@@ -1319,7 +1319,7 @@ READ_ONLY NO
COMMAND_LINE_ARGUMENT REQUIRED
VARIABLE_NAME INNODB_MAX_DIRTY_PAGES_PCT_LWM
SESSION_VALUE NULL
DEFAULT_VALUE 0.000000
DEFAULT_VALUE 75.000000
VARIABLE_SCOPE GLOBAL
VARIABLE_TYPE DOUBLE
VARIABLE_COMMENT Percentage of dirty pages at which flushing kicks in.

24
mysql-test/suite/sys_vars/t/innodb_doublewrite_batch_size_basic.test
View File

@@ -1,24 +0,0 @@
--source include/have_innodb.inc
--source include/have_debug.inc
#
# exists as global only
#
select @@global.innodb_doublewrite_batch_size between 1 and 127;
select @@global.innodb_doublewrite_batch_size;
--error ER_INCORRECT_GLOBAL_LOCAL_VAR
select @@session.innodb_doublewrite_batch_size;
show global variables like 'innodb_doublewrite_batch_size';
show session variables like 'innodb_doublewrite_batch_size';
--disable_warnings
select * from information_schema.global_variables where variable_name='innodb_doublewrite_batch_size';
select * from information_schema.session_variables where variable_name='innodb_doublewrite_batch_size';
--enable_warnings
#
# show that it's read-only
#
--error ER_INCORRECT_GLOBAL_LOCAL_VAR
set global innodb_doublewrite_batch_size=1;
--error ER_INCORRECT_GLOBAL_LOCAL_VAR
set @@session.innodb_doublewrite_batch_size='some';

17
mysql-test/suite/sys_vars/t/innodb_max_dirty_pages_pct_func.test
View File

@@ -25,6 +25,7 @@
--source include/have_innodb.inc
# safe initial value
SET @innodb_max_dirty_pages_pct_lwm = @@global.innodb_max_dirty_pages_pct_lwm;
SET @innodb_max_dirty_pages_pct = @@global.innodb_max_dirty_pages_pct;
--echo '#--------------------FN_DYNVARS_044_02-------------------------#'
@@ -32,23 +33,19 @@ SET @innodb_max_dirty_pages_pct = @@global.innodb_max_dirty_pages_pct;
# Check if setting innodb_max_dirty_pages_pct is changed in new connection #
############################################################################
SET @@global.innodb_max_dirty_pages_pct_lwm = 0;
SET @@global.innodb_max_dirty_pages_pct = 80;
--echo 'connect (con1,localhost,root,,,,)'
SET @@global.innodb_max_dirty_pages_pct_lwm = 80;
connect (con1,localhost,root,,,,);
--echo 'connection con1'
connection con1;
SELECT @@global.innodb_max_dirty_pages_pct;
SET @@global.innodb_max_dirty_pages_pct = 70;
--echo 'connect (con2,localhost,root,,,,)'
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
connect (con2,localhost,root,,,,);
--echo 'connection con2'
connection con2;
SELECT @@global.innodb_max_dirty_pages_pct;
--echo 'connection default'
connection default;
--echo 'disconnect con2'
disconnect con2;
--echo 'disconnect con1'
disconnect con1;
# restore initial value
SET @@global.innodb_max_dirty_pages_pct = @innodb_max_dirty_pages_pct;
@@ -138,6 +135,10 @@ b CHAR(200)
#==========================================================
SET @@global.innodb_max_dirty_pages_pct = 10;
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
SET GLOBAL innodb_max_dirty_pages_pct_lwm = 15;
SELECT @@global.innodb_max_dirty_pages_pct_lwm;
SELECT @@global.innodb_max_dirty_pages_pct;
FLUSH STATUS;
@@ -164,7 +165,9 @@ DROP FUNCTION dirty_pct;
DROP TABLE t1;
# restore initial value
SET GLOBAL innodb_max_dirty_pages_pct_lwm = 0;
SET @@global.innodb_max_dirty_pages_pct = @innodb_max_dirty_pages_pct;
SET @@global.innodb_max_dirty_pages_pct_lwm = @innodb_max_dirty_pages_pct_lwm;
##################################################################
# End of functionality Testing for innodb_max_dirty_pages_pct #

4
storage/innobase/btr/btr0bulk.cc
View File

@@ -981,7 +981,7 @@ inline void BtrBulk::logFreeCheck()
if (log_sys.check_flush_or_checkpoint()) {
release();
log_free_check();
log_check_margins();
latch();
}
@@ -1113,7 +1113,7 @@ BtrBulk::insert(
/* Wake up page cleaner to flush dirty pages. */
srv_inc_activity_count();
os_event_set(buf_flush_event);
mysql_cond_signal(&buf_pool.do_flush_list);
logFreeCheck();
}

6
storage/innobase/btr/btr0cur.cc
View File

@@ -7058,7 +7058,7 @@ static void btr_blob_free(buf_block_t *block, bool all, mtr_t *mtr)
const ulint fold= page_id.fold();
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
if (buf_page_t *bpage= buf_pool.page_hash_get_low(page_id, fold))
if(!buf_LRU_free_page(bpage, all) && all && bpage->zip.data)
@@ -7066,7 +7066,7 @@ static void btr_blob_free(buf_block_t *block, bool all, mtr_t *mtr)
if the whole ROW_FORMAT=COMPRESSED block cannot be deallocted. */
buf_LRU_free_page(bpage, false);
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
}
/** Helper class used while writing blob pages, during insert or update. */
@@ -8253,7 +8253,7 @@ btr_rec_copy_externally_stored_field(
field_ref_zero, BTR_EXTERN_FIELD_REF_SIZE))) {
/* The externally stored field was not written yet.
This record should only be seen by
recv_recovery_rollback_active() or any
trx_rollback_recovered() or any
TRX_ISO_READ_UNCOMMITTED transactions. */
return(NULL);
}

16
storage/innobase/btr/btr0sea.cc
View File

@@ -261,9 +261,9 @@ void btr_search_disable()
void btr_search_enable(bool resize)
{
if (!resize) {
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
bool changed = srv_buf_pool_old_size != srv_buf_pool_size;
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
if (changed) {
return;
}
@@ -2133,7 +2133,7 @@ btr_search_hash_table_validate(ulint hash_table_id)
rec_offs_init(offsets_);
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
auto &part = btr_search_sys.parts[hash_table_id];
@@ -2144,7 +2144,7 @@ btr_search_hash_table_validate(ulint hash_table_id)
give other queries a chance to run. */
if ((i != 0) && ((i % chunk_size) == 0)) {
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
btr_search_x_unlock_all();
os_thread_yield();
@@ -2156,7 +2156,7 @@ btr_search_hash_table_validate(ulint hash_table_id)
goto func_exit;
}
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
ulint curr_cell_count = part.table.n_cells;
@@ -2252,7 +2252,7 @@ state_ok:
/* We release search latches every once in a while to
give other queries a chance to run. */
if (i != 0) {
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
btr_search_x_unlock_all();
os_thread_yield();
@@ -2264,7 +2264,7 @@ state_ok:
goto func_exit;
}
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
ulint curr_cell_count = part.table.n_cells;
@@ -2285,7 +2285,7 @@ state_ok:
}
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
func_exit:
btr_search_x_unlock_all();

20
storage/innobase/buf/buf0buddy.cc
View File

@@ -192,7 +192,7 @@ static bool buf_buddy_check_free(const buf_buddy_free_t* buf, ulint i)
{
const ulint size = BUF_BUDDY_LOW << i;
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(!ut_align_offset(buf, size));
ut_ad(i >= buf_buddy_get_slot(UNIV_ZIP_SIZE_MIN));
@@ -261,7 +261,7 @@ UNIV_INLINE
void
buf_buddy_add_to_free(buf_buddy_free_t* buf, ulint i)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(buf_pool.zip_free[i].start != buf);
buf_buddy_stamp_free(buf, i);
@@ -276,7 +276,7 @@ UNIV_INLINE
void
buf_buddy_remove_from_free(buf_buddy_free_t* buf, ulint i)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(buf_buddy_check_free(buf, i));
UT_LIST_REMOVE(buf_pool.zip_free[i], buf);
@@ -290,7 +290,7 @@ static buf_buddy_free_t* buf_buddy_alloc_zip(ulint i)
{
buf_buddy_free_t* buf;
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_a(i < BUF_BUDDY_SIZES);
ut_a(i >= buf_buddy_get_slot(UNIV_ZIP_SIZE_MIN));
@@ -350,7 +350,7 @@ buf_buddy_block_free(void* buf)
buf_page_t* bpage;
buf_block_t* block;
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_a(!ut_align_offset(buf, srv_page_size));
HASH_SEARCH(hash, &buf_pool.zip_hash, fold, buf_page_t*, bpage,
@@ -433,7 +433,7 @@ byte *buf_buddy_alloc_low(ulint i, bool *lru)
{
buf_block_t* block;
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(i >= buf_buddy_get_slot(UNIV_ZIP_SIZE_MIN));
if (i < BUF_BUDDY_SIZES) {
@@ -483,7 +483,7 @@ static bool buf_buddy_relocate(void* src, void* dst, ulint i, bool force)
ulint space;
ulint offset;
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(!ut_align_offset(src, size));
ut_ad(!ut_align_offset(dst, size));
ut_ad(i >= buf_buddy_get_slot(UNIV_ZIP_SIZE_MIN));
@@ -584,7 +584,7 @@ void buf_buddy_free_low(void* buf, ulint i)
{
buf_buddy_free_t* buddy;
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(i <= BUF_BUDDY_SIZES);
ut_ad(i >= buf_buddy_get_slot(UNIV_ZIP_SIZE_MIN));
ut_ad(buf_pool.buddy_stat[i].used > 0);
@@ -670,7 +670,7 @@ buf_buddy_realloc(void* buf, ulint size)
buf_block_t* block = NULL;
ulint i = buf_buddy_get_slot(size);
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(i <= BUF_BUDDY_SIZES);
ut_ad(i >= buf_buddy_get_slot(UNIV_ZIP_SIZE_MIN));
@@ -711,7 +711,7 @@ buf_buddy_realloc(void* buf, ulint size)
/** Combine all pairs of free buddies. */
void buf_buddy_condense_free()
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(buf_pool.curr_size < buf_pool.old_size);
for (ulint i = 0; i < UT_ARR_SIZE(buf_pool.zip_free); ++i) {

262
storage/innobase/buf/buf0buf.cc
View File

@@ -525,7 +525,7 @@ decrypt_failed:
@retval 0 if all modified persistent pages have been flushed */
lsn_t buf_pool_t::get_oldest_modification()
{
mutex_enter(&flush_list_mutex);
mysql_mutex_lock(&flush_list_mutex);
/* FIXME: Keep temporary tablespace pages in a separate flush
list. We would only need to write out temporary pages if the
@@ -538,7 +538,7 @@ lsn_t buf_pool_t::get_oldest_modification()
ut_ad(bpage->oldest_modification());
lsn_t oldest_lsn= bpage ? bpage->oldest_modification() : 0;
mutex_exit(&flush_list_mutex);
mysql_mutex_unlock(&flush_list_mutex);
/* The result may become stale as soon as we released the mutex.
On log checkpoint, also log_sys.flush_order_mutex will be needed. */
@@ -1063,14 +1063,14 @@ buf_madvise_do_dump()
ret+= madvise(recv_sys.buf, recv_sys.len, MADV_DODUMP);
}
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
auto chunk = buf_pool.chunks;
for (ulint n = buf_pool.n_chunks; n--; chunk++) {
ret+= madvise(chunk->mem, chunk->mem_size(), MADV_DODUMP);
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
return ret;
}
#endif
@@ -1546,7 +1546,7 @@ bool buf_pool_t::create()
while (++chunk < chunks + n_chunks);
ut_ad(is_initialised());
mutex_create(LATCH_ID_BUF_POOL, &mutex);
mysql_mutex_init(buf_pool_mutex_key, &mutex, MY_MUTEX_INIT_FAST);
UT_LIST_INIT(LRU, &buf_page_t::LRU);
UT_LIST_INIT(withdraw, &buf_page_t::list);
@@ -1570,17 +1570,18 @@ bool buf_pool_t::create()
zip_hash.create(2 * curr_size);
last_printout_time= time(NULL);
mutex_create(LATCH_ID_FLUSH_LIST, &flush_list_mutex);
mysql_mutex_init(flush_list_mutex_key, &flush_list_mutex,
MY_MUTEX_INIT_FAST);
for (int i= 0; i < 3; i++)
no_flush[i]= os_event_create(0);
mysql_cond_init(0, &done_flush_LRU, nullptr);
mysql_cond_init(0, &done_flush_list, nullptr);
mysql_cond_init(0, &do_flush_list, nullptr);
try_LRU_scan= true;
ut_d(flush_hp.m_mutex= &flush_list_mutex;);
ut_d(lru_hp.m_mutex= &mutex);
ut_d(lru_scan_itr.m_mutex= &mutex);
ut_d(single_scan_itr.m_mutex= &mutex);
io_buf.create((srv_n_read_io_threads + srv_n_write_io_threads) *
OS_AIO_N_PENDING_IOS_PER_THREAD);
@@ -1604,8 +1605,8 @@ void buf_pool_t::close()
if (!is_initialised())
return;
mutex_free(&mutex);
mutex_free(&flush_list_mutex);
mysql_mutex_destroy(&mutex);
mysql_mutex_destroy(&flush_list_mutex);
for (buf_page_t *bpage= UT_LIST_GET_LAST(LRU), *prev_bpage= nullptr; bpage;
bpage= prev_bpage)
@@ -1633,8 +1634,9 @@ void buf_pool_t::close()
allocator.deallocate_large_dodump(chunk->mem, &chunk->mem_pfx);
}
for (int i= 0; i < 3; ++i)
os_event_destroy(no_flush[i]);
mysql_cond_destroy(&done_flush_LRU);
mysql_cond_destroy(&done_flush_list);
mysql_cond_destroy(&do_flush_list);
ut_free(chunks);
chunks= nullptr;
@@ -1661,7 +1663,7 @@ inline bool buf_pool_t::realloc(buf_block_t *block)
buf_block_t* new_block;
ut_ad(withdrawing);
ut_ad(mutex_own(&mutex));
mysql_mutex_assert_owner(&mutex);
ut_ad(block->page.state() == BUF_BLOCK_FILE_PAGE);
new_block = buf_LRU_get_free_only();
@@ -1732,13 +1734,8 @@ inline bool buf_pool_t::realloc(buf_block_t *block)
+ FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID, 0xff, 4);
MEM_UNDEFINED(block->frame, srv_page_size);
block->page.set_state(BUF_BLOCK_REMOVE_HASH);
/* Relocate flush_list. */
if (block->page.oldest_modification()) {
buf_flush_relocate_on_flush_list(
&block->page, &new_block->page);
}
buf_flush_relocate_on_flush_list(&block->page,
&new_block->page);
block->page.set_corrupt_id();
/* set other flags of buf_block_t */
@@ -1804,16 +1801,16 @@ inline bool buf_pool_t::withdraw_blocks()
<< withdraw_target << " blocks";
/* Minimize zip_free[i] lists */
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
buf_buddy_condense_free();
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
while (UT_LIST_GET_LEN(withdraw) < withdraw_target) {
/* try to withdraw from free_list */
ulint count1 = 0;
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
block = reinterpret_cast<buf_block_t*>(
UT_LIST_GET_FIRST(free));
while (block != NULL
@@ -1828,7 +1825,7 @@ inline bool buf_pool_t::withdraw_blocks()
UT_LIST_GET_NEXT(
list, &block->page));
if (buf_pool.will_be_withdrawn(block->page)) {
if (will_be_withdrawn(block->page)) {
/* This should be withdrawn */
UT_LIST_REMOVE(free, &block->page);
UT_LIST_ADD_LAST(withdraw, &block->page);
@@ -1838,40 +1835,29 @@ inline bool buf_pool_t::withdraw_blocks()
block = next_block;
}
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
/* reserve free_list length */
if (UT_LIST_GET_LEN(withdraw) < withdraw_target) {
ulint scan_depth;
flush_counters_t n;
/* cap scan_depth with current LRU size. */
mutex_enter(&mutex);
scan_depth = UT_LIST_GET_LEN(LRU);
mutex_exit(&mutex);
scan_depth = ut_min(
ut_max(withdraw_target
ulint n_flushed = buf_flush_lists(
std::max<ulint>(withdraw_target
- UT_LIST_GET_LEN(withdraw),
static_cast<ulint>(srv_LRU_scan_depth)),
scan_depth);
srv_LRU_scan_depth), 0);
buf_flush_wait_batch_end_acquiring_mutex(true);
buf_flush_do_batch(true, scan_depth, 0, &n);
buf_flush_wait_batch_end(true);
if (n.flushed) {
if (n_flushed) {
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_LRU_BATCH_FLUSH_TOTAL_PAGE,
MONITOR_LRU_BATCH_FLUSH_COUNT,
MONITOR_LRU_BATCH_FLUSH_PAGES,
n.flushed);
n_flushed);
}
}
/* relocate blocks/buddies in withdrawn area */
ulint count2 = 0;
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
buf_page_t* bpage;
bpage = UT_LIST_GET_FIRST(LRU);
while (bpage != NULL) {
@@ -1892,7 +1878,7 @@ inline bool buf_pool_t::withdraw_blocks()
}
if (bpage->state() == BUF_BLOCK_FILE_PAGE
&& buf_pool.will_be_withdrawn(*bpage)) {
&& will_be_withdrawn(*bpage)) {
if (bpage->can_relocate()) {
buf_pool_mutex_exit_forbid();
if (!realloc(
@@ -1911,7 +1897,7 @@ inline bool buf_pool_t::withdraw_blocks()
bpage = next_bpage;
}
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
buf_resize_status(
"withdrawing blocks. (" ULINTPF "/" ULINTPF ")",
@@ -2031,7 +2017,7 @@ inline void buf_pool_t::page_hash_table::write_unlock_all()
inline void buf_pool_t::write_lock_all_page_hash()
{
ut_ad(mutex_own(&mutex));
mysql_mutex_assert_owner(&mutex);
page_hash.write_lock_all();
for (page_hash_table *old_page_hash= freed_page_hash; old_page_hash;
old_page_hash= static_cast<page_hash_table*>
@@ -2103,7 +2089,7 @@ inline void buf_pool_t::resize()
srv_buf_pool_old_size, srv_buf_pool_size,
srv_buf_pool_chunk_unit);
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
ut_ad(curr_size == old_size);
ut_ad(n_chunks_new == n_chunks);
ut_ad(UT_LIST_GET_LEN(withdraw) == 0);
@@ -2111,7 +2097,7 @@ inline void buf_pool_t::resize()
n_chunks_new = (new_instance_size << srv_page_size_shift)
/ srv_buf_pool_chunk_unit;
curr_size = n_chunks_new * chunks->size;
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
#ifdef BTR_CUR_HASH_ADAPT
/* disable AHI if needed */
@@ -2225,7 +2211,7 @@ withdraw_retry:
/* Indicate critical path */
resizing.store(true, std::memory_order_relaxed);
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
write_lock_all_page_hash();
chunk_t::map_reg = UT_NEW_NOKEY(chunk_t::map());
@@ -2389,7 +2375,7 @@ calc_buf_pool_size:
ib::info() << "hash tables were resized";
}
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
write_unlock_all_page_hash();
UT_DELETE(chunk_map_old);
@@ -2454,10 +2440,10 @@ static void buf_resize_callback(void *)
{
DBUG_ENTER("buf_resize_callback");
ut_a(srv_shutdown_state == SRV_SHUTDOWN_NONE);
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
const auto size= srv_buf_pool_size;
const bool work= srv_buf_pool_old_size != size;
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
if (work)
buf_pool.resize();
@@ -2495,7 +2481,7 @@ static void buf_relocate(buf_page_t *bpage, buf_page_t *dpage)
{
const ulint fold= bpage->id().fold();
ut_ad(bpage->state() == BUF_BLOCK_ZIP_PAGE);
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(buf_pool.hash_lock_get(bpage->id())->is_write_locked());
ut_a(bpage->io_fix() == BUF_IO_NONE);
ut_a(!bpage->buf_fix_count());
@@ -2568,7 +2554,7 @@ retry:
(*hash_lock)->write_unlock();
/* Allocate a watch[] and then try to insert it into the page_hash. */
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
/* The maximum number of purge tasks should never exceed
the UT_ARR_SIZE(watch) - 1, and there is no way for a purge task to hold a
@@ -2592,17 +2578,17 @@ retry:
*hash_lock= page_hash.lock_get(fold);
(*hash_lock)->write_lock();
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
buf_page_t *bpage= page_hash_get_low(id, fold);
if (UNIV_LIKELY_NULL(bpage))
{
(*hash_lock)->write_unlock();
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
w->set_state(BUF_BLOCK_NOT_USED);
*hash_lock= page_hash.lock_get(fold);
(*hash_lock)->write_lock();
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
goto retry;
}
@@ -2615,7 +2601,7 @@ retry:
}
ut_error;
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
return nullptr;
}
@@ -2733,10 +2719,10 @@ err_exit:
{
discard_attempted= true;
hash_lock->read_unlock();
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
if (buf_page_t *bpage= buf_pool.page_hash_get_low(page_id, fold))
buf_LRU_free_page(bpage, false);
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
goto lookup;
}
@@ -3261,14 +3247,14 @@ got_block:
if (UNIV_UNLIKELY(mode == BUF_EVICT_IF_IN_POOL)) {
evict_from_pool:
ut_ad(!fix_block->page.oldest_modification());
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
fix_block->unfix();
if (!buf_LRU_free_page(&fix_block->page, true)) {
ut_ad(0);
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
return(NULL);
}
@@ -3317,7 +3303,7 @@ evict_from_pool:
block = buf_LRU_get_free_block(false);
buf_block_init_low(block);
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
hash_lock = buf_pool.page_hash.lock_get(fold);
hash_lock->write_lock();
@@ -3336,7 +3322,7 @@ evict_from_pool:
hash_lock->write_unlock();
buf_LRU_block_free_non_file_page(block);
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
/* Try again */
goto loop;
@@ -3355,9 +3341,7 @@ evict_from_pool:
/* Set after buf_relocate(). */
block->page.set_buf_fix_count(1);
if (block->page.oldest_modification()) {
buf_flush_relocate_on_flush_list(bpage, &block->page);
}
/* Buffer-fix, I/O-fix, and X-latch the block
for the duration of the decompression.
@@ -3372,7 +3356,7 @@ evict_from_pool:
MEM_UNDEFINED(bpage, sizeof *bpage);
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
hash_lock->write_unlock();
buf_pool.n_pend_unzip++;
@@ -3412,7 +3396,7 @@ evict_from_pool:
ut_ad(fix_block->page.state() == BUF_BLOCK_FILE_PAGE);
#if defined UNIV_DEBUG || defined UNIV_IBUF_DEBUG
re_evict:
if (mode != BUF_GET_IF_IN_POOL
&& mode != BUF_GET_IF_IN_POOL_OR_WATCH) {
} else if (!ibuf_debug) {
@@ -3421,18 +3405,19 @@ evict_from_pool:
/* Try to evict the block from the buffer pool, to use the
insert buffer (change buffer) as much as possible. */
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
fix_block->unfix();
/* Blocks cannot be relocated or enter or exit the
buf_pool while we are holding the buf_pool.mutex. */
if (buf_LRU_free_page(&fix_block->page, true)) {
const bool evicted = buf_LRU_free_page(&fix_block->page, true);
space->release_for_io();
if (evicted) {
hash_lock = buf_pool.page_hash.lock_get(fold);
hash_lock->write_lock();
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
/* We may set the watch, as it would have
been set if the page were not in the
buffer pool in the first place. */
@@ -3456,20 +3441,16 @@ evict_from_pool:
return(NULL);
}
bool flushed = fix_block->page.ready_for_flush()
&& buf_flush_page(&fix_block->page,
IORequest::SINGLE_PAGE, space, true);
space->release_for_io();
if (flushed) {
guess = fix_block;
goto loop;
fix_block->fix();
mysql_mutex_unlock(&buf_pool.mutex);
buf_flush_lists(ULINT_UNDEFINED, LSN_MAX);
buf_flush_wait_batch_end_acquiring_mutex(false);
if (!fix_block->page.oldest_modification()) {
goto re_evict;
}
fix_block->fix();
/* Failed to evict the page; change it directly */
mutex_exit(&buf_pool.mutex);
}
#endif /* UNIV_DEBUG || UNIV_IBUF_DEBUG */
@@ -3793,23 +3774,22 @@ FILE_PAGE (the other is buf_page_get_gen).
@param[in] offset offset of the tablespace
@param[in] zip_size ROW_FORMAT=COMPRESSED page size, or 0
@param[in,out] mtr mini-transaction
@param[in,out] free_block pre-allocated buffer block
@return pointer to the block, page bufferfixed */
buf_block_t*
buf_page_create(fil_space_t *space, uint32_t offset,
ulint zip_size, mtr_t *mtr)
ulint zip_size, mtr_t *mtr, buf_block_t *free_block)
{
page_id_t page_id(space->id, offset);
ut_ad(mtr->is_active());
ut_ad(page_id.space() != 0 || !zip_size);
space->free_page(offset, false);
loop:
buf_block_t *free_block= buf_LRU_get_free_block(false);
free_block->initialise(page_id, zip_size, 1);
const ulint fold= page_id.fold();
mutex_enter(&buf_pool.mutex);
loop:
mysql_mutex_lock(&buf_pool.mutex);
buf_block_t *block= reinterpret_cast<buf_block_t*>
(buf_pool.page_hash_get_low(page_id, fold));
@@ -3820,7 +3800,7 @@ loop:
#ifdef BTR_CUR_HASH_ADAPT
const dict_index_t *drop_hash_entry= nullptr;
#endif
switch (block->page.state()) {
switch (UNIV_EXPECT(block->page.state(), BUF_BLOCK_FILE_PAGE)) {
default:
ut_ad(0);
break;
@@ -3831,16 +3811,15 @@ loop:
while (block->page.io_fix() != BUF_IO_NONE ||
num_fix_count != block->page.buf_fix_count())
{
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
os_thread_yield();
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
}
}
rw_lock_x_lock(&block->lock);
#ifdef BTR_CUR_HASH_ADAPT
drop_hash_entry= block->index;
#endif
buf_LRU_block_free_non_file_page(free_block);
break;
case BUF_BLOCK_ZIP_PAGE:
page_hash_latch *hash_lock= buf_pool.page_hash.lock_get(fold);
@@ -3849,14 +3828,12 @@ loop:
{
hash_lock->write_unlock();
buf_LRU_block_free_non_file_page(free_block);
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
goto loop;
}
rw_lock_x_lock(&free_block->lock);
buf_relocate(&block->page, &free_block->page);
if (block->page.oldest_modification() > 0)
buf_flush_relocate_on_flush_list(&block->page, &free_block->page);
free_block->page.set_state(BUF_BLOCK_FILE_PAGE);
@@ -3868,7 +3845,7 @@ loop:
break;
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
#ifdef BTR_CUR_HASH_ADAPT
if (drop_hash_entry)
@@ -3926,7 +3903,7 @@ loop:
else
hash_lock->write_unlock();
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
mtr->memo_push(block, MTR_MEMO_PAGE_X_FIX);
block->page.set_accessed();
@@ -4074,12 +4051,12 @@ static void buf_mark_space_corrupt(buf_page_t* bpage, const fil_space_t& space)
/** Release and evict a corrupted page.
@param bpage page that was being read */
void buf_pool_t::corrupted_evict(buf_page_t *bpage)
ATTRIBUTE_COLD void buf_pool_t::corrupted_evict(buf_page_t *bpage)
{
const page_id_t id(bpage->id());
page_hash_latch *hash_lock= hash_lock_get(id);
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
hash_lock->write_lock();
ut_ad(bpage->io_fix() == BUF_IO_READ);
@@ -4094,7 +4071,7 @@ void buf_pool_t::corrupted_evict(buf_page_t *bpage)
/* remove from LRU and page_hash */
buf_LRU_free_one_page(bpage, id, hash_lock);
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
ut_d(auto n=) n_pend_reads--;
ut_ad(n > 0);
@@ -4104,6 +4081,7 @@ void buf_pool_t::corrupted_evict(buf_page_t *bpage)
@param[in] bpage Corrupted page
@param[in] node data file
Also remove the bpage from LRU list. */
ATTRIBUTE_COLD
static void buf_corrupt_page_release(buf_page_t *bpage, const fil_node_t &node)
{
ut_ad(bpage->id().space() == node.space->id);
@@ -4220,7 +4198,7 @@ dberr_t buf_page_read_complete(buf_page_t *bpage, const fil_node_t &node)
{
const page_id_t id(bpage->id());
ut_ad(bpage->in_file());
ut_ad(id.space() || !buf_dblwr_page_inside(id.page_no()));
ut_ad(!buf_dblwr.is_inside(id));
ut_ad(id.space() == node.space->id);
ut_ad(bpage->zip_size() == node.space->zip_size());
@@ -4287,7 +4265,7 @@ dberr_t buf_page_read_complete(buf_page_t *bpage, const fil_node_t &node)
}
err= buf_page_check_corrupt(bpage, node);
if (err != DB_SUCCESS)
if (UNIV_UNLIKELY(err != DB_SUCCESS))
{
database_corrupted:
/* Not a real corruption if it was triggered by error injection */
@@ -4375,12 +4353,12 @@ release_page:
@retval nullptr if all freed */
void buf_pool_t::assert_all_freed()
{
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
const chunk_t *chunk= chunks;
for (auto i= n_chunks; i--; chunk++)
if (const buf_block_t* block= chunk->not_freed())
ib::fatal() << "Page " << block->page.id() << " still fixed or dirty";
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
}
#endif /* UNIV_DEBUG */
@@ -4395,33 +4373,20 @@ void buf_refresh_io_stats()
All pages must be in a replaceable state (not modified or latched). */
void buf_pool_invalidate()
{
mutex_enter(&buf_pool.mutex);
ut_ad(!buf_pool.init_flush[IORequest::LRU]);
ut_ad(!buf_pool.init_flush[IORequest::FLUSH_LIST]);
ut_ad(!buf_pool.init_flush[IORequest::SINGLE_PAGE]);
ut_ad(!buf_pool.n_flush[IORequest::SINGLE_PAGE]);
mysql_mutex_lock(&buf_pool.mutex);
if (buf_pool.n_flush[IORequest::LRU]) {
mutex_exit(&buf_pool.mutex);
buf_flush_wait_batch_end(true);
mutex_enter(&buf_pool.mutex);
}
if (buf_pool.n_flush[IORequest::FLUSH_LIST]) {
mutex_exit(&buf_pool.mutex);
buf_flush_wait_batch_end(false);
mutex_enter(&buf_pool.mutex);
}
/* It is possible that a write batch that has been posted
earlier is still not complete. For buffer pool invalidation to
proceed we must ensure there is NO write activity happening. */
ut_d(mutex_exit(&buf_pool.mutex));
ut_d(mysql_mutex_unlock(&buf_pool.mutex));
ut_d(buf_pool.assert_all_freed());
ut_d(mutex_enter(&buf_pool.mutex));
ut_d(mysql_mutex_lock(&buf_pool.mutex));
while (buf_LRU_scan_and_free_block(true));
while (buf_LRU_scan_and_free_block());
ut_ad(UT_LIST_GET_LEN(buf_pool.LRU) == 0);
ut_ad(UT_LIST_GET_LEN(buf_pool.unzip_LRU) == 0);
@@ -4432,7 +4397,7 @@ void buf_pool_invalidate()
memset(&buf_pool.stat, 0x00, sizeof(buf_pool.stat));
buf_refresh_io_stats();
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
}
#ifdef UNIV_DEBUG
@@ -4444,7 +4409,7 @@ void buf_pool_t::validate()
ulint n_free = 0;
ulint n_zip = 0;
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
chunk_t* chunk = chunks;
@@ -4485,7 +4450,7 @@ void buf_pool_t::validate()
/* Check dirty blocks. */
mutex_enter(&flush_list_mutex);
mysql_mutex_lock(&flush_list_mutex);
for (buf_page_t* b = UT_LIST_GET_FIRST(flush_list); b;
b = UT_LIST_GET_NEXT(list, b)) {
ut_ad(b->oldest_modification());
@@ -4511,7 +4476,7 @@ void buf_pool_t::validate()
ut_ad(UT_LIST_GET_LEN(flush_list) == n_flushing);
mutex_exit(&flush_list_mutex);
mysql_mutex_unlock(&flush_list_mutex);
if (curr_size == old_size
&& n_lru + n_free > curr_size + n_zip) {
@@ -4531,7 +4496,7 @@ void buf_pool_t::validate()
<< ", free blocks " << n_free << ". Aborting...";
}
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
ut_d(buf_LRU_validate());
ut_d(buf_flush_validate());
@@ -4559,8 +4524,8 @@ void buf_pool_t::print()
counts = static_cast<ulint*>(ut_malloc_nokey(sizeof(ulint) * size));
mutex_enter(&mutex);
mutex_enter(&flush_list_mutex);
mysql_mutex_lock(&mutex);
mysql_mutex_lock(&flush_list_mutex);
ib::info()
<< "[buffer pool: size=" << curr_size
@@ -4570,16 +4535,15 @@ void buf_pool_t::print()
<< UT_LIST_GET_LEN(flush_list)
<< ", n pending decompressions=" << n_pend_unzip
<< ", n pending reads=" << n_pend_reads
<< ", n pending flush LRU=" << n_flush[IORequest::LRU]
<< " list=" << n_flush[IORequest::FLUSH_LIST]
<< " single page=" << n_flush[IORequest::SINGLE_PAGE]
<< ", n pending flush LRU=" << n_flush_LRU
<< " list=" << n_flush_list
<< ", pages made young=" << stat.n_pages_made_young
<< ", not young=" << stat.n_pages_not_made_young
<< ", pages read=" << stat.n_pages_read
<< ", created=" << stat.n_pages_created
<< ", written=" << stat.n_pages_written << "]";
mutex_exit(&flush_list_mutex);
mysql_mutex_unlock(&flush_list_mutex);
/* Count the number of blocks belonging to each index in the buffer */
@@ -4620,7 +4584,7 @@ void buf_pool_t::print()
}
}
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
for (i = 0; i < n_found; i++) {
index = dict_index_get_if_in_cache(index_ids[i]);
@@ -4650,14 +4614,14 @@ ulint buf_get_latched_pages_number()
{
ulint fixed_pages_number= 0;
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
for (buf_page_t *b= UT_LIST_GET_FIRST(buf_pool.LRU); b;
b= UT_LIST_GET_NEXT(LRU, b))
if (b->in_file() && (b->buf_fix_count() || b->io_fix() != BUF_IO_NONE))
fixed_pages_number++;
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
return fixed_pages_number;
}
@@ -4670,8 +4634,8 @@ void buf_stats_get_pool_info(buf_pool_info_t *pool_info)
time_t current_time;
double time_elapsed;
mutex_enter(&buf_pool.mutex);
mutex_enter(&buf_pool.flush_list_mutex);
mysql_mutex_lock(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.flush_list_mutex);
pool_info->pool_size = buf_pool.curr_size;
@@ -4687,19 +4651,11 @@ void buf_stats_get_pool_info(buf_pool_info_t *pool_info)
pool_info->n_pend_reads = buf_pool.n_pend_reads;
pool_info->n_pending_flush_lru =
(buf_pool.n_flush[IORequest::LRU]
+ buf_pool.init_flush[IORequest::LRU]);
pool_info->n_pending_flush_lru = buf_pool.n_flush_LRU;
pool_info->n_pending_flush_list =
(buf_pool.n_flush[IORequest::FLUSH_LIST]
+ buf_pool.init_flush[IORequest::FLUSH_LIST]);
pool_info->n_pending_flush_list = buf_pool.n_flush_list;
pool_info->n_pending_flush_single_page =
(buf_pool.n_flush[IORequest::SINGLE_PAGE]
+ buf_pool.init_flush[IORequest::SINGLE_PAGE]);
mutex_exit(&buf_pool.flush_list_mutex);
mysql_mutex_unlock(&buf_pool.flush_list_mutex);
current_time = time(NULL);
time_elapsed = 0.001 + difftime(current_time,
@@ -4790,7 +4746,7 @@ void buf_stats_get_pool_info(buf_pool_info_t *pool_info)
pool_info->unzip_cur = buf_LRU_stat_cur.unzip;
buf_refresh_io_stats();
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
}
/*********************************************************************//**
@@ -4813,8 +4769,7 @@ buf_print_io_instance(
"Percent of dirty pages(LRU & free pages): %.3f\n"
"Max dirty pages percent: %.3f\n"
"Pending reads " ULINTPF "\n"
"Pending writes: LRU " ULINTPF ", flush list " ULINTPF
", single page " ULINTPF "\n",
"Pending writes: LRU " ULINTPF ", flush list " ULINTPF "\n",
pool_info->pool_size,
pool_info->free_list_len,
pool_info->lru_len,
@@ -4827,8 +4782,7 @@ buf_print_io_instance(
srv_max_buf_pool_modified_pct,
pool_info->n_pend_reads,
pool_info->n_pending_flush_lru,
pool_info->n_pending_flush_list,
pool_info->n_pending_flush_single_page);
pool_info->n_pending_flush_list);
fprintf(file,
"Pages made young " ULINTPF ", not young " ULINTPF "\n"

1198
storage/innobase/buf/buf0dblwr.cc
View File

File diff suppressed because it is too large Load Diff

8
storage/innobase/buf/buf0dump.cc
View File

@@ -276,13 +276,13 @@ buf_dump(
ulint n_pages;
ulint j;
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
n_pages = UT_LIST_GET_LEN(buf_pool.LRU);
/* skip empty buffer pools */
if (n_pages == 0) {
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
goto done;
}
@@ -310,7 +310,7 @@ buf_dump(
n_pages * sizeof(*dump)));
if (dump == NULL) {
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
fclose(f);
buf_dump_status(STATUS_ERR,
"Cannot allocate " ULINTPF " bytes: %s",
@@ -335,7 +335,7 @@ buf_dump(
dump[j++] = id;
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
ut_a(j <= n_pages);
n_pages = j;

1823
storage/innobase/buf/buf0flu.cc
View File

File diff suppressed because it is too large Load Diff

534
storage/innobase/buf/buf0lru.cc
View File

@@ -39,6 +39,9 @@ Created 11/5/1995 Heikki Tuuri
#include "srv0srv.h"
#include "srv0mon.h"
/** Flush this many pages in buf_LRU_get_free_block() */
size_t innodb_lru_flush_size;
/** The number of blocks from the LRU_old pointer onward, including
the block pointed to, must be buf_pool.LRU_old_ratio/BUF_LRU_OLD_RATIO_DIV
of the whole LRU list length, except that the tolerance defined below
@@ -46,28 +49,13 @@ is allowed. Note that the tolerance must be small enough such that for
even the BUF_LRU_OLD_MIN_LEN long LRU list, the LRU_old pointer is not
allowed to point to either end of the LRU list. */
static const ulint BUF_LRU_OLD_TOLERANCE = 20;
static constexpr ulint BUF_LRU_OLD_TOLERANCE = 20;
/** The minimum amount of non-old blocks when the LRU_old list exists
(that is, when there are more than BUF_LRU_OLD_MIN_LEN blocks).
@see buf_LRU_old_adjust_len */
#define BUF_LRU_NON_OLD_MIN_LEN 5
#ifdef BTR_CUR_HASH_ADAPT
/** When dropping the search hash index entries before deleting an ibd
file, we build a local array of pages belonging to that tablespace
in the buffer pool. Following is the size of that array.
We also release buf_pool.mutex after scanning this many pages of the
flush_list when dropping a table. This is to ensure that other threads
are not blocked for extended period of time when using very large
buffer pools. */
static const ulint BUF_LRU_DROP_SEARCH_SIZE = 1024;
#endif /* BTR_CUR_HASH_ADAPT */
/** We scan these many blocks when looking for a clean page to evict
during LRU eviction. */
static const ulint BUF_LRU_SEARCH_SCAN_THRESHOLD = 100;
/** If we switch on the InnoDB monitor because there are too few available
frames in the buffer pool, we set this to TRUE */
static bool buf_lru_switched_on_innodb_mon = false;
@@ -149,7 +137,7 @@ static void buf_LRU_block_free_hashed_page(buf_block_t *block)
static inline void incr_LRU_size_in_bytes(const buf_page_t* bpage)
{
/* FIXME: use atomics, not mutex */
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
buf_pool.stat.LRU_bytes += bpage->physical_size();
@@ -160,7 +148,7 @@ static inline void incr_LRU_size_in_bytes(const buf_page_t* bpage)
instead of the general LRU list */
bool buf_LRU_evict_from_unzip_LRU()
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
/* If the unzip_LRU list is empty, we can only use the LRU. */
if (UT_LIST_GET_LEN(buf_pool.unzip_LRU) == 0) {
@@ -196,280 +184,19 @@ bool buf_LRU_evict_from_unzip_LRU()
return(unzip_avg <= io_avg * BUF_LRU_IO_TO_UNZIP_FACTOR);
}
#ifdef BTR_CUR_HASH_ADAPT
/**
While flushing (or removing dirty) pages from a tablespace we don't
want to hog the CPU and resources. Release the buffer pool and block
mutex and try to force a context switch. Then reacquire the same mutexes.
The current page is "fixed" before the release of the mutexes and then
"unfixed" again once we have reacquired the mutexes.
@param[in,out] bpage current page */
static void buf_flush_yield(buf_page_t *bpage)
{
mutex_exit(&buf_pool.flush_list_mutex);
ut_ad(bpage->oldest_modification());
ut_ad(bpage->in_file());
ut_ad(bpage->io_fix() == BUF_IO_NONE);
/** Make the block sticky, so that even after we release buf_pool.mutex:
(1) it cannot be removed from the buf_pool.flush_list
(2) bpage cannot be relocated in buf_pool
(3) bpage->in_LRU_list cannot change
However, bpage->LRU can change. */
bpage->set_io_fix(BUF_IO_PIN);
mutex_exit(&buf_pool.mutex);
/* Try and force a context switch. */
os_thread_yield();
mutex_enter(&buf_pool.mutex);
bpage->io_unfix();
mutex_enter(&buf_pool.flush_list_mutex);
/* Should not have been removed from the flush
list during the yield. However, this check is
not sufficient to catch a remove -> add. */
ut_ad(bpage->oldest_modification());
}
/******************************************************************//**
If we have hogged the resources for too long then release the buffer
pool and flush list mutex and do a thread yield. Set the current page
to "sticky" so that it is not relocated during the yield.
@return true if yielded */
static MY_ATTRIBUTE((warn_unused_result))
bool
buf_flush_try_yield(
/*================*/
buf_page_t* bpage, /*!< in/out: bpage to remove */
ulint processed) /*!< in: number of pages processed */
{
/* Every BUF_LRU_DROP_SEARCH_SIZE iterations in the
loop we release buf_pool.mutex to let other threads
do their job but only if the block is not IO fixed. This
ensures that the block stays in its position in the
flush_list. */
if (bpage != NULL
&& processed >= BUF_LRU_DROP_SEARCH_SIZE
&& bpage->io_fix() == BUF_IO_NONE) {
/* Release the buf_pool.mutex
to give the other threads a go. */
buf_flush_yield(bpage);
return(true);
}
return(false);
}
#endif /* BTR_CUR_HASH_ADAPT */
/** Remove a single page from flush_list.
@param[in,out] bpage buffer page to remove
@param[in] flush whether to flush the page before removing
@return true if page was removed. */
static bool buf_flush_or_remove_page(buf_page_t *bpage, bool flush)
{
ut_ad(mutex_own(&buf_pool.mutex));
ut_ad(mutex_own(&buf_pool.flush_list_mutex));
/* bpage->id and bpage->io_fix are protected by
buf_pool.mutex (and bpage->id additionally by hash_lock).
It is safe to check them while holding buf_pool.mutex only. */
if (bpage->io_fix() != BUF_IO_NONE) {
/* We cannot remove this page during this scan
yet; maybe the system is currently reading it
in, or flushing the modifications to the file */
return(false);
}
bool processed = false;
/* We have to release the flush_list_mutex to obey the
latching order. We are however guaranteed that the page
will stay in the flush_list and won't be relocated because
buf_flush_remove() and buf_flush_relocate_on_flush_list()
need buf_pool.mutex as well. */
mutex_exit(&buf_pool.flush_list_mutex);
ut_ad(bpage->oldest_modification());
if (!flush) {
buf_flush_remove(bpage);
processed = true;
} else if (bpage->ready_for_flush()) {
processed = buf_flush_page(bpage, IORequest::SINGLE_PAGE,
nullptr, false);
if (processed) {
mutex_enter(&buf_pool.mutex);
}
}
mutex_enter(&buf_pool.flush_list_mutex);
ut_ad(mutex_own(&buf_pool.mutex));
return(processed);
}
/** Remove all dirty pages belonging to a given tablespace when we are
deleting the data file of that tablespace.
The pages still remain a part of LRU and are evicted from
the list as they age towards the tail of the LRU.
@param[in] id tablespace identifier
@param[in] flush whether to flush the pages before removing
@param[in] first first page to be flushed or evicted
@return whether all matching dirty pages were removed */
static bool buf_flush_or_remove_pages(ulint id, bool flush, ulint first)
{
buf_page_t* prev;
buf_page_t* bpage;
ulint processed = 0;
mutex_enter(&buf_pool.flush_list_mutex);
rescan:
bool all_freed = true;
for (bpage = UT_LIST_GET_LAST(buf_pool.flush_list);
bpage != NULL;
bpage = prev) {
ut_a(bpage->in_file());
/* Save the previous link because once we free the
page we can't rely on the links. */
prev = UT_LIST_GET_PREV(list, bpage);
const page_id_t bpage_id(bpage->id());
if (id != bpage_id.space()) {
/* Skip this block, because it is for a
different tablespace. */
} else if (bpage_id.page_no() < first) {
/* Skip this block, because it is below the limit. */
} else if (!buf_flush_or_remove_page(bpage, flush)) {
/* Remove was unsuccessful, we have to try again
by scanning the entire list from the end.
This also means that we never released the
buf_pool mutex. Therefore we can trust the prev
pointer.
buf_flush_or_remove_page() released the
flush list mutex but not the buf_pool mutex.
Therefore it is possible that a new page was
added to the flush list. For example, in case
where we are at the head of the flush list and
prev == NULL. That is OK because we have the
tablespace quiesced and no new pages for this
space-id should enter flush_list. This is
because the only callers of this function are
DROP TABLE and FLUSH TABLE FOR EXPORT.
We know that we'll have to do at least one more
scan but we don't break out of loop here and
try to do as much work as we can in this
iteration. */
all_freed = false;
} else if (flush) {
/* The processing was successful. And during the
processing we have released the buf_pool mutex
when calling buf_page_flush(). We cannot trust
prev pointer. */
goto rescan;
}
#ifdef BTR_CUR_HASH_ADAPT
++processed;
/* Yield if we have hogged the CPU and mutexes for too long. */
if (buf_flush_try_yield(prev, processed)) {
/* Reset the batch size counter if we had to yield. */
processed = 0;
}
#endif /* BTR_CUR_HASH_ADAPT */
}
mutex_exit(&buf_pool.flush_list_mutex);
return(all_freed);
}
/** Remove or flush all the dirty pages that belong to a given tablespace.
The pages will remain in the LRU list and will be evicted from the LRU list
as they age and move towards the tail of the LRU list.
@param[in] id tablespace identifier
@param[in] flush whether to flush the pages before removing
@param[in] first first page to be flushed or evicted */
static void buf_flush_dirty_pages(ulint id, bool flush, ulint first)
{
mutex_enter(&buf_pool.mutex);
while (!buf_flush_or_remove_pages(id, flush, first))
{
mutex_exit(&buf_pool.mutex);
ut_d(buf_flush_validate());
os_thread_sleep(2000);
mutex_enter(&buf_pool.mutex);
}
#ifdef UNIV_DEBUG
if (!first)
{
mutex_enter(&buf_pool.flush_list_mutex);
for (buf_page_t *bpage= UT_LIST_GET_FIRST(buf_pool.flush_list); bpage;
bpage= UT_LIST_GET_NEXT(list, bpage))
{
ut_ad(bpage->in_file());
ut_ad(bpage->oldest_modification());
ut_ad(id != bpage->id().space());
}
mutex_exit(&buf_pool.flush_list_mutex);
}
#endif
mutex_exit(&buf_pool.mutex);
}
/** Empty the flush list for all pages belonging to a tablespace.
@param[in] id tablespace identifier
@param[in] flush whether to write the pages to files
@param[in] first first page to be flushed or evicted */
void buf_LRU_flush_or_remove_pages(ulint id, bool flush, ulint first)
{
/* Pages in the system tablespace must never be discarded. */
ut_ad(id || flush);
buf_flush_dirty_pages(id, flush, first);
if (flush) {
/* Ensure that all asynchronous IO is completed. */
os_aio_wait_until_no_pending_writes();
fil_flush(id);
}
}
/** Try to free an uncompressed page of a compressed block from the unzip
LRU list. The compressed page is preserved, and it need not be clean.
@param[in] scan_all true=scan the whole list;
false=scan srv_LRU_scan_depth / 2 blocks
@param limit maximum number of blocks to scan
@return true if freed */
static bool buf_LRU_free_from_unzip_LRU_list(bool scan_all)
static bool buf_LRU_free_from_unzip_LRU_list(ulint limit)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
if (!buf_LRU_evict_from_unzip_LRU()) {
return(false);
}
ulint scanned = 0;
const ulint limit = scan_all ? ULINT_UNDEFINED : srv_LRU_scan_depth;
bool freed = false;
for (buf_block_t* block = UT_LIST_GET_LAST(buf_pool.unzip_LRU);
@@ -500,31 +227,24 @@ static bool buf_LRU_free_from_unzip_LRU_list(bool scan_all)
}
/** Try to free a clean page from the common LRU list.
@param[in] scan_all true=scan the whole LRU list
false=use BUF_LRU_SEARCH_SCAN_THRESHOLD
@param limit maximum number of blocks to scan
@return whether a page was freed */
static bool buf_LRU_free_from_common_LRU_list(bool scan_all)
static bool buf_LRU_free_from_common_LRU_list(ulint limit)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ulint scanned = 0;
bool freed = false;
for (buf_page_t* bpage = buf_pool.lru_scan_itr.start();
bpage && (scan_all || scanned < BUF_LRU_SEARCH_SCAN_THRESHOLD);
bpage && scanned < limit;
++scanned, bpage = buf_pool.lru_scan_itr.get()) {
buf_page_t* prev = UT_LIST_GET_PREV(LRU, bpage);
buf_pool.lru_scan_itr.set(prev);
const auto accessed = bpage->is_accessed();
freed = bpage->ready_for_replace();
if (freed) {
freed = buf_LRU_free_page(bpage, true);
if (!freed) {
continue;
}
if (!bpage->oldest_modification()
&& buf_LRU_free_page(bpage, true)) {
if (!accessed) {
/* Keep track of pages that are evicted without
ever being accessed. This gives us a measure of
@@ -532,6 +252,7 @@ static bool buf_LRU_free_from_common_LRU_list(bool scan_all)
++buf_pool.stat.n_ra_pages_evicted;
}
freed = true;
break;
}
}
@@ -548,15 +269,14 @@ static bool buf_LRU_free_from_common_LRU_list(bool scan_all)
}
/** Try to free a replaceable block.
@param[in] scan_all true=scan the whole LRU list,
false=use BUF_LRU_SEARCH_SCAN_THRESHOLD
@param limit maximum number of blocks to scan
@return true if found and freed */
bool buf_LRU_scan_and_free_block(bool scan_all)
bool buf_LRU_scan_and_free_block(ulint limit)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
return(buf_LRU_free_from_unzip_LRU_list(scan_all)
|| buf_LRU_free_from_common_LRU_list(scan_all));
return buf_LRU_free_from_unzip_LRU_list(limit) ||
buf_LRU_free_from_common_LRU_list(limit);
}
/** @return a buffer block from the buf_pool.free list
@@ -565,7 +285,7 @@ buf_block_t* buf_LRU_get_free_only()
{
buf_block_t* block;
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
block = reinterpret_cast<buf_block_t*>(
UT_LIST_GET_FIRST(buf_pool.free));
@@ -611,106 +331,89 @@ function will either assert or issue a warning and switch on the
status monitor. */
static void buf_LRU_check_size_of_non_data_objects()
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
if (!recv_recovery_is_on()
&& buf_pool.curr_size == buf_pool.old_size
&& UT_LIST_GET_LEN(buf_pool.free)
+ UT_LIST_GET_LEN(buf_pool.LRU) < buf_pool.curr_size / 20) {
if (recv_recovery_is_on() || buf_pool.curr_size != buf_pool.old_size)
return;
const auto s= UT_LIST_GET_LEN(buf_pool.free) + UT_LIST_GET_LEN(buf_pool.LRU);
if (s < buf_pool.curr_size / 20)
ib::fatal() << "Over 95 percent of the buffer pool is"
" occupied by lock heaps"
#ifdef BTR_CUR_HASH_ADAPT
" or the adaptive hash index!"
" or the adaptive hash index"
#endif /* BTR_CUR_HASH_ADAPT */
" Check that your transactions do not set too many"
" row locks, or review if"
" innodb_buffer_pool_size="
"! Check that your transactions do not set too many"
" row locks, or review if innodb_buffer_pool_size="
<< (buf_pool.curr_size >> (20U - srv_page_size_shift))
<< "M could be bigger.";
} else if (!recv_recovery_is_on()
&& buf_pool.curr_size == buf_pool.old_size
&& (UT_LIST_GET_LEN(buf_pool.free)
+ UT_LIST_GET_LEN(buf_pool.LRU))
< buf_pool.curr_size / 3) {
if (!buf_lru_switched_on_innodb_mon && srv_monitor_timer) {
/* Over 67 % of the buffer pool is occupied by lock
heaps or the adaptive hash index. This may be a memory
leak! */
if (s < buf_pool.curr_size / 3)
{
if (!buf_lru_switched_on_innodb_mon && srv_monitor_timer)
{
/* Over 67 % of the buffer pool is occupied by lock heaps or
the adaptive hash index. This may be a memory leak! */
ib::warn() << "Over 67 percent of the buffer pool is"
" occupied by lock heaps"
#ifdef BTR_CUR_HASH_ADAPT
" or the adaptive hash index!"
" or the adaptive hash index"
#endif /* BTR_CUR_HASH_ADAPT */
" Check that your transactions do not"
" set too many row locks."
"! Check that your transactions do not set too many row locks."
" innodb_buffer_pool_size="
<< (buf_pool.curr_size >>
(20U - srv_page_size_shift)) << "M."
" Starting the InnoDB Monitor to print"
" diagnostics.";
buf_lru_switched_on_innodb_mon = true;
srv_print_innodb_monitor = TRUE;
<< (buf_pool.curr_size >> (20U - srv_page_size_shift))
<< "M. Starting the InnoDB Monitor to print diagnostics.";
buf_lru_switched_on_innodb_mon= true;
srv_print_innodb_monitor= TRUE;
srv_monitor_timer_schedule_now();
}
} else if (buf_lru_switched_on_innodb_mon) {
/* Switch off the InnoDB Monitor; this is a simple way
to stop the monitor if the situation becomes less urgent,
but may also surprise users if the user also switched on the
monitor! */
buf_lru_switched_on_innodb_mon = false;
srv_print_innodb_monitor = FALSE;
}
else if (buf_lru_switched_on_innodb_mon)
{
/* Switch off the InnoDB Monitor; this is a simple way to stop the
monitor if the situation becomes less urgent, but may also
surprise users who did SET GLOBAL innodb_status_output=ON earlier! */
buf_lru_switched_on_innodb_mon= false;
srv_print_innodb_monitor= FALSE;
}
}
/** Get a free block from the buf_pool. The block is taken off the
free list. If free list is empty, blocks are moved from the end of the
LRU list to the free list.
/** Get a block from the buf_pool.free list.
If the list is empty, blocks will be moved from the end of buf_pool.LRU
to buf_pool.free.
This function is called from a user thread when it needs a clean
block to read in a page. Note that we only ever get a block from
the free list. Even when we flush a page or find a page in LRU scan
we put it to free list to be used.
* iteration 0:
* get a block from free list, success:done
* get a block from the buf_pool.free list, success:done
* if buf_pool.try_LRU_scan is set
* scan LRU up to srv_LRU_scan_depth to find a clean block
* the above will put the block on free list
* scan LRU up to 100 pages to free a clean block
* success:retry the free list
* flush one dirty page from tail of LRU to disk
* the above will put the block on free list
* flush up to innodb_lru_flush_size LRU blocks to data files
(until UT_LIST_GET_GEN(buf_pool.free) < innodb_lru_scan_depth)
* on buf_page_write_complete() the blocks will put on buf_pool.free list
* success: retry the free list
* iteration 1:
* same as iteration 0 except:
* subsequent iterations: same as iteration 0 except:
* scan whole LRU list
* scan LRU list even if buf_pool.try_LRU_scan is not set
* iteration > 1:
* same as iteration 1 but sleep 10ms
@param have_mutex whether buf_pool.mutex is already being held
@return the free control block, in state BUF_BLOCK_MEMORY */
buf_block_t* buf_LRU_get_free_block(bool have_mutex)
{
buf_block_t* block = NULL;
bool freed = false;
ulint n_iterations = 0;
ulint flush_failures = 0;
MONITOR_INC(MONITOR_LRU_GET_FREE_SEARCH);
if (have_mutex) {
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
goto got_mutex;
}
loop:
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
got_mutex:
buf_LRU_check_size_of_non_data_objects();
DBUG_EXECUTE_IF("ib_lru_force_no_free_page",
@@ -718,49 +421,38 @@ got_mutex:
n_iterations = 21;
goto not_found;});
retry:
/* If there is a block in the free list, take it */
block = buf_LRU_get_free_only();
if (block) {
if (buf_block_t* block = buf_LRU_get_free_only()) {
if (!have_mutex) {
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
}
memset(&block->page.zip, 0, sizeof block->page.zip);
return(block);
return block;
}
MONITOR_INC( MONITOR_LRU_GET_FREE_LOOPS );
freed = false;
if (n_iterations || buf_pool.try_LRU_scan) {
/* If no block was in the free list, search from the
end of the LRU list and try to free a block there.
If we are doing for the first time we'll scan only
tail of the LRU list otherwise we scan the whole LRU
list. */
freed = buf_LRU_scan_and_free_block(n_iterations > 0);
if (!freed && n_iterations == 0) {
/* Tell other threads that there is no point
in scanning the LRU list. This flag is set to
TRUE again when we flush a batch from this
buffer pool. */
buf_pool.try_LRU_scan = false;
/* Also tell the page_cleaner thread that
there is work for it to do. */
os_event_set(buf_flush_event);
if (buf_LRU_scan_and_free_block(n_iterations
? ULINT_UNDEFINED : 100)) {
goto retry;
}
/* Tell other threads that there is no point
in scanning the LRU list. */
buf_pool.try_LRU_scan = false;
}
#ifndef DBUG_OFF
not_found:
#endif
mutex_exit(&buf_pool.mutex);
if (freed) {
goto loop;
}
mysql_mutex_unlock(&buf_pool.mutex);
buf_flush_wait_batch_end_acquiring_mutex(true);
if (n_iterations > 20 && !buf_lru_free_blocks_error_printed
&& srv_buf_pool_old_size == srv_buf_pool_size) {
@@ -782,18 +474,8 @@ not_found:
buf_lru_free_blocks_error_printed = true;
}
/* If we have scanned the whole LRU and still are unable to
find a free block then we should sleep here to let the
page_cleaner do an LRU batch for us. */
if (!srv_read_only_mode) {
os_event_set(buf_flush_event);
}
if (n_iterations > 1) {
MONITOR_INC( MONITOR_LRU_GET_FREE_WAITS );
os_thread_sleep(10000);
}
/* No free block was found: try to flush the LRU list.
@@ -804,10 +486,10 @@ not_found:
TODO: A more elegant way would have been to return the freed
up block to the caller here but the code that deals with
removing the block from page_hash and LRU_list is fairly
involved (particularly in case of compressed pages). We
involved (particularly in case of ROW_FORMAT=COMPRESSED pages). We
can do that in a separate patch sometime in future. */
if (!buf_flush_single_page_from_LRU()) {
if (!buf_flush_lists(innodb_lru_flush_size, 0)) {
MONITOR_INC(MONITOR_LRU_SINGLE_FLUSH_FAILURE_COUNT);
++flush_failures;
}
@@ -827,7 +509,7 @@ static void buf_LRU_old_adjust_len()
ulint new_len;
ut_a(buf_pool.LRU_old);
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(buf_pool.LRU_old_ratio >= BUF_LRU_OLD_RATIO_MIN);
ut_ad(buf_pool.LRU_old_ratio <= BUF_LRU_OLD_RATIO_MAX);
compile_time_assert(BUF_LRU_OLD_RATIO_MIN * BUF_LRU_OLD_MIN_LEN
@@ -888,7 +570,7 @@ static void buf_LRU_old_adjust_len()
called when the LRU list grows to BUF_LRU_OLD_MIN_LEN length. */
static void buf_LRU_old_init()
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_a(UT_LIST_GET_LEN(buf_pool.LRU) == BUF_LRU_OLD_MIN_LEN);
/* We first initialize all blocks in the LRU list as old and then use
@@ -917,7 +599,7 @@ static void buf_LRU_old_init()
static void buf_unzip_LRU_remove_block_if_needed(buf_page_t* bpage)
{
ut_ad(bpage->in_file());
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
if (bpage->belongs_to_unzip_LRU()) {
buf_block_t* block = reinterpret_cast<buf_block_t*>(bpage);
@@ -1000,7 +682,7 @@ buf_unzip_LRU_add_block(
ibool old) /*!< in: TRUE if should be put to the end
of the list, else put to the start */
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_a(block->page.belongs_to_unzip_LRU());
ut_ad(!block->in_unzip_LRU_list);
ut_d(block->in_unzip_LRU_list = true);
@@ -1024,7 +706,7 @@ buf_LRU_add_block(
LRU list is very short, the block is added to
the start, regardless of this parameter */
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(!bpage->in_LRU_list);
if (!old || (UT_LIST_GET_LEN(buf_pool.LRU) < BUF_LRU_OLD_MIN_LEN)) {
@@ -1084,7 +766,7 @@ void buf_page_make_young(buf_page_t *bpage)
{
ut_ad(bpage->in_file());
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
if (UNIV_UNLIKELY(bpage->old))
buf_pool.stat.n_pages_made_young++;
@@ -1092,7 +774,7 @@ void buf_page_make_young(buf_page_t *bpage)
buf_LRU_remove_block(bpage);
buf_LRU_add_block(bpage, false);
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
}
/** Try to free a block. If bpage is a descriptor of a compressed-only
@@ -1107,7 +789,7 @@ bool buf_LRU_free_page(buf_page_t *bpage, bool zip)
const page_id_t id(bpage->id());
buf_page_t* b = nullptr;
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(bpage->in_file());
ut_ad(bpage->in_LRU_list);
@@ -1148,7 +830,7 @@ func_exit:
b->set_state(BUF_BLOCK_ZIP_PAGE);
}
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(bpage->in_file());
ut_ad(bpage->in_LRU_list);
@@ -1238,9 +920,7 @@ func_exit:
buf_LRU_add_block(b, b->old);
}
if (b->oldest_modification()) {
buf_flush_relocate_on_flush_list(bpage, b);
}
bpage->zip.data = nullptr;
@@ -1253,14 +933,17 @@ func_exit:
hash_lock->write_unlock();
}
mutex_exit(&buf_pool.mutex);
buf_block_t* block = reinterpret_cast<buf_block_t*>(bpage);
/* Remove possible adaptive hash index on the page.
#ifdef BTR_CUR_HASH_ADAPT
if (block->index) {
mysql_mutex_unlock(&buf_pool.mutex);
/* Remove the adaptive hash index on the page.
The page was declared uninitialized by
buf_LRU_block_remove_hashed(). We need to flag
the contents of the page valid (which it still is) in
order to avoid bogus Valgrind or MSAN warnings.*/
buf_block_t* block = reinterpret_cast<buf_block_t*>(bpage);
MEM_MAKE_DEFINED(block->frame, srv_page_size);
btr_search_drop_page_hash_index(block);
@@ -1271,7 +954,14 @@ func_exit:
b->io_unfix();
}
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
} else
#endif
if (UNIV_LIKELY_NULL(b)) {
ut_ad(b->zip_size());
b->io_unfix();
}
buf_LRU_block_free_hashed_page(block);
return(true);
@@ -1332,6 +1022,16 @@ buf_LRU_block_free_non_file_page(
MEM_NOACCESS(block->frame, srv_page_size);
}
/** Release a memory block to the buffer pool. */
ATTRIBUTE_COLD void buf_pool_t::free_block(buf_block_t *block)
{
ut_ad(this == &buf_pool);
mysql_mutex_lock(&mutex);
buf_LRU_block_free_non_file_page(block);
mysql_mutex_unlock(&mutex);
}
/** Remove bpage from buf_pool.LRU and buf_pool.page_hash.
If bpage->state() == BUF_BLOCK_ZIP_PAGE && !bpage->oldest_modification(),
@@ -1350,7 +1050,7 @@ this case the block is already returned to the buddy allocator. */
static bool buf_LRU_block_remove_hashed(buf_page_t *bpage, const page_id_t id,
page_hash_latch *hash_lock, bool zip)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(hash_lock->is_write_locked());
ut_a(bpage->io_fix() == BUF_IO_NONE);
@@ -1545,7 +1245,7 @@ uint buf_LRU_old_ratio_update(uint old_pct, bool adjust)
}
if (adjust) {
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
if (ratio != buf_pool.LRU_old_ratio) {
buf_pool.LRU_old_ratio = ratio;
@@ -1556,7 +1256,7 @@ uint buf_LRU_old_ratio_update(uint old_pct, bool adjust)
}
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
} else {
buf_pool.LRU_old_ratio = ratio;
}
@@ -1609,7 +1309,7 @@ void buf_LRU_validate()
ulint old_len;
ulint new_len;
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
if (UT_LIST_GET_LEN(buf_pool.LRU) >= BUF_LRU_OLD_MIN_LEN) {
@@ -1687,7 +1387,7 @@ void buf_LRU_validate()
ut_a(block->page.belongs_to_unzip_LRU());
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
}
#endif /* UNIV_DEBUG */
@@ -1695,7 +1395,7 @@ void buf_LRU_validate()
/** Dump the LRU list to stderr. */
void buf_LRU_print()
{
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
for (buf_page_t* bpage = UT_LIST_GET_FIRST(buf_pool.LRU);
bpage != NULL;
@@ -1744,6 +1444,6 @@ void buf_LRU_print()
}
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
}
#endif /* UNIV_DEBUG_PRINT || UNIV_DEBUG */

10
storage/innobase/buf/buf0rea.cc
View File

@@ -121,7 +121,7 @@ static buf_page_t* buf_page_init_for_read(ulint mode, const page_id_t page_id,
const ulint fold= page_id.fold();
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
/* We must acquire hash_lock this early to prevent
a race condition with buf_pool_t::watch_remove() */
@@ -239,11 +239,11 @@ static buf_page_t* buf_page_init_for_read(ulint mode, const page_id_t page_id,
buf_LRU_add_block(bpage, true/* to old blocks */);
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
buf_pool.n_pend_reads++;
goto func_exit_no_mutex;
func_exit:
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
func_exit_no_mutex:
if (mode == BUF_READ_IBUF_PAGES_ONLY)
ibuf_mtr_commit(&mtr);
@@ -286,10 +286,10 @@ buf_read_page_low(
*err = DB_SUCCESS;
if (!page_id.space() && buf_dblwr_page_inside(page_id.page_no())) {
if (buf_dblwr.is_inside(page_id)) {
ib::error() << "Trying to read doublewrite buffer page "
<< page_id;
ut_ad(0);
return(0);
}

33
storage/innobase/fil/fil0crypt.cc
View File

@@ -1080,20 +1080,8 @@ static bool fil_crypt_start_encrypting_space(fil_space_t* space)
mtr.commit();
/* record lsn of update */
lsn_t end_lsn = mtr.commit_lsn();
/* 4 - sync tablespace before publishing crypt data */
bool success = false;
ulint sum_pages = 0;
do {
ulint n_pages = 0;
success = buf_flush_lists(ULINT_MAX, end_lsn, &n_pages);
buf_flush_wait_batch_end(false);
sum_pages += n_pages;
} while (!success);
while (buf_flush_dirty_pages(space->id));
/* 5 - publish crypt data */
mutex_enter(&fil_crypt_threads_mutex);
@@ -1156,7 +1144,6 @@ struct rotate_thread_t {
case SRV_SHUTDOWN_CLEANUP:
case SRV_SHUTDOWN_INITIATED:
return true;
case SRV_SHUTDOWN_FLUSH_PHASE:
case SRV_SHUTDOWN_LAST_PHASE:
break;
}
@@ -1945,8 +1932,7 @@ fil_crypt_rotate_pages(
* real pages, they will be updated anyway when the
* real page is updated
*/
if (space == TRX_SYS_SPACE &&
buf_dblwr_page_inside(state->offset)) {
if (buf_dblwr.is_inside(page_id_t(space, state->offset))) {
continue;
}
@@ -1978,20 +1964,19 @@ fil_crypt_flush_space(
lsn_t end_lsn = crypt_data->rotate_state.end_lsn;
if (end_lsn > 0 && !space->is_stopping()) {
bool success = false;
ulint n_pages = 0;
ulint sum_pages = 0;
const ulonglong start = my_interval_timer();
do {
success = buf_flush_lists(ULINT_MAX, end_lsn, &n_pages);
buf_flush_wait_batch_end(false);
sum_pages += n_pages;
} while (!success && !space->is_stopping());
ulint n_dirty= buf_flush_dirty_pages(state->space->id);
if (!n_dirty) {
break;
}
sum_pages += n_dirty;
} while (!space->is_stopping());
if (sum_pages) {
const ulonglong end = my_interval_timer();
if (sum_pages && end > start) {
state->cnt_waited += sum_pages;
state->sum_waited_us += (end - start) / 1000;

56
storage/innobase/fil/fil0fil.cc
View File

@@ -171,7 +171,6 @@ fil_system_t fil_system;
/** At this age or older a space/page will be rotated */
UNIV_INTERN extern uint srv_fil_crypt_rotate_key_age;
UNIV_INTERN extern ib_mutex_t fil_crypt_threads_mutex;
/** Determine if the space id is a user tablespace id or not.
@param[in] space_id Space ID to check
@@ -713,8 +712,8 @@ fil_space_extend_must_retry(
const ulint page_size = space->physical_size();
/* fil_read_first_page() expects srv_page_size bytes.
fil_node_open_file() expects at least 4 * srv_page_size bytes.*/
/* Datafile::read_first_page() expects srv_page_size bytes.
fil_node_t::read_page0() expects at least 4 * srv_page_size bytes.*/
os_offset_t new_size = std::max(
os_offset_t(size - file_start_page_no) * page_size,
os_offset_t(FIL_IBD_FILE_INITIAL_SIZE << srv_page_size_shift));
@@ -2097,7 +2096,10 @@ void fil_close_tablespace(ulint id)
Thus we can clean the tablespace out of buf_pool
completely and permanently. The flag stop_new_ops also prevents
fil_flush() from being applied to this tablespace. */
buf_LRU_flush_or_remove_pages(id, true);
while (buf_flush_dirty_pages(id));
/* Ensure that all asynchronous IO is completed. */
os_aio_wait_until_no_pending_writes();
fil_flush(id);
/* If the free is successful, the X lock will be released before
the space memory data structure is freed. */
@@ -2186,7 +2188,7 @@ dberr_t fil_delete_tablespace(ulint id, bool if_exists,
::stop_new_ops flag in fil_io(). */
err = DB_SUCCESS;
buf_LRU_flush_or_remove_pages(id, false);
buf_flush_remove_pages(id);
/* If it is a delete then also delete any generated files, otherwise
when we drop the database the remove directory will fail. */
@@ -3683,15 +3685,6 @@ fil_report_invalid_page_access(const page_id_t id, const char *name,
<< ". Byte offset " << byte_offset << ", len " << len;
}
inline void IORequest::set_fil_node(fil_node_t* node)
{
if (!node->space->punch_hole) {
clear_punch_hole();
}
m_fil_node = node;
}
/** Reads or writes data. This operation could be asynchronous (aio).
@param[in,out] type IO context
@@ -3726,9 +3719,8 @@ fil_io(
bool punch_hole)
{
os_offset_t offset;
IORequest req_type(type);
ut_ad(req_type.validate());
ut_ad(type.validate());
ut_ad(len > 0);
ut_ad(byte_offset < srv_page_size);
@@ -3742,7 +3734,7 @@ fil_io(
/* ibuf bitmap pages must be read in the sync AIO mode: */
ut_ad(recv_no_ibuf_operations
|| req_type.is_write()
|| type.is_write()
|| !ibuf_bitmap_page(page_id, zip_size)
|| sync);
@@ -3750,7 +3742,7 @@ fil_io(
if (sync) {
mode = OS_AIO_SYNC;
} else if (req_type.is_read()
} else if (type.is_read()
&& !recv_no_ibuf_operations
&& ibuf_page(page_id, zip_size, NULL)) {
mode = OS_AIO_IBUF;
@@ -3758,11 +3750,11 @@ fil_io(
mode = OS_AIO_NORMAL;
}
if (req_type.is_read()) {
if (type.is_read()) {
srv_stats.data_read.add(len);
} else if (req_type.is_write()) {
} else if (type.is_write()) {
ut_ad(!srv_read_only_mode
|| fsp_is_system_temporary(page_id.space()));
@@ -3776,7 +3768,7 @@ fil_io(
page_id.space());
if (!space
|| (req_type.is_read()
|| (type.is_read()
&& !sync
&& space->is_stopping()
&& !space->is_being_truncated)) {
@@ -3786,7 +3778,7 @@ fil_io(
ib::error()
<< "Trying to do I/O to a tablespace which"
" does not exist. I/O type: "
<< (req_type.is_read() ? "read" : "write")
<< (type.is_read() ? "read" : "write")
<< ", page: " << page_id
<< ", I/O length: " << len << " bytes";
}
@@ -3807,7 +3799,7 @@ fil_io(
fil_report_invalid_page_access(
page_id, space->name, byte_offset, len,
req_type.is_read());
type.is_read());
} else if (fil_is_user_tablespace_id(space->id)
&& node->size == 0) {
@@ -3838,7 +3830,7 @@ fil_io(
<< space->name
<< "' which exists without .ibd data file."
" I/O type: "
<< (req_type.is_read()
<< (type.is_read()
? "read" : "write")
<< ", page: "
<< page_id
@@ -3853,14 +3845,14 @@ fil_io(
/* If we can tolerate the non-existent pages, we
should return with DB_ERROR and let caller decide
what to do. */
node->complete_io(req_type.is_write());
node->complete_io(type.is_write());
mutex_exit(&fil_system.mutex);
return {DB_ERROR, nullptr};
}
fil_report_invalid_page_access(
page_id, space->name, byte_offset, len,
req_type.is_read());
type.is_read());
}
space->acquire_for_io();
@@ -3879,9 +3871,7 @@ fil_io(
const char* name = node->name == NULL ? space->name : node->name;
req_type.set_fil_node(node);
ut_ad(!req_type.is_write()
ut_ad(!type.is_write()
|| !fil_is_user_tablespace_id(page_id.space())
|| offset == page_id.page_no() * zip_size);
@@ -3897,6 +3887,8 @@ fil_io(
err = DB_SUCCESS;
}
} else {
IORequest req_type(type);
req_type.set_fil_node(node);
/* Queue the aio request */
err = os_aio(
req_type,
@@ -3909,10 +3901,10 @@ fil_io(
/* We an try to recover the page from the double write buffer if
the decompression fails or the page is corrupt. */
ut_a(req_type.is_dblwr_recover() || err == DB_SUCCESS);
ut_a(type.is_dblwr_recover() || err == DB_SUCCESS);
if (sync) {
mutex_enter(&fil_system.mutex);
node->complete_io(req_type.is_write());
node->complete_io(type.is_write());
mutex_exit(&fil_system.mutex);
ut_ad(fil_validate_skip());
}
@@ -3960,7 +3952,7 @@ write_completed:
bpage->status= buf_page_t::NORMAL;
dblwr= false;
}
buf_page_write_complete(bpage, data->type, dblwr, false);
buf_page_write_complete(bpage, data->type, dblwr);
goto write_completed;
}

9
storage/innobase/fsp/fsp0file.cc
View File

@@ -296,14 +296,13 @@ Datafile::read_first_page(bool read_only_mode)
m_first_page = static_cast<byte*>(
aligned_malloc(UNIV_PAGE_SIZE_MAX, srv_page_size));
IORequest request;
constexpr IORequest request(IORequest::READ |
IORequest::DISABLE_PARTIAL_IO_WARNINGS);
dberr_t err = DB_ERROR;
size_t page_size = UNIV_PAGE_SIZE_MAX;
/* Don't want unnecessary complaints about partial reads. */
request.disable_partial_io_warnings();
while (page_size >= UNIV_PAGE_SIZE_MIN) {
ulint n_read = 0;
@@ -805,10 +804,8 @@ Datafile::restore_from_doublewrite()
<< physical_size << " bytes into file '"
<< m_filepath << "'";
IORequest request(IORequest::WRITE);
return(os_file_write(
request,
IORequestWrite,
m_filepath, m_handle, page, 0, physical_size)
!= DB_SUCCESS);
}

28
storage/innobase/fsp/fsp0fsp.cc
View File

@@ -553,11 +553,18 @@ void fsp_header_init(fil_space_t* space, ulint size, mtr_t* mtr)
const page_id_t page_id(space->id, 0);
const ulint zip_size = space->zip_size();
buf_block_t *free_block = buf_LRU_get_free_block(false);
mtr_x_lock_space(space, mtr);
buf_block_t* block = buf_page_create(space, 0, zip_size, mtr);
buf_block_t* block = buf_page_create(space, 0, zip_size, mtr,
free_block);
buf_block_dbg_add_level(block, SYNC_FSP_PAGE);
if (UNIV_UNLIKELY(block != free_block)) {
buf_pool.free_block(free_block);
}
space->size_in_header = size;
space->free_len = 0;
space->free_limit = 0;
@@ -874,11 +881,14 @@ fsp_fill_free_list(
pages should be ignored. */
if (i > 0) {
buf_block_t *f= buf_LRU_get_free_block(false);
block= buf_page_create(
space, static_cast<uint32_t>(i),
zip_size, mtr);
zip_size, mtr, f);
buf_block_dbg_add_level(block, SYNC_FSP_PAGE);
if (UNIV_UNLIKELY(block != f)) {
buf_pool.free_block(f);
}
fsp_init_file_page(space, block, mtr);
mtr->write<2>(*block,
FIL_PAGE_TYPE + block->frame,
@@ -886,13 +896,16 @@ fsp_fill_free_list(
}
if (space->purpose != FIL_TYPE_TEMPORARY) {
buf_block_t *f= buf_LRU_get_free_block(false);
block = buf_page_create(
space,
static_cast<uint32_t>(
i + FSP_IBUF_BITMAP_OFFSET),
zip_size, mtr);
zip_size, mtr, f);
buf_block_dbg_add_level(block, SYNC_FSP_PAGE);
if (UNIV_UNLIKELY(block != f)) {
buf_pool.free_block(f);
}
fsp_init_file_page(space, block, mtr);
mtr->write<2>(*block,
block->frame + FIL_PAGE_TYPE,
@@ -1042,8 +1055,11 @@ static
buf_block_t*
fsp_page_create(fil_space_t *space, page_no_t offset, mtr_t *mtr)
{
buf_block_t *free_block= buf_LRU_get_free_block(false);
buf_block_t *block= buf_page_create(space, static_cast<uint32_t>(offset),
space->zip_size(), mtr);
space->zip_size(), mtr, free_block);
if (UNIV_UNLIKELY(block != free_block))
buf_pool.free_block(free_block);
fsp_init_file_page(space, block, mtr);
return block;
}

2
storage/innobase/fsp/fsp0sysspace.cc
View File

@@ -559,7 +559,7 @@ SysTablespace::read_lsn_and_check_flags(lsn_t* flushed_lsn)
ut_a(it->order() == 0);
if (srv_operation == SRV_OPERATION_NORMAL) {
buf_dblwr_init_or_load_pages(it->handle(), it->filepath());
buf_dblwr.init_or_load_pages(it->handle(), it->filepath());
}
/* Check the contents of the first page of the

40
storage/innobase/handler/ha_innodb.cc
View File

@@ -878,8 +878,7 @@ static SHOW_VAR innodb_status_variables[]= {
&export_vars.innodb_buffer_pool_pages_dirty, SHOW_SIZE_T},
{"buffer_pool_bytes_dirty",
&export_vars.innodb_buffer_pool_bytes_dirty, SHOW_SIZE_T},
{"buffer_pool_pages_flushed",
&export_vars.innodb_buffer_pool_pages_flushed, SHOW_SIZE_T},
{"buffer_pool_pages_flushed", &buf_flush_page_count, SHOW_SIZE_T},
{"buffer_pool_pages_free",
&export_vars.innodb_buffer_pool_pages_free, SHOW_SIZE_T},
#ifdef UNIV_DEBUG
@@ -17505,6 +17504,7 @@ func_exit:
block->frame[FIL_PAGE_SPACE_ID]);
}
mtr.commit();
log_write_up_to(mtr.commit_lsn(), true);
goto func_exit;
}
#endif // UNIV_DEBUG
@@ -17966,7 +17966,7 @@ static bool innodb_buffer_pool_evict_uncompressed()
{
bool all_evicted = true;
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
for (buf_block_t* block = UT_LIST_GET_LAST(buf_pool.unzip_LRU);
block != NULL; ) {
@@ -17986,7 +17986,7 @@ static bool innodb_buffer_pool_evict_uncompressed()
}
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
return(all_evicted);
}
@@ -19063,13 +19063,13 @@ static MYSQL_SYSVAR_ULONG(page_cleaners, deprecated::innodb_page_cleaners,
static MYSQL_SYSVAR_DOUBLE(max_dirty_pages_pct, srv_max_buf_pool_modified_pct,
PLUGIN_VAR_RQCMDARG,
"Percentage of dirty pages allowed in bufferpool.",
NULL, innodb_max_dirty_pages_pct_update, 75.0, 0, 99.999, 0);
NULL, innodb_max_dirty_pages_pct_update, 90.0, 0, 99.999, 0);
static MYSQL_SYSVAR_DOUBLE(max_dirty_pages_pct_lwm,
srv_max_dirty_pages_pct_lwm,
PLUGIN_VAR_RQCMDARG,
"Percentage of dirty pages at which flushing kicks in.",
NULL, innodb_max_dirty_pages_pct_lwm_update, 0, 0, 99.999, 0);
NULL, innodb_max_dirty_pages_pct_lwm_update, 75.0, 0, 99.999, 0);
static MYSQL_SYSVAR_DOUBLE(adaptive_flushing_lwm,
srv_adaptive_flushing_lwm,
@@ -19234,13 +19234,6 @@ static MYSQL_SYSVAR_ULONG(buffer_pool_chunk_size, srv_buf_pool_chunk_unit,
NULL, NULL,
128 * 1024 * 1024, 1024 * 1024, LONG_MAX, 1024 * 1024);
#if defined UNIV_DEBUG || defined UNIV_PERF_DEBUG
static MYSQL_SYSVAR_ULONG(doublewrite_batch_size, srv_doublewrite_batch_size,
PLUGIN_VAR_OPCMDARG | PLUGIN_VAR_READONLY,
"Number of pages reserved in doublewrite buffer for batch flushing",
NULL, NULL, 120, 1, 127, 0);
#endif /* defined UNIV_DEBUG || defined UNIV_PERF_DEBUG */
static MYSQL_SYSVAR_ENUM(lock_schedule_algorithm, innodb_lock_schedule_algorithm,
PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY,
"The algorithm Innodb uses for deciding which locks to grant next when"
@@ -19364,7 +19357,12 @@ static MYSQL_SYSVAR_UINT(defragment_frequency, srv_defragment_frequency,
static MYSQL_SYSVAR_ULONG(lru_scan_depth, srv_LRU_scan_depth,
PLUGIN_VAR_RQCMDARG,
"How deep to scan LRU to keep it clean",
NULL, NULL, 1024, 100, ~0UL, 0);
NULL, NULL, 1536, 100, ~0UL, 0);
static MYSQL_SYSVAR_SIZE_T(lru_flush_size, innodb_lru_flush_size,
PLUGIN_VAR_RQCMDARG,
"How many pages to flush on LRU eviction",
NULL, NULL, 32, 1, SIZE_T_MAX, 0);
static MYSQL_SYSVAR_ULONG(flush_neighbors, srv_flush_neighbors,
PLUGIN_VAR_OPCMDARG,
@@ -19994,6 +19992,7 @@ static struct st_mysql_sys_var* innobase_system_variables[]= {
MYSQL_SYSVAR(defragment_fill_factor_n_recs),
MYSQL_SYSVAR(defragment_frequency),
MYSQL_SYSVAR(lru_scan_depth),
MYSQL_SYSVAR(lru_flush_size),
MYSQL_SYSVAR(flush_neighbors),
MYSQL_SYSVAR(checksum_algorithm),
MYSQL_SYSVAR(log_checksums),
@@ -20115,9 +20114,6 @@ static struct st_mysql_sys_var* innobase_system_variables[]= {
MYSQL_SYSVAR(buf_flush_list_now),
MYSQL_SYSVAR(merge_threshold_set_all_debug),
#endif /* UNIV_DEBUG */
#if defined UNIV_DEBUG || defined UNIV_PERF_DEBUG
MYSQL_SYSVAR(doublewrite_batch_size),
#endif /* defined UNIV_DEBUG || defined UNIV_PERF_DEBUG */
MYSQL_SYSVAR(status_output),
MYSQL_SYSVAR(status_output_locks),
MYSQL_SYSVAR(print_all_deadlocks),
@@ -21191,10 +21187,10 @@ innodb_buffer_pool_size_validate(
#endif /* UNIV_DEBUG */
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
if (srv_buf_pool_old_size != srv_buf_pool_size) {
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
my_printf_error(ER_WRONG_ARGUMENTS,
"Another buffer pool resize is already in progress.", MYF(0));
return(1);
@@ -21205,13 +21201,13 @@ innodb_buffer_pool_size_validate(
*static_cast<ulonglong*>(save) = requested_buf_pool_size;
if (srv_buf_pool_size == ulint(intbuf)) {
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
/* nothing to do */
return(0);
}
if (srv_buf_pool_size == requested_buf_pool_size) {
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN,
ER_WRONG_ARGUMENTS,
"innodb_buffer_pool_size must be at least"
@@ -21222,7 +21218,7 @@ innodb_buffer_pool_size_validate(
}
srv_buf_pool_size = requested_buf_pool_size;
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
if (intbuf != static_cast<longlong>(requested_buf_pool_size)) {
char buf[64];

12
storage/innobase/handler/i_s.cc
View File

@@ -1635,7 +1635,7 @@ i_s_cmpmem_fill_low(
buf_buddy_stat_t buddy_stat_local[BUF_BUDDY_SIZES_MAX + 1];
/* Save buddy stats for buffer pool in local variables. */
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
for (uint x = 0; x <= BUF_BUDDY_SIZES; x++) {
zip_free_len_local[x] = (x < BUF_BUDDY_SIZES) ?
@@ -1650,7 +1650,7 @@ i_s_cmpmem_fill_low(
}
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
for (uint x = 0; x <= BUF_BUDDY_SIZES; x++) {
buf_buddy_stat_t* buddy_stat = &buddy_stat_local[x];
@@ -4270,7 +4270,7 @@ static int i_s_innodb_buffer_page_fill(THD *thd, TABLE_LIST *tables, Item *)
buffer pool info printout, we are not required to
preserve the overall consistency, so we can
release mutex periodically */
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
/* GO through each block in the chunk */
for (n_blocks = num_to_process; n_blocks--; block++) {
@@ -4281,7 +4281,7 @@ static int i_s_innodb_buffer_page_fill(THD *thd, TABLE_LIST *tables, Item *)
num_page++;
}
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
/* Fill in information schema table with information
just collected from the buffer chunk scan */
@@ -4604,7 +4604,7 @@ static int i_s_innodb_fill_buffer_lru(THD *thd, TABLE_LIST *tables, Item *)
/* Aquire the mutex before allocating info_buffer, since
UT_LIST_GET_LEN(buf_pool.LRU) could change */
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
lru_len = UT_LIST_GET_LEN(buf_pool.LRU);
@@ -4636,7 +4636,7 @@ static int i_s_innodb_fill_buffer_lru(THD *thd, TABLE_LIST *tables, Item *)
ut_ad(lru_pos == UT_LIST_GET_LEN(buf_pool.LRU));
exit:
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
if (info_buffer) {
status = i_s_innodb_buf_page_lru_fill(

149
storage/innobase/include/buf0buf.h
View File

@@ -104,10 +104,6 @@ struct buf_pool_info_t
ulint n_pend_reads; /*!< buf_pool.n_pend_reads, pages
pending read */
ulint n_pending_flush_lru; /*!< Pages pending flush in LRU */
ulint n_pending_flush_single_page;/*!< Pages pending to be
flushed as part of single page
flushes issued by various user
threads */
ulint n_pending_flush_list; /*!< Pages pending flush in FLUSH
LIST */
ulint n_pages_made_young; /*!< number of pages made young */
@@ -339,10 +335,11 @@ FILE_PAGE (the other is buf_page_get_gen).
@param[in] offset offset of the tablespace
@param[in] zip_size ROW_FORMAT=COMPRESSED page size, or 0
@param[in,out] mtr mini-transaction
@param[in,out] free_block pre-allocated buffer block
@return pointer to the block, page bufferfixed */
buf_block_t*
buf_page_create(fil_space_t *space, uint32_t offset,
ulint zip_size, mtr_t *mtr);
ulint zip_size, mtr_t *mtr, buf_block_t *free_block);
/********************************************************************//**
Releases a compressed-only page acquired with buf_page_get_zip(). */
@@ -1038,7 +1035,7 @@ struct buf_block_t{
is of size srv_page_size, and
aligned to an address divisible by
srv_page_size */
BPageLock lock; /*!< read-write lock of the buffer
rw_lock_t lock; /*!< read-write lock of the buffer
frame */
#ifdef UNIV_DEBUG
/** whether page.list is in buf_pool.withdraw
@@ -1214,13 +1211,13 @@ public:
virtual ~HazardPointer() {}
/** @return current value */
buf_page_t *get() const { ut_ad(mutex_own(m_mutex)); return m_hp; }
buf_page_t *get() const { mysql_mutex_assert_owner(m_mutex); return m_hp; }
/** Set current value
@param bpage buffer block to be set as hp */
void set(buf_page_t *bpage)
{
ut_ad(mutex_own(m_mutex));
mysql_mutex_assert_owner(m_mutex);
ut_ad(!bpage || bpage->in_file());
m_hp= bpage;
}
@@ -1229,7 +1226,7 @@ public:
@param bpage buffer block to be compared
@return true if it is hp */
bool is_hp(const buf_page_t *bpage) const
{ ut_ad(mutex_own(m_mutex)); return bpage == m_hp; }
{ mysql_mutex_assert_owner(m_mutex); return bpage == m_hp; }
/** Adjust the value of hp. This happens when some
other thread working on the same list attempts to
@@ -1238,7 +1235,7 @@ public:
#ifdef UNIV_DEBUG
/** mutex that protects access to the m_hp. */
const ib_mutex_t *m_mutex= nullptr;
const mysql_mutex_t *m_mutex= nullptr;
#endif /* UNIV_DEBUG */
protected:
@@ -1494,7 +1491,10 @@ public:
bool will_be_withdrawn(const byte *ptr) const
{
ut_ad(curr_size < old_size);
ut_ad(!resizing.load(std::memory_order_relaxed) || mutex_own(&mutex));
#ifdef SAFE_MUTEX
if (resizing.load(std::memory_order_relaxed))
mysql_mutex_assert_owner(&mutex);
#endif /* SAFE_MUTEX */
for (const chunk_t *chunk= chunks + n_chunks_new,
* const echunk= chunks + n_chunks;
@@ -1511,7 +1511,10 @@ public:
bool will_be_withdrawn(const buf_page_t &bpage) const
{
ut_ad(curr_size < old_size);
ut_ad(!resizing.load(std::memory_order_relaxed) || mutex_own(&mutex));
#ifdef SAFE_MUTEX
if (resizing.load(std::memory_order_relaxed))
mysql_mutex_assert_owner(&mutex);
#endif /* SAFE_MUTEX */
for (const chunk_t *chunk= chunks + n_chunks_new,
* const echunk= chunks + n_chunks;
@@ -1524,7 +1527,10 @@ public:
/** Release and evict a corrupted page.
@param bpage page that was being read */
void corrupted_evict(buf_page_t *bpage);
ATTRIBUTE_COLD void corrupted_evict(buf_page_t *bpage);
/** Release a memory block to the buffer pool. */
ATTRIBUTE_COLD void free_block(buf_block_t *block);
#ifdef UNIV_DEBUG
/** Find a block that points to a ROW_FORMAT=COMPRESSED page
@@ -1533,7 +1539,7 @@ public:
@retval nullptr if not found */
const buf_block_t *contains_zip(const void *data) const
{
ut_ad(mutex_own(&mutex));
mysql_mutex_assert_owner(&mutex);
for (const chunk_t *chunk= chunks, * const end= chunks + n_chunks;
chunk != end; chunk++)
if (const buf_block_t *block= chunk->contains_zip(data))
@@ -1556,8 +1562,8 @@ public:
inline buf_block_t *block_from_ahi(const byte *ptr) const;
#endif /* BTR_CUR_HASH_ADAPT */
bool is_block_lock(const BPageLock *l) const
{ return is_block_field(reinterpret_cast<const void*>(l)); }
bool is_block_lock(const rw_lock_t *l) const
{ return is_block_field(static_cast<const void*>(l)); }
/**
@return the smallest oldest_modification lsn for any page
@@ -1588,7 +1594,10 @@ public:
buf_page_t *page_hash_get_low(const page_id_t id, const ulint fold)
{
ut_ad(id.fold() == fold);
ut_ad(mutex_own(&mutex) || page_hash.lock_get(fold)->is_locked());
#ifdef SAFE_MUTEX
DBUG_ASSERT(mysql_mutex_is_owner(&mutex) ||
page_hash.lock_get(fold)->is_locked());
#endif /* SAFE_MUTEX */
buf_page_t *bpage;
/* Look for the page in the hash table */
HASH_SEARCH(hash, &page_hash, fold, buf_page_t*, bpage,
@@ -1655,7 +1664,10 @@ public:
@return whether bpage a sentinel for a buffer pool watch */
bool watch_is_sentinel(const buf_page_t &bpage)
{
ut_ad(mutex_own(&mutex) || hash_lock_get(bpage.id())->is_locked());
#ifdef SAFE_MUTEX
DBUG_ASSERT(mysql_mutex_is_owner(&mutex) ||
hash_lock_get(bpage.id())->is_locked());
#endif /* SAFE_MUTEX */
ut_ad(bpage.in_file());
if (&bpage < &watch[0] || &bpage >= &watch[UT_ARR_SIZE(watch)])
@@ -1712,7 +1724,7 @@ public:
HASH_DELETE(buf_page_t, hash, &page_hash, fold, watch);
hash_lock->write_unlock();
// Now that the watch is detached from page_hash, release it to watch[].
mutex_enter(&mutex);
mysql_mutex_lock(&mutex);
/* It is possible that watch_remove() already removed the watch. */
if (watch->id_ == id)
{
@@ -1720,7 +1732,7 @@ public:
ut_ad(watch->state() == BUF_BLOCK_ZIP_PAGE);
watch->set_state(BUF_BLOCK_NOT_USED);
}
mutex_exit(&mutex);
mysql_mutex_unlock(&mutex);
}
else
hash_lock->write_unlock();
@@ -1753,14 +1765,13 @@ public:
@return the predecessor in the LRU list */
buf_page_t *LRU_remove(buf_page_t *bpage)
{
ut_ad(mutex_own(&mutex));
mysql_mutex_assert_owner(&mutex);
ut_ad(bpage->in_LRU_list);
ut_ad(bpage->in_page_hash);
ut_ad(!bpage->in_zip_hash);
ut_ad(bpage->in_file());
lru_hp.adjust(bpage);
lru_scan_itr.adjust(bpage);
single_scan_itr.adjust(bpage);
ut_d(bpage->in_LRU_list= false);
buf_page_t *prev= UT_LIST_GET_PREV(LRU, bpage);
UT_LIST_REMOVE(LRU, bpage);
@@ -1770,9 +1781,19 @@ public:
/** Number of pages to read ahead */
static constexpr uint32_t READ_AHEAD_PAGES= 64;
/** Buffer pool mutex */
mysql_mutex_t mutex;
/** Number of pending LRU flush. */
Atomic_counter<ulint> n_flush_LRU;
/** broadcast when n_flush_LRU reaches 0; protected by mutex */
mysql_cond_t done_flush_LRU;
/** Number of pending flush_list flush. */
Atomic_counter<ulint> n_flush_list;
/** broadcast when n_flush_list reaches 0; protected by mutex */
mysql_cond_t done_flush_list;
/** @name General fields */
/* @{ */
BufPoolMutex mutex; /*!< Buffer pool mutex */
ulint curr_pool_size; /*!< Current pool size in bytes */
ulint LRU_old_ratio; /*!< Reserve this much of the buffer
pool for "old" blocks */
@@ -1904,35 +1925,22 @@ public:
/* @} */
/** @name Page flushing algorithm fields */
/* @{ */
FlushListMutex flush_list_mutex;/*!< mutex protecting the
flush list access. This mutex
protects flush_list
and bpage::list pointers when
the bpage is on flush_list. It
also protects writes to
bpage::oldest_modification and
flush_list_hp */
FlushHp flush_hp;/*!< "hazard pointer"
used during scan of flush_list
while doing flush list batch.
Protected by flush_list_mutex */
/** mutex protecting flush_list, buf_page_t::set_oldest_modification()
and buf_page_t::list pointers when !oldest_modification() */
mysql_mutex_t flush_list_mutex;
/** "hazard pointer" for flush_list scans; protected by flush_list_mutex */
FlushHp flush_hp;
/** modified blocks (a subset of LRU) */
UT_LIST_BASE_NODE_T(buf_page_t) flush_list;
/*!< base node of the modified block
list */
/** set if a flush of the type is being initialized */
Atomic_relaxed<bool> init_flush[3];
/** Number of pending writes of a flush type.
The sum of these is approximately the sum of BUF_IO_WRITE blocks. */
Atomic_counter<ulint> n_flush[3];
os_event_t no_flush[3];
/*!< this is in the set state
when there is no flush batch
of the given type running;
os_event_set() and os_event_reset()
are protected by buf_pool_t::mutex */
/** signalled to wake up the page_cleaner; protected by flush_list_mutex */
mysql_cond_t do_flush_list;
// n_flush_LRU + n_flush_list is approximately COUNT(io_fix()==BUF_IO_WRITE)
// in flush_list
unsigned freed_page_clock;/*!< a sequence number used
to count the number of buffer
blocks removed from the end of
@@ -1978,10 +1986,6 @@ public:
replacable victim. Protected by buf_pool_t::mutex. */
LRUItr lru_scan_itr;
/** Iterator used to scan the LRU list when searching for
single page flushing victim. Protected by buf_pool_t::mutex. */
LRUItr single_scan_itr;
UT_LIST_BASE_NODE_T(buf_page_t) LRU;
/*!< base node of the LRU list */
@@ -2020,16 +2024,12 @@ public:
/** @return whether any I/O is pending */
bool any_io_pending() const
{
return n_pend_reads ||
n_flush[IORequest::LRU] || n_flush[IORequest::FLUSH_LIST] ||
n_flush[IORequest::SINGLE_PAGE];
return n_pend_reads || n_flush_LRU || n_flush_list;
}
/** @return total amount of pending I/O */
ulint io_pending() const
{
return n_pend_reads +
n_flush[IORequest::LRU] + n_flush[IORequest::FLUSH_LIST] +
n_flush[IORequest::SINGLE_PAGE];
return n_pend_reads + n_flush_LRU + n_flush_list;
}
private:
/** Temporary memory for page_compressed and encrypted I/O */
@@ -2086,7 +2086,7 @@ extern buf_pool_t buf_pool;
inline void page_hash_latch::read_lock()
{
ut_ad(!mutex_own(&buf_pool.mutex));
mysql_mutex_assert_not_owner(&buf_pool.mutex);
if (!read_trylock())
read_lock_wait();
}
@@ -2099,19 +2099,19 @@ inline void page_hash_latch::write_lock()
inline void buf_page_t::add_buf_fix_count(uint32_t count)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
buf_fix_count_+= count;
}
inline void buf_page_t::set_buf_fix_count(uint32_t count)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
buf_fix_count_= count;
}
inline void buf_page_t::set_state(buf_page_state state)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
#ifdef UNIV_DEBUG
switch (state) {
case BUF_BLOCK_REMOVE_HASH:
@@ -2140,7 +2140,7 @@ inline void buf_page_t::set_state(buf_page_state state)
inline void buf_page_t::set_io_fix(buf_io_fix io_fix)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
io_fix_= io_fix;
}
@@ -2166,7 +2166,7 @@ inline void buf_page_t::set_corrupt_id()
/** Set oldest_modification when adding to buf_pool.flush_list */
inline void buf_page_t::set_oldest_modification(lsn_t lsn)
{
ut_ad(mutex_own(&buf_pool.flush_list_mutex));
mysql_mutex_assert_owner(&buf_pool.flush_list_mutex);
ut_ad(!oldest_modification());
oldest_modification_= lsn;
}
@@ -2174,7 +2174,7 @@ inline void buf_page_t::set_oldest_modification(lsn_t lsn)
/** Clear oldest_modification when removing from buf_pool.flush_list */
inline void buf_page_t::clear_oldest_modification()
{
ut_ad(mutex_own(&buf_pool.flush_list_mutex));
mysql_mutex_assert_owner(&buf_pool.flush_list_mutex);
ut_d(const auto state= state_);
ut_ad(state == BUF_BLOCK_FILE_PAGE || state == BUF_BLOCK_ZIP_PAGE ||
state == BUF_BLOCK_REMOVE_HASH);
@@ -2185,7 +2185,7 @@ inline void buf_page_t::clear_oldest_modification()
/** @return whether the block is modified and ready for flushing */
inline bool buf_page_t::ready_for_flush() const
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(in_LRU_list);
ut_a(in_file());
return oldest_modification() && io_fix_ == BUF_IO_NONE;
@@ -2195,7 +2195,7 @@ inline bool buf_page_t::ready_for_flush() const
The block can be dirty, but it must not be I/O-fixed or bufferfixed. */
inline bool buf_page_t::can_relocate() const
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(in_file());
ut_ad(in_LRU_list);
return io_fix_ == BUF_IO_NONE && !buf_fix_count_;
@@ -2204,7 +2204,7 @@ inline bool buf_page_t::can_relocate() const
/** @return whether the block has been flagged old in buf_pool.LRU */
inline bool buf_page_t::is_old() const
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(in_file());
ut_ad(in_LRU_list);
return old;
@@ -2213,7 +2213,7 @@ inline bool buf_page_t::is_old() const
/** Set whether a block is old in buf_pool.LRU */
inline void buf_page_t::set_old(bool old)
{
ut_ad(mutex_own(&buf_pool.mutex));
mysql_mutex_assert_owner(&buf_pool.mutex);
ut_ad(in_LRU_list);
#ifdef UNIV_LRU_DEBUG
@@ -2241,12 +2241,12 @@ inline void buf_page_t::set_old(bool old)
#ifdef UNIV_DEBUG
/** Forbid the release of the buffer pool mutex. */
# define buf_pool_mutex_exit_forbid() do { \
ut_ad(mutex_own(&buf_pool.mutex)); \
mysql_mutex_assert_owner(&buf_pool.mutex); \
buf_pool.mutex_exit_forbidden++; \
} while (0)
/** Allow the release of the buffer pool mutex. */
# define buf_pool_mutex_exit_allow() do { \
ut_ad(mutex_own(&buf_pool.mutex)); \
mysql_mutex_assert_owner(&buf_pool.mutex); \
ut_ad(buf_pool.mutex_exit_forbidden--); \
} while (0)
#else
@@ -2265,8 +2265,7 @@ MEMORY: is not in free list, LRU list, or flush list, nor page
hash table
FILE_PAGE: space and offset are defined, is in page hash table
if io_fix == BUF_IO_WRITE,
pool: no_flush[flush_type] is in reset state,
pool: n_flush[flush_type] > 0
buf_pool.n_flush_LRU > 0 || buf_pool.n_flush_list > 0
(1) if buf_fix_count == 0, then
is in LRU list, not in free list
@@ -2303,7 +2302,7 @@ of the LRU list.
@return buf_page_t from where to start scan. */
inline buf_page_t *LRUItr::start()
{
ut_ad(mutex_own(m_mutex));
mysql_mutex_assert_owner(m_mutex);
if (!m_hp || m_hp->old)
m_hp= UT_LIST_GET_LAST(buf_pool.LRU);

14
storage/innobase/include/buf0buf.ic
View File

@@ -198,9 +198,9 @@ buf_block_free(
/*===========*/
buf_block_t* block) /*!< in, own: block to be freed */
{
mutex_enter(&buf_pool.mutex);
mysql_mutex_lock(&buf_pool.mutex);
buf_LRU_block_free_non_file_page(block);
mutex_exit(&buf_pool.mutex);
mysql_mutex_unlock(&buf_pool.mutex);
}
/********************************************************************//**
@@ -213,12 +213,20 @@ buf_block_modify_clock_inc(
/*=======================*/
buf_block_t* block) /*!< in: block */
{
#ifdef SAFE_MUTEX
/* No latch is acquired for the shared temporary tablespace. */
ut_ad(fsp_is_system_temporary(block->page.id().space())
|| (mutex_own(&buf_pool.mutex)
|| (mysql_mutex_is_owner(&buf_pool.mutex)
&& !block->page.buf_fix_count())
|| rw_lock_own_flagged(&block->lock,
RW_LOCK_FLAG_X | RW_LOCK_FLAG_SX));
#else /* SAFE_MUTEX */
/* No latch is acquired for the shared temporary tablespace. */
ut_ad(fsp_is_system_temporary(block->page.id().space())
|| !block->page.buf_fix_count()
|| rw_lock_own_flagged(&block->lock,
RW_LOCK_FLAG_X | RW_LOCK_FLAG_SX));
#endif /* SAFE_MUTEX */
assert_block_ahi_valid(block);
block->modify_clock++;

183
storage/innobase/include/buf0dblwr.h
View File

@@ -24,101 +24,38 @@ Doublewrite buffer module
Created 2011/12/19 Inaam Rana
*******************************************************/
#ifndef buf0dblwr_h
#define buf0dblwr_h
#pragma once
#include "ut0byte.h"
#include "log0log.h"
#include "os0file.h"
#include "buf0types.h"
#include "log0recv.h"
/** Doublewrite system */
extern buf_dblwr_t* buf_dblwr;
/** Create the doublewrite buffer if the doublewrite buffer header
is not present in the TRX_SYS page.
@return whether the operation succeeded
@retval true if the doublewrite buffer exists or was created
@retval false if the creation failed (too small first data file) */
MY_ATTRIBUTE((warn_unused_result))
bool
buf_dblwr_create();
/**
At database startup initializes the doublewrite buffer memory structure if
we already have a doublewrite buffer created in the data files. If we are
upgrading to an InnoDB version which supports multiple tablespaces, then this
function performs the necessary update operations. If we are in a crash
recovery, this function loads the pages from double write buffer into memory.
@param[in] file File handle
@param[in] path Path name of file
@return DB_SUCCESS or error code */
dberr_t
buf_dblwr_init_or_load_pages(
pfs_os_file_t file,
const char* path);
/** Process and remove the double write buffer pages for all tablespaces. */
void
buf_dblwr_process();
/****************************************************************//**
frees doublewrite buffer. */
void
buf_dblwr_free();
/** Update the doublewrite buffer on write completion. */
void buf_dblwr_update(const buf_page_t &bpage, bool single_page);
/****************************************************************//**
Determines if a page number is located inside the doublewrite buffer.
@return TRUE if the location is inside the two blocks of the
doublewrite buffer */
ibool
buf_dblwr_page_inside(
/*==================*/
ulint page_no); /*!< in: page number */
/********************************************************************//**
Flushes possible buffered writes from the doublewrite memory buffer to disk.
It is very important to call this function after a batch of writes
has been posted, and also when we may have to wait for a page latch!
Otherwise a deadlock of threads can occur. */
void
buf_dblwr_flush_buffered_writes();
/** Doublewrite control struct */
struct buf_dblwr_t{
ib_mutex_t mutex; /*!< mutex protecting the first_free
field and write_buf */
ulint block1; /*!< the page number of the first
doublewrite block (64 pages) */
ulint block2; /*!< page number of the second block */
ulint first_free;/*!< first free position in write_buf
measured in units of srv_page_size */
ulint b_reserved;/*!< number of slots currently reserved
for batch flush. */
os_event_t b_event;/*!< event where threads wait for a
batch flush to end;
os_event_set() and os_event_reset()
are protected by buf_dblwr_t::mutex */
ulint s_reserved;/*!< number of slots currently
reserved for single page flushes. */
os_event_t s_event;/*!< event where threads wait for a
single page flush slot. Protected by mutex. */
bool batch_running;/*!< set to TRUE if currently a batch
is being written from the doublewrite
buffer. */
byte* write_buf;/*!< write buffer used in writing to the
doublewrite buffer, aligned to an
address divisible by srv_page_size
(which is required by Windows aio) */
class buf_dblwr_t
{
/** the page number of the first doublewrite block (block_size() pages) */
page_id_t block1= page_id_t(0, 0);
/** the page number of the second doublewrite block (block_size() pages) */
page_id_t block2= page_id_t(0, 0);
/** mutex protecting the data members below */
mysql_mutex_t mutex;
/** condition variable for !batch_running */
mysql_cond_t cond;
/** whether a batch is being written from the doublewrite buffer */
bool batch_running;
/** first free position in write_buf measured in units of srv_page_size */
ulint first_free;
/** number of slots reserved for the current write batch */
ulint reserved;
/** the doublewrite buffer, aligned to srv_page_size */
byte *write_buf;
struct element
{
/** block descriptor */
buf_page_t *bpage;
/** flush type */
IORequest::flush_t flush;
/** true=buf_pool.flush_list, false=buf_pool.LRU */
bool lru;
/** payload size in bytes */
size_t size;
};
@@ -126,23 +63,67 @@ struct buf_dblwr_t{
/** buffer blocks to be written via write_buf */
element *buf_block_arr;
/** Initialize the doublewrite buffer data structure.
@param header doublewrite page header in the TRX_SYS page */
inline void init(const byte *header);
/** Flush possible buffered writes to persistent storage. */
bool flush_buffered_writes(const ulint size);
public:
/** Create or restore the doublewrite buffer in the TRX_SYS page.
@return whether the operation succeeded */
bool create();
/** Free the doublewrite buffer. */
void close();
/** Initialize the doublewrite buffer memory structure on recovery.
If we are upgrading from a version before MySQL 4.1, then this
function performs the necessary update operations to support
innodb_file_per_table. If we are in a crash recovery, this function
loads the pages from double write buffer into memory.
@param file File handle
@param path Path name of file
@return DB_SUCCESS or error code */
dberr_t init_or_load_pages(pfs_os_file_t file, const char *path);
/** Process and remove the double write buffer pages for all tablespaces. */
void recover();
/** Update the doublewrite buffer on write completion. */
void write_completed();
/** Flush possible buffered writes to persistent storage.
It is very important to call this function after a batch of writes has been
posted, and also when we may have to wait for a page latch!
Otherwise a deadlock of threads can occur. */
void flush_buffered_writes();
/** Size of the doublewrite block in pages */
uint32_t block_size() const { return FSP_EXTENT_SIZE; }
/** Schedule a page write. If the doublewrite memory buffer is full,
buf_dblwr_flush_buffered_writes() will be invoked to make space.
flush_buffered_writes() will be invoked to make space.
@param bpage buffer pool page to be written
@param flush type of flush
@param lru true=buf_pool.LRU; false=buf_pool.flush_list
@param size payload size in bytes */
void add_to_batch(buf_page_t *bpage, IORequest::flush_t flush, size_t size);
/** Write a page to the doublewrite buffer on disk, sync it, then write
the page to the datafile and sync the datafile. This function is used
for single page flushes. If all the buffers allocated for single page
flushes in the doublewrite buffer are in use we wait here for one to
become free. We are guaranteed that a slot will become free because any
thread that is using a slot must also release the slot before leaving
this function.
@param bpage buffer pool page to be written
@param sync whether synchronous operation is requested
@param size payload size in bytes */
void write_single_page(buf_page_t *bpage, bool sync, size_t size);
void add_to_batch(buf_page_t *bpage, bool lru, size_t size);
/** Determine whether the doublewrite buffer is initialized */
bool is_initialised() const
{ return UNIV_LIKELY(block1 != page_id_t(0, 0)); }
/** @return whether a page identifier is part of the doublewrite buffer */
bool is_inside(const page_id_t id) const
{
if (!is_initialised())
return false;
ut_ad(block1 < block2);
if (id < block1)
return false;
const uint32_t size= block_size();
return id < block1 + size || (id >= block2 && id < block2 + size);
}
};
#endif
/** The doublewrite buffer */
extern buf_dblwr_t buf_dblwr;

101
storage/innobase/include/buf0flu.h
View File

@@ -31,7 +31,10 @@ Created 11/5/1995 Heikki Tuuri
#include "log0log.h"
#include "buf0types.h"
/** Number of pages flushed through non flush_list flushes. */
/** Number of pages flushed. Protected by buf_pool.mutex. */
extern ulint buf_flush_page_count;
/** Number of pages flushed via LRU. Protected by buf_pool.mutex.
Also included in buf_flush_page_count. */
extern ulint buf_lru_flush_page_count;
/** Flag indicating if the page_cleaner is in active state. */
@@ -44,27 +47,24 @@ extern my_bool innodb_page_cleaner_disabled_debug;
#endif /* UNIV_DEBUG */
/** Event to synchronise with the flushing. */
extern os_event_t buf_flush_event;
/** Remove all dirty pages belonging to a given tablespace when we are
deleting the data file of that tablespace.
The pages still remain a part of LRU and are evicted from
the list as they age towards the tail of the LRU.
@param id tablespace identifier */
void buf_flush_remove_pages(ulint id);
class ut_stage_alter_t;
/** Handled page counters for a single flush */
struct flush_counters_t {
ulint flushed; /*!< number of dirty pages flushed */
ulint evicted; /*!< number of clean pages evicted */
ulint unzip_LRU_evicted;/*!< number of uncompressed page images
evicted */
};
/** Remove a block from the flush list of modified blocks.
@param[in] bpage block to be removed from the flush list */
void buf_flush_remove(buf_page_t* bpage);
/** Try to flush all the dirty pages that belong to a given tablespace.
@param id tablespace identifier
@return number dirty pages that there were for this tablespace */
ulint buf_flush_dirty_pages(ulint id)
MY_ATTRIBUTE((warn_unused_result));
/*******************************************************************//**
Relocates a buffer control block on the flush_list.
Note that it is assumed that the contents of bpage has already been
copied to dpage. */
ATTRIBUTE_COLD
void
buf_flush_relocate_on_flush_list(
/*=============================*/
@@ -74,10 +74,9 @@ buf_flush_relocate_on_flush_list(
/** Complete write of a file page from buf_pool.
@param bpage written page
@param request write request
@param dblwr whether the doublewrite buffer was used
@param evict whether or not to evict the page from LRU list */
@param dblwr whether the doublewrite buffer was used */
void buf_page_write_complete(buf_page_t *bpage, const IORequest &request,
bool dblwr, bool evict);
bool dblwr);
/** Assign the full crc32 checksum for non-compressed page.
@param[in,out] page page to be updated */
@@ -95,46 +94,15 @@ buf_flush_init_for_writing(
void* page_zip_,
bool use_full_checksum);
/** Do flushing batch of a given type.
NOTE: The calling thread is not allowed to own any latches on pages!
@param[in] lru true=buf_pool.LRU; false=buf_pool.flush_list
@param[in] min_n wished minimum mumber of blocks flushed
(it is not guaranteed that the actual number is that big, though)
@param[in] lsn_limit in the case BUF_FLUSH_LIST all blocks whose
oldest_modification is smaller than this should be flushed (if their number
does not exceed min_n), otherwise ignored
@param[out] n the number of pages which were processed is
passed back to caller. Ignored if NULL
@retval true if a batch was queued successfully.
@retval false if another batch of same type was already running. */
bool buf_flush_do_batch(bool lru, ulint min_n, lsn_t lsn_limit,
flush_counters_t *n);
/** This utility flushes dirty blocks from the end of the flush list.
NOTE: The calling thread is not allowed to own any latches on pages!
@param[in] min_n wished minimum mumber of blocks flushed (it is
not guaranteed that the actual number is that big, though)
@param[in] lsn_limit in the case BUF_FLUSH_LIST all blocks whose
oldest_modification is smaller than this should be flushed (if their number
does not exceed min_n), otherwise ignored
@param[out] n_processed the number of pages which were processed is
passed back to caller. Ignored if NULL.
@retval true if a batch was queued successfully
@retval false if another batch of same type was already running */
bool buf_flush_lists(ulint min_n, lsn_t lsn_limit, ulint *n_processed);
/******************************************************************//**
This function picks up a single page from the tail of the LRU
list, flushes it (if it is dirty), removes it from page_hash and LRU
list and puts it on the free list. It is called from user threads when
they are unable to find a replaceable page at the tail of the LRU
list i.e.: when the background LRU flushing in the page_cleaner thread
is not fast enough to keep pace with the workload.
@return true if success. */
bool buf_flush_single_page_from_LRU();
/** Write out dirty blocks from buf_pool.flush_list.
@param max_n wished maximum mumber of blocks flushed
@param lsn buf_pool.get_oldest_modification(LSN_MAX) target (0=LRU flush)
@return the number of processed pages
@retval 0 if a batch of the same type (lsn==0 or lsn!=0) is already running */
ulint buf_flush_lists(ulint max_n, lsn_t lsn);
/** Wait until a flush batch ends.
@param[in] lru true=buf_pool.LRU; false=buf_pool.flush_list */
@param lru true=buf_pool.LRU; false=buf_pool.flush_list */
void buf_flush_wait_batch_end(bool lru);
/** Wait until a flush batch of the given lsn ends
@param[in] new_oldest target oldest_modified_lsn to wait for */
@@ -156,33 +124,18 @@ buf_flush_note_modification(
/** Initialize page_cleaner. */
void buf_flush_page_cleaner_init();
/** Wait for any possible LRU flushes to complete. */
void buf_flush_wait_LRU_batch_end();
/** Wait for pending flushes to complete. */
void buf_flush_wait_batch_end_acquiring_mutex(bool lru);
#ifdef UNIV_DEBUG
/** Validate the flush list. */
void buf_flush_validate();
#endif /* UNIV_DEBUG */
/** Write a flushable page from buf_pool to a file.
buf_pool.mutex must be held.
@param bpage buffer control block
@param flush_type type of flush
@param space tablespace (or nullptr if not known)
@param sync whether this is a synchronous request
(only for flush_type=SINGLE_PAGE)
@return whether the page was flushed and buf_pool.mutex was released */
bool buf_flush_page(buf_page_t *bpage, IORequest::flush_t flush_type,
fil_space_t *space, bool sync);
/** Synchronously flush dirty blocks.
NOTE: The calling thread is not allowed to hold any buffer page latches! */
void buf_flush_sync();
/** Request IO burst and wake page_cleaner up.
@param[in] lsn_limit upper limit of LSN to be flushed */
void buf_flush_request_force(lsn_t lsn_limit);
#include "buf0flu.ic"
#endif

35
storage/innobase/include/buf0lru.h
View File

@@ -34,6 +34,9 @@ Created 11/5/1995 Heikki Tuuri
struct trx_t;
struct fil_space_t;
/** Flush this many pages in buf_LRU_get_free_block() */
extern size_t innodb_lru_flush_size;
/*#######################################################################
These are low-level functions
#########################################################################*/
@@ -41,12 +44,6 @@ These are low-level functions
/** Minimum LRU list length for which the LRU_old pointer is defined */
#define BUF_LRU_OLD_MIN_LEN 512 /* 8 megabytes of 16k pages */
/** Empty the flush list for all pages belonging to a tablespace.
@param[in] id tablespace identifier
@param[in] flush whether to write the pages to files
@param[in] first first page to be flushed or evicted */
void buf_LRU_flush_or_remove_pages(ulint id, bool flush, ulint first = 0);
/** Try to free a block. If bpage is a descriptor of a compressed-only
ROW_FORMAT=COMPRESSED page, the buf_page_t object will be freed as well.
The caller must hold buf_pool.mutex.
@@ -58,38 +55,34 @@ bool buf_LRU_free_page(buf_page_t *bpage, bool zip)
MY_ATTRIBUTE((nonnull));
/** Try to free a replaceable block.
@param[in] scan_all true=scan the whole LRU list,
false=use BUF_LRU_SEARCH_SCAN_THRESHOLD
@param limit maximum number of blocks to scan
@return true if found and freed */
bool buf_LRU_scan_and_free_block(bool scan_all);
bool buf_LRU_scan_and_free_block(ulint limit= ULINT_UNDEFINED);
/** @return a buffer block from the buf_pool.free list
@retval NULL if the free list is empty */
buf_block_t* buf_LRU_get_free_only();
/** Get a free block from the buf_pool. The block is taken off the
free list. If free list is empty, blocks are moved from the end of the
LRU list to the free list.
/** Get a block from the buf_pool.free list.
If the list is empty, blocks will be moved from the end of buf_pool.LRU
to buf_pool.free.
This function is called from a user thread when it needs a clean
block to read in a page. Note that we only ever get a block from
the free list. Even when we flush a page or find a page in LRU scan
we put it to free list to be used.
* iteration 0:
* get a block from free list, success:done
* get a block from the buf_pool.free list, success:done
* if buf_pool.try_LRU_scan is set
* scan LRU up to srv_LRU_scan_depth to find a clean block
* the above will put the block on free list
* scan LRU up to 100 pages to free a clean block
* success:retry the free list
* flush one dirty page from tail of LRU to disk
* the above will put the block on free list
* flush up to innodb_lru_flush_size LRU blocks to data files
(until UT_LIST_GET_GEN(buf_pool.free) < innodb_lru_scan_depth)
* on buf_page_write_complete() the blocks will put on buf_pool.free list
* success: retry the free list
* iteration 1:
* same as iteration 0 except:
* subsequent iterations: same as iteration 0 except:
* scan whole LRU list
* scan LRU list even if buf_pool.try_LRU_scan is not set
* iteration > 1:
* same as iteration 1 but sleep 10ms
@param have_mutex whether buf_pool.mutex is already being held
@return the free control block, in state BUF_BLOCK_MEMORY */

9
storage/innobase/include/buf0types.h
View File

@@ -27,8 +27,7 @@ Created 11/17/1995 Heikki Tuuri
#ifndef buf0types_h
#define buf0types_h
#include "os0event.h"
#include "ut0ut.h"
#include "univ.i"
/** Buffer page (uncompressed or compressed) */
class buf_page_t;
@@ -38,8 +37,6 @@ struct buf_block_t;
struct buf_pool_stat_t;
/** Buffer pool buddy statistics struct */
struct buf_buddy_stat_t;
/** Doublewrite memory struct */
struct buf_dblwr_t;
/** A buffer frame. @see page_t */
typedef byte buf_frame_t;
@@ -194,10 +191,6 @@ extern const byte field_ref_zero[UNIV_PAGE_SIZE_MAX];
#include "sync0rw.h"
#include "rw_lock.h"
typedef ib_mutex_t BufPoolMutex;
typedef ib_mutex_t FlushListMutex;
typedef rw_lock_t BPageLock;
class page_hash_latch : public rw_lock
{
public:

6
storage/innobase/include/fil0fil.h
View File

@@ -33,6 +33,7 @@ Created 10/25/1995 Heikki Tuuri
#ifndef UNIV_INNOCHECKSUM
#include "buf0dblwr.h"
#include "hash0hash.h"
#include "log0recv.h"
#include "dict0types.h"
@@ -92,7 +93,6 @@ inline bool srv_is_undo_tablespace(ulint space_id)
space_id < srv_undo_space_id_start + srv_undo_tablespaces_open;
}
extern struct buf_dblwr_t* buf_dblwr;
class page_id_t;
/** Structure containing encryption specification */
@@ -418,8 +418,8 @@ public:
/** @return whether doublewrite buffering is needed */
bool use_doublewrite() const
{
return !atomic_write_supported
&& srv_use_doublewrite_buf && buf_dblwr;
return !atomic_write_supported && srv_use_doublewrite_buf &&
buf_dblwr.is_initialised();
}
/** Append a file to the chain of files of a space.

12
storage/innobase/include/log0recv.h
View File

@@ -50,21 +50,12 @@ ATTRIBUTE_COLD void recv_recover_page(fil_space_t* space, buf_page_t* bpage)
MY_ATTRIBUTE((nonnull));
/** Start recovering from a redo log checkpoint.
@see recv_recovery_from_checkpoint_finish
@param[in] flush_lsn FIL_PAGE_FILE_FLUSH_LSN
of first system tablespace page
@return error code or DB_SUCCESS */
dberr_t
recv_recovery_from_checkpoint_start(
lsn_t flush_lsn);
/** Complete recovery from a checkpoint. */
void
recv_recovery_from_checkpoint_finish(void);
/********************************************************//**
Initiates the rollback of active transactions. */
void
recv_recovery_rollback_active(void);
/*===============================*/
/** Whether to store redo log records in recv_sys.pages */
enum store_t {
@@ -296,9 +287,10 @@ private:
@param page_id page identifier
@param p iterator pointing to page_id
@param mtr mini-transaction
@param b pre-allocated buffer pool block
@return whether the page was successfully initialized */
inline buf_block_t *recover_low(const page_id_t page_id, map::iterator &p,
mtr_t &mtr);
mtr_t &mtr, buf_block_t *b);
/** Attempt to initialize a page based on redo log records.
@param page_id page identifier
@return the recovered block

82
storage/innobase/include/os0file.h
View File

@@ -182,32 +182,14 @@ static const ulint OS_FILE_OPERATION_NOT_SUPPORTED = 125;
static const ulint OS_FILE_ERROR_MAX = 200;
/* @} */
/** Types for AIO operations @{ */
/** No transformations during read/write, write as is. */
#define IORequestRead IORequest(IORequest::READ)
#define IORequestWrite IORequest(IORequest::WRITE)
/**
The I/O context that is passed down to the low level IO code */
class IORequest
{
public:
/** Buffer pool flush types */
enum flush_t
{
/** via buf_pool.LRU */
LRU= 0,
/** via buf_pool.flush_list */
FLUSH_LIST,
/** single page of buf_poof.LRU */
SINGLE_PAGE
};
IORequest(ulint type= READ, buf_page_t *bpage= nullptr,
flush_t flush_type= LRU) :
m_bpage(bpage), m_type(static_cast<uint16_t>(type)),
m_flush_type(flush_type) {}
constexpr IORequest(ulint type= READ, buf_page_t *bpage= nullptr,
bool lru= false) :
m_bpage(bpage), m_type(static_cast<uint16_t>(type)), m_LRU(lru) {}
/** Flags passed in the request, they can be ORred together. */
enum {
@@ -243,12 +225,6 @@ public:
return((m_type & WRITE) == WRITE);
}
/** Clear the punch hole flag */
void clear_punch_hole()
{
m_type &= uint16_t(~PUNCH_HOLE);
}
/** @return true if partial read warning disabled */
bool is_partial_io_warning_disabled() const
MY_ATTRIBUTE((warn_unused_result))
@@ -256,12 +232,6 @@ public:
return !!(m_type & DISABLE_PARTIAL_IO_WARNINGS);
}
/** Disable partial read warnings */
void disable_partial_io_warnings()
{
m_type |= DISABLE_PARTIAL_IO_WARNINGS;
}
/** @return true if punch hole should be used */
bool punch_hole() const
MY_ATTRIBUTE((warn_unused_result))
@@ -276,29 +246,15 @@ public:
return(is_read() ^ is_write());
}
/** Set the punch hole flag */
void set_punch_hole()
{
if (is_punch_hole_supported()) {
m_type |= PUNCH_HOLE;
}
}
/** Set the pointer to file node for IO
@param[in] node File node */
inline void set_fil_node(fil_node_t* node);
void set_fil_node(fil_node_t *node) { m_fil_node= node; }
bool operator==(const IORequest& rhs) const
{
return(m_type == rhs.m_type);
}
/** Note that the IO is for double write recovery. */
void dblwr_recover()
{
m_type |= DBLWR_RECOVER;
}
/** @return true if the request is from the dblwr recovery */
bool is_dblwr_recover() const
MY_ATTRIBUTE((warn_unused_result))
@@ -306,24 +262,6 @@ public:
return((m_type & DBLWR_RECOVER) == DBLWR_RECOVER);
}
/** @return true if punch hole is supported */
static bool is_punch_hole_supported()
{
/* In this debugging mode, we act as if punch hole is supported,
and then skip any calls to actually punch a hole here.
In this way, Transparent Page Compression is still being tested. */
DBUG_EXECUTE_IF("ignore_punch_hole",
return(true);
);
#if defined(HAVE_FALLOC_PUNCH_HOLE_AND_KEEP_SIZE) || defined(_WIN32)
return(true);
#else
return(false);
#endif /* HAVE_FALLOC_PUNCH_HOLE_AND_KEEP_SIZE || _WIN32 */
}
ulint get_trim_length(ulint write_length) const
{
return (m_bpage ?
@@ -340,8 +278,8 @@ public:
@return DB_SUCCESS or error code */
dberr_t punch_hole(os_file_t fh, os_offset_t off, ulint len);
/** @return the flush type */
flush_t flush_type() const { return m_flush_type; }
/** @return type of page flush (for writes) */
bool is_LRU() const { return m_LRU; }
private:
/** Page to be written on write operation. */
@@ -351,13 +289,15 @@ private:
fil_node_t* m_fil_node= nullptr;
/** Request type bit flags */
uint16_t m_type= READ;
const uint16_t m_type;
/** for writes, type of page flush */
flush_t m_flush_type= LRU;
const bool m_LRU= false;
};
/* @} */
constexpr IORequest IORequestRead(IORequest::READ);
constexpr IORequest IORequestWrite(IORequest::WRITE);
/** Sparse file size information. */
struct os_file_size_t {

8
storage/innobase/include/srv0mon.h
View File

@@ -196,16 +196,11 @@ enum monitor_id_t {
MONITOR_FLUSH_N_TO_FLUSH_BY_AGE,
MONITOR_FLUSH_ADAPTIVE_AVG_TIME_SLOT,
MONITOR_LRU_BATCH_FLUSH_AVG_TIME_SLOT,
MONITOR_FLUSH_ADAPTIVE_AVG_TIME_THREAD,
MONITOR_LRU_BATCH_FLUSH_AVG_TIME_THREAD,
MONITOR_FLUSH_ADAPTIVE_AVG_TIME_EST,
MONITOR_LRU_BATCH_FLUSH_AVG_TIME_EST,
MONITOR_FLUSH_AVG_TIME,
MONITOR_FLUSH_ADAPTIVE_AVG_PASS,
MONITOR_LRU_BATCH_FLUSH_AVG_PASS,
MONITOR_FLUSH_AVG_PASS,
MONITOR_LRU_GET_FREE_LOOPS,
@@ -234,9 +229,6 @@ enum monitor_id_t {
MONITOR_LRU_BATCH_EVICT_TOTAL_PAGE,
MONITOR_LRU_BATCH_EVICT_COUNT,
MONITOR_LRU_BATCH_EVICT_PAGES,
MONITOR_LRU_SINGLE_FLUSH_SCANNED,
MONITOR_LRU_SINGLE_FLUSH_SCANNED_NUM_CALL,
MONITOR_LRU_SINGLE_FLUSH_SCANNED_PER_CALL,
MONITOR_LRU_SINGLE_FLUSH_FAILURE_COUNT,
MONITOR_LRU_GET_FREE_SEARCH,
MONITOR_LRU_SEARCH_SCANNED,

6
storage/innobase/include/srv0srv.h
View File

@@ -100,10 +100,6 @@ struct srv_stats_t
need to make a flush, in order to be able to read or create a page. */
ulint_ctr_1_t buf_pool_wait_free;
/** Count the number of pages that were written from buffer
pool to the disk */
ulint_ctr_1_t buf_pool_flushed;
/** Number of buffer pool reads that led to the reading of
a disk page */
ulint_ctr_1_t buf_pool_reads;
@@ -409,7 +405,6 @@ extern unsigned long long srv_stats_modified_counter;
extern my_bool srv_stats_sample_traditional;
extern my_bool srv_use_doublewrite_buf;
extern ulong srv_doublewrite_batch_size;
extern ulong srv_checksum_algorithm;
extern double srv_max_buf_pool_modified_pct;
@@ -764,7 +759,6 @@ struct export_var_t{
ulint innodb_buffer_pool_read_requests; /*!< buf_pool.stat.n_page_gets */
ulint innodb_buffer_pool_reads; /*!< srv_buf_pool_reads */
ulint innodb_buffer_pool_wait_free; /*!< srv_buf_pool_wait_free */
ulint innodb_buffer_pool_pages_flushed; /*!< srv_buf_pool_flushed */
ulint innodb_buffer_pool_write_requests;/*!< srv_buf_pool_write_requests */
ulint innodb_buffer_pool_read_ahead_rnd;/*!< srv_read_ahead_rnd */
ulint innodb_buffer_pool_read_ahead; /*!< srv_read_ahead */

5
storage/innobase/include/srv0start.h
View File

@@ -112,11 +112,6 @@ enum srv_shutdown_t {
SRV_SHUTDOWN_INITIATED,
SRV_SHUTDOWN_CLEANUP, /*!< Cleaning up in
logs_empty_and_mark_files_at_shutdown() */
SRV_SHUTDOWN_FLUSH_PHASE,/*!< At this phase the master and the
purge threads must have completed their
work. Once we enter this phase the
page_cleaner can clean up the buffer
pool and exit */
SRV_SHUTDOWN_LAST_PHASE,/*!< Last phase after ensuring that
the buffer pool can be freed: flush
all file spaces and close all files */

6
storage/innobase/include/sync0debug.h
View File

@@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 2013, 2015, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2017, MariaDB Corporation.
Copyright (c) 2017, 2020, MariaDB Corporation.
Portions of this file contain modifications contributed and copyrighted by
Google, Inc. Those modifications are gratefully acknowledged and are described
@@ -44,10 +44,6 @@ void
sync_check_close();
#ifdef UNIV_DEBUG
/** Enable sync order checking. */
void
sync_check_enable();
/** Check if it is OK to acquire the latch.
@param[in] latch latch type */
void

1
storage/innobase/include/sync0sync.h
View File

@@ -63,7 +63,6 @@ extern mysql_pfs_key_t log_sys_mutex_key;
extern mysql_pfs_key_t log_cmdq_mutex_key;
extern mysql_pfs_key_t log_flush_order_mutex_key;
extern mysql_pfs_key_t recalc_pool_mutex_key;
extern mysql_pfs_key_t page_cleaner_mutex_key;
extern mysql_pfs_key_t purge_sys_pq_mutex_key;
extern mysql_pfs_key_t recv_sys_mutex_key;
extern mysql_pfs_key_t rtr_active_mutex_key;

12
storage/innobase/include/sync0types.h
View File

@@ -188,12 +188,6 @@ enum latch_level_t {
SYNC_ANY_LATCH,
SYNC_DOUBLEWRITE,
SYNC_BUF_FLUSH_LIST,
SYNC_BUF_POOL,
SYNC_POOL,
SYNC_POOL_MANAGER,
@@ -208,7 +202,6 @@ enum latch_level_t {
SYNC_RECV,
SYNC_LOG_FLUSH_ORDER,
SYNC_LOG,
SYNC_PAGE_CLEANER,
SYNC_PURGE_QUEUE,
SYNC_TRX_SYS_HEADER,
SYNC_REC_LOCK,
@@ -271,11 +264,9 @@ enum latch_level_t {
up its meta-data. See sync0debug.cc. */
enum latch_id_t {
LATCH_ID_NONE = 0,
LATCH_ID_BUF_POOL,
LATCH_ID_DICT_FOREIGN_ERR,
LATCH_ID_DICT_SYS,
LATCH_ID_FIL_SYSTEM,
LATCH_ID_FLUSH_LIST,
LATCH_ID_FTS_BG_THREADS,
LATCH_ID_FTS_DELETE,
LATCH_ID_FTS_DOC_ID,
@@ -285,7 +276,6 @@ enum latch_id_t {
LATCH_ID_IBUF_PESSIMISTIC_INSERT,
LATCH_ID_LOG_SYS,
LATCH_ID_LOG_FLUSH_ORDER,
LATCH_ID_PAGE_CLEANER,
LATCH_ID_PURGE_SYS_PQ,
LATCH_ID_RECALC_POOL,
LATCH_ID_RECV_SYS,
@@ -299,7 +289,6 @@ enum latch_id_t {
LATCH_ID_SRV_INNODB_MONITOR,
LATCH_ID_SRV_MISC_TMPFILE,
LATCH_ID_SRV_MONITOR_FILE,
LATCH_ID_BUF_DBLWR,
LATCH_ID_TRX_POOL,
LATCH_ID_TRX_POOL_MANAGER,
LATCH_ID_TRX,
@@ -982,6 +971,7 @@ struct sync_checker : public sync_check_functor_t
{
if (some_allowed) {
switch (level) {
case SYNC_FSP:
case SYNC_DICT:
case SYNC_DICT_OPERATION:
case SYNC_FTS_CACHE:

5
storage/innobase/include/trx0sys.h
View File

@@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2017, 2019, MariaDB Corporation.
Copyright (c) 2017, 2020, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -340,9 +340,6 @@ FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID. */
constexpr uint32_t TRX_SYS_DOUBLEWRITE_MAGIC_N= 536853855;
/** Contents of TRX_SYS_DOUBLEWRITE_SPACE_ID_STORED */
constexpr uint32_t TRX_SYS_DOUBLEWRITE_SPACE_ID_STORED_N= 1783657386;
/** Size of the doublewrite block in pages */
#define TRX_SYS_DOUBLEWRITE_BLOCK_SIZE FSP_EXTENT_SIZE
/* @} */
trx_t* current_trx();

108
storage/innobase/log/log0log.cc
View File

@@ -493,7 +493,7 @@ void log_t::create()
log record has a non-zero start lsn, a fact which we will use */
set_lsn(LOG_START_LSN + LOG_BLOCK_HDR_SIZE);
set_flushed_lsn(0);
set_flushed_lsn(LOG_START_LSN + LOG_BLOCK_HDR_SIZE);
ut_ad(srv_log_buffer_size >= 16 * OS_FILE_LOG_BLOCK_SIZE);
ut_ad(srv_log_buffer_size >= 4U << srv_page_size_shift);
@@ -936,6 +936,7 @@ loop:
and invoke log_mutex_enter(). */
static void log_write_flush_to_disk_low(lsn_t lsn)
{
if (!log_sys.log.writes_are_durable())
log_sys.log.flush();
ut_a(lsn >= log_sys.get_flushed_lsn());
log_sys.set_flushed_lsn(lsn);
@@ -1129,12 +1130,7 @@ void log_write_up_to(lsn_t lsn, bool flush_to_disk, bool rotate_key)
/* Flush the highest written lsn.*/
auto flush_lsn = write_lock.value();
flush_lock.set_pending(flush_lsn);
if (!log_sys.log.writes_are_durable())
{
log_write_flush_to_disk_low(flush_lsn);
}
flush_lock.release(flush_lsn);
innobase_mysql_log_notify(flush_lsn);
@@ -1204,29 +1200,22 @@ static bool log_preflush_pool_modified_pages(lsn_t new_oldest)
|| !buf_page_cleaner_is_active
|| srv_is_being_started) {
ulint n_pages;
ulint n_pages = buf_flush_lists(ULINT_UNDEFINED, new_oldest);
success = buf_flush_lists(ULINT_MAX, new_oldest, &n_pages);
buf_flush_wait_batch_end_acquiring_mutex(false);
buf_flush_wait_batch_end(false);
if (!success) {
MONITOR_INC(MONITOR_FLUSH_SYNC_WAITS);
}
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_FLUSH_SYNC_TOTAL_PAGE,
MONITOR_FLUSH_SYNC_COUNT,
MONITOR_FLUSH_SYNC_PAGES,
n_pages);
const lsn_t oldest = buf_pool.get_oldest_modification();
success = !oldest || oldest >= new_oldest;
} else {
/* better to wait for flushed by page cleaner */
if (srv_flush_sync) {
/* wake page cleaner for IO burst */
buf_flush_request_force(new_oldest);
}
buf_flush_wait_flushed(new_oldest);
success = true;
@@ -1515,6 +1504,41 @@ void log_check_margins()
extern void buf_resize_shutdown();
/** @return the number of dirty pages in the buffer pool */
static ulint flush_list_length()
{
mysql_mutex_lock(&buf_pool.flush_list_mutex);
const ulint len= UT_LIST_GET_LEN(buf_pool.flush_list);
mysql_mutex_unlock(&buf_pool.flush_list_mutex);
return len;
}
static void flush_buffer_pool()
{
service_manager_extend_timeout(INNODB_EXTEND_TIMEOUT_INTERVAL,
"Waiting to flush the buffer pool");
while (buf_pool.n_flush_list || flush_list_length())
{
buf_flush_lists(ULINT_UNDEFINED, LSN_MAX);
timespec abstime;
if (buf_pool.n_flush_list)
{
service_manager_extend_timeout(INNODB_EXTEND_TIMEOUT_INTERVAL,
"Waiting to flush " ULINTPF " pages",
flush_list_length());
set_timespec(abstime, INNODB_EXTEND_TIMEOUT_INTERVAL / 2);
mysql_mutex_lock(&buf_pool.mutex);
while (buf_pool.n_flush_list)
mysql_cond_timedwait(&buf_pool.done_flush_list, &buf_pool.mutex,
&abstime);
mysql_mutex_unlock(&buf_pool.mutex);
}
}
ut_ad(!buf_pool.any_io_pending());
}
/** Make a checkpoint at the latest lsn on shutdown. */
void logs_empty_and_mark_files_at_shutdown()
{
@@ -1616,30 +1640,26 @@ wait_suspend_loop:
goto wait_suspend_loop;
}
if (buf_page_cleaner_is_active) {
thread_name = "page cleaner thread";
mysql_cond_signal(&buf_pool.do_flush_list);
goto wait_suspend_loop;
}
buf_load_dump_end();
srv_shutdown_state = SRV_SHUTDOWN_FLUSH_PHASE;
/* At this point only page_cleaner should be active. We wait
here to let it complete the flushing of the buffer pools
before proceeding further. */
count = 0;
service_manager_extend_timeout(COUNT_INTERVAL * CHECK_INTERVAL/1000000 * 2,
"Waiting for page cleaner");
while (buf_page_cleaner_is_active) {
++count;
os_thread_sleep(CHECK_INTERVAL);
if (srv_print_verbose_log && count > COUNT_INTERVAL) {
service_manager_extend_timeout(COUNT_INTERVAL * CHECK_INTERVAL/1000000 * 2,
"Waiting for page cleaner");
ib::info() << "Waiting for page_cleaner to "
"finish flushing of buffer pool";
/* This is a workaround to avoid the InnoDB hang
when OS datetime changed backwards */
os_event_set(buf_flush_event);
if (!buf_pool.is_initialised()) {
ut_ad(!srv_was_started);
} else if (ulint pending_io = buf_pool.io_pending()) {
if (srv_print_verbose_log && count > 600) {
ib::info() << "Waiting for " << pending_io << " buffer"
" page I/Os to complete";
count = 0;
}
goto loop;
} else {
flush_buffer_pool();
}
if (log_sys.is_initialised()) {
@@ -1660,18 +1680,6 @@ wait_suspend_loop:
}
}
if (!buf_pool.is_initialised()) {
ut_ad(!srv_was_started);
} else if (ulint pending_io = buf_pool.io_pending()) {
if (srv_print_verbose_log && count > 600) {
ib::info() << "Waiting for " << pending_io << " buffer"
" page I/Os to complete";
count = 0;
}
goto loop;
}
if (srv_fast_shutdown == 2 || !srv_was_started) {
if (!srv_read_only_mode && srv_was_started) {
ib::info() << "MySQL has requested a very fast"

64
storage/innobase/log/log0recv.cc
View File

@@ -987,9 +987,9 @@ void recv_sys_t::debug_free()
{
ut_ad(this == &recv_sys);
ut_ad(is_initialised());
ut_ad(!recv_recovery_is_on());
mutex_enter(&mutex);
recovery_on= false;
pages.clear();
ut_free_dodump(buf, RECV_PARSING_BUF_SIZE);
@@ -2497,9 +2497,11 @@ static void recv_read_in_area(page_id_t page_id)
@param page_id page identifier
@param p iterator pointing to page_id
@param mtr mini-transaction
@param b pre-allocated buffer pool block
@return whether the page was successfully initialized */
inline buf_block_t *recv_sys_t::recover_low(const page_id_t page_id,
map::iterator &p, mtr_t &mtr)
map::iterator &p, mtr_t &mtr,
buf_block_t *b)
{
ut_ad(mutex_own(&mutex));
ut_ad(p->first == page_id);
@@ -2515,20 +2517,21 @@ inline buf_block_t *recv_sys_t::recover_low(const page_id_t page_id,
{
mtr.start();
mtr.set_log_mode(MTR_LOG_NO_REDO);
block= buf_page_create(space, page_id.page_no(), space->zip_size(), &mtr);
p= recv_sys.pages.find(page_id);
if (p == recv_sys.pages.end())
block= buf_page_create(space, page_id.page_no(), space->zip_size(), &mtr,
b);
if (UNIV_UNLIKELY(block != b))
{
/* The page happened to exist in the buffer pool, or it was just
being read in. Before buf_page_get_with_no_latch() returned to
buf_page_create(), all changes must have been applied to the
page already. */
ut_ad(recv_sys.pages.find(page_id) == recv_sys.pages.end());
mtr.commit();
block= nullptr;
}
else
{
ut_ad(&recs == &p->second);
ut_ad(&recs == &recv_sys.pages.find(page_id)->second);
i.created= true;
buf_block_dbg_add_level(block, SYNC_NO_ORDER_CHECK);
recv_recover_page(block, mtr, p, space, &i);
@@ -2548,6 +2551,7 @@ inline buf_block_t *recv_sys_t::recover_low(const page_id_t page_id,
@return whether the page was successfully initialized */
buf_block_t *recv_sys_t::recover_low(const page_id_t page_id)
{
buf_block_t *free_block= buf_LRU_get_free_block(false);
buf_block_t *block= nullptr;
mutex_enter(&mutex);
@@ -2556,10 +2560,13 @@ buf_block_t *recv_sys_t::recover_low(const page_id_t page_id)
if (p != pages.end() && p->second.state == page_recv_t::RECV_WILL_NOT_READ)
{
mtr_t mtr;
block= recover_low(page_id, p, mtr);
block= recover_low(page_id, p, mtr, free_block);
ut_ad(!block || block == free_block);
}
mutex_exit(&mutex);
if (UNIV_UNLIKELY(!block))
buf_pool.free_block(free_block);
return block;
}
@@ -2614,6 +2621,8 @@ void recv_sys_t::apply(bool last_batch)
trim(page_id_t(id + srv_undo_space_id_start, t.pages), t.lsn);
}
buf_block_t *free_block= buf_LRU_get_free_block(false);
for (map::iterator p= pages.begin(); p != pages.end(); )
{
const page_id_t page_id= p->first;
@@ -2626,7 +2635,14 @@ void recv_sys_t::apply(bool last_batch)
p++;
continue;
case page_recv_t::RECV_WILL_NOT_READ:
recover_low(page_id, p, mtr);
if (UNIV_LIKELY(!!recover_low(page_id, p, mtr, free_block)))
{
mutex_exit(&mutex);
free_block= buf_LRU_get_free_block(false);
mutex_enter(&mutex);
next_page:
p= pages.lower_bound(page_id);
}
continue;
case page_recv_t::RECV_NOT_PROCESSED:
mtr.start();
@@ -2652,9 +2668,11 @@ void recv_sys_t::apply(bool last_batch)
continue;
}
p= pages.lower_bound(page_id);
goto next_page;
}
buf_pool.free_block(free_block);
/* Wait until all the pages have been processed */
while (!pages.empty())
{
@@ -2686,7 +2704,6 @@ void recv_sys_t::apply(bool last_batch)
/* Instead of flushing, last_batch could sort the buf_pool.flush_list
in ascending order of buf_page_t::oldest_modification. */
buf_flush_wait_LRU_batch_end();
buf_flush_sync();
if (!last_batch)
@@ -3250,7 +3267,6 @@ recv_init_crash_recovery_spaces(bool rescan, bool& missing_tablespace)
}
/** Start recovering from a redo log checkpoint.
@see recv_recovery_from_checkpoint_finish
@param[in] flush_lsn FIL_PAGE_FILE_FLUSH_LSN
of first system tablespace page
@return error code or DB_SUCCESS */
@@ -3268,10 +3284,10 @@ recv_recovery_from_checkpoint_start(lsn_t flush_lsn)
ut_ad(srv_operation == SRV_OPERATION_NORMAL
|| srv_operation == SRV_OPERATION_RESTORE
|| srv_operation == SRV_OPERATION_RESTORE_EXPORT);
ut_d(mutex_enter(&buf_pool.flush_list_mutex));
ut_d(mysql_mutex_lock(&buf_pool.flush_list_mutex));
ut_ad(UT_LIST_GET_LEN(buf_pool.LRU) == 0);
ut_ad(UT_LIST_GET_LEN(buf_pool.unzip_LRU) == 0);
ut_d(mutex_exit(&buf_pool.flush_list_mutex));
ut_d(mysql_mutex_unlock(&buf_pool.flush_list_mutex));
if (srv_force_recovery >= SRV_FORCE_NO_LOG_REDO) {
@@ -3424,6 +3440,11 @@ completed:
}
log_sys.set_lsn(recv_sys.recovered_lsn);
if (UNIV_LIKELY(log_sys.get_flushed_lsn() < recv_sys.recovered_lsn)) {
/* This may already have been set by create_log_file()
if no logs existed when the server started up. */
log_sys.set_flushed_lsn(recv_sys.recovered_lsn);
}
if (recv_needed_recovery) {
bool missing_tablespace = false;
@@ -3472,7 +3493,7 @@ completed:
recv_sys.parse_start_lsn = checkpoint_lsn;
if (srv_operation == SRV_OPERATION_NORMAL) {
buf_dblwr_process();
buf_dblwr.recover();
}
ut_ad(srv_force_recovery <= SRV_FORCE_NO_UNDO_LOG_SCAN);
@@ -3556,19 +3577,6 @@ completed:
return(DB_SUCCESS);
}
/** Complete recovery from a checkpoint. */
void recv_recovery_from_checkpoint_finish()
{
/* Free the resources of the recovery system */
recv_sys.recovery_on= false;
recv_sys.debug_free();
/* Enable innodb_sync_debug checks */
ut_d(sync_check_enable());
}
bool recv_dblwr_t::validate_page(const page_id_t page_id,
const byte *page,
const fil_space_t *space,
@@ -3651,7 +3659,7 @@ byte *recv_dblwr_t::find_page(const page_id_t page_id,
if (lsn <= max_lsn ||
!validate_page(page_id, page, space, tmp_buf))
{
/* Mark processed for subsequent iterations in buf_dblwr_process() */
/* Mark processed for subsequent iterations in buf_dblwr_t::recover() */
memset(page + FIL_PAGE_LSN, 0, 8);
continue;
}

5
storage/innobase/os/os0file.cc
View File

@@ -153,9 +153,6 @@ static ulint os_innodb_umask = 0;
#endif /* _WIN32 */
/** Flag indicating if the page_cleaner is in active state. */
extern bool buf_page_cleaner_is_active;
#ifdef WITH_INNODB_DISALLOW_WRITES
#define WAIT_ALLOW_WRITES() os_event_wait(srv_allow_writes_event)
#else
@@ -3829,9 +3826,7 @@ IORequest::punch_hole(os_file_t fh, os_offset_t off, ulint len)
/* If punch hole is not supported,
set space so that it is not used. */
if (err == DB_IO_NO_PUNCH_HOLE) {
if (m_fil_node) {
m_fil_node->space->punch_hole = false;
}
err = DB_SUCCESS;
}
}

18
storage/innobase/row/row0import.cc
View File

@@ -3423,11 +3423,10 @@ fil_iterate(
? iter.crypt_io_buffer : io_buffer;
byte* const writeptr = readptr;
IORequest read_request(IORequest::READ);
read_request.disable_partial_io_warnings();
err = os_file_read_no_error_handling(
read_request, iter.file, readptr, offset, n_bytes, 0);
IORequest(IORequest::READ
| IORequest::DISABLE_PARTIAL_IO_WARNINGS),
iter.file, readptr, offset, n_bytes, 0);
if (err != DB_SUCCESS) {
ib::error() << iter.filepath
<< ": os_file_read() failed";
@@ -3760,11 +3759,10 @@ fil_tablespace_iterate(
/* Read the first page and determine the page and zip size. */
IORequest request(IORequest::READ);
request.disable_partial_io_warnings();
err = os_file_read_no_error_handling(request, file, page, 0,
srv_page_size, 0);
err = os_file_read_no_error_handling(
IORequest(IORequest::READ
| IORequest::DISABLE_PARTIAL_IO_WARNINGS),
file, page, 0, srv_page_size, 0);
if (err == DB_SUCCESS) {
err = callback.init(file_size, block);
@@ -4175,7 +4173,7 @@ row_import_for_mysql(
/* Ensure that all pages dirtied during the IMPORT make it to disk.
The only dirty pages generated should be from the pessimistic purge
of delete marked records that couldn't be purged in Phase I. */
buf_LRU_flush_or_remove_pages(prebuilt->table->space_id, true);
while (buf_flush_dirty_pages(prebuilt->table->space_id));
ib::info() << "Phase IV - Flush complete";
prebuilt->table->space->set_imported();

13
storage/innobase/row/row0log.cc
View File

@@ -410,7 +410,6 @@ row_log_online_op(
const os_offset_t byte_offset
= (os_offset_t) log->tail.blocks
* srv_sort_buf_size;
IORequest request(IORequest::WRITE);
byte* buf = log->tail.block;
if (byte_offset + srv_sort_buf_size >= srv_online_max_size) {
@@ -448,7 +447,7 @@ row_log_online_op(
log->tail.blocks++;
if (os_file_write(
request,
IORequestWrite,
"(modification log)",
log->fd,
buf, byte_offset, srv_sort_buf_size)
@@ -549,7 +548,6 @@ row_log_table_close_func(
const os_offset_t byte_offset
= (os_offset_t) log->tail.blocks
* srv_sort_buf_size;
IORequest request(IORequest::WRITE);
byte* buf = log->tail.block;
if (byte_offset + srv_sort_buf_size >= srv_online_max_size) {
@@ -587,7 +585,7 @@ row_log_table_close_func(
log->tail.blocks++;
if (os_file_write(
request,
IORequestWrite,
"(modification log)",
log->fd,
buf, byte_offset, srv_sort_buf_size)
@@ -2874,11 +2872,10 @@ all_done:
goto func_exit;
}
IORequest request(IORequest::READ);
byte* buf = index->online_log->head.block;
if (os_file_read_no_error_handling(
request, index->online_log->fd,
IORequestRead, index->online_log->fd,
buf, ofs, srv_sort_buf_size, 0) != DB_SUCCESS) {
ib::error()
<< "Unable to read temporary file"
@@ -3767,8 +3764,6 @@ all_done:
os_offset_t ofs = static_cast<os_offset_t>(
index->online_log->head.blocks)
* srv_sort_buf_size;
IORequest request(IORequest::READ);
ut_ad(has_index_lock);
has_index_lock = false;
rw_lock_x_unlock(dict_index_get_lock(index));
@@ -3783,7 +3778,7 @@ all_done:
byte* buf = index->online_log->head.block;
if (os_file_read_no_error_handling(
request, index->online_log->fd,
IORequestRead, index->online_log->fd,
buf, ofs, srv_sort_buf_size, 0) != DB_SUCCESS) {
ib::error()
<< "Unable to read temporary file"

6
storage/innobase/row/row0merge.cc
View File

@@ -1080,9 +1080,8 @@ row_merge_read(
DBUG_LOG("ib_merge_sort", "fd=" << fd << " ofs=" << ofs);
DBUG_EXECUTE_IF("row_merge_read_failure", DBUG_RETURN(FALSE););
IORequest request(IORequest::READ);
const bool success = DB_SUCCESS == os_file_read_no_error_handling(
request, fd, buf, ofs, srv_sort_buf_size, 0);
IORequestRead, fd, buf, ofs, srv_sort_buf_size, 0);
/* If encryption is enabled decrypt buffer */
if (success && log_tmp_is_encrypted()) {
@@ -1144,9 +1143,8 @@ row_merge_write(
out_buf = crypt_buf;
}
IORequest request(IORequest::WRITE);
const bool success = DB_SUCCESS == os_file_write(
request, "(merge)", fd, out_buf, ofs, buf_len);
IORequestWrite, "(merge)", fd, out_buf, ofs, buf_len);
#ifdef POSIX_FADV_DONTNEED
/* The block will be needed on the next merge pass,

3
storage/innobase/row/row0mysql.cc
View File

@@ -3391,8 +3391,7 @@ row_drop_table_for_mysql(
dict_stats_recalc_pool_del(table);
dict_stats_defrag_pool_del(table, NULL);
if (btr_defragment_active) {
/* During fts_drop_orphaned_tables() in
recv_recovery_rollback_active() the
/* During fts_drop_orphaned_tables() the
btr_defragment_mutex has not yet been
initialized by btr_defragment_init(). */
btr_defragment_remove_table(table);

17
storage/innobase/row/row0quiesce.cc
View File

@@ -525,17 +525,27 @@ row_quiesce_table_start(
}
for (ulint count = 0;
ibuf_merge_space(table->space_id) != 0
&& !trx_is_interrupted(trx);
ibuf_merge_space(table->space_id);
++count) {
if (trx_is_interrupted(trx)) {
goto aborted;
}
if (!(count % 20)) {
ib::info() << "Merging change buffer entries for "
<< table->name;
}
}
while (buf_flush_dirty_pages(table->space_id)) {
if (trx_is_interrupted(trx)) {
goto aborted;
}
}
if (!trx_is_interrupted(trx)) {
buf_LRU_flush_or_remove_pages(table->space_id, true);
/* Ensure that all asynchronous IO is completed. */
os_aio_wait_until_no_pending_writes();
fil_flush(table->space_id);
if (row_quiesce_write_cfg(table, trx->mysql_thd)
!= DB_SUCCESS) {
@@ -546,6 +556,7 @@ row_quiesce_table_start(
<< " flushed to disk";
}
} else {
aborted:
ib::warn() << "Quiesce aborted!";
}

6
storage/innobase/row/row0sel.cc
View File

@@ -121,7 +121,7 @@ row_sel_sec_rec_is_for_blob(
field_ref_zero, BTR_EXTERN_FIELD_REF_SIZE)) {
/* The externally stored field was not written yet.
This record should only be seen by
recv_recovery_rollback_active() or any
trx_rollback_recovered() or any
TRX_ISO_READ_UNCOMMITTED transactions. */
return(FALSE);
}
@@ -528,7 +528,7 @@ row_sel_fetch_columns(
externally stored field was not
written yet. This record
should only be seen by
recv_recovery_rollback_active() or any
trx_rollback_recovered() or any
TRX_ISO_READ_UNCOMMITTED
transactions. The InnoDB SQL parser
(the sole caller of this function)
@@ -2891,7 +2891,7 @@ row_sel_store_mysql_field(
if (UNIV_UNLIKELY(!data)) {
/* The externally stored field was not written
yet. This record should only be seen by
recv_recovery_rollback_active() or any
trx_rollback_recovered() or any
TRX_ISO_READ_UNCOMMITTED transactions. */
if (heap != prebuilt->blob_heap) {

8
storage/innobase/row/row0undo.cc
View File

@@ -466,13 +466,7 @@ row_undo_step(
{
dberr_t err;
undo_node_t* node;
trx_t* trx;
ut_ad(thr);
srv_inc_activity_count();
trx = thr_get_trx(thr);
trx_t* trx = thr_get_trx(thr);
node = static_cast<undo_node_t*>(thr->run_node);

2
storage/innobase/row/row0upd.cc
View File

@@ -1539,7 +1539,7 @@ row_upd_changes_ord_field_binary_func(
/* The externally stored field
was not written yet. This
record should only be seen by
recv_recovery_rollback_active(),
trx_rollback_recovered()
when the server had crashed before
storing the field. */
ut_ad(thr->graph->trx->is_recovered);

40
storage/innobase/srv/srv0mon.cc
View File

@@ -386,31 +386,16 @@ static monitor_info_t innodb_counter_info[] =
MONITOR_NONE,
MONITOR_DEFAULT_START, MONITOR_FLUSH_ADAPTIVE_AVG_TIME_SLOT},
{"buffer_LRU_batch_flush_avg_time_slot", "buffer",
"Avg time (ms) spent for LRU batch flushing recently per slot.",
MONITOR_NONE,
MONITOR_DEFAULT_START, MONITOR_LRU_BATCH_FLUSH_AVG_TIME_SLOT},
{"buffer_flush_adaptive_avg_time_thread", "buffer",
"Avg time (ms) spent for adaptive flushing recently per thread.",
MONITOR_NONE,
MONITOR_DEFAULT_START, MONITOR_FLUSH_ADAPTIVE_AVG_TIME_THREAD},
{"buffer_LRU_batch_flush_avg_time_thread", "buffer",
"Avg time (ms) spent for LRU batch flushing recently per thread.",
MONITOR_NONE,
MONITOR_DEFAULT_START, MONITOR_LRU_BATCH_FLUSH_AVG_TIME_THREAD},
{"buffer_flush_adaptive_avg_time_est", "buffer",
"Estimated time (ms) spent for adaptive flushing recently.",
MONITOR_NONE,
MONITOR_DEFAULT_START, MONITOR_FLUSH_ADAPTIVE_AVG_TIME_EST},
{"buffer_LRU_batch_flush_avg_time_est", "buffer",
"Estimated time (ms) spent for LRU batch flushing recently.",
MONITOR_NONE,
MONITOR_DEFAULT_START, MONITOR_LRU_BATCH_FLUSH_AVG_TIME_EST},
{"buffer_flush_avg_time", "buffer",
"Avg time (ms) spent for flushing recently.",
MONITOR_NONE,
@@ -421,11 +406,6 @@ static monitor_info_t innodb_counter_info[] =
MONITOR_NONE,
MONITOR_DEFAULT_START, MONITOR_FLUSH_ADAPTIVE_AVG_PASS},
{"buffer_LRU_batch_flush_avg_pass", "buffer",
"Number of LRU batch flushes passed during the recent Avg period.",
MONITOR_NONE,
MONITOR_DEFAULT_START, MONITOR_LRU_BATCH_FLUSH_AVG_PASS},
{"buffer_flush_avg_pass", "buffer",
"Number of flushes passed during the recent Avg period.",
MONITOR_NONE,
@@ -562,23 +542,6 @@ static monitor_info_t innodb_counter_info[] =
MONITOR_SET_MEMBER, MONITOR_LRU_BATCH_EVICT_TOTAL_PAGE,
MONITOR_LRU_BATCH_EVICT_PAGES},
/* Cumulative counter for single page LRU scans */
{"buffer_LRU_single_flush_scanned", "buffer",
"Total pages scanned as part of single page LRU flush",
MONITOR_SET_OWNER,
MONITOR_LRU_SINGLE_FLUSH_SCANNED_NUM_CALL,
MONITOR_LRU_SINGLE_FLUSH_SCANNED},
{"buffer_LRU_single_flush_num_scan", "buffer",
"Number of times single page LRU flush is called",
MONITOR_SET_MEMBER, MONITOR_LRU_SINGLE_FLUSH_SCANNED,
MONITOR_LRU_SINGLE_FLUSH_SCANNED_NUM_CALL},
{"buffer_LRU_single_flush_scanned_per_call", "buffer",
"Page scanned per single LRU flush",
MONITOR_SET_MEMBER, MONITOR_LRU_SINGLE_FLUSH_SCANNED,
MONITOR_LRU_SINGLE_FLUSH_SCANNED_PER_CALL},
{"buffer_LRU_single_flush_failure_count", "Buffer",
"Number of times attempt to flush a single page from LRU failed",
MONITOR_NONE,
@@ -1468,7 +1431,8 @@ srv_mon_set_module_control(
ibool set_current_module = FALSE;
ut_a(module_id <= NUM_MONITOR);
ut_a(UT_ARR_SIZE(innodb_counter_info) == NUM_MONITOR);
compile_time_assert(array_elements(innodb_counter_info)
== NUM_MONITOR);
/* The module_id must be an ID of MONITOR_MODULE type */
ut_a(innodb_counter_info[module_id].monitor_type & MONITOR_MODULE);

16
storage/innobase/srv/srv0srv.cc
View File

@@ -342,11 +342,6 @@ my_bool srv_stats_sample_traditional;
my_bool srv_use_doublewrite_buf;
/** innodb_doublewrite_batch_size (a debug parameter) specifies the
number of pages to use in LRU and flush_list batch flushing.
The rest of the doublewrite buffer is used for single-page flushing. */
ulong srv_doublewrite_batch_size = 120;
/** innodb_sync_spin_loops */
ulong srv_n_spin_wait_rounds;
/** innodb_spin_wait_delay */
@@ -707,9 +702,6 @@ static void srv_init()
if (!srv_read_only_mode) {
mutex_create(LATCH_ID_SRV_SYS_TASKS, &srv_sys.tasks_mutex);
buf_flush_event = os_event_create("buf_flush_event");
UT_LIST_INIT(srv_sys.tasks, &que_thr_t::queue);
}
@@ -755,7 +747,6 @@ srv_free(void)
if (!srv_read_only_mode) {
mutex_free(&srv_sys.tasks_mutex);
os_event_destroy(buf_flush_event);
}
ut_d(os_event_destroy(srv_master_thread_disabled_event));
@@ -1119,9 +1110,6 @@ srv_export_innodb_status(void)
export_vars.innodb_buffer_pool_wait_free =
srv_stats.buf_pool_wait_free;
export_vars.innodb_buffer_pool_pages_flushed =
srv_stats.buf_pool_flushed;
export_vars.innodb_buffer_pool_reads = srv_stats.buf_pool_reads;
export_vars.innodb_buffer_pool_read_ahead_rnd =
@@ -1871,10 +1859,6 @@ srv_master_do_idle_tasks(void)
log_checkpoint();
MONITOR_INC_TIME_IN_MICRO_SECS(MONITOR_SRV_CHECKPOINT_MICROSECOND,
counter_time);
/* This is a workaround to avoid the InnoDB hang when OS datetime
changed backwards.*/
os_event_set(buf_flush_event);
}
/**

90
storage/innobase/srv/srv0start.cc
View File

@@ -140,30 +140,8 @@ UNIV_INTERN uint srv_sys_space_size_debug;
UNIV_INTERN bool srv_log_file_created;
#endif /* UNIV_DEBUG */
/** Bit flags for tracking background thread creation. They are used to
determine which threads need to be stopped if we need to abort during
the initialisation step. */
enum srv_start_state_t {
/** No thread started */
SRV_START_STATE_NONE = 0, /*!< No thread started */
/** lock_wait_timeout timer task started */
SRV_START_STATE_LOCK_SYS = 1,
/** buf_flush_page_cleaner_coordinator,
buf_flush_page_cleaner_worker started */
SRV_START_STATE_IO = 2,
/** srv_error_monitor_thread, srv_print_monitor_task started */
SRV_START_STATE_MONITOR = 4,
/** srv_master_thread started */
SRV_START_STATE_MASTER = 8,
/** srv_purge_coordinator_thread, srv_worker_thread started */
SRV_START_STATE_PURGE = 16,
/** fil_crypt_thread,
(all background threads that can generate redo log but not undo log */
SRV_START_STATE_REDO = 32
};
/** Track server thrd starting phases */
static ulint srv_start_state;
/** whether some background threads that create redo log have been started */
static bool srv_started_redo;
/** At a shutdown this value climbs from SRV_SHUTDOWN_NONE to
SRV_SHUTDOWN_CLEANUP and then to SRV_SHUTDOWN_LAST_PHASE, and so on */
@@ -844,30 +822,6 @@ srv_open_tmp_tablespace(bool create_new_db)
return(err);
}
/****************************************************************//**
Set state to indicate start of particular group of threads in InnoDB. */
UNIV_INLINE
void
srv_start_state_set(
/*================*/
srv_start_state_t state) /*!< in: indicate current state of
thread startup */
{
srv_start_state |= ulint(state);
}
/****************************************************************//**
Check if following group of threads is started.
@return true if started */
UNIV_INLINE
bool
srv_start_state_is_set(
/*===================*/
srv_start_state_t state) /*!< in: state to check for */
{
return(srv_start_state & ulint(state));
}
/**
Shutdown all background threads created by InnoDB. */
static
@@ -899,11 +853,11 @@ srv_shutdown_all_bg_threads()
}
}
if (srv_start_state_is_set(SRV_START_STATE_IO)) {
if (buf_page_cleaner_is_active) {
ut_ad(!srv_read_only_mode);
/* e. Exit the i/o threads */
os_event_set(buf_flush_event);
/* e. Exit the buf_flush_page_cleaner */
mysql_cond_signal(&buf_pool.do_flush_list);
}
if (!os_thread_count) {
@@ -1145,8 +1099,7 @@ dberr_t srv_start(bool create_new_db)
|| srv_force_recovery > SRV_FORCE_NO_IBUF_MERGE
|| srv_sys_space.created_new_raw();
/* Reset the start state. */
srv_start_state = SRV_START_STATE_NONE;
srv_started_redo = false;
compile_time_assert(sizeof(ulint) == sizeof(void*));
@@ -1328,7 +1281,7 @@ dberr_t srv_start(bool create_new_db)
if (!srv_read_only_mode) {
buf_flush_page_cleaner_init();
srv_start_state_set(SRV_START_STATE_IO);
ut_ad(buf_page_cleaner_is_active);
}
srv_startup_is_before_trx_rollback_phase = !create_new_db;
@@ -1377,11 +1330,10 @@ dberr_t srv_start(bool create_new_db)
std::string logfile0;
if (create_new_db) {
flushed_lsn = log_sys.get_lsn();
log_sys.set_flushed_lsn(flushed_lsn);
buf_flush_sync();
flushed_lsn = log_get_lsn();
err = create_log_file(flushed_lsn, logfile0);
if (err != DB_SUCCESS) {
@@ -1652,10 +1604,7 @@ file_checked:
}
}
/* recv_recovery_from_checkpoint_finish needs trx lists which
are initialized in trx_lists_init_at_db_start(). */
recv_recovery_from_checkpoint_finish();
recv_sys.debug_free();
if (srv_operation == SRV_OPERATION_RESTORE
|| srv_operation == SRV_OPERATION_RESTORE_EXPORT) {
@@ -1757,7 +1706,7 @@ file_checked:
/* Create the doublewrite buffer to a new tablespace */
if (!srv_read_only_mode && srv_force_recovery < SRV_FORCE_NO_TRX_UNDO
&& !buf_dblwr_create()) {
&& !buf_dblwr.create()) {
return(srv_init_abort(DB_ERROR));
}
@@ -1896,9 +1845,6 @@ file_checked:
srv_start_periodic_timer(srv_error_monitor_timer, srv_error_monitor_task, 1000);
srv_start_periodic_timer(srv_monitor_timer, srv_monitor_task, 5000);
srv_start_state |= SRV_START_STATE_LOCK_SYS
| SRV_START_STATE_MONITOR;
#ifndef DBUG_OFF
skip_monitors:
#endif
@@ -1957,7 +1903,6 @@ skip_monitors:
srv_init_purge_tasks();
purge_sys.coordinator_startup();
srv_wake_purge_thread_if_not_active();
srv_start_state_set(SRV_START_STATE_PURGE);
}
srv_is_being_started = false;
@@ -2016,7 +1961,7 @@ skip_monitors:
/* Initialize online defragmentation. */
btr_defragment_init();
srv_start_state |= SRV_START_STATE_REDO;
srv_started_redo = true;
}
return(DB_SUCCESS);
@@ -2103,7 +2048,8 @@ void innodb_shutdown()
ut_ad(dict_sys.is_initialised() || !srv_was_started);
ut_ad(trx_sys.is_initialised() || !srv_was_started);
ut_ad(buf_dblwr || !srv_was_started || srv_read_only_mode
ut_ad(buf_dblwr.is_initialised() || !srv_was_started
|| srv_read_only_mode
|| srv_force_recovery >= SRV_FORCE_NO_TRX_UNDO);
ut_ad(lock_sys.is_initialised() || !srv_was_started);
ut_ad(log_sys.is_initialised() || !srv_was_started);
@@ -2111,7 +2057,7 @@ void innodb_shutdown()
dict_stats_deinit();
if (srv_start_state_is_set(SRV_START_STATE_REDO)) {
if (srv_started_redo) {
ut_ad(!srv_read_only_mode);
/* srv_shutdown_bg_undo_sources() already invoked
fts_optimize_shutdown(); dict_stats_shutdown(); */
@@ -2132,9 +2078,7 @@ void innodb_shutdown()
log_sys.close();
purge_sys.close();
trx_sys.close();
if (buf_dblwr) {
buf_dblwr_free();
}
buf_dblwr.close();
lock_sys.close();
trx_pool_close();
@@ -2161,7 +2105,7 @@ void innodb_shutdown()
<< "; transaction id " << trx_sys.get_max_trx_id();
}
srv_thread_pool_end();
srv_start_state = SRV_START_STATE_NONE;
srv_started_redo = false;
srv_was_started = false;
srv_start_has_been_called = false;
}

29
storage/innobase/sync/sync0debug.cc
View File

@@ -453,9 +453,6 @@ LatchDebug::LatchDebug()
LEVEL_MAP_INSERT(RW_LOCK_X);
LEVEL_MAP_INSERT(RW_LOCK_NOT_LOCKED);
LEVEL_MAP_INSERT(SYNC_ANY_LATCH);
LEVEL_MAP_INSERT(SYNC_DOUBLEWRITE);
LEVEL_MAP_INSERT(SYNC_BUF_FLUSH_LIST);
LEVEL_MAP_INSERT(SYNC_BUF_POOL);
LEVEL_MAP_INSERT(SYNC_POOL);
LEVEL_MAP_INSERT(SYNC_POOL_MANAGER);
LEVEL_MAP_INSERT(SYNC_SEARCH_SYS);
@@ -467,7 +464,6 @@ LatchDebug::LatchDebug()
LEVEL_MAP_INSERT(SYNC_RECV);
LEVEL_MAP_INSERT(SYNC_LOG_FLUSH_ORDER);
LEVEL_MAP_INSERT(SYNC_LOG);
LEVEL_MAP_INSERT(SYNC_PAGE_CLEANER);
LEVEL_MAP_INSERT(SYNC_PURGE_QUEUE);
LEVEL_MAP_INSERT(SYNC_TRX_SYS_HEADER);
LEVEL_MAP_INSERT(SYNC_REC_LOCK);
@@ -741,10 +737,8 @@ LatchDebug::check_order(
case SYNC_FTS_OPTIMIZE:
case SYNC_FTS_CACHE:
case SYNC_FTS_CACHE_INIT:
case SYNC_PAGE_CLEANER:
case SYNC_LOG:
case SYNC_LOG_FLUSH_ORDER:
case SYNC_DOUBLEWRITE:
case SYNC_SEARCH_SYS:
case SYNC_LOCK_SYS:
case SYNC_LOCK_WAIT_SYS:
@@ -802,15 +796,6 @@ LatchDebug::check_order(
}
break;
case SYNC_BUF_FLUSH_LIST:
case SYNC_BUF_POOL:
/* We can have multiple mutexes of this type therefore we
can only check whether the greater than condition holds. */
basic_check(latches, level, level - 1);
break;
case SYNC_REC_LOCK:
if (find(latches, SYNC_LOCK_SYS) != 0) {
@@ -1169,8 +1154,7 @@ sync_check_iterate(const sync_check_functor_t& functor)
Note: We don't enforce any synchronisation checks. The caller must ensure
that no races can occur */
void
sync_check_enable()
static void sync_check_enable()
{
if (!srv_sync_debug) {
@@ -1243,8 +1227,6 @@ sync_latch_meta_init()
/* The latches should be ordered on latch_id_t. So that we can
index directly into the vector to update and fetch meta-data. */
LATCH_ADD_MUTEX(BUF_POOL, SYNC_BUF_POOL, buf_pool_mutex_key);
LATCH_ADD_MUTEX(DICT_FOREIGN_ERR, SYNC_NO_ORDER_CHECK,
dict_foreign_err_mutex_key);
@@ -1252,8 +1234,6 @@ sync_latch_meta_init()
LATCH_ADD_MUTEX(FIL_SYSTEM, SYNC_ANY_LATCH, fil_system_mutex_key);
LATCH_ADD_MUTEX(FLUSH_LIST, SYNC_BUF_FLUSH_LIST, flush_list_mutex_key);
LATCH_ADD_MUTEX(FTS_BG_THREADS, SYNC_FTS_BG_THREADS,
fts_bg_threads_mutex_key);
@@ -1277,9 +1257,6 @@ sync_latch_meta_init()
LATCH_ADD_MUTEX(LOG_FLUSH_ORDER, SYNC_LOG_FLUSH_ORDER,
log_flush_order_mutex_key);
LATCH_ADD_MUTEX(PAGE_CLEANER, SYNC_PAGE_CLEANER,
page_cleaner_mutex_key);
LATCH_ADD_MUTEX(PURGE_SYS_PQ, SYNC_PURGE_QUEUE,
purge_sys_pq_mutex_key);
@@ -1320,8 +1297,6 @@ sync_latch_meta_init()
LATCH_ADD_MUTEX(SRV_MONITOR_FILE, SYNC_NO_ORDER_CHECK,
srv_monitor_file_mutex_key);
LATCH_ADD_MUTEX(BUF_DBLWR, SYNC_DOUBLEWRITE, buf_dblwr_mutex_key);
LATCH_ADD_MUTEX(TRX_POOL, SYNC_POOL, trx_pool_mutex_key);
LATCH_ADD_MUTEX(TRX_POOL_MANAGER, SYNC_POOL_MANAGER,
@@ -1456,6 +1431,8 @@ sync_check_init()
ut_d(LatchDebug::init());
sync_array_init();
ut_d(sync_check_enable());
}
/** Free the InnoDB synchronization data structures. */

1
storage/innobase/sync/sync0sync.cc
View File

@@ -52,7 +52,6 @@ mysql_pfs_key_t log_sys_mutex_key;
mysql_pfs_key_t log_cmdq_mutex_key;
mysql_pfs_key_t log_flush_order_mutex_key;
mysql_pfs_key_t recalc_pool_mutex_key;
mysql_pfs_key_t page_cleaner_mutex_key;
mysql_pfs_key_t purge_sys_pq_mutex_key;
mysql_pfs_key_t recv_sys_mutex_key;
mysql_pfs_key_t redo_rseg_mutex_key;

4
storage/innobase/trx/trx0purge.cc
View File

@@ -262,7 +262,7 @@ trx_purge_add_undo_to_history(const trx_t* trx, trx_undo_t*& undo, mtr_t* mtr)
or in trx_rollback_recovered() in slow shutdown.
Before any transaction-generating background threads or the
purge have been started, recv_recovery_rollback_active() can
purge have been started, we can
start transactions in row_merge_drop_temp_indexes() and
fts_drop_orphaned_tables(), and roll back recovered transactions.
@@ -680,7 +680,7 @@ not_free:
mini-transaction commit and the server was killed, then
discarding the to-be-trimmed pages without flushing would
break crash recovery. So, we cannot avoid the write. */
buf_LRU_flush_or_remove_pages(space.id, true);
while (buf_flush_dirty_pages(space.id));
log_free_check();

7
storage/rocksdb/mysql-test/rocksdb/r/innodb_i_s_tables_disabled.result
View File

@@ -72,14 +72,10 @@ buffer_flush_neighbor_pages buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL N
buffer_flush_n_to_flush_requested buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of pages requested for flushing.
buffer_flush_n_to_flush_by_age buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of pages target by LSN Age for flushing.
buffer_flush_adaptive_avg_time_slot buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for adaptive flushing recently per slot.
buffer_LRU_batch_flush_avg_time_slot buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for LRU batch flushing recently per slot.
buffer_flush_adaptive_avg_time_thread buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for adaptive flushing recently per thread.
buffer_LRU_batch_flush_avg_time_thread buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for LRU batch flushing recently per thread.
buffer_flush_adaptive_avg_time_est buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Estimated time (ms) spent for adaptive flushing recently.
buffer_LRU_batch_flush_avg_time_est buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Estimated time (ms) spent for LRU batch flushing recently.
buffer_flush_avg_time buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Avg time (ms) spent for flushing recently.
buffer_flush_adaptive_avg_pass buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of adaptive flushes passed during the recent Avg period.
buffer_LRU_batch_flush_avg_pass buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of LRU batch flushes passed during the recent Avg period.
buffer_flush_avg_pass buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of flushes passed during the recent Avg period.
buffer_LRU_get_free_loops buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Total loops in LRU get free.
buffer_LRU_get_free_waits buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Total sleep waits in LRU get free.
@@ -106,9 +102,6 @@ buffer_LRU_batch_flush_pages buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL
buffer_LRU_batch_evict_total_pages buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_owner Total pages evicted as part of LRU batches
buffer_LRU_batches_evict buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_member Number of LRU batches
buffer_LRU_batch_evict_pages buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_member Pages queued as an LRU batch
buffer_LRU_single_flush_scanned buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_owner Total pages scanned as part of single page LRU flush
buffer_LRU_single_flush_num_scan buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_member Number of times single page LRU flush is called
buffer_LRU_single_flush_scanned_per_call buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_member Page scanned per single LRU flush
buffer_LRU_single_flush_failure_count Buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of times attempt to flush a single page from LRU failed
buffer_LRU_get_free_search Buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of searches performed for a clean page
buffer_LRU_search_scanned buffer 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 set_owner Total pages scanned as part of LRU search