database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Code Activity

MDEV-32286 Reuse remembered clustered leaves in secondary-index scans

Browse Source 
Row_sel_get_clust_rec_for_mysql::operator() descends the clustered B-tree
from the root for every row whose clustered-index record a secondary-index
scan must read, although consecutive rows land on the same clustered leaf
page wherever the secondary order tracks the clustered one. A non-covering
scan needs one for every row, a locking read needs one whatever the
secondary index holds, because an exclusive select lock type makes
ha_innobase::build_template() build its template against the clustered
index, and a covering scan needs one for every row of a secondary leaf
whose PAGE_MAX_TRX_ID its read view cannot see. ANALYZE FORMAT=JSON
charges each descent its full height, and those descents are nearly the
whole cost: the secondary index is charged its own descent and one page
for each further leaf, and nothing per row, because the position that its
cursor holds between two rows is restored optimistically, which latches
the leaf again without counting an access. So pages_accessed is the row
count times the height of the clustered index, plus a handful: 1000 rows
over a 2-level clustered index cost 2006 and 750 rows over a 3-level one
cost 2291, where a full table scan of the same data costs 23 and 110.

Let a handle remember the clustered leaves that the lookups of one
statement reached, and let the next lookup try them before it descends
again. The reasoning behind each value and each rejection is in the
comments beside it.

row0mysql.h defines clust_leaf_hint_slot, which names one leaf: its page
number, copies of its first and last user record truncated to the key
fields, which bound the key range that the leaf held when it was
remembered, and the rec_get_offsets() of both. A slot of a
leaf that had no right sibling names no last key, because every key above
the last record of the rightmost leaf still belongs to it.
CLUST_LEAF_HINT_SLOTS (4) slots hang off the new
row_prebuilt_t::clust_leaf_hint, beside clust_leaf_hint_mru, the most
recently used order held as slot numbers, and clust_leaf_hint_n and
clust_leaf_hint_miss, the used-slot count and the miss counter.

row0sel.cc holds the policy. row_sel_clust_leaf_hint_covers() compares a
key against the remembered ranges, so a lookup that no slot can answer
costs no buffer pool access and no pages_accessed.
row_sel_clust_leaf_hint_search() probes the first slot that covers the key
and moves it to the front of the order.
row_sel_clust_leaf_hint_remember() records the leaf that a descent landed
on, and refreshes the slot of a leaf that is remembered already rather
than spend a second one on the same page. Two descents fill no slot: the
lookups of a statement before lookup CLUST_LEAF_HINT_MIN_LOOKUPS (4), and
a leaf that is the root. row_sel_clust_leaf_hint_armed() stands a scan
down once the slots stop paying for themselves: a miss adds
CLUST_LEAF_HINT_MISS_WEIGHT (2) to the miss counter and a hit takes one
away, so a scan gives the slots up where it answers too little of its
lookups to pay for them, CLUST_LEAF_HINT_MAX_MISSES (8) misses with no hit
between them still reach the threshold, and one lookup in
CLUST_LEAF_HINT_RETRY (1024) starts the count again, so a scan whose order
becomes correlated only later recovers. Both halves of the cost stop
there, the test of the slots and the copies that refresh them.
Row_sel_get_clust_rec_for_mysql::operator() calls all of this in place of
its btr_pcur_open_with_no_init(), and only where the adaptive hash index
is disabled. A guess of that index lands on the record with no page-local
search and no page access to charge, but neither of the two is faster for
every shape of index and scan, and a hint that runs first stops that index
from adding entries. The adaptive hash index is off by default, so the
hints are active in a default configuration.

btr0cur.h and btr0cur.cc add btr_cur_t::try_leaf_hint(), a PAGE_CUR_LE,
BTR_SEARCH_LEAF search on one named leaf, which starts from the record
where the last search of that leaf landed while that record is still valid,
and from a binary search of the page otherwise. It acquires the page with
buf_page_try_get(): a hint is never derived from a latched parent page,
so by the time it is tried it can precede the caller's already-latched
secondary-index leaf in the latching order, where a blocking wait can
deadlock. It then rejects the page unless the checks that it makes on the
latched frame put the match on it. Those checks are the sole authority on
the result, so a stale range costs a wasted probe or a needless descent,
never a wrong result, and the ranges need no invalidation protocol.

A slot also holds a btr_leaf_step, where the last search of its leaf
landed: the block, its buf_block_t::modify_clock() and the record. A
record pointer stays valid while the block holds the same page and the
clock has not moved, because eviction, deletion and reorganization
advance the clock and an insertion moves no record. While the step is
valid, btr_cur_t::try_leaf_hint() starts from that record instead of a
binary search of the page. page0cur.cc adds page_cur_search_near(),
which compares the key with the record and with one neighbour, and
positions the cursor on the record, on the record after it or on the
record before it. It refuses any other answer and leaves it to
page_cur_search_with_match(). So a scan whose consecutive rows are
neighbours in the clustered index finds each row with two comparisons. A
scan that reads the clustered leaf in falling key order does the same: a
secondary key that falls as the primary key rises, or a secondary index
read backward. Where the secondary order tracks the clustered one with
gaps between the rows, the step fails, and btr_leaf_step::expect (below)
leaves the leaf to the binary search. The step refuses the metadata
pseudo-record, which compares below every key on 0 fields, and a record
before it that is the infimum. A debug build checks every successful
step against the binary search.

A failed step costs comparisons whose outcome no branch predictor can
guess, so btr_leaf_step::expect gates it. The next search tries the step
only where this one landed where a step would have: on the remembered
record, or right before or after it. The successor link of the landing
record tells the record right before, because finding a predecessor takes
a walk of the page directory. So a scan in random order over a few leaves,
which the slots answer in full, keeps the binary search and pays only the
tests that set expect. row_sel_clust_leaf_hint_remember() sets the step on
the record where a descent landed, with expect set.

ha_innodb.cc: ha_innobase::reset() zeroes the used-slot count and the miss
counter per statement, matching autoinc_last_value. row0mysql.cc:
row_prebuilt_free() frees the key buffers that the slots own.

innodb.clust_leaf_hint measures pages_accessed over key orders that differ
in how closely the secondary order tracks the clustered one, and further
tables check query results over the record formats and key shapes
that a clustered-index lookup has to read, down to the metadata
pseudo-record of instant ALTER TABLE, to leaves that split and merge while
a locking read walks them, and to a record that a remembered leaf supplies
for a scan that must then rebuild an older version of it. Two of the
tables scan a covering index, which reads a clustered record under an
exclusive select lock type and under a PAGE_MAX_TRX_ID that the read view
cannot see. clust_leaf_hint_off_debug runs the same body with the hints
turned off, through a debug switch that returns before a lookup tests or
refreshes the slots, so a diff of the two .result files is what the hints
save: 2006 to 1031 (2-level clustered index), 2291 to 1011 (3-level), 4006
to 2015 (two interleaved key ranges), 12016 to 9078 (locality in the
second half alone) and 20020 to 10045 for a covering scan that FOR UPDATE
makes non-covering, where the same scan without FOR UPDATE costs 20 in
both files. Two orders with too little locality to pay for the slots give
them up early and end within a hundred accesses of the unhinted count:
20020 to 19966 (decorrelated) and 20020 to 19999 (shuffled).

Two further tables in innodb.clust_leaf_hint cover the step back. t16
scans a secondary index whose keys fall as the primary key rises, and
reads an ascending secondary index backward: 20020 to 10047 for both. t17
runs the inverse scan on ROW_FORMAT=REDUNDANT with an instantly added
column. innodb.clust_leaf_hint_backward_debug reaches the two refusals of
the step back that no count shows, at the infimum and at the metadata
pseudo-record. A lookup reaches either only for a key that left the
clustered index while the slot still covers it. The rollback of an insert
over a delete-marked record removes the key while purge leaves the
secondary entry, and a PAGE_MAX_TRX_ID that the read view cannot see makes
the reader look that entry up. A debug switch writes each refusal to the
error log, and the case reads both back.

A slot stays valid when dict_index_t::clear_instant_alter() changes the
index under a reader that holds a shared metadata lock, between two
lookups or during one. That clear runs where purge or rollback removes
the last user record of a single-page table whose only instant change was
ADD COLUMN, and it raises dict_index_t::n_core_fields.
rec_copy_prefix_to_buf() writes the copies in the format of an index
without instant ADD COLUMN, their offsets are made with them, and the
comparisons depend on the key fields alone, which no instant change moves.
So the slots do not record n_core_fields, and the hints read it only under
a latch on a clustered leaf.

innodb.clust_leaf_hint_instant_alter makes a lookup compare against a slot
that a clear has outlived. The clear needs the clustered index to lose the
last user record of its root page, while no leaf that is the root fills a
slot, so the tree has to shrink between the two, which purge does there.
The reader therefore reads uncommitted rows at READ UNCOMMITTED and waits
in a stored function while a rollback and purge take them away, and one
row that arrives above the position it stopped at is the lookup that
tests the slots.

main.rowid_filter_innodb: 90 to 88, and its ahi combination unchanged.

A static_assert in Row_sel_get_clust_rec_for_mysql::operator() fails to
compile if btr_search.enabled stops being a Boolean that applies to every
index, because the test !btr_search.enabled that stands the hints down
would then have to become !btr_search.is_enabled(clust_index).
This commit is contained in:
Alessandro Vetere
2026-09-10 11:44:29 +02:00
parent 9a1123dc45
commit 6838067af5
19 changed files with 2359 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
mysql-test/main/rowid_filter_innodb,ahi.rdiff
View File

@@ -1,5 +1,5 @@
--- rowid_filter_innodb.result
+++ rowid_filter_innodb,ahi.result
+++ rowid_filter_innodb,ahi.reject
@@ -2,7 +2,7 @@
SET @save_stats_persistent=@@GLOBAL.innodb_stats_persistent;
SET GLOBAL innodb_stats_persistent=0;
@@ -9,20 +9,20 @@
DROP DATABASE IF EXISTS dbt3_s001;
CREATE DATABASE dbt3_s001;
use dbt3_s001;
@@ -2059,7 +2059,7 @@
@@ -2061,7 +2061,7 @@
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"r_engine_stats": {
- "pages_accessed": 90
- "pages_accessed": 88
+ "pages_accessed": 51
},
"filtered": "REPLACED",
"r_total_filtered": 2.43902439,
@@ -2227,7 +2227,7 @@
@@ -2229,7 +2229,7 @@
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"r_engine_stats": {
- "pages_accessed": 90
- "pages_accessed": 88
+ "pages_accessed": 49
},
"filtered": "REPLACED",

4
mysql-test/main/rowid_filter_innodb.result
View File

@@ -2061,7 +2061,7 @@ ANALYZE
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"r_engine_stats": {
"pages_accessed": 90
"pages_accessed": 88
},
"filtered": "REPLACED",
"r_total_filtered": 2.43902439,
@@ -2229,7 +2229,7 @@ ANALYZE
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"r_engine_stats": {
"pages_accessed": 90
"pages_accessed": 88
},
"filtered": "REPLACED",
"r_total_filtered": 2.43902439,

301
mysql-test/suite/innodb/r/clust_leaf_hint.result Normal file
View File

@@ -0,0 +1,301 @@
create table t1 (
pk int not null primary key,
domain_grp int,
val int
) engine=innodb row_format=dynamic;
insert into t1 select seq, mod(seq,10), seq from seq_1_to_10000;
create index domain_idx on t1(domain_grp);
analyze table t1 persistent for all;
Table Op Msg_type Msg_text
test.t1 analyze status Engine-independent statistics collected
test.t1 analyze status OK
set @js='$out_scan';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_FULL_SCAN;
PAGES_ACCESSED_FULL_SCAN
23
set @js='$out_idx';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SEC_INDEX;
PAGES_ACCESSED_SEC_INDEX
1031
drop table t1;
create table t2 (
pk varchar(500) character set utf8mb4 collate utf8mb4_general_ci
not null primary key,
domain_grp int,
val int
) engine=innodb row_format=dynamic;
insert into t2 select lpad(seq,500,'0'), mod(seq,4), seq from seq_1_to_3000;
create index domain_idx on t2(domain_grp);
analyze table t2 persistent for all;
Table Op Msg_type Msg_text
test.t2 analyze status Engine-independent statistics collected
test.t2 analyze status OK
select stat_value into @leaf from mysql.innodb_index_stats
where database_name='test' and table_name='t2' and index_name='PRIMARY'
and stat_name='n_leaf_pages';
select @leaf > 100 as THREE_LEVEL_TREE;
THREE_LEVEL_TREE
1
set @js='$out_scan2';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_FULL_SCAN_3LEVEL;
PAGES_ACCESSED_FULL_SCAN_3LEVEL
110
set @js='$out_idx2';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SEC_INDEX_3LEVEL;
PAGES_ACCESSED_SEC_INDEX_3LEVEL
1011
drop table t2;
create table t3 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t3 select seq, mod(seq*997+13,10007), seq from seq_1_to_10000;
create index k_idx on t3(k);
analyze table t3 persistent for all;
Table Op Msg_type Msg_text
test.t3 analyze status Engine-independent statistics collected
test.t3 analyze status OK
set @js='$out_uncorr';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_UNCORRELATED;
PAGES_ACCESSED_UNCORRELATED
19966
drop table t3;
create table t4 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t4 select seq, 2*seq-1, seq from seq_1_to_1000;
insert into t4 select 1000000+seq, 2*seq, seq from seq_1_to_1000;
create index k_idx on t4(k);
analyze table t4 persistent for all;
Table Op Msg_type Msg_text
test.t4 analyze status Engine-independent statistics collected
test.t4 analyze status OK
set @js='$out_interleaved';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_INTERLEAVED;
PAGES_ACCESSED_INTERLEAVED
2015
drop table t4;
create table t5 (
pk int not null primary key,
k int unsigned,
val int
) engine=innodb row_format=dynamic;
insert into t5 select seq, crc32(seq), seq from seq_1_to_10000;
create index k_idx on t5(k);
analyze table t5 persistent for all;
Table Op Msg_type Msg_text
test.t5 analyze status Engine-independent statistics collected
test.t5 analyze status OK
set @js='$out_shuffled';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SHUFFLED;
PAGES_ACCESSED_SHUFFLED
19999
drop table t5;
create table t6 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t6 select seq, mod(seq,10), seq from seq_1_to_10000;
alter table t6 add column c int default 42, algorithm=instant;
create index k_idx on t6(k);
select count(val) as ROWS_INSTANT, sum(val) as SUM_INSTANT, min(c) as DEFAULT_C
from t6 force index(k_idx) where k=3;
ROWS_INSTANT SUM_INSTANT DEFAULT_C
1000 4998000 42
drop table t6;
create table t7 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=redundant;
insert into t7 select seq, mod(seq,10), seq from seq_1_to_10000;
alter table t7 add column c int default 42, algorithm=instant;
create index k_idx on t7(k);
select count(val) as ROWS_REDUNDANT, sum(val) as SUM_REDUNDANT
from t7 force index(k_idx) where k=3;
ROWS_REDUNDANT SUM_REDUNDANT
1000 4998000
drop table t7;
create table t8 (
pk int not null primary key,
k int,
val int,
drop_me int
) engine=innodb row_format=dynamic;
insert into t8 select seq, mod(seq,10), seq, seq from seq_1_to_10000;
alter table t8 drop column drop_me, algorithm=instant;
create index k_idx on t8(k);
select count(val) as ROWS_DROPPED, sum(val) as SUM_DROPPED
from t8 force index(k_idx) where k=3;
ROWS_DROPPED SUM_DROPPED
1000 4998000
drop table t8;
create table t9 (
pk int not null primary key,
k int,
val int,
drop_me int
) engine=innodb row_format=redundant;
insert into t9 select seq, mod(seq,10), seq, seq from seq_1_to_10000;
alter table t9 drop column drop_me, algorithm=instant;
create index k_idx on t9(k);
select count(val) as ROWS_REDUNDANT_DROPPED, sum(val) as SUM_REDUNDANT_DROPPED
from t9 force index(k_idx) where k=3;
ROWS_REDUNDANT_DROPPED SUM_REDUNDANT_DROPPED
1000 4998000
drop table t9;
create table t10 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=compressed key_block_size=8;
insert into t10 select seq, mod(seq,10), seq from seq_1_to_10000;
create index k_idx on t10(k);
select count(val) as ROWS_COMPRESSED, sum(val) as SUM_COMPRESSED
from t10 force index(k_idx) where k=3;
ROWS_COMPRESSED SUM_COMPRESSED
1000 4998000
drop table t10;
create table t11 (
k int,
val int
) engine=innodb row_format=dynamic;
insert into t11 select mod(seq,10), seq from seq_1_to_10000;
create index k_idx on t11(k);
select count(val) as ROWS_ROW_ID, sum(val) as SUM_ROW_ID
from t11 force index(k_idx) where k=3;
ROWS_ROW_ID SUM_ROW_ID
1000 4998000
drop table t11;
create table t12 (
pk int not null primary key,
k int,
val varchar(255)
) engine=innodb row_format=dynamic;
insert into t12 select seq, seq, repeat('x',30) from seq_1_to_10000;
create index k_idx on t12(k);
update t12 force index(k_idx) set val=repeat('y',200) where k > 0;
select count(*) as ROWS_GROWN, sum(pk) as SUM_PK from t12
where val=repeat('y',200);
ROWS_GROWN SUM_PK
10000 50005000
update t12 force index(k_idx) set val=repeat('x',30) where k > 0;
select count(*) as ROWS_SHRUNK, sum(pk) as SUM_PK from t12
where val=repeat('x',30);
ROWS_SHRUNK SUM_PK
10000 50005000
check table t12;
Table Op Msg_type Msg_text
test.t12 check status OK
drop table t12;
create table t13 (
pk int not null primary key,
k bigint unsigned,
val int
) engine=innodb row_format=dynamic;
insert into t13 select seq, crc32(seq), seq from seq_1_to_3000;
insert into t13 select 3000+seq, 4294967296+seq, seq from seq_1_to_3000;
create index k_idx on t13(k);
analyze table t13 persistent for all;
Table Op Msg_type Msg_text
test.t13 analyze status Engine-independent statistics collected
test.t13 analyze status OK
set @js='$out_split';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_LATE_LOCALITY;
PAGES_ACCESSED_LATE_LOCALITY
9078
drop table t13;
create table t14 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t14 select seq, seq, seq from seq_1_to_10000;
create index k_idx on t14(k);
analyze table t14 persistent for all;
Table Op Msg_type Msg_text
test.t14 analyze status Engine-independent statistics collected
test.t14 analyze status OK
set @js='$out_cov';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_COVERING;
PAGES_ACCESSED_COVERING
20
set @js='$out_cov_x';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_COVERING_FOR_UPDATE;
PAGES_ACCESSED_COVERING_FOR_UPDATE
10045
drop table t14;
create table t15 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t15 select seq, seq, seq from seq_1_to_10000;
create index k_idx on t15(k);
start transaction with consistent snapshot;
connect con2,localhost,root,,test;
update t15 set k=k+100000 where pk <= 5000;
disconnect con2;
connection default;
select count(k) as ROWS_SNAPSHOT, sum(k) as SUM_SNAPSHOT
from t15 force index(k_idx) where k > 0;
ROWS_SNAPSHOT SUM_SNAPSHOT
10000 50005000
commit;
drop table t15;
create table t16 (
pk int not null primary key,
k_inv int,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t16 select seq, 10001-seq, seq, seq from seq_1_to_10000;
create index k_inv_idx on t16(k_inv);
create index k_idx on t16(k);
analyze table t16 persistent for all;
Table Op Msg_type Msg_text
test.t16 analyze status Engine-independent statistics collected
test.t16 analyze status OK
set @js='$out_inv';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_INVERSE;
PAGES_ACCESSED_INVERSE
10047
set @js='$out_desc';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_DESCENDING;
PAGES_ACCESSED_DESCENDING
10047
select val as LAST_ROW_DESCENDING from t16 force index(k_idx)
where k > 0 order by k desc limit 9999,1;
LAST_ROW_DESCENDING
1
drop table t16;
create table t17 (
pk int not null primary key,
k_inv int,
val int
) engine=innodb row_format=redundant;
insert into t17 select seq, 10001-seq, seq from seq_1_to_10000;
alter table t17 add column c int default 42, algorithm=instant;
create index k_inv_idx on t17(k_inv);
select count(val) as ROWS_REDUNDANT_INVERSE, sum(val) as SUM_REDUNDANT_INVERSE,
min(c) as DEFAULT_C_INVERSE
from t17 force index(k_inv_idx) where k_inv > 0;
ROWS_REDUNDANT_INVERSE SUM_REDUNDANT_INVERSE DEFAULT_C_INVERSE
10000 50005000 42
drop table t17;

50
mysql-test/suite/innodb/r/clust_leaf_hint_backward_debug.result Normal file
View File

@@ -0,0 +1,50 @@
create table t1 (
pk int not null primary key,
k int,
val text,
key k_idx(k)
) engine=innodb row_format=dynamic stats_persistent=0 comment='MERGE_THRESHOLD=1';
set @save_limit= @@global.innodb_limit_optimistic_insert_debug;
set global innodb_limit_optimistic_insert_debug= 100;
insert into t1 select seq, 2*(20001-seq), 'v' from seq_1_to_10000;
set global innodb_limit_optimistic_insert_debug= @save_limit;
alter table t1 add column extra int, algorithm=instant;
set global innodb_max_purge_lag_wait=0;
set @save_view_update= @@global.innodb_trx_purge_view_update_only_debug;
set global innodb_trx_purge_view_update_only_debug= 1;
delete from t1 where pk in (1, 4951);
connect ins,localhost,root;
begin;
insert into t1 (pk, k, val) values (1, 2*(20001-1)+1, 'w'), (4951, 2*(20001-4951)+1, 'w');
connection default;
set statement max_statement_time=1 for set global innodb_max_purge_lag_wait=0;
create function hold(x int) returns int
begin
if x = 4956 then
set debug_sync= 'now SIGNAL reader_paused';
set debug_sync= 'now WAIT_FOR reader_go';
end if;
return 1;
end|
connect reader,localhost,root;
set session optimizer_switch='index_condition_pushdown=off';
set session debug_dbug='+d,ib_log_clust_leaf_hint_step_back';
select count(left(val,1)) as n, sum(pk) as s from t1 force index(k_idx) where k > 0 and hold(pk);
connection default;
set debug_sync='now WAIT_FOR reader_paused';
connection ins;
rollback;
connection default;
set debug_sync='now SIGNAL reader_go';
connection reader;
n s
9998 50000048
connection default;
Clustered leaf hint step back reached the infimum
Clustered leaf hint step back reached the metadata record
disconnect reader;
disconnect ins;
set global innodb_trx_purge_view_update_only_debug= @save_view_update;
drop function hold;
drop table t1;
set debug_sync='reset';

58
mysql-test/suite/innodb/r/clust_leaf_hint_instant_alter.result Normal file
View File

@@ -0,0 +1,58 @@
create table t1 (
pk int not null primary key,
domain_grp int,
val text,
key domain_idx(domain_grp)
) engine=innodb row_format=dynamic stats_persistent=0;
insert into t1 select seq, 9, repeat('f', 4000) from seq_1_to_500;
alter table t1 add column extra int, algorithm=instant;
connect prevent_purge,localhost,root;
start transaction with consistent snapshot;
connection default;
delete from t1 where domain_grp=9;
connect ins,localhost,root;
begin;
insert into t1 (pk, domain_grp, val) select 1000000+seq, 1, 'v' from seq_1_to_8;
connection default;
create function hold(x int) returns int
begin
if x = 1000008 then
set debug_sync= 'now SIGNAL reader_paused';
set debug_sync= 'now WAIT_FOR reader_go';
end if;
return 1;
end|
connect reader,localhost,root;
set session transaction isolation level read uncommitted;
set session optimizer_switch='index_condition_pushdown=off';
select pk, left(val,1) from t1 force index(domain_idx) where domain_grp=1 and hold(pk);
connection default;
set debug_sync='now WAIT_FOR reader_paused';
connection prevent_purge;
commit;
connection default;
set global innodb_max_purge_lag_wait=0;
InnoDB 0 transactions not purged
connection ins;
rollback;
connection default;
insert into t1 values (1000009, 1, 'w', null);
set debug_sync='now SIGNAL reader_go';
connection reader;
pk left(val,1)
1000001 v
1000002 v
1000003 v
1000004 v
1000005 v
1000006 v
1000007 v
1000008 v
1000009 w
connection default;
disconnect reader;
disconnect ins;
disconnect prevent_purge;
drop function hold;
drop table t1;
set debug_sync='reset';

302
mysql-test/suite/innodb/r/clust_leaf_hint_off_debug.result Normal file
View File

@@ -0,0 +1,302 @@
# clustered leaf hints disabled
create table t1 (
pk int not null primary key,
domain_grp int,
val int
) engine=innodb row_format=dynamic;
insert into t1 select seq, mod(seq,10), seq from seq_1_to_10000;
create index domain_idx on t1(domain_grp);
analyze table t1 persistent for all;
Table Op Msg_type Msg_text
test.t1 analyze status Engine-independent statistics collected
test.t1 analyze status OK
set @js='$out_scan';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_FULL_SCAN;
PAGES_ACCESSED_FULL_SCAN
23
set @js='$out_idx';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SEC_INDEX;
PAGES_ACCESSED_SEC_INDEX
2006
drop table t1;
create table t2 (
pk varchar(500) character set utf8mb4 collate utf8mb4_general_ci
not null primary key,
domain_grp int,
val int
) engine=innodb row_format=dynamic;
insert into t2 select lpad(seq,500,'0'), mod(seq,4), seq from seq_1_to_3000;
create index domain_idx on t2(domain_grp);
analyze table t2 persistent for all;
Table Op Msg_type Msg_text
test.t2 analyze status Engine-independent statistics collected
test.t2 analyze status OK
select stat_value into @leaf from mysql.innodb_index_stats
where database_name='test' and table_name='t2' and index_name='PRIMARY'
and stat_name='n_leaf_pages';
select @leaf > 100 as THREE_LEVEL_TREE;
THREE_LEVEL_TREE
1
set @js='$out_scan2';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_FULL_SCAN_3LEVEL;
PAGES_ACCESSED_FULL_SCAN_3LEVEL
110
set @js='$out_idx2';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SEC_INDEX_3LEVEL;
PAGES_ACCESSED_SEC_INDEX_3LEVEL
2291
drop table t2;
create table t3 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t3 select seq, mod(seq*997+13,10007), seq from seq_1_to_10000;
create index k_idx on t3(k);
analyze table t3 persistent for all;
Table Op Msg_type Msg_text
test.t3 analyze status Engine-independent statistics collected
test.t3 analyze status OK
set @js='$out_uncorr';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_UNCORRELATED;
PAGES_ACCESSED_UNCORRELATED
20020
drop table t3;
create table t4 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t4 select seq, 2*seq-1, seq from seq_1_to_1000;
insert into t4 select 1000000+seq, 2*seq, seq from seq_1_to_1000;
create index k_idx on t4(k);
analyze table t4 persistent for all;
Table Op Msg_type Msg_text
test.t4 analyze status Engine-independent statistics collected
test.t4 analyze status OK
set @js='$out_interleaved';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_INTERLEAVED;
PAGES_ACCESSED_INTERLEAVED
4006
drop table t4;
create table t5 (
pk int not null primary key,
k int unsigned,
val int
) engine=innodb row_format=dynamic;
insert into t5 select seq, crc32(seq), seq from seq_1_to_10000;
create index k_idx on t5(k);
analyze table t5 persistent for all;
Table Op Msg_type Msg_text
test.t5 analyze status Engine-independent statistics collected
test.t5 analyze status OK
set @js='$out_shuffled';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SHUFFLED;
PAGES_ACCESSED_SHUFFLED
20020
drop table t5;
create table t6 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t6 select seq, mod(seq,10), seq from seq_1_to_10000;
alter table t6 add column c int default 42, algorithm=instant;
create index k_idx on t6(k);
select count(val) as ROWS_INSTANT, sum(val) as SUM_INSTANT, min(c) as DEFAULT_C
from t6 force index(k_idx) where k=3;
ROWS_INSTANT SUM_INSTANT DEFAULT_C
1000 4998000 42
drop table t6;
create table t7 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=redundant;
insert into t7 select seq, mod(seq,10), seq from seq_1_to_10000;
alter table t7 add column c int default 42, algorithm=instant;
create index k_idx on t7(k);
select count(val) as ROWS_REDUNDANT, sum(val) as SUM_REDUNDANT
from t7 force index(k_idx) where k=3;
ROWS_REDUNDANT SUM_REDUNDANT
1000 4998000
drop table t7;
create table t8 (
pk int not null primary key,
k int,
val int,
drop_me int
) engine=innodb row_format=dynamic;
insert into t8 select seq, mod(seq,10), seq, seq from seq_1_to_10000;
alter table t8 drop column drop_me, algorithm=instant;
create index k_idx on t8(k);
select count(val) as ROWS_DROPPED, sum(val) as SUM_DROPPED
from t8 force index(k_idx) where k=3;
ROWS_DROPPED SUM_DROPPED
1000 4998000
drop table t8;
create table t9 (
pk int not null primary key,
k int,
val int,
drop_me int
) engine=innodb row_format=redundant;
insert into t9 select seq, mod(seq,10), seq, seq from seq_1_to_10000;
alter table t9 drop column drop_me, algorithm=instant;
create index k_idx on t9(k);
select count(val) as ROWS_REDUNDANT_DROPPED, sum(val) as SUM_REDUNDANT_DROPPED
from t9 force index(k_idx) where k=3;
ROWS_REDUNDANT_DROPPED SUM_REDUNDANT_DROPPED
1000 4998000
drop table t9;
create table t10 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=compressed key_block_size=8;
insert into t10 select seq, mod(seq,10), seq from seq_1_to_10000;
create index k_idx on t10(k);
select count(val) as ROWS_COMPRESSED, sum(val) as SUM_COMPRESSED
from t10 force index(k_idx) where k=3;
ROWS_COMPRESSED SUM_COMPRESSED
1000 4998000
drop table t10;
create table t11 (
k int,
val int
) engine=innodb row_format=dynamic;
insert into t11 select mod(seq,10), seq from seq_1_to_10000;
create index k_idx on t11(k);
select count(val) as ROWS_ROW_ID, sum(val) as SUM_ROW_ID
from t11 force index(k_idx) where k=3;
ROWS_ROW_ID SUM_ROW_ID
1000 4998000
drop table t11;
create table t12 (
pk int not null primary key,
k int,
val varchar(255)
) engine=innodb row_format=dynamic;
insert into t12 select seq, seq, repeat('x',30) from seq_1_to_10000;
create index k_idx on t12(k);
update t12 force index(k_idx) set val=repeat('y',200) where k > 0;
select count(*) as ROWS_GROWN, sum(pk) as SUM_PK from t12
where val=repeat('y',200);
ROWS_GROWN SUM_PK
10000 50005000
update t12 force index(k_idx) set val=repeat('x',30) where k > 0;
select count(*) as ROWS_SHRUNK, sum(pk) as SUM_PK from t12
where val=repeat('x',30);
ROWS_SHRUNK SUM_PK
10000 50005000
check table t12;
Table Op Msg_type Msg_text
test.t12 check status OK
drop table t12;
create table t13 (
pk int not null primary key,
k bigint unsigned,
val int
) engine=innodb row_format=dynamic;
insert into t13 select seq, crc32(seq), seq from seq_1_to_3000;
insert into t13 select 3000+seq, 4294967296+seq, seq from seq_1_to_3000;
create index k_idx on t13(k);
analyze table t13 persistent for all;
Table Op Msg_type Msg_text
test.t13 analyze status Engine-independent statistics collected
test.t13 analyze status OK
set @js='$out_split';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_LATE_LOCALITY;
PAGES_ACCESSED_LATE_LOCALITY
12016
drop table t13;
create table t14 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t14 select seq, seq, seq from seq_1_to_10000;
create index k_idx on t14(k);
analyze table t14 persistent for all;
Table Op Msg_type Msg_text
test.t14 analyze status Engine-independent statistics collected
test.t14 analyze status OK
set @js='$out_cov';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_COVERING;
PAGES_ACCESSED_COVERING
20
set @js='$out_cov_x';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_COVERING_FOR_UPDATE;
PAGES_ACCESSED_COVERING_FOR_UPDATE
20020
drop table t14;
create table t15 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t15 select seq, seq, seq from seq_1_to_10000;
create index k_idx on t15(k);
start transaction with consistent snapshot;
connect con2,localhost,root,,test;
update t15 set k=k+100000 where pk <= 5000;
disconnect con2;
connection default;
select count(k) as ROWS_SNAPSHOT, sum(k) as SUM_SNAPSHOT
from t15 force index(k_idx) where k > 0;
ROWS_SNAPSHOT SUM_SNAPSHOT
10000 50005000
commit;
drop table t15;
create table t16 (
pk int not null primary key,
k_inv int,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t16 select seq, 10001-seq, seq, seq from seq_1_to_10000;
create index k_inv_idx on t16(k_inv);
create index k_idx on t16(k);
analyze table t16 persistent for all;
Table Op Msg_type Msg_text
test.t16 analyze status Engine-independent statistics collected
test.t16 analyze status OK
set @js='$out_inv';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_INVERSE;
PAGES_ACCESSED_INVERSE
20020
set @js='$out_desc';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_DESCENDING;
PAGES_ACCESSED_DESCENDING
20020
select val as LAST_ROW_DESCENDING from t16 force index(k_idx)
where k > 0 order by k desc limit 9999,1;
LAST_ROW_DESCENDING
1
drop table t16;
create table t17 (
pk int not null primary key,
k_inv int,
val int
) engine=innodb row_format=redundant;
insert into t17 select seq, 10001-seq, seq from seq_1_to_10000;
alter table t17 add column c int default 42, algorithm=instant;
create index k_inv_idx on t17(k_inv);
select count(val) as ROWS_REDUNDANT_INVERSE, sum(val) as SUM_REDUNDANT_INVERSE,
min(c) as DEFAULT_C_INVERSE
from t17 force index(k_inv_idx) where k_inv > 0;
ROWS_REDUNDANT_INVERSE SUM_REDUNDANT_INVERSE DEFAULT_C_INVERSE
10000 50005000 42
drop table t17;

518
mysql-test/suite/innodb/t/clust_leaf_hint.test Normal file
View File

@@ -0,0 +1,518 @@
#
# MDEV-32286 ANALYZE displays a huge number of InnoDB secondary index pages_accessed
#
# The clustered leaf hints of row_prebuilt_t::clust_leaf_hint: what they save
# on a scan through a secondary index that must read clustered-index records,
# covering or not, and what such a scan reads while they are in place.
#
# The .result records raw pages_accessed values, so every server option they
# depend on is fixed by the test itself, because mariadb-test-run appends the
# options of its own --mysqld arguments after any that a test file requests,
# and the last value on the command line wins. The page size decides the tree
# shape, so a run of another page size skips. The row format decides the
# record layout, so every table names its own. The whole working set must stay
# resident, so that no access path depends on eviction timing, and it fits the
# buffer pool of a default run; a run with a smaller pool skips. The adaptive
# hash index, whose hash guesses reach a record without reading the pages that
# a descent reads, is set below, where no command line reaches it.
--source include/have_innodb.inc
--source include/have_innodb_16k.inc
--source include/have_sequence.inc
if (`select @@global.innodb_buffer_pool_size < 8*1024*1024`)
{
--skip Test requires innodb_buffer_pool_size of 8M or more
}
# Kept out of the .result, so that a build without the adaptive hash index,
# where the variable does not exist, records the same result.
--disable_query_log
let $adaptive_hash_index= `select count(*) from information_schema.global_variables
where variable_name = 'innodb_adaptive_hash_index'`;
if ($adaptive_hash_index)
{
set @save_adaptive_hash_index= @@global.innodb_adaptive_hash_index;
set global innodb_adaptive_hash_index= 0;
}
# The file that runs these shapes with the hints turned off asks for it here,
# and not around its own source of this file, because every skip above leaves
# mariadb-test no way to restore what a caller had set: the setting would
# outlive the test and reach every later test of the same server. Global, and
# not of this session: an embedded server answers the backticks of a let on
# another thread than the rest, and only a global DBUG setting reaches both.
if ($clust_leaf_hint_off)
{
set @save_debug_dbug= @@global.debug_dbug;
set global debug_dbug='+d,ib_no_clust_leaf_hint';
}
--enable_query_log
create table t1 (
pk int not null primary key,
domain_grp int,
val int
) engine=innodb row_format=dynamic;
insert into t1 select seq, mod(seq,10), seq from seq_1_to_10000;
create index domain_idx on t1(domain_grp);
analyze table t1 persistent for all;
# "val" is outside domain_idx, so each matching row needs a clustered-index
# lookup either way.
let $out_scan=`analyze format=json select sql_no_cache val from t1 ignore index(domain_idx) where domain_grp=3`;
evalp set @js='$out_scan';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_FULL_SCAN;
# ~1000 rows share key 3
let $out_idx=`analyze format=json select sql_no_cache val from t1 force index(domain_idx) where domain_grp=3`;
evalp set @js='$out_idx';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SEC_INDEX;
drop table t1;
# Wide primary key: the clustered index needs a 3-level tree. The character
# set and collation are pinned, because they decide the maximum key length
# and the comparison rules, and with them the recorded counts.
create table t2 (
pk varchar(500) character set utf8mb4 collate utf8mb4_general_ci
not null primary key,
domain_grp int,
val int
) engine=innodb row_format=dynamic;
insert into t2 select lpad(seq,500,'0'), mod(seq,4), seq from seq_1_to_3000;
create index domain_idx on t2(domain_grp);
analyze table t2 persistent for all;
# A node pointer for this 500-byte key takes about 507 bytes, so a 16k root
# page holds about 32 of them: a leaf count far above that cannot hang off a
# single root, and the tree has at least three levels. The 'size' statistic
# cannot show this, because it counts the pages that both segments reserve,
# the free ones included, and so exceeds the leaf count of a two-level tree
# as well.
select stat_value into @leaf from mysql.innodb_index_stats
where database_name='test' and table_name='t2' and index_name='PRIMARY'
and stat_name='n_leaf_pages';
select @leaf > 100 as THREE_LEVEL_TREE;
let $out_scan2=`analyze format=json select sql_no_cache val from t2 ignore index(domain_idx) where domain_grp=3`;
evalp set @js='$out_scan2';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_FULL_SCAN_3LEVEL;
# ~750 rows share key 3
let $out_idx2=`analyze format=json select sql_no_cache val from t2 force index(domain_idx) where domain_grp=3`;
evalp set @js='$out_idx2';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SEC_INDEX_3LEVEL;
drop table t2;
# Secondary index decorrelated from clustered key order by a multiplier: the
# scan walks the clustered index in two slowly drifting strides, so
# consecutive rows keep sharing a clustered leaf for a while.
create table t3 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t3 select seq, mod(seq*997+13,10007), seq from seq_1_to_10000;
create index k_idx on t3(k);
analyze table t3 persistent for all;
let $out_uncorr=`analyze format=json select sql_no_cache val from t3 force index(k_idx) where k between 0 and 10006`;
evalp set @js='$out_uncorr';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_UNCORRELATED;
drop table t3;
# Two well separated clustered key ranges, interleaved in secondary key order:
# consecutive rows alternate between two clustered leaves, so the whole scan
# needs no more than two of them at a time.
create table t4 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t4 select seq, 2*seq-1, seq from seq_1_to_1000;
insert into t4 select 1000000+seq, 2*seq, seq from seq_1_to_1000;
create index k_idx on t4(k);
analyze table t4 persistent for all;
let $out_interleaved=`analyze format=json select sql_no_cache val from t4 force index(k_idx) where k between 1 and 2000`;
evalp set @js='$out_interleaved';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_INTERLEAVED;
drop table t4;
# Secondary key order shuffled against clustered key order, over five times
# as many clustered leaves as there are slots. Some lookups do land on a
# remembered leaf, but too few in a row: the miss counter reaches its limit
# early in the scan and the slots stop being tested on every row. The count
# therefore shows that the scan abandoned pays almost nothing, which is what
# this case is for, rather than what a hinted scan saves.
create table t5 (
pk int not null primary key,
k int unsigned,
val int
) engine=innodb row_format=dynamic;
insert into t5 select seq, crc32(seq), seq from seq_1_to_10000;
create index k_idx on t5(k);
analyze table t5 persistent for all;
let $out_shuffled=`analyze format=json select sql_no_cache val from t5 force index(k_idx) where k > 0`;
evalp set @js='$out_shuffled';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_SHUFFLED;
drop table t5;
# The cases below cover the record formats and key shapes that the remembered
# leaves have to read. They check query results rather than counts, because a
# count would only repeat what t1 measures over a record layout that the
# counts do not depend on.
#
# Instant ALTER TABLE leaves a metadata pseudo-record as the first user record
# of the leftmost clustered leaf, which a remembered leaf then copies and
# compares its keys against. Nothing between that ALTER TABLE and the query
# may rebuild the clustered index, here or in the ROW_FORMAT=REDUNDANT case
# below: a rebuild drops the record, and the case then measures what t1
# already measures, with the same query result to show it.
create table t6 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t6 select seq, mod(seq,10), seq from seq_1_to_10000;
alter table t6 add column c int default 42, algorithm=instant;
create index k_idx on t6(k);
select count(val) as ROWS_INSTANT, sum(val) as SUM_INSTANT, min(c) as DEFAULT_C
from t6 force index(k_idx) where k=3;
drop table t6;
# ROW_FORMAT=REDUNDANT, with a metadata pseudo-record as well: the keys are
# copied out of the other record format.
create table t7 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=redundant;
insert into t7 select seq, mod(seq,10), seq from seq_1_to_10000;
alter table t7 add column c int default 42, algorithm=instant;
create index k_idx on t7(k);
select count(val) as ROWS_REDUNDANT, sum(val) as SUM_REDUNDANT
from t7 force index(k_idx) where k=3;
drop table t7;
# The same, for a column dropped instantly rather than added. That sets
# dict_table_t::instant and leaves an alter metadata pseudo-record, whose
# header carries an added-field count and a metadata BLOB that the add
# metadata record above does not, so rec_copy_prefix_to_buf() converts a
# different header when a remembered leaf copies its first record.
# ALGORITHM=INSTANT is the assertion: without it the record would not exist.
create table t8 (
pk int not null primary key,
k int,
val int,
drop_me int
) engine=innodb row_format=dynamic;
insert into t8 select seq, mod(seq,10), seq, seq from seq_1_to_10000;
alter table t8 drop column drop_me, algorithm=instant;
create index k_idx on t8(k);
select count(val) as ROWS_DROPPED, sum(val) as SUM_DROPPED
from t8 force index(k_idx) where k=3;
drop table t8;
# The dropped column under ROW_FORMAT=REDUNDANT.
create table t9 (
pk int not null primary key,
k int,
val int,
drop_me int
) engine=innodb row_format=redundant;
insert into t9 select seq, mod(seq,10), seq, seq from seq_1_to_10000;
alter table t9 drop column drop_me, algorithm=instant;
create index k_idx on t9(k);
select count(val) as ROWS_REDUNDANT_DROPPED, sum(val) as SUM_REDUNDANT_DROPPED
from t9 force index(k_idx) where k=3;
drop table t9;
# ROW_FORMAT=COMPRESSED: the keys are copied out of a third record layout,
# and its page can carry a page_zip_des_t. Whether the buffer pool holds such
# a page without the uncompressed frame that a hint needs is not a state a
# test can ask for, so this case does not reach that rejection; it covers the
# record layout.
create table t10 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=compressed key_block_size=8;
insert into t10 select seq, mod(seq,10), seq from seq_1_to_10000;
create index k_idx on t10(k);
select count(val) as ROWS_COMPRESSED, sum(val) as SUM_COMPRESSED
from t10 force index(k_idx) where k=3;
drop table t10;
# No user primary key: the clustered index is keyed by the hidden row id, so
# a remembered key is that single field.
create table t11 (
k int,
val int
) engine=innodb row_format=dynamic;
insert into t11 select mod(seq,10), seq from seq_1_to_10000;
create index k_idx on t11(k);
select count(val) as ROWS_ROW_ID, sum(val) as SUM_ROW_ID
from t11 force index(k_idx) where k=3;
drop table t11;
# Leaves that move while they are read. The secondary key order follows the
# clustered one, so every row is looked up through a remembered leaf, and the
# writes of the same statement grow each row from 30 to 200 bytes, which
# splits those leaves, and shrink it back, which merges and frees them. A
# remembered range then promises a leaf that the latched page denies, and a
# page that has left the tree keeps contents that only the check for a freed
# page rejects. Being a locking read, it also stores the position of a cursor
# that a hint placed.
create table t12 (
pk int not null primary key,
k int,
val varchar(255)
) engine=innodb row_format=dynamic;
insert into t12 select seq, seq, repeat('x',30) from seq_1_to_10000;
create index k_idx on t12(k);
update t12 force index(k_idx) set val=repeat('y',200) where k > 0;
select count(*) as ROWS_GROWN, sum(pk) as SUM_PK from t12
where val=repeat('y',200);
update t12 force index(k_idx) set val=repeat('x',30) where k > 0;
select count(*) as ROWS_SHRUNK, sum(pk) as SUM_PK from t12
where val=repeat('x',30);
check table t12;
drop table t12;
# Locality that appears only half way through the scan, which is what the
# retry that starts the miss count again is for. The secondary key order
# visits 3000 shuffled rows first, which makes the scan give the slots up,
# and then 3000 whose order follows the clustered one. crc32() cannot reach
# 2**32, so the second group sorts wholly above the first. Without the retry
# the scan would stay unhinted over that second group and keep no more than
# the few hits it made before it gave up, so nearly the whole saving that
# this count records is the retry's. A count again, placed after the cases
# above because it needs a table of its own shape.
create table t13 (
pk int not null primary key,
k bigint unsigned,
val int
) engine=innodb row_format=dynamic;
insert into t13 select seq, crc32(seq), seq from seq_1_to_3000;
insert into t13 select 3000+seq, 4294967296+seq, seq from seq_1_to_3000;
create index k_idx on t13(k);
analyze table t13 persistent for all;
let $out_split=`analyze format=json select sql_no_cache val from t13 force index(k_idx) where k >= 0`;
evalp set @js='$out_split';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_LATE_LOCALITY;
drop table t13;
# A locking read cannot be covering. Every column that this query reads is in
# k_idx, so the first count is a scan of the secondary index alone, with no
# clustered-index lookup at all, and the hints cannot change it. The same scan
# under FOR UPDATE needs one lookup per row, because an exclusive select lock
# type makes ha_innobase::build_template() retrieve the whole clustered index
# record whatever the index holds, and the remembered leaves answer those
# lookups like any others: the secondary key order follows the clustered one
# here. ha_innobase::external_lock() decides the lock type per statement and
# not from the transaction state, so this case starts no transaction.
create table t14 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t14 select seq, seq, seq from seq_1_to_10000;
create index k_idx on t14(k);
analyze table t14 persistent for all;
let $out_cov=`analyze format=json select sql_no_cache k from t14 force index(k_idx) where k > 0`;
evalp set @js='$out_cov';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_COVERING;
let $out_cov_x=`analyze format=json select sql_no_cache k from t14 force index(k_idx) where k > 0 for update`;
evalp set @js='$out_cov_x';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_COVERING_FOR_UPDATE;
drop table t14;
# A covering scan reaches a clustered-index record as well, when the secondary
# leaf carries a PAGE_MAX_TRX_ID that the read view cannot see: the visibility
# test of row_search_mvcc() then enters the lookup past the check of
# need_to_access_clustered, whatever the query reads. A lookup whose clustered
# record is too new for the read view then rebuilds an older version of it
# through row_sel_build_prev_vers_for_mysql(), which
# row_sel_sec_rec_is_for_clust_rec() matches against the secondary entry. No
# other case here has a concurrent writer, so this is the only one that makes
# a remembered leaf answer such a lookup.
# The update below moves half the rows to another key range, so the older
# transaction meets both verdicts in one scan: it accepts the delete-marked
# entries that its version still carries, and rejects the new entries that no
# version of it does. Its result is the snapshot it opened on, and no record
# from a remembered leaf other than the right one produces that. A result
# rather than a count, because a count here follows the write pattern and
# the depth of the version rebuilds, and so reports what the writer did
# rather than what the hints save.
create table t15 (
pk int not null primary key,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t15 select seq, seq, seq from seq_1_to_10000;
create index k_idx on t15(k);
start transaction with consistent snapshot;
--connect (con2,localhost,root,,test)
update t15 set k=k+100000 where pk <= 5000;
--disconnect con2
--connection default
select count(k) as ROWS_SNAPSHOT, sum(k) as SUM_SNAPSHOT
from t15 force index(k_idx) where k > 0;
commit;
drop table t15;
# The clustered order walked backward. Every lookup lands on the record right
# before the one that the lookup before it landed on, so the remembered leaf
# answers it, and the search inside that leaf starts from the previous landing
# and steps back one record. The counts are those of a forward walk, because
# the step changes only the search inside a leaf; the comparisons that a debug
# build makes against a full search of the leaf are what check the step. The
# first scan follows a secondary key that falls as the primary key rises, and
# the second reads a secondary key that rises with it in descending order.
# The row that the second query returns is the last one that it reads, which
# shows the order of the scan.
create table t16 (
pk int not null primary key,
k_inv int,
k int,
val int
) engine=innodb row_format=dynamic;
insert into t16 select seq, 10001-seq, seq, seq from seq_1_to_10000;
create index k_inv_idx on t16(k_inv);
create index k_idx on t16(k);
analyze table t16 persistent for all;
let $out_inv=`analyze format=json select sql_no_cache val from t16 force index(k_inv_idx) where k_inv > 0`;
evalp set @js='$out_inv';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_INVERSE;
let $out_desc=`analyze format=json select sql_no_cache val from t16 force index(k_idx) where k > 0 order by k desc limit 9999,1`;
evalp set @js='$out_desc';
set @out=(select json_extract(@js,'$**.r_engine_stats.pages_accessed'));
select cast(json_extract(@out,'$[0]') as UNSIGNED) as PAGES_ACCESSED_DESCENDING;
select val as LAST_ROW_DESCENDING from t16 force index(k_idx)
where k > 0 order by k desc limit 9999,1;
drop table t16;
# The backward walk under ROW_FORMAT=REDUNDANT, where the step finds the
# record before through the other record format. The instant ADD COLUMN puts
# a metadata pseudo-record at the start of the leftmost leaf, where the walk
# ends, and leaves every row with fewer fields than the table has. No lookup
# steps back onto that pseudo-record, because the row with the lowest key is
# the last one that the walk looks up.
create table t17 (
pk int not null primary key,
k_inv int,
val int
) engine=innodb row_format=redundant;
insert into t17 select seq, 10001-seq, seq from seq_1_to_10000;
alter table t17 add column c int default 42, algorithm=instant;
create index k_inv_idx on t17(k_inv);
select count(val) as ROWS_REDUNDANT_INVERSE, sum(val) as SUM_REDUNDANT_INVERSE,
min(c) as DEFAULT_C_INVERSE
from t17 force index(k_inv_idx) where k_inv > 0;
drop table t17;
--disable_query_log
if ($adaptive_hash_index)
{
set global innodb_adaptive_hash_index= @save_adaptive_hash_index;
}
if ($clust_leaf_hint_off)
{
set global debug_dbug= @save_debug_dbug;
}
--enable_query_log

142
mysql-test/suite/innodb/t/clust_leaf_hint_backward_debug.test Normal file
View File

@@ -0,0 +1,142 @@
#
# MDEV-32286 ANALYZE displays a huge number of InnoDB secondary index pages_accessed
#
# A clustered lookup that a hint answers may step from the expected record
# to the record before it. page_cur_search_near() refuses that step when the
# record before is the page infimum or the metadata pseudo-record. A lookup
# reaches either only for a key that is not in the clustered index while
# row_sel_clust_leaf_hint_covers() still accepts the slot, and these are the
# constraints that leave one shape for that:
#
# - The key must leave the clustered index under a reader that has a
# secondary index entry for it to look up. Purge removes the entry together
# with the record, so the record must be removed by the rollback of an
# insert over a delete-marked record, which removes it when
# row_undo_mod_must_purge() holds, while purge is not let to remove the
# delete-marked secondary index entry.
# - The reader must look that entry up and not skip it, so the secondary
# index page must carry a PAGE_MAX_TRX_ID that its read view does not see.
# The insert gives the row a new secondary key next to the old one, which
# puts it on the same page.
# - The key must be the first user record of its leaf, so that the step back
# lands on the infimum, or the first key of the leftmost leaf, where it
# lands on the metadata pseudo-record.
#
--source include/have_innodb.inc
--source include/have_debug.inc
--source include/have_debug_sync.inc
--source include/have_sequence.inc
# The notes below are read from the error log, and an embedded server writes
# its stderr to the log of mariadb-test instead of that file.
--source include/not_embedded.inc
--source include/count_sessions.inc
# The hints are collected and tested only while the adaptive hash index is
# off, so a run with it on would reach none of this. Kept out of the .result,
# so that a build without the adaptive hash index records the same result.
--disable_query_log
let $adaptive_hash_index= `select count(*) from information_schema.global_variables
where variable_name = 'innodb_adaptive_hash_index'`;
if ($adaptive_hash_index)
{
set @save_adaptive_hash_index= @@global.innodb_adaptive_hash_index;
set global innodb_adaptive_hash_index= 0;
}
--enable_query_log
# k sorts in the inverse order of pk, so a scan of k_idx walks the clustered
# index backward. MERGE_THRESHOLD=1 keeps the removals below from merging a
# leaf into its neighbour, which would move the keys away from the start of
# their leaves.
create table t1 (
pk int not null primary key,
k int,
val text,
key k_idx(k)
) engine=innodb row_format=dynamic stats_persistent=0 comment='MERGE_THRESHOLD=1';
# An ascending load with this limit puts pk 4951 first on its leaf.
set @save_limit= @@global.innodb_limit_optimistic_insert_debug;
set global innodb_limit_optimistic_insert_debug= 100;
insert into t1 select seq, 2*(20001-seq), 'v' from seq_1_to_10000;
set global innodb_limit_optimistic_insert_debug= @save_limit;
# This adds the metadata pseudo-record before pk 1.
alter table t1 add column extra int, algorithm=instant;
set global innodb_max_purge_lag_wait=0;
# From here on, purge advances its view and removes nothing.
set @save_view_update= @@global.innodb_trx_purge_view_update_only_debug;
set global innodb_trx_purge_view_update_only_debug= 1;
delete from t1 where pk in (1, 4951);
connect (ins,localhost,root);
begin;
# Each insert goes over a delete-marked record. The new key differs from the
# old one, so the secondary index entry of the delete stays in place,
# delete-marked.
insert into t1 (pk, k, val) values (1, 2*(20001-1)+1, 'w'), (4951, 2*(20001-4951)+1, 'w');
connection default;
# The purge view now includes the delete, so row_undo_mod_must_purge() holds
# and the rollback below removes the clustered index records.
set statement max_statement_time=1 for set global innodb_max_purge_lag_wait=0;
delimiter |;
create function hold(x int) returns int
begin
if x = 4956 then
set debug_sync= 'now SIGNAL reader_paused';
set debug_sync= 'now WAIT_FOR reader_go';
end if;
return 1;
end|
delimiter ;|
connect (reader,localhost,root);
# Without this the condition would be evaluated inside the engine, before the
# clustered lookup of the row it stops on rather than after it.
set session optimizer_switch='index_condition_pushdown=off';
set session debug_dbug='+d,ib_log_clust_leaf_hint_step_back';
# val is outside k_idx, so every row costs a clustered lookup, and its type
# turns the prefetch cache off.
--send select count(left(val,1)) as n, sum(pk) as s from t1 force index(k_idx) where k > 0 and hold(pk)
connection default;
set debug_sync='now WAIT_FOR reader_paused';
connection ins;
# This changes the modify clock of the leaf of pk 4951, which the reader has
# remembered already. The first lookup on it after that is a binary search,
# the second is one too and arms the step again, and the reader is stopped far
# enough above pk 4951 for both to come before it. The reader remembers the
# leaf of pk 1 only after this, so the change does not reach it.
rollback;
connection default;
set debug_sync='now SIGNAL reader_go';
connection reader;
# pk 1 and pk 4951 are deleted for the reader. Their secondary index entries
# are delete-marked, so the lookups that find no record skip them.
--reap
connection default;
--let SEARCH_FILE = $MYSQLTEST_VARDIR/log/mysqld.1.err
--let SEARCH_PATTERN = Clustered leaf hint step back reached the [a-z ]+
--let SEARCH_OUTPUT = matches
--source include/search_pattern_in_file.inc
disconnect reader;
disconnect ins;
set global innodb_trx_purge_view_update_only_debug= @save_view_update;
drop function hold;
drop table t1;
set debug_sync='reset';
--disable_query_log
if ($adaptive_hash_index)
{
set global innodb_adaptive_hash_index= @save_adaptive_hash_index;
}
--enable_query_log
--source include/wait_until_count_sessions.inc

147
mysql-test/suite/innodb/t/clust_leaf_hint_instant_alter.test Normal file
View File

@@ -0,0 +1,147 @@
#
# MDEV-32286 ANALYZE displays a huge number of InnoDB secondary index pages_accessed
#
# dict_index_t::clear_instant_alter() raises dict_index_t::n_core_fields under
# a reader that holds no more than a shared metadata lock. The key copies of a
# clustered leaf hint that the reader filled before that stay valid, and
# row_sel_clust_leaf_hint_covers() compares the next lookup against them. The
# case below is the shape that reaches that comparison, and these are the
# constraints that leave no other:
#
# - The clear needs the clustered index to lose the last user record of its
# root page, so the tree must be one page by then, while a slot is only
# filled from a leaf that is not the root. The tree therefore has to shrink
# between the two, which purge does here.
# - The rows that the reader can see must be rows that something may remove
# while its statement runs. Purge may remove only what was delete-marked and
# committed before the read view of the reader, and a row in that state is
# one that no isolation level shows it. A locking read would wait for the
# inserting transaction rather than read it. That leaves uncommitted inserts
# read at READ UNCOMMITTED and taken away by a rollback.
# - The reader must look up one more row after the clear, so a row has to
# arrive after it, above the key that the reader is stopped at.
#
--source include/have_innodb.inc
--source include/have_debug.inc
--source include/have_debug_sync.inc
--source include/have_sequence.inc
# The hints are collected and tested only while the adaptive hash index is
# off, so a run with it on would reach none of this. Kept out of the .result,
# so that a build without the adaptive hash index records the same result.
--disable_query_log
let $adaptive_hash_index= `select count(*) from information_schema.global_variables
where variable_name = 'innodb_adaptive_hash_index'`;
if ($adaptive_hash_index)
{
set @save_adaptive_hash_index= @@global.innodb_adaptive_hash_index;
set global innodb_adaptive_hash_index= 0;
}
--enable_query_log
create table t1 (
pk int not null primary key,
domain_grp int,
val text,
key domain_idx(domain_grp)
) engine=innodb row_format=dynamic stats_persistent=0;
# Filler rows, enough of them that the clustered index has more than one leaf
# at every page size: what a lookup lands on is then not the root, and can be
# remembered. Where the value is stored off page, which is every page size
# below 16k, the count and not the width is what gives the tree its leaves.
insert into t1 select seq, 9, repeat('f', 4000) from seq_1_to_500;
# The instant ADD COLUMN needs a non-empty table. On an empty one it is
# carried out by emptying the root page, which leaves n_core_fields at
# n_fields and nothing for the clear to change.
alter table t1 add column extra int, algorithm=instant;
connect (prevent_purge,localhost,root);
start transaction with consistent snapshot;
connection default;
# Delete-marked here, and committed before the reader opens its read view, so
# that purge may remove these rows while the reader is stopped. Until
# prevent_purge commits it may not, which is what keeps the tree tall enough
# for the reader to fill a slot.
delete from t1 where domain_grp=9;
connect (ins,localhost,root);
begin;
# Consecutive keys above every filler key, and narrow, so that all of them
# share one leaf. The lookups of the reader then answer each other from the
# first slot, and the miss counter stays below CLUST_LEAF_HINT_MAX_MISSES,
# which a scan that gives up would exceed.
insert into t1 (pk, domain_grp, val) select 1000000+seq, 1, 'v' from seq_1_to_8;
connection default;
delimiter |;
create function hold(x int) returns int
begin
if x = 1000008 then
set debug_sync= 'now SIGNAL reader_paused';
set debug_sync= 'now WAIT_FOR reader_go';
end if;
return 1;
end|
delimiter ;|
connect (reader,localhost,root);
# READ UNCOMMITTED is what lets the reader see the uncommitted rows above.
set session transaction isolation level read uncommitted;
# Without this the condition would be evaluated inside the engine, before the
# clustered lookup of the row it stops on rather than after it.
set session optimizer_switch='index_condition_pushdown=off';
# val is outside domain_idx, so every row costs a clustered lookup, and its
# type keeps row_prebuilt_t::templ_contains_blob set, which turns the prefetch
# cache off: with it on, one fetch reads ahead past the row that hold() stops
# on.
--send select pk, left(val,1) from t1 force index(domain_idx) where domain_grp=1 and hold(pk)
connection default;
set debug_sync='now WAIT_FOR reader_paused';
# Eight lookups have been made and at least one slot is filled, all of them
# with the n_core_fields that the instant ADD COLUMN left.
connection prevent_purge;
commit;
connection default;
# Purge now removes the filler rows, which collapses the clustered index onto
# its root. The eight rows of the reader keep the root from losing its last
# user record, so nothing is cleared yet.
set global innodb_max_purge_lag_wait=0;
let $wait_all_purged= 0;
--source include/wait_all_purged.inc
connection ins;
# The last of these removals empties the root page, where
# dict_index_t::clear_instant_alter() raises n_core_fields past the value that
# every slot of the reader was filled with.
rollback;
connection default;
# One row for the reader to look up after that change. Its key sorts above the
# position the reader is stopped at, so the scan reaches it, and the lookup it
# costs is the one that tests the slots.
insert into t1 values (1000009, 1, 'w', null);
set debug_sync='now SIGNAL reader_go';
connection reader;
--reap
connection default;
disconnect reader;
disconnect ins;
disconnect prevent_purge;
drop function hold;
drop table t1;
set debug_sync='reset';
--disable_query_log
if ($adaptive_hash_index)
{
set global innodb_adaptive_hash_index= @save_adaptive_hash_index;
}
--enable_query_log

13
mysql-test/suite/innodb/t/clust_leaf_hint_off_debug.test Normal file
View File

@@ -0,0 +1,13 @@
#
# MDEV-32286 ANALYZE displays a huge number of InnoDB secondary index pages_accessed
#
# The same shapes with the clustered leaf hints turned off, which only a debug
# build can do. The .result is a line for line counterpart of the one of
# clust_leaf_hint, so a diff of the two files is what the hints save.
#
--source include/have_debug.inc
--echo # clustered leaf hints disabled
# The file below sets the DBUG flag itself, past its own skips, and restores
# it at its end.
--let $clust_leaf_hint_off= 1
--source suite/innodb/t/clust_leaf_hint.test

147
storage/innobase/btr/btr0cur.cc
View File

@@ -1601,6 +1601,153 @@ release_tree:
goto search_loop;
}
bool btr_cur_t::try_leaf_hint(const dtuple_t *tuple, page_id_t hint_page_id,
btr_leaf_step &step, mtr_t *mtr) noexcept
{
/* A clustered index only: the checks below accept FIL_PAGE_RTREE, which
search_leaf() rejects, and no page of a clustered index is an R-tree. */
ut_ad(index()->is_primary());
/* A complete unique key. The search below takes a match on every compared
field as the answer without examining the successor record, which only a
key that cannot repeat allows: the PAGE_CUR_LE match of a repeating key can
be on a later leaf. */
ut_ad(dtuple_get_n_fields_cmp(tuple) == dict_index_get_n_unique(index()));
/* A user key, not the search key of the metadata pseudo-record, which
page_cur_search_with_match() would report as a full match on it. */
ut_ad(!(tuple->info_bits & REC_INFO_MIN_REC_FLAG));
/* hint_page_id was not read from a latched parent page, so it may now
precede the caller's already-latched secondary-index leaf in the
B-tree latching order: never block on its latch (page latches have no
deadlock detection) and never read it from disk. */
buf_block_t *const block= buf_page_try_get(hint_page_id, mtr);
if (!block)
return false;
const page_t *const page= block->page.frame;
if (block->page.is_freed() || !fil_page_index_page_check(page) ||
!page_is_leaf(page) ||
!!page_is_comp(page) != index()->table->not_redundant() ||
btr_page_get_index_id(page) != index()->id)
{
/* Stale hint or, for search_leaf()'s own checks, corruption; we cannot
tell here, so fall back either way, and the full descent still reports a
corrupt live leaf. is_freed() is the guard that descent omits: a freed
but unreused page keeps old contents that pass the other checks. */
mtr->release_last_page();
return false;
}
/* Search on a copy, so that a rejection below leaves this cursor as it
was. */
page_cur_t cur{page_cur};
cur.block= block;
uint16_t up= 0, low= 0;
/* A record pointer stays valid while the block holds the same page and
its modify_clock has not moved: eviction, deletion and reorganization
all advance the clock, and an insertion moves no record. Try the record
where the previous search of this leaf landed and the records before and
after it, and nothing further, because the binary search is what the step
replaces. */
const bool step_valid= step.block == block &&
step.modify_clock == block->modify_clock();
const bool stepped= step_valid && step.expect &&
page_cur_search_near(tuple, step.rec, &up, &low, &cur);
#ifdef UNIV_DEBUG
if (stepped)
{
page_cur_t check{page_cur};
check.block= block;
uint16_t check_up= 0, check_low= 0;
ut_a(!page_cur_search_with_match(tuple, PAGE_CUR_LE, &check_up,
&check_low, &check, nullptr));
ut_a(check.rec == cur.rec);
ut_a(check_low == low);
}
#endif /* UNIV_DEBUG */
if (!stepped &&
(page_cur_search_with_match(tuple, PAGE_CUR_LE, &up, &low, &cur,
nullptr) ||
page_rec_is_infimum(cur.rec)))
{
/* Corruption, or tuple precedes every record on this page: its
predecessor, if any, is on an earlier leaf. */
mtr->release_last_page();
return false;
}
/* A user key sorts above the metadata pseudo-record on 0 fields. Landing
on it means that tuple precedes every key of the leftmost leaf, so that
it is not in the index, which a full descent reports the same way. */
ut_ad(!rec_is_metadata(cur.rec, *index()) || !low);
if (page_has_next(page) && low < dtuple_get_n_fields_cmp(tuple))
{
/* The record found is strictly less than tuple: PAGE_CUR_LE lands on
the greatest record <= tuple, and a full match would have made
low == n_fields_cmp. If it is the last user record of a leaf
with a right sibling, the true match may be on a later leaf; we cannot
resolve that from here, so reject the hint. The rightmost leaf needs
no such check: its last record is that match for any larger tuple. */
const rec_t *const next_rec= page_rec_get_next_const(cur.rec);
if (UNIV_UNLIKELY(!next_rec) || page_rec_is_supremum(next_rec))
{
mtr->release_last_page();
return false;
}
}
/* This feeds no btr_search_info_update(). The caller tries a hint only
while the adaptive hash index is disabled, and search_leaf() feeds
nothing then either. */
/* Age the page as the buf_page_get_gen() of a descent would, which
buf_page_try_get() does not do: a leaf that a correlated scan reads once
per row must not look less recently used than one reached by descent. */
block->page.touch();
/* A failed step costs comparisons whose outcome no branch predictor can
guess, so try the next one only where this search landed where a step
would have: on the previous record or right before or after it. The record
right before it is told by its successor, one link away, because finding a
predecessor takes a walk of the page directory. */
step.expect= stepped ||
(step_valid && (cur.rec == step.rec ||
cur.rec == page_rec_get_next_const(step.rec) ||
page_rec_get_next_const(cur.rec) == step.rec));
step.block= block;
step.modify_clock= block->modify_clock();
step.rec= cur.rec;
page_cur= cur;
up_match= up;
low_match= low;
/* The byte counts stay 0 instead of being computed: the search above does
not report them, and their only reader is the adaptive hash index, which
the hints stand down for. */
up_bytes= 0;
low_bytes= 0;
/* search_leaf() also sets tree_height, which the hint cannot know because
it never walks the levels above the leaf. The value that the last descent
of this cursor left stands, and it is a real height of this tree: a hint is
only tried where a descent of this cursor already remembered a leaf of this
index. That matters because the value can travel: apart from search_leaf()
itself, its readers are the extent reservations of the pessimistic insert,
update and delete, which btr_pcur_copy_stored_position() reaches by copying
the whole cursor into the cursor of an update node. None of the three ever
sees a value that a hint left: each holds the index latch, which this
cursor can only take by descending again under BTR_MODIFY_TREE, and that
descent sets tree_height afresh. */
#ifdef BTR_CUR_HASH_ADAPT
flag= BTR_CUR_BINARY;
#endif
return true;
}
ATTRIBUTE_COLD void mtr_t::index_lock_upgrade()
{
auto &slot= m_memo[get_savepoint() - 1];

7
storage/innobase/handler/ha_innodb.cc
View File

@@ -16132,6 +16132,13 @@ ha_innobase::reset()
/* This is a statement level counter. */
m_prebuilt->autoinc_last_value = 0;
/* The clustered leaf hints are scoped to one statement. The slots
keep their key buffers, which are already sized for this table, and
their contents, which the next statement overwrites before reading:
a slot is only read once an insertion has counted it. */
m_prebuilt->clust_leaf_hint_n = 0;
m_prebuilt->clust_leaf_hint_miss = 0;
m_prebuilt->skip_locked = false;
return(0);
}

24
storage/innobase/include/btr0cur.h
View File

@@ -781,6 +781,30 @@ struct btr_cur_t {
will be set correctly. */
bool check_mismatch(const dtuple_t &tuple, bool ge, ulint comp) noexcept;
#endif
/** Try a PAGE_CUR_LE, BTR_SEARCH_LEAF lookup directly on a previously
remembered leaf page instead of descending from the root.
The hint is a guess, never derived from a latched parent page, so the
page is acquired via a non-blocking, no-I/O buf_page_try_get() and a
miss (stale hint) is expected, not corruption; the caller falls back
to a normal search.
Unlike search_leaf(), no index()->lock is acquired: that latch protects
the tree structure, which only a descent reads, while the contents of a
leaf page are protected by the page latch alone, and this never
dereferences a node pointer. A caller cannot rely on that latch after
search_leaf() either: it releases the latch for BTR_SEARCH_LEAF before
it returns, so both leave the caller the leaf latch alone.
For a clustered index only.
@param tuple key to search for: a complete unique key of the index,
compared over its n_fields_cmp fields
@param hint_page_id remembered leaf page id
@param step where the previous search of this leaf landed, used
as the start of this one while still valid, and
updated on success
@param mtr mini-transaction
@return whether the cursor was positioned on the hinted leaf page */
bool try_leaf_hint(const dtuple_t *tuple, page_id_t hint_page_id,
btr_leaf_step &step, mtr_t *mtr) noexcept;
};
/** Modify the delete-mark flag of a record.

15
storage/innobase/include/btr0types.h
View File

@@ -34,6 +34,21 @@ struct btr_pcur_t;
/** B-tree cursor */
struct btr_cur_t;
/** A user record that a search of a leaf page landed on, valid while the
block holds the same page and its modify_clock has not moved */
struct btr_leaf_step
{
/** the block that held the leaf, or nullptr */
const buf_block_t *block;
/** buf_block_t::modify_clock of block when rec was found */
uint64_t modify_clock;
/** the record that the search landed on */
const rec_t *rec;
/** whether the search landed on or right after the record of the search
before it, which is when the next one is worth starting from rec */
bool expect;
};
/** The size of a reference to data stored on a different page.
The reference is stored at the end of the prefix of the field
in the index record. */

15
storage/innobase/include/page0cur.h
View File

@@ -211,6 +211,21 @@ page_cur_delete_rec() for a ROW_FORMAT=COMPACT or DYNAMIC page.
bool page_apply_delete_dynamic(const buf_block_t &block, ulint prev,
size_t hdr_size, size_t data_size);
MY_ATTRIBUTE((warn_unused_result))
/** Position a cursor on a leaf page for PAGE_CUR_LE on a user record or on
the record before or after it, as a search that starts where the previous one
landed.
@param tuple search key
@param rec user record on cursor->block to start from
@param iup_fields matched fields in the upper limit record
@param ilow_fields matched fields in the low limit record
@param cursor page cursor
@return whether the cursor was positioned; false when the search would land
on none of the three records, and the cursor is then unchanged */
bool page_cur_search_near(const dtuple_t *tuple, const rec_t *rec,
uint16_t *iup_fields, uint16_t *ilow_fields,
page_cur_t *cursor) noexcept;
MY_ATTRIBUTE((warn_unused_result))
/** Search the right position for a page cursor.
@param tuple search key

101
storage/innobase/include/row0mysql.h
View File

@@ -456,6 +456,70 @@ struct mysql_row_templ_t {
#define ROW_PREBUILT_ALLOCATED 78540783
#define ROW_PREBUILT_FREED 26423527
/** A remembered clustered-index leaf page and the key range it covered when
it was remembered, for btr_cur_t::try_leaf_hint(). The keys are copies,
because the page is unlatched between two lookups.
The range is a filter only: a stale one (the page was split, merged, or a
record was inserted below its old minimum) can cost a wasted probe or a
needless descent, never a wrong result, because the checks that
btr_cur_t::try_leaf_hint() makes on the latched page remain the sole
authority. It therefore needs no invalidation protocol and no
modify_clock guard.
The fields that a lookup compares come first and the buffers that only
row_sel_clust_leaf_hint_remember() writes come last, each group widest
first, which is the order that keeps the slot smallest. */
struct clust_leaf_hint_slot {
const rec_t* first; /*!< copy of the leaf's first user
record, truncated to the key fields */
rec_offs* first_offs; /*!< rec_get_offsets() of first */
const rec_t* last; /*!< copy of the leaf's last user
record, truncated to the key fields, or
NULL where the leaf had no right
sibling: no lookup reads the last key
of the rightmost leaf, so a null here
is what says the leaf was that one */
rec_offs* last_offs; /*!< rec_get_offsets() of last. Both
arrays are made where the copies are,
so that a lookup compares against a
slot without parsing its records
again; they are sized for the key of
this index and never grow */
uint32_t page_no; /*!< the remembered leaf page number in
the clustered index's own tablespace;
page 0 is the FSP header, never a
leaf */
byte* first_buf; /*!< buffer owning first */
byte* last_buf; /*!< buffer owning last */
uint32_t first_buf_size; /*!< allocated size of first_buf */
uint32_t last_buf_size; /*!< allocated size of last_buf. A
record prefix is bounded by the page
size, so 32 bits hold either size */
btr_leaf_step step; /*!< where the last search of this leaf
landed */
};
/** Number of clustered leaf pages that a handle remembers, kept in most
recently used order. It buys the access patterns that come back to a few
leaves; it cannot buy a scan that is random over a table with many more
leaves than this, at any size.
The count is a cost limit and not a shape that some order is expected to
take. Every lookup that the hints do not answer scans the slots in use,
comparing at most two keys per slot and only one where the key sorts above
the range, so the count is kept small enough for that scan to stay under
the page-local searches of the descent that follows it.
Eviction takes the last position of the order, so a strict cycle over more
distinct leaves than there are slots finds each of them evicted again
before it comes back, and answers nothing, which is the least that any
order can be answered. An uneven order over the same leaves still answers a
part of its lookups, and the part grows with the count of slots. Which of
the two a scan meets is not decided here: one that answers too little pays
the miss count instead, which stands the hints down. */
constexpr ulint CLUST_LEAF_HINT_SLOTS= 4;
/** A struct for (sometimes lazily) prebuilt structures in an Innobase table
handle used within MySQL; these are used to save CPU time. */
@@ -574,11 +638,34 @@ struct row_prebuilt_t {
sel/upd/del */
lock_mode select_lock_type;/*!< LOCK_NONE, LOCK_S, or LOCK_X */
bool skip_locked; /*!< TL_{READ,WRITE}_SKIP_LOCKED */
uint8_t clust_leaf_hint_n;/*!< how many leading positions of
clust_leaf_hint_mru name a slot in use;
zeroed per statement in
ha_innobase::reset(), matching
autoinc_last_value */
uint16_t clust_leaf_hint_miss;/*!< lookups that no slot of
clust_leaf_hint answered, weighted
against the lookups that one did; see
CLUST_LEAF_HINT_MISS_WEIGHT in
row0sel.cc. Zeroed per statement with
clust_leaf_hint_n */
lock_mode stored_select_lock_type;/*!< this field is used to
remember the original select_lock_type
that was decided in ha_innodb.cc,
::store_lock(), ::external_lock(),
etc. */
uint8_t clust_leaf_hint_mru[CLUST_LEAF_HINT_SLOTS];
/*!< the slots of clust_leaf_hint in
most recently used order, held as slot
numbers: a lookup reads the slots in
this order and a hit moves the number
of the slot that answered to the
front, so that the order costs a few
bytes to keep and a slot never moves,
which lets it own its key buffers for
good. Set to the identity where
clust_leaf_hint is allocated, and a
permutation of the slots from there */
ulint row_read_type; /*!< ROW_READ_WITH_LOCKS if row locks
should be the obtained for records
under an UPDATE or DELETE cursor.
@@ -695,6 +782,20 @@ struct row_prebuilt_t {
/** The MySQL table object */
TABLE* m_mysql_table;
/** CLUST_LEAF_HINT_SLOTS clustered leaves remembered from the
Row_sel_get_clust_rec_for_mysql() lookups of this statement, in no
order of their own, because clust_leaf_hint_mru holds the most recently
used order, or NULL if no lookup of this handle has descended yet.
Allocated from heap on the first descent, so that a
handle that never needs a clustered lookup allocates nothing, and one
that does pays for the slots once, not per statement; the key buffers,
which are already sized for this table, outlive the statement and are
released in row_prebuilt_free().
Declared last, apart from the counters that go with it, so that every
field that precedes it keeps the offset, and with it the cache line,
that it had before this pointer existed. */
clust_leaf_hint_slot* clust_leaf_hint;
/** Get template by dict_table_t::cols[] number */
const mysql_row_templ_t* get_template_by_col(ulint col) const
{

87
storage/innobase/page/page0cur.cc
View File

@@ -807,6 +807,93 @@ static bool page_cur_try_search_shortcut(const page_t *page, const rec_t *rec,
return true;
}
bool page_cur_search_near(const dtuple_t *tuple, const rec_t *rec,
uint16_t *iup_fields, uint16_t *ilow_fields,
page_cur_t *cursor) noexcept
{
const dict_index_t &index= *cursor->index;
const page_t *const page= cursor->block->page.frame;
ut_ad(page_is_leaf(page));
ut_ad(page_align(rec) == page);
ut_ad(page_rec_is_user_rec(rec));
const auto comp= page_is_comp(page);
/* The metadata pseudo-record compares below every key on 0 fields, which
only the binary search accounts for. */
if (rec_get_info_bits(rec, comp) & REC_INFO_MIN_REC_FLAG)
return false;
uint16_t low= 0, up= 0;
int cmp= page_cur_dtuple_cmp(*tuple, rec, index, &low, comp);
if (cmp < 0)
{
/* rec sorts above tuple. The search lands on the record before rec when
that record does not sort above tuple, and rec is then the upper limit.
Nothing bounds the fields that tuple shares with the record before rec,
so that comparison starts from 0 fields. */
up= low;
low= 0;
const rec_t *const prev= page_rec_get_prev_const(rec);
if (!prev)
return false;
if (page_rec_is_infimum(prev))
{
DBUG_EXECUTE_IF("ib_log_clust_leaf_hint_step_back",
ib::info() << "Clustered leaf hint step back "
"reached the infimum";);
return false;
}
if (rec_get_info_bits(prev, comp) & REC_INFO_MIN_REC_FLAG)
{
DBUG_EXECUTE_IF("ib_log_clust_leaf_hint_step_back",
ib::info() << "Clustered leaf hint step back "
"reached the metadata record";);
return false;
}
if (page_cur_dtuple_cmp(*tuple, prev, index, &low, comp) < 0)
return false;
rec= prev;
}
else if (cmp)
{
/* rec sorts below tuple. The search lands on rec when the record after
it is the supremum or sorts above tuple, and on that record when it
equals tuple; a record after it that sorts below tuple is left to the
binary search. Nothing bounds the fields that tuple shares with the
record after rec, so that comparison starts from 0 fields. */
const rec_t *const next= comp
? page_rec_next_get<true>(page, rec)
: page_rec_next_get<false>(page, rec);
if (!next)
return false;
if (next != page + (comp ? PAGE_NEW_SUPREMUM : PAGE_OLD_SUPREMUM))
{
if (comp)
switch (rec_get_status(next)) {
case REC_STATUS_INSTANT:
case REC_STATUS_ORDINARY:
break;
default:
return false;
}
cmp= page_cur_dtuple_cmp(*tuple, next, index, &up, comp);
if (cmp > 0)
return false;
if (!cmp)
{
rec= next;
low= up;
up= 0;
}
}
}
page_cur_position(rec, cursor->block, cursor);
*iup_fields= up;
*ilow_fields= low;
return true;
}
bool page_cur_search_with_match(const dtuple_t *tuple, page_cur_mode_t mode,
uint16_t *iup_fields, uint16_t *ilow_fields,
page_cur_t *cursor, rtr_info_t *rtr_info)

11
storage/innobase/row/row0mysql.cc
View File

@@ -940,6 +940,17 @@ void row_prebuilt_free(row_prebuilt_t *prebuilt)
ut_free(prebuilt->mysql_template);
if (prebuilt->clust_leaf_hint) {
/* The slots are on prebuilt->heap, but the buffers that
rec_copy_prefix_to_buf() allocated for their keys are not.
Every slot is freed, not only the ones in use: a slot that
was discarded or evicted keeps the buffers it owned. */
for (ulint i = 0; i < CLUST_LEAF_HINT_SLOTS; i++) {
ut_free(prebuilt->clust_leaf_hint[i].first_buf);
ut_free(prebuilt->clust_leaf_hint[i].last_buf);
}
}
if (prebuilt->ins_graph) {
que_graph_free_recursive(prebuilt->ins_graph);
}

415
storage/innobase/row/row0sel.cc
View File

@@ -73,6 +73,57 @@ to que_run_threads: this is to allow canceling runaway queries */
#define SEL_EXHAUSTED 1
#define SEL_RETRY 2
/** Lookups that no slot of row_prebuilt_t::clust_leaf_hint answered, with no
lookup that one did between them, after which a scan is taken to have too
little locality to pay for the slots. It then stops testing them and stops
refreshing them, which is the larger half of their cost: two key copies for
every lookup that they do not answer.
What a hit saves is one buffer pool access and one page-local search for
each level above the leaf, so a scan that seldom hits pays more for the
slots than it takes from them. This count is what a scan spends before it
finds that out. Misses are weighted against hits, so a scan that answers a
part of its lookups reaches the threshold later than one that answers none,
and a scan that answers enough of them never reaches it; see
CLUST_LEAF_HINT_MISS_WEIGHT. */
constexpr unsigned CLUST_LEAF_HINT_MAX_MISSES= 8;
/** What a lookup that no slot answered adds to the miss counter, against
the one that a lookup that a slot answered takes away. The counter therefore
follows the answer rate of the scan and not the length of its longest run of
misses. Without the weight a hit would clear the count, and a scan that
answered one lookup in ten would keep the slots for the whole statement.
At this weight the counter climbs while the scan answers less than two
thirds of its lookups and falls while it answers more, so two thirds is the
rate that a scan has to hold to keep the slots. The floor at zero holds a
scan that answers well away from the threshold, so a run of misses alone
does not stand it down. */
constexpr unsigned CLUST_LEAF_HINT_MISS_WEIGHT= 2;
/** The miss counter at which a scan gives the slots up. A scan reaches it
after CLUST_LEAF_HINT_MAX_MISSES lookups that no slot answered where no
lookup between them was answered, and later than that where some were. */
constexpr unsigned CLUST_LEAF_HINT_GIVE_UP=
CLUST_LEAF_HINT_MAX_MISSES * CLUST_LEAF_HINT_MISS_WEIGHT;
/** How often the clustered leaf hints are tested again once
CLUST_LEAF_HINT_GIVE_UP has been reached, in lookups. A scan whose order
becomes correlated only later recovers after at most this many rows, instead
of losing the hints for the rest of the statement. The trial that a retry
begins starts the count again, so a scan that is still uncorrelated pays
CLUST_LEAF_HINT_MAX_MISSES lookups of the slots where none of them is
answered, and more where some are, before it stands them down once more. */
constexpr unsigned CLUST_LEAF_HINT_RETRY= 1024;
/** The clustered-index lookup of a statement that fills its first slot. The
lookups before it fill none. A slot is filled by copying two keys, and that is
paid where a later lookup may read it, so a statement with fewer lookups than
this copies no keys. The value is that of MYSQL_FETCH_CACHE_THRESHOLD, for the
reason that row_search_mvcc() holds its fetch cache back: under it a statement
is a lookup or a short list of them and not a scan. */
constexpr unsigned CLUST_LEAF_HINT_MIN_LOOKUPS= 4;
/********************************************************************//**
Returns TRUE if the user-defined column in a secondary index record
is alphabetically the same as the corresponding BLOB column in the clustered
@@ -3353,6 +3404,315 @@ public:
dtuple_t **vrow, mtr_t *mtr);
};
/** Determine whether a key can be on the leaf that a clustered leaf hint
remembers, comparing the key against the copies of that leaf's boundary
records. This decides a miss without any buffer pool access, so an
uncorrelated scan pays no page access for the hint it cannot use.
@param hint a non-empty clustered leaf hint
@param tuple key to search for
@param index the clustered index
@return whether the hinted leaf is worth probing */
static bool row_sel_clust_leaf_hint_covers(const clust_leaf_hint_slot &hint,
const dtuple_t *tuple,
const dict_index_t *index)
{
ut_ad(hint.page_no);
/* A used slot always names the first key of its leaf. It names the last
one only where a lookup can read it, so a slot without one is a slot of
the rightmost leaf. */
ut_ad(hint.first);
/* The copies and their offsets remain valid when
dict_index_t::clear_instant_alter() raises dict_index_t::n_core_fields
during the statement: rec_copy_prefix_to_buf() writes them without
instant ADD COLUMN, and the comparisons depend on the key fields alone. */
/* This is the field count that row_sel_clust_leaf_hint_remember() copied,
and the one that its offsets describe. */
ut_ad(dtuple_get_n_fields_cmp(tuple) == dict_index_get_n_unique(index));
/* The upper bound comes first: an ascending scan leaves each slot for a
key above its range, which one comparison rejects. A key above the last
record of the rightmost leaf still belongs to that leaf, mirroring the
page_has_next() test in btr_cur_t::try_leaf_hint(), and a slot of such a
leaf names no last key. */
if (hint.last &&
cmp_dtuple_rec(tuple, hint.last, index, hint.last_offs) > 0)
return false;
/* On the leftmost leaf of a table that was subjected to instant ALTER
TABLE, the first user record is the metadata pseudo-record.
cmp_dtuple_rec_with_match_low() settles that comparison from
REC_INFO_MIN_REC_FLAG alone, which rec_copy_prefix_to_buf() preserves,
and reports every key as sorting above it: the correct lower bound for
the leaf that precedes all others. */
return cmp_dtuple_rec(tuple, hint.first, index, hint.first_offs) >= 0;
}
/** Move a clustered leaf hint to another position of the most recently used
order, shifting every position in between by one. The order holds slot
numbers, so this moves those and never a slot: a slot keeps the key buffers
that it owns for the life of the handle, and a scan that alternates between
a few leaves reorders a few bytes per row rather than the slots themselves.
@param mru the most recently used order
@param from the position to move
@param to where to move it */
static void row_sel_clust_leaf_hint_move(uint8_t *mru, ulint from, ulint to)
{
ut_ad(from < CLUST_LEAF_HINT_SLOTS);
ut_ad(to < CLUST_LEAF_HINT_SLOTS);
if (from == to)
return;
const uint8_t moved= mru[from];
if (from > to)
memmove(mru + to + 1, mru + to, from - to);
else
memmove(mru + from, mru + from + 1, to - from);
mru[to]= moved;
}
/** Decide whether the clustered leaf hints take part in this lookup.
Both halves of their cost are governed here: the test of the slots before
the descent, and the copies that refresh them after it. A scan that has
given up must pay for neither.
@param prebuilt prebuilt struct of the handle
@return whether the slots are to be tested and refreshed */
static bool row_sel_clust_leaf_hint_armed(row_prebuilt_t *prebuilt)
{
DBUG_EXECUTE_IF("ib_no_clust_leaf_hint", return false;);
const unsigned misses= prebuilt->clust_leaf_hint_miss;
if (misses < CLUST_LEAF_HINT_GIVE_UP)
return true;
if (misses % CLUST_LEAF_HINT_RETRY)
{
/* This scan has shown that it has no locality to exploit. */
prebuilt->clust_leaf_hint_miss= uint16_t(misses + 1);
return false;
}
/* One lookup in CLUST_LEAF_HINT_RETRY starts the count again, so that a
scan whose order becomes correlated only later recovers, after at most
that many rows, instead of losing the hints for the rest of the
statement. The trial that this begins is what makes the recovery
possible: the slots hold the leaves of the row where the scan gave up,
which nothing has refreshed since, so it takes a miss that remembers the
leaf the scan is on now before a later lookup can be answered. */
prebuilt->clust_leaf_hint_miss= 0;
return true;
}
/** Try the leaves that this statement remembered, most recently used first.
@param prebuilt prebuilt struct of the handle
@param index the clustered index
@param mtr mini-transaction
@return whether prebuilt->clust_pcur was positioned on a remembered leaf */
static bool row_sel_clust_leaf_hint_search(row_prebuilt_t *prebuilt,
const dict_index_t *index,
mtr_t *mtr)
{
const ulint n= prebuilt->clust_leaf_hint_n;
ut_ad(n <= CLUST_LEAF_HINT_SLOTS);
const unsigned misses= prebuilt->clust_leaf_hint_miss;
ut_ad(misses < CLUST_LEAF_HINT_GIVE_UP);
clust_leaf_hint_slot *const hints= prebuilt->clust_leaf_hint;
ut_ad(hints || !n);
uint8_t *const mru= prebuilt->clust_leaf_hint_mru;
for (ulint i= 0; i < n; i++)
{
clust_leaf_hint_slot &hint= hints[mru[i]];
if (!row_sel_clust_leaf_hint_covers(hint, prebuilt->clust_ref, index))
continue;
/* Two ranges can cover the same key only if one of them is stale, so
there is nothing to gain from looking past the first candidate: the
descent resolves whatever this one cannot. */
if (prebuilt->clust_pcur->btr_cur.try_leaf_hint(
prebuilt->clust_ref,
page_id_t(index->table->space_id, hint.page_no), hint.step, mtr))
{
/* A hit takes one off the count rather than clearing it, so that the
count follows the answer rate of the scan and not the length of its
longest run of misses; see CLUST_LEAF_HINT_MISS_WEIGHT. */
prebuilt->clust_leaf_hint_miss= uint16_t(misses ? misses - 1 : 0);
row_sel_clust_leaf_hint_move(mru, i, 0);
return true;
}
/* The probe did not answer: the page is not in the buffer pool with an
uncompressed frame, a writer holds its latch, or the checks that
btr_cur_t::try_leaf_hint() makes on the latched page rejected it.
Discard the slot past the end of the used slots, where its key buffers
are the ones that the next insertion takes over: the descent that
follows remembers the leaf it lands on, so a slot dropped for a passing
reason is restored at once, and one whose range was stale is replaced by
a correct one. This counts as a miss, like a key that no range covered:
only a page that answered is locality. A page that never answers, as a
ROW_FORMAT=COMPRESSED page that the buffer pool holds without an
uncompressed frame never does, would otherwise hold the counter at zero
and be probed once per row for the whole statement. */
prebuilt->clust_leaf_hint_n= uint8_t(n - 1);
row_sel_clust_leaf_hint_move(mru, i, n - 1);
break;
}
prebuilt->clust_leaf_hint_miss=
uint16_t(misses + CLUST_LEAF_HINT_MISS_WEIGHT);
return false;
}
/** Remember the clustered leaf that a descent landed on, together with the
keys of its first and last user record, at the front of the most recently
used order. The key buffers grow in place and travel with their slot, so a
scan allocates at most twice per slot, and nothing per row.
@param prebuilt prebuilt struct of the handle
@param block the clustered index leaf page the cursor is positioned on
@param index the clustered index */
static void row_sel_clust_leaf_hint_remember(row_prebuilt_t *prebuilt,
const buf_block_t *block,
const dict_index_t *index)
{
if (!prebuilt->clust_leaf_hint_n &&
prebuilt->clust_leaf_hint_miss <
CLUST_LEAF_HINT_MIN_LOOKUPS * CLUST_LEAF_HINT_MISS_WEIGHT)
/* One of the first lookups of this statement, counted by the misses that
a lookup makes while no slot is in use, both of which ha_innobase::reset()
zeroes. A statement that ends here copied keys that it never tested, so
the slots begin above the threshold. A statement that discarded its last
slot passes here again and pays the same descents, on the same terms. */
return;
if (block->page.id().page_no() == index->page)
/* The leaf is the root, so the clustered index is one page. A lookup
that a slot answers acquires that page and searches it, which is all
that the descent it replaces does, so there is no saving to pay for the
copies below or for the test of the slots that each row would make. */
return;
const page_t *const page= block->page.frame;
ut_ad(page_is_leaf(page));
const rec_t *const first= page_rec_get_next_const(page_get_infimum_rec(page));
const rec_t *const last= page_rec_get_prev_const(page_get_supremum_rec(page));
if (UNIV_UNLIKELY(!first || !last || page_rec_is_supremum(first) ||
page_rec_is_infimum(last)))
/* A corrupted record list; there is nothing worth remembering. Only the
root of an empty tree is empty, and the check above left that one. */
return;
const ulint n_fields= dict_index_get_n_unique(index);
clust_leaf_hint_slot *hints= prebuilt->clust_leaf_hint;
if (!hints)
{
/* Allocated on the first descent rather than with the handle, so that a
handle that never needs a clustered lookup allocates nothing.
The offsets arrays are allocated here with the slots and never grow,
unlike the key buffers, because their size follows the key field count
of the index and not the length of a key: rec_get_offsets() describes at
most the n_fields fields that it is asked for. The order starts as the
identity and stays a permutation of the slots from there. */
const ulint n_offs= n_fields + (1 + REC_OFFS_HEADER_SIZE);
hints= static_cast<clust_leaf_hint_slot*>
(mem_heap_zalloc(prebuilt->heap, CLUST_LEAF_HINT_SLOTS * sizeof *hints));
rec_offs *offs= static_cast<rec_offs*>
(mem_heap_alloc(prebuilt->heap,
2 * CLUST_LEAF_HINT_SLOTS * n_offs * sizeof *offs));
for (ulint i= 0; i < CLUST_LEAF_HINT_SLOTS; i++)
{
rec_offs_set_n_alloc(offs, n_offs);
hints[i].first_offs= offs;
offs+= n_offs;
rec_offs_set_n_alloc(offs, n_offs);
hints[i].last_offs= offs;
offs+= n_offs;
prebuilt->clust_leaf_hint_mru[i]= uint8_t(i);
}
prebuilt->clust_leaf_hint= hints;
}
const ulint n= prebuilt->clust_leaf_hint_n;
ut_ad(n <= CLUST_LEAF_HINT_SLOTS);
const uint32_t page_no= block->page.id().page_no();
ut_ad(page_no);
/* This leaf can be remembered already, because a lookup that no range
covered descends without consulting any page, and the live range of a leaf
grows past the remembered one where a record is inserted above its last, or
where a sibling merges into it. Refresh that slot, rather than spend a
second one of the few on the same page. */
uint8_t *const mru= prebuilt->clust_leaf_hint_mru;
ulint from= n < CLUST_LEAF_HINT_SLOTS ? n : CLUST_LEAF_HINT_SLOTS - 1;
bool remembered= false;
for (ulint i= 0; i < n; i++)
if (hints[mru[i]].page_no == page_no)
{
from= i;
remembered= true;
break;
}
/* Move the slot to the front of the order, taking the least recently used
one once the array is full. The slot that this displaces, or refreshes, is
the one whose buffers the copies below reuse, which makes the eviction
exact and free. */
row_sel_clust_leaf_hint_move(mru, from, 0);
if (!remembered && n < CLUST_LEAF_HINT_SLOTS)
prebuilt->clust_leaf_hint_n= uint8_t(n + 1);
clust_leaf_hint_slot &hint= hints[mru[0]];
/* The offsets go with the copy they describe, and both stay with the slot,
which the order above moves without moving; rec_copy_prefix_to_buf() can
still move a copy inside its slot when it grows the buffer.
prebuilt->heap is passed for a growth that the sizing above rules out,
so that an array which did grow would still outlive the statement. A
buffer size travels through a ulint, which is what
rec_copy_prefix_to_buf() takes, and a slot keeps it in 32 bits, wide
enough for a prefix of at most one page. */
ulint buf_size= hint.first_buf_size;
hint.first= rec_copy_prefix_to_buf(first, index, n_fields, &hint.first_buf,
&buf_size);
ut_ad(buf_size <= UINT32_MAX);
hint.first_buf_size= uint32_t(buf_size);
hint.first_offs= rec_get_offsets(hint.first, index, hint.first_offs,
index->n_core_fields, n_fields,
&prebuilt->heap);
if (!page_has_next(page))
/* Every key above the last record of the rightmost leaf still belongs
to that leaf, so row_sel_clust_leaf_hint_covers() never reads the last
key of one, and the null that this leaves is what tells it so. The
buffer that the slot owns stays with it for the next occupant to
reuse. */
hint.last= nullptr;
else
{
buf_size= hint.last_buf_size;
hint.last= rec_copy_prefix_to_buf(last, index, n_fields, &hint.last_buf,
&buf_size);
ut_ad(buf_size <= UINT32_MAX);
hint.last_buf_size= uint32_t(buf_size);
hint.last_offs= rec_get_offsets(hint.last, index, hint.last_offs,
index->n_core_fields, n_fields,
&prebuilt->heap);
}
hint.page_no= page_no;
const rec_t *const rec= prebuilt->clust_pcur->btr_cur.page_cur.rec;
if (!page_rec_is_user_rec(rec))
hint.step.block= nullptr;
else
{
hint.step.block= block;
hint.step.modify_clock= block->modify_clock();
hint.step.rec= rec;
hint.step.expect= true;
}
}
/*********************************************************************//**
Retrieves the clustered index record corresponding to a record in a
non-clustered index. Does the necessary locking. Used in the MySQL
@@ -3397,9 +3757,62 @@ Row_sel_get_clust_rec_for_mysql::operator()(
clust_index = dict_table_get_first_index(sec_index->table);
prebuilt->clust_pcur->btr_cur.page_cur.index = clust_index;
dberr_t err = btr_pcur_open_with_no_init(prebuilt->clust_ref,
/* The rows of a non-covering secondary-index scan often share a few
clustered leaf pages, so try the leaves that this statement already
visited before descending again. The remembered key ranges decide the
uncorrelated case in memory, so a scan that the hints cannot serve
pays neither a buffer pool access nor a pages_accessed for them.
A probe would come before the descent, and therefore also before the
adaptive hash index guess that the descent tries first. A guess lands
directly on the record, with no page-local search and no buffer pool
access to charge, where a hint hit costs both. But neither of the two
is faster for every shape of index and scan, so no fixed order of the
two is correct for every index. Also, a hint that answers a lookup
before the guess stops the adaptive hash index from adding entries for
that index. So where the adaptive hash index is enabled, the hints stay
out of its way entirely, neither used nor collected.
The flag is read again at every lookup, so a change in the middle of
a statement takes effect at the next lookup, and the slots that the
change leaves behind are reset in ha_innobase::reset(). The test is
coarse, because the flag is global: an index that the adaptive hash
index does not serve loses the hints too. Both are heuristics, so the
price of either answer is a descent, never a wrong result. */
#ifdef BTR_CUR_HASH_ADAPT
static_assert(std::is_same<decltype(btr_search.enabled),
Atomic_relaxed<bool>>::value,
"use !btr_search.is_enabled(clust_index)");
const bool use_hints = !btr_search.enabled;
#else
const bool use_hints = true;
#endif /* BTR_CUR_HASH_ADAPT */
dberr_t err;
const bool hints_armed = use_hints
&& row_sel_clust_leaf_hint_armed(prebuilt);
if (hints_armed
&& row_sel_clust_leaf_hint_search(prebuilt, clust_index, mtr)) {
err = DB_SUCCESS;
/* Set what btr_pcur_open_with_no_init() below would set for
the same (PAGE_CUR_LE, BTR_SEARCH_LEAF) arguments, except
trx_if_known, which is assigned unconditionally further
down. */
prebuilt->clust_pcur->latch_mode
= BTR_LATCH_MODE_WITHOUT_INTENTION(BTR_SEARCH_LEAF);
prebuilt->clust_pcur->search_mode = PAGE_CUR_LE;
prebuilt->clust_pcur->pos_state = BTR_PCUR_IS_POSITIONED;
} else {
err = btr_pcur_open_with_no_init(prebuilt->clust_ref,
PAGE_CUR_LE, BTR_SEARCH_LEAF,
prebuilt->clust_pcur, mtr);
if (hints_armed && err == DB_SUCCESS) {
row_sel_clust_leaf_hint_remember(
prebuilt,
prebuilt->clust_pcur->btr_cur.page_cur.block,
clust_index);
}
}
if (UNIV_UNLIKELY(err != DB_SUCCESS)) {
return err;
}