mirror of
https://github.com/MariaDB/server.git
synced 2026-10-23 18:49:29 +03:00
1f71dcfc3c
Problem:
=======
When persistent statistics are enabled (innodb_stats_persistent=ON),
the innodb_stats_method setting is not properly utilized during
statistics calculation.
The statistics collection functions always use a hardcoded default
behavior for NULL value comparison instead of respecting the
configured stats method. This affects the accuracy of
n_diff_key_vals (distinct key count), particularly for
indexes with nullable columns containing NULL values.
Moreover, stat_n_non_null_key_vals[] was never computed for
persistent statistics; it stayed at the 0 that
dict_stats_empty_index() assigns.
With innodb_stats_method=nulls_ignored, innodb_rec_per_key()
therefore always found n_diff <= n_null and reported one record
per key for every index. This impacts the query optimizer,
which makes decisions based on inaccurate cardinality estimates.
Solution:
========
Introduced IndexLevelStats to collect statistics at a specific
B-tree level during index analysis.
Introduced PageStats to collect statistics for leaf page analysis.
Refactored the following functions:
dict_stats_analyze_index_level() to IndexLevelStats::analyze_level()
dict_stats_analyze_index_for_n_prefix() to IndexLevelStats::sample_leaf_pages()
dict_stats_analyze_index_below_cur() to PageStats::scan_below()
dict_stats_scan_page() to PageStats::scan()
The innodb_stats_method value is read once per table in
dict_stats_update_persistent() and passed down, so that all
indexes of a table are analyzed with the same method.
Add the stats method name to stat_description when
innodb_stats_method has a non-default value. The suffix is
dropped when the description is already full.
Added the new stat name n_nonnull_fld01, n_nonnull_fld02, etc.
with a stats description, to indicate how many non-null values
exist for the nth field of the index. This value is retrieved
and stored in the index statistics
in dict_stats_fetch_index_stats_step(). The counts are per
column, not per n-column prefix.
rec_get_n_blob_pages(): Calculate the number of
externally stored pages for a record, using ceiling division
by the usable BLOB page payload (blob_part_size), which differs
between ROW_FORMAT=COMPRESSED (zip_size minus FIL_PAGE_DATA) and
the other formats (srv_page_size minus the BLOB header and the
page trailer). For ROW_FORMAT=COMPRESSED the length in
the field reference is the uncompressed length, so the result is an
upper bound.
When the leaf level is scanned in full, the number of leaf pages that
were scanned is reported as n_leaf_pages for a multi level index.
Before, result.n_leaf_pages was overwritten with
index->stat_n_leaf_pages, which dict_stats_empty_index() had
just set to 1, so every index that took the full scan path
reported n_leaf_pages=1. Single page indexes report 1.
This changes cardinality estimates and
therefore leads to multiple changes in existing test cases.
Non-null values are counted only at the leaf level, since only leaf
pages hold actual records. A full scan of the leaf level counts them
exactly. When the level is sampled, the per column count is derived
from the sampled leaves with the same formula as n_diff:
n_ordinary_leaf_pages * n_non_null_all_analyzed_pages
/ n_leaf_pages_to_analyze
This is an estimate for NOT NULL columns as well: the sampled leaves
may hold fewer or more records than the average, and a dive that
stops at a boring page contributes nothing to the sum while still
counting in the divisor.
innodb_rec_per_key(): stat_n_non_null_key_vals[i] holds the
number of records in which the i-th indexed column alone is
not NULL, while what has to be excluded here is the number
of records whose first i+1 columns are all not NULL,
because that is the population which the n-column
prefix statistic stat_n_diff_key_vals[i] has to be
corrected against when innodb_stats_method=nulls_ignored:
with NULLs compared as unequal, every record carrying a NULL
anywhere in the prefix adds a distinct value of its own to n_diff.
PageStats::scan(): n_non_null is accumulated and assigned only
for leaf pages, so that a non-leaf scan cannot leave a node
pointer count behind when scan_below() stops at a boring page
without reaching a leaf.
IndexLevelStats::reset_for_level() also clears n_diff[], and
dict_stats_analyze_index() zero initializes the buffer backing it, so
that a level scan which finds no records (a failed
btr_pcur_open_level(), or a non-leaf page whose first record is not
marked as the leftmost one on the level) leaves n_diff[] at 0 instead
of stale values.
IndexLevelStats::sample_leaf_pages() returns early when the group
boundaries for the prefix are empty, which is the same condition.
IndexLevelStats::analyze_level(): Instead of copying the last record
of the page, retain the latch on the page until the record has been
compared with the first record of the next page
dict_stats_fetch_index_stats_step() no longer resets
stat_n_non_null_key_vals[] while processing an n_diff_pfxNN row:
dict_stats_empty_table() has already cleared the array before the
fetch, and with n_nonnull_fldNN rows now being read too,
that reset would make the result depend on the order in which the
rows arrive.
dict_stats_save(): now static function in dict0stats.cc that
takes the innodb_stats_method value, and is removed from dict0stats.h.
dict_stats_update_persistent() saves the statistics itself, so its
callers no longer have to.
Replaced btr_rec_get_externally_stored_len() with
rec_get_n_blob_pages() in dict0stats.cc.
btr_rec_get_field_ref_offs() and btr_rec_get_field_ref(),
together with the BTR_BLOB_HDR_* macros, were moved from btr0cur.cc
to btr0cur.h so that rec_get_n_blob_pages() can reuse them;
btr_rec_get_field_ref_offs() is now a noexcept function
returning size_t.
Changed stat_n_diff_key_vals and stat_n_non_null_key_vals from
ib_uint64_t* to uint64_t*
len_is_stored(): simplified to a single comparison, which is
equivalent for the unsigned lengths that it is used with.
Removed the unused UNIV_STATS_DEBUG build macro (univ.i) and
turned the DEBUG_PRINTF() helper in dict0stats.cc into an
unconditional no-op
54 lines
1.5 KiB
Plaintext
54 lines
1.5 KiB
Plaintext
CREATE TABLE autorecalc (a INT, PRIMARY KEY (a)) ENGINE=INNODB;
|
|
SELECT n_rows, clustered_index_size FROM mysql.innodb_table_stats WHERE table_name = 'autorecalc';
|
|
n_rows 0
|
|
clustered_index_size 1
|
|
SELECT index_name, stat_name, stat_value FROM mysql.innodb_index_stats WHERE table_name = 'autorecalc';
|
|
index_name PRIMARY
|
|
stat_name n_diff_pfx01
|
|
stat_value 0
|
|
index_name PRIMARY
|
|
stat_name n_leaf_pages
|
|
stat_value 1
|
|
index_name PRIMARY
|
|
stat_name n_nonnull_fld01
|
|
stat_value 0
|
|
index_name PRIMARY
|
|
stat_name size
|
|
stat_value 1
|
|
INSERT INTO autorecalc VALUES (1);
|
|
INSERT INTO autorecalc VALUES (2);
|
|
SELECT n_rows, clustered_index_size FROM mysql.innodb_table_stats WHERE table_name = 'autorecalc';
|
|
n_rows 2
|
|
clustered_index_size 1
|
|
SELECT index_name, stat_name, stat_value FROM mysql.innodb_index_stats WHERE table_name = 'autorecalc';
|
|
index_name PRIMARY
|
|
stat_name n_diff_pfx01
|
|
stat_value 2
|
|
index_name PRIMARY
|
|
stat_name n_leaf_pages
|
|
stat_value 1
|
|
index_name PRIMARY
|
|
stat_name n_nonnull_fld01
|
|
stat_value 2
|
|
index_name PRIMARY
|
|
stat_name size
|
|
stat_value 1
|
|
DELETE FROM autorecalc;
|
|
SELECT n_rows, clustered_index_size FROM mysql.innodb_table_stats WHERE table_name = 'autorecalc';
|
|
n_rows 0
|
|
clustered_index_size 1
|
|
SELECT index_name, stat_name, stat_value FROM mysql.innodb_index_stats WHERE table_name = 'autorecalc';
|
|
index_name PRIMARY
|
|
stat_name n_diff_pfx01
|
|
stat_value 0
|
|
index_name PRIMARY
|
|
stat_name n_leaf_pages
|
|
stat_value 1
|
|
index_name PRIMARY
|
|
stat_name n_nonnull_fld01
|
|
stat_value 0
|
|
index_name PRIMARY
|
|
stat_name size
|
|
stat_value 1
|
|
DROP TABLE autorecalc;
|