This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 12.3.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 11.8.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
A join of only const tables stored the WHERE conjuncts that refer to a
column of an outer query or call rand() in pseudo_bits_cond. The
executor read pseudo_bits_cond only when the query needed no temporary
table, and read outer_ref_cond otherwise. A window function or
DISTINCT with ROLLUP needs a temporary table, so those conjuncts were
never checked, and the query returned rows that the WHERE clause
rejects.
pseudo_bits_cond came from the fix for LP bug 908269 in 2012. For a
join of only const tables, that fix also collected conjuncts that call
rand(), and it stored them in a new field without stating a reason.
The code that filled the two fields and the code that read them chose
between them by the same test. The tests have differed since
MDEV-8646 in 2016, which added the temporary table condition to the
executor's test. Later changes only display the field, in EXPLAIN
FORMAT=JSON (MDEV-19714) and in the optimizer trace (MDEV-21095). In
MDEV-19714, its author wrote that it was not clear why pseudo_bits_cond
was needed in addition to outer_ref_cond.
pseudo_bits_cond is removed, and outer_ref_cond holds these conjuncts
for every join. The executor checks it on both paths, so where the
conjuncts are stored no longer depends on the executor's test.
Conjuncts that call rand() are still collected only for a join of only
const tables. EXPLAIN FORMAT=JSON shows these conjuncts as
outer_ref_condition instead of pseudo_bits_condition.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
valgrind reported use of an un-initialized value, when trying to write
analyzed plan to json. Instead of fetching c_str() from the string
object and passing it to the writer object's add_string() method, one could
use an overridden add_string() that accepts a different string object.
This would avoid the valgrind warnings.
There are two types of partition index scans. Ordered and unordered.
Ordered scan is used in common cases, including when the partitioned
table is the first in the join order and when ORDER BY ... DESC.
Whereas unordered scan is used when the partitioned table is not the
first in the join order, or in mrr when mrr_mode does not have
HA_MRR_SORTED. It can be unclear whether ordered or unordered scan has
been used. Therefore in this patch we add a field
"partitions_index_scan_method" to ANALYZE FORMAT=JSON to reveal with
method has been used. Possible values include
"merge_ordered_scans" (for ordered scan)
"iterate_over_partitions" (for unordered scan) and "both" (when both
have been used).
This commits adds the "materialization" block to the output of
EXPLAIN/ANALYZE FORMAT=JSON when materialized subqueries are involved
into processing. In the case of ANALYZE additional runtime information
is displayed, such as:
- chosen strategy of materialization
- number of partial match/index lookup loops
- sizes of partial match buffers
The symptoms were: take a server with no activity and a table that's
not in the buffer pool. Run a query that reads the whole table and
observe that r_engine_stats.pages_read_count shows about 2% of the table
was read. Who reads the rest?
The cause was that page prefetching done inside InnoDB was not counted.
This counts page prefetch requests made in buf_read_ahead_random() and
buf_read_ahead_linear() and makes them visible in:
- ANALYZE: r_engine_stats.pages_prefetch_read_count
- Slow Query Log: Pages_prefetched:
This patch intentionally doesn't attempt to count the time to read the
prefetched pages:
* there's no obvious place where one can do it
* prefetch reads may be done in parallel (right?), it is not clear how
to count the time in this case.
- Change the comments in class ha_handler_stats to say the members
are in ticks, not milliseconds.
- In sql_explain.cc, adjust the scale to print milliseconds.
Part#2, variant 2: Make the printed r_ values in JSON output consistent.
After this patch, ANALYZE output has:
- r_index_rows (NEW) - Observed number of rows before ICP or Rowid Filtering
checks. This is a per-scan average. like r_rows and "rows" are.
- r_rows (AS BEFORE) - Observed number of rows after ICP and Rowid Filtering.
- r_icp_filtered (NEW) - Observed selectivity of ICP condition.
- (AS BEFORE) observed selectivity of Rowid Filter is in
$.rowid_filter.r_selectivity_pct
- r_total_filtered - Observed combined selectivity: fraction of rows left
after applying ICP condition, Rowid Filter, and attached_condition.
This is now comparable with "filtered" and is printed right after it.
- r_filtered (AS BEFORE) - Observed selectivity of "attached_condition".
Tabular ANALYZE output is not changed. Note that JSON's r_filtered and
r_rows have the same meanings as before and have the same meaning as in
tabular output.
(Based on the original patch by Jason Cu)
Part #1:
- Add ha_handler_stats::{icp_attempts,icp_match}, make
handler_index_cond_check() increment them.
- ANALYZE FORMAT=JSON now prints r_icp_filtered based on these counters.
Before this patch, the code in Item_field::print() used
this convention (described in sql_explain.h:ExplainDataStructureLifetime):
- By default, the table that Item_field refers to is accessible.
- ANALYZE and SHOW {EXPLAIN|ANALYZE} may print Items after some
temporary tables have been dropped. They use
QT_DONT_ACCESS_TMP_TABLES flag. When it is ON, Item_field::print
will not access the table it refers to, if it is a temp.table
The bug was that EXPLAIN statement also may compute subqueries (depending
on subquery context and @@expensive_subquery_limit setting). After the
computation, the subquery calls JOIN::cleanup(true) which drops some of
its temporary tables. Calling Item_field::print() that refer to such table
will cause an access to free'd memory.
In this patch, we take into account that query optimization can compute
a subquery and discard its temporary tables. Item_field::print() now
assumes that any temporary table might have already been dropped.
This means QT_DONT_ACCESS_TMP_TABLES flag is not needed - we imply it is
always present.
But we also make one exception: derived tables are not freed in
JOIN::cleanup() call. They are freed later in close_thread_tables(),
at the same time when regular tables are closed.
Because of that, Item_field::print may assume that temp.tables
representing derived tables are available.
Initial patch by: Rex Jonston
Reviewed by: Monty <monty@mariadb.org>
