database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-11-19 13:42:17 +03:00
Code Activity

Show index search count in EXPLAIN ANALYZE, take 2.

Browse Source 
Expose the count of index searches/index descents in EXPLAIN ANALYZE's
output for index scan/index-only scan/bitmap index scan nodes.  This
information is particularly useful with scans that use ScalarArrayOp
quals, where the number of index searches can be unpredictable due to
implementation details that interact with physical index characteristics
(at least with nbtree SAOP scans, since Postgres 17 commit 5bf748b8).
The information shown also provides useful context when EXPLAIN ANALYZE
runs a plan with an index scan node that successfully applied the skip
scan optimization (set to be added to nbtree by an upcoming patch).

The instrumentation works by teaching all index AMs to increment a new
nsearches counter whenever a new index search begins.  The counter is
incremented at exactly the same point that index AMs already increment
the pg_stat_*_indexes.idx_scan counter (we're counting the same event,
but at the scan level rather than the relation level).  Parallel queries
have workers copy their local counter struct into shared memory when an
index scan node ends -- even when it isn't a parallel aware scan node.
An earlier version of this patch that only worked with parallel aware
scans became commit 5ead85fb (though that was quickly reverted by commit
d00107cd following "debug_parallel_query=regress" buildfarm failures).

Our approach doesn't match the approach used when tracking other index
scan related costs (e.g., "Rows Removed by Filter:").  It is comparable
to the approach used in similar cases involving costs that are only
readily accessible inside an access method, not from the executor proper
(e.g., "Heap Blocks:" output for a Bitmap Heap Scan, which was recently
enhanced to show per-worker costs by commit 5a1e6df3, using essentially
the same scheme as the one used here).  It is necessary for index AMs to
have direct responsibility for maintaining the new counter, since the
counter might need to be incremented multiple times per amgettuple call
(or per amgetbitmap call).  But it is also necessary for the executor
proper to manage the shared memory now used to transfer each worker's
counter struct to the leader.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Reviewed-By: Tomas Vondra <tomas@vondra.me>
Reviewed-By: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-WzkRqvaqR2CTNqTZP0z6FuL4-3ED6eQB0yx38XBNj1v-4Q@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-Wz=PKR6rB7qbx+Vnd7eqeB5VTcrW=iJvAsTsKbdG+kW_UA@mail.gmail.com
This commit is contained in:
Peter Geoghegan
2025-03-11 09:20:50 -04:00
parent 12c5f797ea
commit 0fbceae841
37 changed files with 797 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
doc/src/sgml/monitoring.sgml
View File

@@ -4234,16 +4234,32 @@ description | Waiting for a newly initialized WAL file to reach durable storage
<note>
<para>
Queries that use certain <acronym>SQL</acronym> constructs to search for
rows matching any value out of a list or array of multiple scalar values
(see <xref linkend="functions-comparisons"/>) perform multiple
<quote>primitive</quote> index scans (up to one primitive scan per scalar
value) during query execution. Each internal primitive index scan
increments <structname>pg_stat_all_indexes</structname>.<structfield>idx_scan</structfield>,
Index scans may sometimes perform multiple index searches per execution.
Each index search increments <structname>pg_stat_all_indexes</structname>.<structfield>idx_scan</structfield>,
so it's possible for the count of index scans to significantly exceed the
total number of index scan executor node executions.
</para>
<para>
This can happen with queries that use certain <acronym>SQL</acronym>
constructs to search for rows matching any value out of a list or array of
multiple scalar values (see <xref linkend="functions-comparisons"/>). It
can also happen to queries with a
<literal><replaceable>column_name</replaceable> =
<replaceable>value1</replaceable> OR
<replaceable>column_name</replaceable> =
<replaceable>value2</replaceable> ...</literal> construct, though only
when the optimizer transforms the construct into an equivalent
multi-valued array representation.
</para>
</note>
<tip>
<para>
<command>EXPLAIN ANALYZE</command> outputs the total number of index
searches performed by each index scan node. See
<xref linkend="using-explain-analyze"/> for an example demonstrating how
this works.
</para>
</tip>
</sect2>