Add table_index_getnext_slot, a new table AM interface that serves as
the single entry point for both plain and index-only index scans that
use amgettuple. The table AM's index_scan_begin callback resolves which
implementation the scan will use and stores it in an IndexScanDesc
function pointer (xs_getnext_slot), which table_index_getnext_slot
dispatches through.
The interface intentionally avoids using fixed TableAmRoutine members,
allowing the table AM to dynamically register an implementation based on
conditions known at the start of each scan. That way a table AM can
provide specialized variants that help the compiler produce more
efficient code. heapam provides two implementations: one for plain
index scans, and one for index-only scans. An upcoming commit that adds
the amgetbatch interface will add two more. (Table AMs can opt to
provide a single generic implementation, or to provide many specialized
implementations, as requirements dictate.)
heapam's xs_getnext_slot implementations make aggressive use of forced
inlining to ensure that plain and index-only code paths are fully
specialized at compile time, without duplicating the code they have in
common. This specialization is necessary to keep icache and iTLB
pressure to a minimum.
The new interface moves visibility map checks required by index-only
scans out of the executor (and selfuncs.c) and into heapam, enabling
batching of visibility map lookups (though for now we continue to just
perform retail lookups). Using the new higher level slot-based
interface greatly simplifies nodeIndexonlyscan.c, which no longer has to
deal with the visibility map directly. More importantly, this is a
significant architectural improvement: table AMs can now implement
index-only scans that are not tied to heapam's visibility map.
Two users of the now removed table_index_fetch_tuple interface,
_bt_check_unique and unique_key_recheck, fundamentally need to pass a
TID to the table AM to perform constraint enforcement. Neither actually
performs an index scan (even though their TIDs are taken from an index),
so neither has any use for most of the index scan machinery. Switch
these callers over to fetch_tid, a new special-purpose table AM
interface for constraint enforcement code. All true index scan callers
now use the new slot-based interface. (Note that fetch_tid doesn't
perform on-access pruning, which matches how things worked prior to
Postgres 12 commits c2fe139c2 and 71bdc99d0.)
The VISITED_PAGES_LIMIT mechanism used by get_actual_variable_range to
cap scan overhead during planning is reworked to go through a new scan
descriptor field (xs_visited_pages_limit), rather than having selfuncs.c
count heap page fetches and terminate the scan itself. This is
necessary because callers that use the new slot-based interface no
longer have direct access to which heap blocks were fetched. Similarly,
nodeIndexonlyscan.c can no longer use InstrCountTuples2 to count heap
fetches during an EXPLAIN ANALYZE. EXPLAIN ANALYZE now obtains this
information from a new IndexScanInstrumentation field, which table AMs
are required to maintain during index-only scans.
Though independently useful, this commit is preparatory work for an
upcoming commit that will add an amgetbatch index AM interface, where
the table AM takes full responsibility for managing the progress of
index scans. The table AM determines when to request the next batch
from the index AM, at a time of its choosing. A follow-on commit will
rely on this to prefetch table blocks during index scans. Having a
single entry point for all index scan callers unambiguously puts the
table AM in control of the scan.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Rui Zhao <zhaorui126@gmail.com>
Discussion: https://postgr.es/m/CAH2-WzmYqhacBH161peAWb5eF=Ja7CFAQ+0jSEMq=qnfLVTOOg@mail.gmail.com
Allows collecting details about AIO / prefetch for scan nodes backed by
a ReadStream. This may be enabled by a new "IO" option in EXPLAIN, and
it shows information about the prefetch distance and I/O requests.
As of this commit this applies only to BitmapHeapScan, because that's
the only scan node using a ReadStream and collecting instrumentation
from workers in a parallel query. Support for SeqScan and TidRangeScan,
the other scan nodes using ReadStream, will be added in subsequent
commits.
The stats are collected only when required by EXPLAIN ANALYZE, with the
IO option (disabled by default). The amount of collected statistics is
very limited, but we don't want to clutter EXPLAIN with too much data.
The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.
It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) whenever the flag SO_SCAN_INSTRUMENT flag is passed to the
scan, so that the executor and ReadStream has access to it.
The collected stats are designed for ReadStream, but are meant to be
reasonably generic in case a TAM manages I/Os in different ways.
Author: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
Previously, parallel index and index-only scans packed the parallel scan
descriptor and shared instrumentation (for EXPLAIN ANALYZE) into a
single DSM allocation. Since scans may be instrumented without being
parallel-aware, and vice versa, using separate DSM chunks -- each with
its own TOC key -- is cleaner. A future commit will extend this pattern
to other scan node types.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
