ed8050370b
GetLocalPinLimit() and GetAdditionalLocalPinLimit(), currently in use
only by the read stream, previously allowed a backend to pin all
num_temp_buffers local buffers. This meant that the read stream could
use every available local buffer for read-ahead, leaving none for other
concurrent pin-holders.
In 18, this was reported as a sequential scan of a temporary table
failing with "no empty local buffer available": with
effective_io_concurrency >= 64 and default io_combine_limit and
temp_buffers, the scan's read stream budget,
(effective_io_concurrency + 1) * io_combine_limit, meets or exceeds
num_temp_buffers, so look-ahead could pin the entire local buffer pool.
Any additional pin request made while returning tuples -- such as
fetching TOASTed values from the table's (also temporary) TOAST table --
then found no unpinned buffer to evict.
Cap the local pin limit to num_temp_buffers / 4, providing some
headroom. This doesn't guarantee that all needed pins will be available
-- for example, a backend can still open more cursors than there are
buffers -- but it makes it less likely that read-ahead will exhaust the
pool.
This is a backpatch of commit da6874635d, which was applied to master
only during the 19 development cycle, where the issue surfaced as a
regression test failure after on-access pruning began setting the
visibility map. Reports of the failure above on 18 prompted
backpatching it now.
No backpatch to 17, even though its read stream also allows a single
stream to pin the whole pool in principle. There, sequential scans pass
READ_STREAM_SEQUENTIAL, which disables fadvise-based look-ahead, so the
look-ahead distance cannot grow past io_combine_limit (at most 32
buffers, versus a minimum temp_buffers of 100) and the reported failure
is unreachable. The remaining theoretical paths require adversarial
settings and have never been reported. Moreover, 17 predates these
functions; the fix would have to be reimplemented in
LimitAdditionalLocalPins(), making the change not a simple cherry-pick.
Reported-by: Induja Sreekanthan <indujas@google.com>
Reported-by: Eduard Stepanov <crtxcz@gmail.com>
Reported-by: Feike Steenbergen <feikesteenbergen@gmail.com>
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/97529f5a-ec10-46b1-ab50-4653126c6889%40gmail.com
Discussion: https://postgr.es/m/flat/CAFMO8-rYPSJbXsDdWDzDdpNi-fQ%2B6bKvgbXwE%2BR%3DsGko4epq0Q%40mail.gmail.com
Backpatch-through: 18
PostgreSQL Database Management System
This directory contains the source code distribution of the PostgreSQL database management system.
PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions. This distribution also contains C language bindings.
Copyright and license information can be found in the file COPYRIGHT.
General documentation about this version of PostgreSQL can be found at https://www.postgresql.org/docs/18/. In particular, information about building PostgreSQL from the source code can be found at https://www.postgresql.org/docs/18/installation.html.
The latest version of this software, and related software, may be obtained at https://www.postgresql.org/download/. For more information look at our web site located at https://www.postgresql.org/.