database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-08-30 06:01:21 +03:00
Code Activity

Eagerly scan all-visible pages to amortize aggressive vacuum

Browse Source 
Aggressive vacuums must scan every unfrozen tuple in order to advance
the relfrozenxid/relminmxid. Because data is often vacuumed before it is
old enough to require freezing, relations may build up a large backlog
of pages that are set all-visible but not all-frozen in the visibility
map. When an aggressive vacuum is triggered, all of these pages must be
scanned. These pages have often been evicted from shared buffers and
even from the kernel buffer cache. Thus, aggressive vacuums often incur
large amounts of extra I/O at the expense of foreground workloads.

To amortize the cost of aggressive vacuums, eagerly scan some
all-visible but not all-frozen pages during normal vacuums.

All-visible pages that are eagerly scanned and set all-frozen in the
visibility map are counted as successful eager freezes and those not
frozen are counted as failed eager freezes.

If too many eager scans fail in a row, eager scanning is temporarily
suspended until a later portion of the relation. The number of failures
tolerated is configurable globally and per table.

To effectively amortize aggressive vacuums, we cap the number of
successes as well. Capping eager freeze successes also limits the amount
of potentially wasted work if these pages are modified again before the
next aggressive vacuum. Once we reach the maximum number of blocks
successfully eager frozen, eager scanning is disabled for the remainder
of the vacuum of the relation.

Original design idea from Robert Haas, with enhancements from
Andres Freund, Tomas Vondra, and me

Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Robert Treat <rob@xzilla.net>
Reviewed-by: Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/flat/CAAKRu_ZF_KCzZuOrPrOqjGVe8iRVWEAJSpzMgRQs%3D5-v84cXUg%40mail.gmail.com
This commit is contained in:
Melanie Plageman
2025-02-11 13:52:19 -05:00
parent 4dd09a1d41
commit 052026c9b9
12 changed files with 552 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
doc/src/sgml/config.sgml
View File

@@ -9147,6 +9147,45 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
<varlistentry id="guc-vacuum-max-eager-freeze-failure-rate" xreflabel="vacuum_max_eager_freeze_failure_rate">
<term><varname>vacuum_max_eager_freeze_failure_rate</varname> (<type>floating point</type>)
<indexterm>
<primary><varname>vacuum_max_eager_freeze_failure_rate</varname> configuration parameter</primary>
</indexterm>
</term>
<listitem>
<para>
Specifies the maximum number of pages (as a fraction of total pages in
the relation) that <command>VACUUM</command> may scan and
<emphasis>fail</emphasis> to set all-frozen in the visibility map
before disabling eager scanning. A value of <literal>0</literal>
disables eager scanning altogether. The default is
<literal>0.03</literal> (3%).
</para>
<para>
Note that when eager scanning is enabled, successful page freezes do
not count against the cap on eager freeze failures. Successful page
freezes are capped internally at 20% of the all-visible but not
all-frozen pages in the relation. Capping successful page freezes helps
amortize the overhead across multiple normal vacuums and limits the
potential downside of wasted eager freezes of pages that are modified
again before the next aggressive vacuum.
</para>
<para>
This parameter can only be set in the
<filename>postgresql.conf</filename> file or on the server command
line; but the setting can be overridden for individual tables by
changing the
<link linkend="reloption-vacuum-max-eager-freeze-failure-rate">
corresponding table storage parameter</link>.
For more information on tuning vacuum's freezing behavior,
see <xref linkend="vacuum-for-wraparound"/>.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
</sect1>

33
doc/src/sgml/maintenance.sgml
View File

@@ -496,9 +496,25 @@
When that happens, <command>VACUUM</command> will eventually need to perform an
<firstterm>aggressive vacuum</firstterm>, which will freeze all eligible unfrozen
XID and MXID values, including those from all-visible but not all-frozen pages.
In practice most tables require periodic aggressive vacuuming.
</para>
<para>
If a table is building up a backlog of all-visible but not all-frozen
pages, a normal vacuum may choose to scan skippable pages in an effort to
freeze them. Doing so decreases the number of pages the next aggressive
vacuum must scan. These are referred to as <firstterm>eagerly
scanned</firstterm> pages. Eager scanning can be tuned to attempt to freeze
more all-visible pages by increasing <xref
linkend="guc-vacuum-max-eager-freeze-failure-rate"/>. Even if eager
scanning has kept the number of all-visible but not all-frozen pages to a
minimum, most tables still require periodic aggressive vacuuming. However,
any pages successfully eager frozen may be skipped during an aggressive
vacuum, so eager freezing may minimize the overhead of aggressive vacuums.
</para>
<para>
<xref linkend="guc-vacuum-freeze-table-age"/>
controls when <command>VACUUM</command> does that: all-visible but not all-frozen
controls when a table is aggressively vacuumed. All all-visible but not all-frozen
pages are scanned if the number of transactions that have passed since the
last such scan is greater than <varname>vacuum_freeze_table_age</varname> minus
<varname>vacuum_freeze_min_age</varname>. Setting
@@ -626,10 +642,12 @@ SELECT datname, age(datfrozenxid) FROM pg_database;
</tip>
<para>
<command>VACUUM</command> normally only scans pages that have been modified
since the last vacuum, but <structfield>relfrozenxid</structfield> can only be
advanced when every page of the table
that might contain unfrozen XIDs is scanned. This happens when
While <command>VACUUM</command> scans mostly pages that have been
modified since the last vacuum, it may also eagerly scan some
all-visible but not all-frozen pages in an attempt to freeze them, but
the <structfield>relfrozenxid</structfield> will only be advanced when
every page of the table that might contain unfrozen XIDs is scanned.
This happens when
<structfield>relfrozenxid</structfield> is more than
<varname>vacuum_freeze_table_age</varname> transactions old, when
<command>VACUUM</command>'s <literal>FREEZE</literal> option is used, or when all
@@ -931,8 +949,7 @@ vacuum insert threshold = vacuum base insert threshold + vacuum insert scale fac
If the <structfield>relfrozenxid</structfield> value of the table
is more than <varname>vacuum_freeze_table_age</varname> transactions old,
an aggressive vacuum is performed to freeze old tuples and advance
<structfield>relfrozenxid</structfield>; otherwise, only pages that have been modified
since the last vacuum are scanned.
<structfield>relfrozenxid</structfield>.
</para>
<para>

15
doc/src/sgml/ref/create_table.sgml
View File

@@ -1950,6 +1950,21 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
<varlistentry id="reloption-vacuum-max-eager-freeze-failure-rate" xreflabel="vacuum_max_eager_freeze_failure_rate">
<term><literal>vacuum_max_eager_freeze_failure_rate</literal>, <literal>toast.vacuum_max_eager_freeze_failure_rate</literal> (<type>floating point</type>)
<indexterm>
<primary><varname>vacuum_max_eager_freeze_failure_rate</varname></primary>
<secondary>storage parameter</secondary>
</indexterm>
</term>
<listitem>
<para>
Per-table value for <xref linkend="guc-vacuum-max-eager-freeze-failure-rate"/>
parameter.
</para>
</listitem>
</varlistentry>
<varlistentry id="reloption-user-catalog-table" xreflabel="user_catalog_table">
<term><literal>user_catalog_table</literal> (<type>boolean</type>)
<indexterm>