database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2026-10-23 13:29:09 +03:00
Code Activity

doc: Documentation updates for online checksums

Browse Source 
A set of documentation updates for the online checksums work, found via
manual as well as LLM-guided review.

  * The state diagram was re-done to be improve readability and be
    more in line with the look and feel of other diagrams.
  * Paragraph about mismatched states moved to its own sect2.
  * The documentation for the functions to enable/ disable checksums
    failed to mention that they are superuser only.
  * The documentation for the checksums progress reporting had
    accidentally omitted the work "fork" and for the blocks_* columns.
    The values are per fork but the documentation made it seem they
    were per relation.
  * Add the state image to the meson buildfile

Reported-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/5b08b4d1-0982-4b77-bac5-3bdffc6583f5@iki.fi
Backpatch-through: 19
This commit is contained in:
Daniel Gustafsson
2026-09-14 15:33:05 +02:00
parent 724477d67e
commit 2bd7519a81
7 changed files with 204 additions and 102 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
doc/src/sgml/func/func-admin.sgml
View File

@@ -3128,7 +3128,8 @@ SELECT convert_from(pg_read_binary_file('file_in_utf8.txt'), 'UTF8');
<para>
The functions shown in <xref linkend="functions-checksums-table" /> can
be used to enable or disable data checksums in a running cluster.
be used to enable or disable data checksums in a running cluster. Use of
these functions is restricted to superusers.
</para>
<para>
Changing data checksums can be done in a cluster with concurrent activity
@@ -3158,7 +3159,7 @@ SELECT convert_from(pg_read_binary_file('file_in_utf8.txt'), 'UTF8');
<indexterm>
<primary>pg_enable_data_checksums</primary>
</indexterm>
<function>pg_enable_data_checksums</function> ( <optional><parameter>cost_delay</parameter> <type>int</type>, <parameter>cost_limit</parameter> <type>int</type></optional> )
<function>pg_enable_data_checksums</function> ( <optional><parameter>cost_delay</parameter> <type>int</type> <optional>, <parameter>cost_limit</parameter> <type>int</type></optional></optional> )
<returnvalue>void</returnvalue>
</para>
<para>
@@ -3174,6 +3175,11 @@ SELECT convert_from(pg_read_binary_file('file_in_utf8.txt'), 'UTF8');
If <parameter>cost_delay</parameter> and <parameter>cost_limit</parameter> are
specified, the process is throttled using the same principles as
<link linkend="runtime-config-resource-vacuum-cost">Cost-based Vacuum Delay</link>.
<parameter>cost_delay</parameter> defaults to 0,
<parameter>cost_limit</parameter> defaults to 100.
</para>
<para>
This function is restricted to superusers.
</para>
</entry>
</row>
@@ -3193,6 +3199,9 @@ SELECT convert_from(pg_read_binary_file('file_in_utf8.txt'), 'UTF8');
stopped validating data checksums, the data checksum state will be
set to <literal>off</literal>.
</para>
<para>
This function is restricted to superusers.
</para>
</entry>
</row>
</tbody>

59
doc/src/sgml/images/datachecksums.gv
View File

@@ -1,14 +1,49 @@
digraph G {
A -> B [label="SELECT pg_enable_data_checksums()"];
B -> C;
D -> A;
C -> D [label="SELECT pg_disable_data_checksums()"];
E -> A [label=" --no-data-checksums"];
E -> C [label=" --data-checksums"];
digraph "datachecksums_states" {
layout=dot;
node [label="", shape=box, style=filled, fillcolor=gray, width=1.0, fontname="sans-serif"];
A [label="off"];
B [label="inprogress-on"];
C [label="on"];
D [label="inprogress-off"];
E [label="initdb"];
m1 [label="initdb", shape=Mdiamond];
subgraph cluster01 {
label="Online Checksums";
subgraph agroup1 {
rank=same;
a1;
a3;
}
subgraph agroup2 {
rank=same;
a2;
a4;
}
a1 -> a2;
a2 -> a3;
a3 -> a4;
a4 -> a1;
a1 [fillcolor=lightblue, label="off"];
a2 [fillcolor=lightgreen, label="inprogress-on"];
a3 [fillcolor=lightblue, label="on"];
a4 [fillcolor=lightgreen, label="inprogress-off"];
}
subgraph cluster05 {
label="Offline Checksums";
subgraph bgroup2 {
rank=same;
b1 -> b2;
}
b2 -> b1;
b1 [fillcolor=lightblue, label="off"];
b2 [fillcolor=lightblue, label="on"];
}
m1 -> a1;
m1 -> a3;
m1 -> b1;
m1 -> b2;
}

165
doc/src/sgml/images/datachecksums.svg
View File

@@ -1,81 +1,126 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml version="1.0"?>
<!-- Generated by graphviz version 14.0.5 (20251129.0259)
-->
<!-- Title: G Pages: 1 -->
<svg width="409pt" height="383pt"
viewBox="0.00 0.00 409.00 383.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 378.5)">
<title>G</title>
<polygon fill="white" stroke="none" points="-4,4 -4,-378.5 404.74,-378.5 404.74,4 -4,4"/>
<!-- A -->
<!-- Title: datachecksums_states Pages: 1 -->
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="441pt" height="209pt" viewBox="0.00 0.00 441.00 209.00">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 204.5)">
<title>datachecksums_states</title>
<polygon fill="white" stroke="none" points="-4,4 -4,-204.5 437,-204.5 437,4 -4,4"/>
<g id="clust1" class="cluster">
<title>cluster01</title>
<polygon fill="none" stroke="black" points="8,-8 8,-156.5 239,-156.5 239,-8 8,-8"/>
<text xml:space="preserve" text-anchor="middle" x="123.5" y="-139.2" font-family="Times,serif" font-size="14.00">Online Checksums</text>
</g>
<g id="clust4" class="cluster">
<title>cluster05</title>
<polygon fill="none" stroke="black" points="247,-80 247,-156.5 425,-156.5 425,-80 247,-80"/>
<text xml:space="preserve" text-anchor="middle" x="336" y="-139.2" font-family="Times,serif" font-size="14.00">Offline Checksums</text>
</g>
<!-- m1 -->
<g id="node1" class="node">
<title>A</title>
<ellipse fill="none" stroke="black" cx="80.12" cy="-268" rx="27" ry="18"/>
<text xml:space="preserve" text-anchor="middle" x="80.12" y="-262.95" font-family="Times,serif" font-size="14.00">off</text>
<title>m1</title>
<polygon fill="gray" stroke="black" points="242,-200.5 197.65,-182.5 242,-164.5 286.35,-182.5 242,-200.5"/>
<polyline fill="none" stroke="black" points="208.77,-187.01 208.77,-177.99"/>
<polyline fill="none" stroke="black" points="230.88,-169.01 253.12,-169.01"/>
<polyline fill="none" stroke="black" points="275.23,-177.99 275.23,-187.01"/>
<polyline fill="none" stroke="black" points="253.12,-195.99 230.88,-195.99"/>
<text xml:space="preserve" text-anchor="middle" x="242" y="-176.7" font-family="sans-serif" font-size="14.00">initdb</text>
</g>
<!-- B -->
<!-- a1 -->
<g id="node2" class="node">
<title>B</title>
<ellipse fill="none" stroke="black" cx="137.12" cy="-179.5" rx="61.59" ry="18"/>
<text xml:space="preserve" text-anchor="middle" x="137.12" y="-174.45" font-family="Times,serif" font-size="14.00">inprogress&#45;on</text>
<title>a1</title>
<polygon fill="lightblue" stroke="black" points="134,-124 62,-124 62,-88 134,-88 134,-124"/>
<text xml:space="preserve" text-anchor="middle" x="98" y="-100.2" font-family="sans-serif" font-size="14.00">off</text>
</g>
<!-- A&#45;&gt;B -->
<g id="edge1" class="edge">
<title>A&#45;&gt;B</title>
<path fill="none" stroke="black" d="M76.5,-249.68C75.22,-239.14 75.3,-225.77 81.12,-215.5 84.2,-210.08 88.49,-205.38 93.35,-201.34"/>
<polygon fill="black" stroke="black" points="95.22,-204.31 101.33,-195.66 91.16,-198.61 95.22,-204.31"/>
<text xml:space="preserve" text-anchor="middle" x="187.62" y="-218.7" font-family="Times,serif" font-size="14.00">SELECT pg_enable_data_checksums()</text>
<!-- m1&#45;&gt;a1 -->
<g id="edge7" class="edge">
<title>m1-&gt;a1</title>
<path fill="none" stroke="black" d="M208.02,-177.91C187.98,-174.61 162.76,-168.35 143,-156.5 132.95,-150.47 123.82,-141.5 116.46,-132.85"/>
<polygon fill="black" stroke="black" points="119.38,-130.89 110.41,-125.26 113.91,-135.26 119.38,-130.89"/>
</g>
<!-- C -->
<!-- a3 -->
<g id="node3" class="node">
<title>C</title>
<ellipse fill="none" stroke="black" cx="137.12" cy="-106.5" rx="27" ry="18"/>
<text xml:space="preserve" text-anchor="middle" x="137.12" y="-101.45" font-family="Times,serif" font-size="14.00">on</text>
<title>a3</title>
<polygon fill="lightblue" stroke="black" points="224,-124 152,-124 152,-88 224,-88 224,-124"/>
<text xml:space="preserve" text-anchor="middle" x="188" y="-100.2" font-family="sans-serif" font-size="14.00">on</text>
</g>
<!-- B&#45;&gt;C -->
<g id="edge2" class="edge">
<title>B&#45;&gt;C</title>
<path fill="none" stroke="black" d="M137.12,-161.31C137.12,-153.73 137.12,-144.6 137.12,-136.04"/>
<polygon fill="black" stroke="black" points="140.62,-136.04 137.12,-126.04 133.62,-136.04 140.62,-136.04"/>
<!-- m1&#45;&gt;a3 -->
<g id="edge8" class="edge">
<title>m1-&gt;a3</title>
<path fill="none" stroke="black" d="M232.35,-168.18C225.36,-158.54 215.68,-145.18 207.14,-133.41"/>
<polygon fill="black" stroke="black" points="210.21,-131.68 201.51,-125.64 204.54,-135.79 210.21,-131.68"/>
</g>
<!-- D -->
<!-- b1 -->
<g id="node6" class="node">
<title>b1</title>
<polygon fill="lightblue" stroke="black" points="327,-124 255,-124 255,-88 327,-88 327,-124"/>
<text xml:space="preserve" text-anchor="middle" x="291" y="-100.2" font-family="sans-serif" font-size="14.00">off</text>
</g>
<!-- m1&#45;&gt;b1 -->
<g id="edge9" class="edge">
<title>m1-&gt;b1</title>
<path fill="none" stroke="black" d="M250.99,-167.84C257.29,-158.25 265.92,-145.13 273.55,-133.53"/>
<polygon fill="black" stroke="black" points="276.25,-135.79 278.82,-125.51 270.4,-131.95 276.25,-135.79"/>
</g>
<!-- b2 -->
<g id="node7" class="node">
<title>b2</title>
<polygon fill="lightblue" stroke="black" points="417,-124 345,-124 345,-88 417,-88 417,-124"/>
<text xml:space="preserve" text-anchor="middle" x="381" y="-100.2" font-family="sans-serif" font-size="14.00">on</text>
</g>
<!-- m1&#45;&gt;b2 -->
<g id="edge10" class="edge">
<title>m1-&gt;b2</title>
<path fill="none" stroke="black" d="M275.15,-177.42C294.03,-173.98 317.53,-167.73 336,-156.5 346.02,-150.41 355.13,-141.42 362.49,-132.78"/>
<polygon fill="black" stroke="black" points="365.04,-135.2 368.56,-125.2 359.57,-130.82 365.04,-135.2"/>
</g>
<!-- a2 -->
<g id="node4" class="node">
<title>D</title>
<ellipse fill="none" stroke="black" cx="63.12" cy="-18" rx="63.12" ry="18"/>
<text xml:space="preserve" text-anchor="middle" x="63.12" y="-12.95" font-family="Times,serif" font-size="14.00">inprogress&#45;off</text>
<title>a2</title>
<polygon fill="lightgreen" stroke="black" points="114.25,-52 15.75,-52 15.75,-16 114.25,-16 114.25,-52"/>
<text xml:space="preserve" text-anchor="middle" x="65" y="-28.2" font-family="sans-serif" font-size="14.00">inprogress-on</text>
</g>
<!-- C&#45;&gt;D -->
<g id="edge4" class="edge">
<title>C&#45;&gt;D</title>
<path fill="none" stroke="black" d="M124.23,-90.43C113.36,-77.73 97.58,-59.28 84.77,-44.31"/>
<polygon fill="black" stroke="black" points="87.78,-42.44 78.62,-37.12 82.46,-46.99 87.78,-42.44"/>
<text xml:space="preserve" text-anchor="middle" x="214.75" y="-57.2" font-family="Times,serif" font-size="14.00">SELECT pg_disable_data_checksums()</text>
<!-- a1&#45;&gt;a2 -->
<g id="edge1" class="edge">
<title>a1-&gt;a2</title>
<path fill="none" stroke="black" d="M89.84,-87.7C86.25,-80.07 81.93,-70.92 77.92,-62.4"/>
<polygon fill="black" stroke="black" points="81.14,-61.03 73.71,-53.47 74.81,-64.01 81.14,-61.03"/>
</g>
<!-- D&#45;&gt;A -->
<g id="edge3" class="edge">
<title>D&#45;&gt;A</title>
<path fill="none" stroke="black" d="M62.52,-36.28C61.62,-68.21 60.54,-138.57 66.12,-197.5 67.43,-211.24 70.27,-226.28 73.06,-238.85"/>
<polygon fill="black" stroke="black" points="69.64,-239.59 75.32,-248.54 76.46,-238 69.64,-239.59"/>
</g>
<!-- E -->
<!-- a4 -->
<g id="node5" class="node">
<title>E</title>
<ellipse fill="none" stroke="black" cx="198.12" cy="-356.5" rx="32.41" ry="18"/>
<text xml:space="preserve" text-anchor="middle" x="198.12" y="-351.45" font-family="Times,serif" font-size="14.00">initdb</text>
<title>a4</title>
<polygon fill="lightgreen" stroke="black" points="231.25,-52 132.75,-52 132.75,-16 231.25,-16 231.25,-52"/>
<text xml:space="preserve" text-anchor="middle" x="182" y="-28.2" font-family="sans-serif" font-size="14.00">inprogress-off</text>
</g>
<!-- E&#45;&gt;A -->
<!-- a3&#45;&gt;a4 -->
<g id="edge3" class="edge">
<title>a3-&gt;a4</title>
<path fill="none" stroke="black" d="M186.52,-87.7C185.89,-80.41 185.15,-71.73 184.45,-63.54"/>
<polygon fill="black" stroke="black" points="187.94,-63.28 183.6,-53.61 180.96,-63.87 187.94,-63.28"/>
</g>
<!-- a2&#45;&gt;a3 -->
<g id="edge2" class="edge">
<title>a2-&gt;a3</title>
<path fill="none" stroke="black" d="M95.33,-52.26C111.09,-61.23 130.56,-72.31 147.59,-82"/>
<polygon fill="black" stroke="black" points="145.54,-84.86 155.96,-86.77 149,-78.78 145.54,-84.86"/>
</g>
<!-- a4&#45;&gt;a1 -->
<g id="edge4" class="edge">
<title>a4-&gt;a1</title>
<path fill="none" stroke="black" d="M161.18,-52.35C150.98,-60.85 138.51,-71.24 127.36,-80.53"/>
<polygon fill="black" stroke="black" points="125.37,-77.64 119.93,-86.73 129.85,-83.01 125.37,-77.64"/>
</g>
<!-- b1&#45;&gt;b2 -->
<g id="edge5" class="edge">
<title>E&#45;&gt;A</title>
<path fill="none" stroke="black" d="M179.16,-341.6C159.64,-327.29 129.05,-304.86 107.03,-288.72"/>
<polygon fill="black" stroke="black" points="109.23,-286 99.1,-282.91 105.09,-291.64 109.23,-286"/>
<text xml:space="preserve" text-anchor="middle" x="208.57" y="-307.2" font-family="Times,serif" font-size="14.00"> &#45;&#45;no&#45;data&#45;checksums</text>
<title>b1-&gt;b2</title>
<path fill="none" stroke="black" d="M327.21,-93.01C329.33,-92.77 331.44,-92.61 333.55,-92.54"/>
<polygon fill="black" stroke="black" points="333.21,-96.03 343.35,-92.96 333.51,-89.03 333.21,-96.03"/>
</g>
<!-- E&#45;&gt;C -->
<!-- b2&#45;&gt;b1 -->
<g id="edge6" class="edge">
<title>E&#45;&gt;C</title>
<path fill="none" stroke="black" d="M227.13,-348.04C242.29,-342.72 259.95,-334.06 271.12,-320.5 301.5,-283.62 316.36,-257.78 294.12,-215.5 268.41,-166.6 209.42,-135.53 171.52,-119.85"/>
<polygon fill="black" stroke="black" points="172.96,-116.65 162.37,-116.21 170.37,-123.16 172.96,-116.65"/>
<text xml:space="preserve" text-anchor="middle" x="350.87" y="-218.7" font-family="Times,serif" font-size="14.00"> &#45;&#45;data&#45;checksums</text>
<title>b2-&gt;b1</title>
<path fill="none" stroke="black" d="M344.86,-118.98C342.75,-119.23 340.63,-119.39 338.52,-119.46"/>
<polygon fill="black" stroke="black" points="338.86,-115.97 328.72,-119.05 338.57,-122.96 338.86,-115.97"/>
</g>
</g>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.2 KiB

After

Width:  |  Height:  |  Size: 6.1 KiB

1
doc/src/sgml/images/meson.build
View File

@@ -11,6 +11,7 @@ image_targets = []
fixup_svg_xsl = files('fixup-svg.xsl')
all_files = [
'datachecksums.gv',
'genetic-algorithm.gv',
'gin.gv',
'pagelayout.txt',

4
doc/src/sgml/monitoring.sgml
View File

@@ -8339,7 +8339,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
<structfield>blocks_total</structfield> <type>bigint</type>
</para>
<para>
The number of blocks in the current relation which will be processed,
The number of blocks in the current relation fork which will be processed,
or <literal>NULL</literal> if the worker process hasn't
calculated the number of blocks yet. The launcher process has
this set to <literal>NULL</literal>.
@@ -8353,7 +8353,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
<structfield>blocks_done</structfield> <type>bigint</type>
</para>
<para>
The number of blocks in the current relation which have been processed.
The number of blocks in the current relation fork which have been processed.
The launcher process has this set to <literal>NULL</literal>.
</para>
</entry>

2
doc/src/sgml/ref/pg_checksums.sgml
View File

@@ -288,7 +288,7 @@ PostgreSQL documentation
</procedure>
<para>
The replay requirement in <xref linkend="shut-down-nodes"/>exists
The replay requirement in <xref linkend="shut-down-nodes"/> exists
because an offline change is recorded only in the control file and
has no defined ordering against WAL the node has not replayed yet; see
<xref linkend="checksums-offline-enable-disable"/>. A node stopped

62
doc/src/sgml/wal.sgml
View File

@@ -316,30 +316,6 @@
application can be used to enable or disable data checksums, as well as
verify checksums, on an offline cluster.
</para>
<para>
An offline change provides durability differently from an
<link linkend="checksums-online-enable-disable">online change</link>.
An online transition is WAL-logged: it is ordered against all other
WAL records, it is replayed after a crash, and it propagates to
standbys. An offline change is recorded only in the cluster's
control file: it writes no WAL, it is invisible to replication, and
it has no defined ordering against WAL the node has not replayed
yet. When a node later replays WAL that contains an online checksum
state change, that change takes effect on the node even if it was
written before the offline change was made.
</para>
<para>
An offline change only affects the data directory it is run on; the
new state does not propagate over replication. In a replication setup
the same change must be applied to all nodes while all of them are
stopped, as described in <xref linkend="app-pgchecksums"/>. A standby
whose state diverges logs a warning but keeps its local setting. The
mismatch persists until the states are brought together again, with
the offline procedure or with an online transition; do this promptly.
</para>
</sect2>
<sect2 id="checksums-online-enable-disable" xreflabel="Online Enabling of Checksums">
@@ -428,7 +404,7 @@
<listitem><para>
I/O: all pages need to have data checksums calculated and written which
will generate a lot of dirty pages that will need to be flushed to disk,
as well as WAL logged.
as well as WAL-logged.
</para></listitem>
<listitem><para>
Replication: When the standby receives the data checksum state change
@@ -456,7 +432,43 @@
still required.
</para>
</sect3>
</sect2>
<sect2 id="checksums-mismatched-states">
<title>Mismatched Data Checksum States in a Replicated Cluster</title>
<para>
The primary and secondaries can end up with different data checksums
states due to the different durability models between offline and online
checksum operations.
</para>
<para>
An online transition is WAL-logged, which means that it is replayed after a
crash, and the transition along with all checksum updates are propagated to
standbys. An offline change is recorded only in the cluster's control
file, it writes no WAL and is invisible to replication. The changes made
to the data files to write checksums are not WAL-logged. This means that it
has no defined ordering against WAL the node has not replayed yet. When a
node later replays WAL that contains an online checksum state change, that
change takes effect on the node even if it was written before the offline
change was made.
</para>
<para>
An offline change only affects the data directory it is run on; the new
state does not propagate over replication. This means that all nodes can
be operated on in parallel, and no additional network traffic or WAL
archive traffic will occur. In a replication setup the same change must be
applied to all nodes while all of them are stopped, as described in
<xref linkend="app-pgchecksums"/>. A standby whose state diverges logs a
warning but keeps its local setting. The mismatch persists until the
states are brought together again, with the offline procedure or with an
online transition.
</para>
<para>
Running a replicated cluster with different data checksum states on the
different nodes is not supported or recommended.
</para>
</sect2>
</sect1>