database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-31 22:04:40 +03:00
Code Activity

Just-in-time background writing strategy. This code avoids re-scanning

Browse Source 
buffers that cannot possibly need to be cleaned, and estimates how many
buffers it should try to clean based on moving averages of recent allocation
requests and density of reusable buffers.  The patch also adds a couple
more columns to pg_stat_bgwriter to help measure the effectiveness of the
bgwriter.

Greg Smith, building on his own work and ideas from several other people,
in particular a much older patch from Itagaki Takahiro.
This commit is contained in:
Tom Lane
2007-09-25 20:03:38 +00:00
parent 588901df84
commit 6f5c38dcd0
16 changed files with 447 additions and 105 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
doc/src/sgml/config.sgml
View File

@ -1,4 +1,4 @@
<!-- $PostgreSQL: pgsql/doc/src/sgml/config.sgml,v 1.146 2007/09/24 03:12:23 tgl Exp $ -->
<!-- $PostgreSQL: pgsql/doc/src/sgml/config.sgml,v 1.147 2007/09/25 20:03:37 tgl Exp $ -->
<chapter Id="runtime-config">
<title>Server Configuration</title>
@ -1205,25 +1205,6 @@ SET ENABLE_SEQSCAN TO OFF;
</listitem>
</varlistentry>
<varlistentry id="guc-bgwriter-lru-percent" xreflabel="bgwriter_lru_percent">
<term><varname>bgwriter_lru_percent</varname> (<type>floating point</type>)</term>
<indexterm>
<primary><varname>bgwriter_lru_percent</> configuration parameter</primary>
</indexterm>
<listitem>
<para>
To reduce the probability that server processes will need to issue
their own writes, the background writer tries to write buffers that
are likely to be recycled soon. In each round, it examines up to
<varname>bgwriter_lru_percent</> of the buffers that are nearest to
being recycled, and writes any that are dirty.
The default value is 1.0 (1% of the total number of shared buffers).
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
</para>
</listitem>
</varlistentry>
<varlistentry id="guc-bgwriter-lru-maxpages" xreflabel="bgwriter_lru_maxpages">
<term><varname>bgwriter_lru_maxpages</varname> (<type>integer</type>)</term>
<indexterm>
@ -1232,8 +1213,34 @@ SET ENABLE_SEQSCAN TO OFF;
<listitem>
<para>
In each round, no more than this many buffers will be written
as a result of scanning soon-to-be-recycled buffers.
The default value is five buffers.
by the background writer. Setting this to zero disables
background writing (except for checkpoint activity).
The default value is 100 buffers.
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
</para>
</listitem>
</varlistentry>
<varlistentry id="guc-bgwriter-lru-multiplier" xreflabel="bgwriter_lru_multiplier">
<term><varname>bgwriter_lru_multiplier</varname> (<type>floating point</type>)</term>
<indexterm>
<primary><varname>bgwriter_lru_multiplier</> configuration parameter</primary>
</indexterm>
<listitem>
<para>
Unless limited by <varname>bgwriter_lru_maxpages</>, the number
of dirty buffers written in each round is determined by reference
to the number of new buffers that have been needed by server
processes during recent rounds. This number is multiplied by
<varname>bgwriter_lru_multiplier</> to arrive at the estimate
of the number of buffers that will be needed during the next round.
Thus, a setting of 1.0 represents a <quote>just in time</> policy
of writing exactly the number of buffers predicted to be needed.
Larger values provide some cushion against spikes in demand,
while smaller values intentionally leave writes to be done by
server processes.
The default is 2.0.
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
</para>
@ -1242,8 +1249,8 @@ SET ENABLE_SEQSCAN TO OFF;
</variablelist>
<para>
Smaller values of <varname>bgwriter_lru_percent</varname> and
<varname>bgwriter_lru_maxpages</varname> reduce the extra I/O load
Smaller values of <varname>bgwriter_lru_maxpages</varname> and
<varname>bgwriter_lru_multiplier</varname> reduce the extra I/O load
caused by the background writer, but make it more likely that server
processes will have to issue writes for themselves, delaying interactive
queries.

24
doc/src/sgml/monitoring.sgml
View File

@ -1,4 +1,4 @@
<!-- $PostgreSQL: pgsql/doc/src/sgml/monitoring.sgml,v 1.53 2007/09/24 03:12:23 tgl Exp $ -->
<!-- $PostgreSQL: pgsql/doc/src/sgml/monitoring.sgml,v 1.54 2007/09/25 20:03:37 tgl Exp $ -->
<chapter id="monitoring">
<title>Monitoring Database Activity</title>
@ -237,7 +237,10 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
background writer: number of scheduled checkpoints, requested
checkpoints, buffers written by checkpoints and cleaning scans,
and the number of times the bgwriter stopped a cleaning scan
because it had written too many buffers.
because it had written too many buffers. Also includes
statistics about the shared buffer pool, including buffers written
by backends (that is, not by the background writer) and total buffers
allocated.
</entry>
</row>
@ -817,6 +820,23 @@ postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
</entry>
</row>
<row>
<entry><literal><function>pg_stat_get_buf_written_backend</function>()</literal></entry>
<entry><type>bigint</type></entry>
<entry>
The number of buffers written by backends because they needed
to allocate a new buffer
</entry>
</row>
<row>
<entry><literal><function>pg_stat_get_buf_alloc</function>()</literal></entry>
<entry><type>bigint</type></entry>
<entry>
The total number of buffer allocations
</entry>
</row>
<row>
<entry><literal><function>pg_stat_clear_snapshot</function>()</literal></entry>
<entry><type>void</type></entry>