database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-07 00:36:50 +03:00
Code Activity

Add function to get memory context stats for processes

Browse Source 
This adds a function for retrieving memory context statistics
and information from backends as well as auxiliary processes.
The intended usecase is cluster debugging when under memory
pressure or unanticipated memory usage characteristics.

When calling the function it sends a signal to the specified
process to submit statistics regarding its memory contexts
into dynamic shared memory.  Each memory context is returned
in detail, followed by a cumulative total in case the number
of contexts exceed the max allocated amount of shared memory.
Each process is limited to use at most 1Mb memory for this.

A summary can also be explicitly requested by the user, this
will return the TopMemoryContext and a cumulative total of
all lower contexts.

In order to not block on busy processes the caller specifies
the number of seconds during which to retry before timing out.
In the case where no statistics are published within the set
timeout,  the last known statistics are returned, or NULL if
no previously published statistics exist.  This allows dash-
board type queries to continually publish even if the target
process is temporarily congested.  Context records contain a
timestamp to indicate when they were submitted.

Author: Rahila Syed <rahilasyed90@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Discussion: https://postgr.es/m/CAH2L28v8mc9HDt8QoSJ8TRmKau_8FM_HKS41NeO9-6ZAkuZKXw@mail.gmail.com
This commit is contained in:
Daniel Gustafsson
2025-04-08 11:06:56 +02:00
parent 15f0cb26b5
commit 042a66291b
26 changed files with 1385 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

172
doc/src/sgml/func.sgml
View File

@ -28663,6 +28663,144 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
</para></entry>
</row>
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
<primary>pg_get_process_memory_contexts</primary>
</indexterm>
<function>pg_get_process_memory_contexts</function> ( <parameter>pid</parameter> <type>integer</type>, <parameter>summary</parameter> <type>boolean</type>, <parameter>timeout</parameter> <type>float</type> )
<returnvalue>setof record</returnvalue>
( <parameter>name</parameter> <type>text</type>,
<parameter>ident</parameter> <type>text</type>,
<parameter>type</parameter> <type>text</type>,
<parameter>path</parameter> <type>integer[]</type>,
<parameter>level</parameter> <type>integer</type>,
<parameter>total_bytes</parameter> <type>bigint</type>,
<parameter>total_nblocks</parameter> <type>bigint</type>,
<parameter>free_bytes</parameter> <type>bigint</type>,
<parameter>free_chunks</parameter> <type>bigint</type>,
<parameter>used_bytes</parameter> <type>bigint</type>,
<parameter>num_agg_contexts</parameter> <type>integer</type>,
<parameter>stats_timestamp</parameter> <type>timestamptz</type> )
</para>
<para>
This function handles requests to display the memory contexts of a
<productname>PostgreSQL</productname> process with the specified
process ID. The function can be used to send requests to backends as
well as <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>.
</para>
<para>
The returned record contains extended statistics per each memory
context:
<itemizedlist spacing="compact">
<listitem>
<para>
<parameter>name</parameter> - The name of the memory context.
</para>
</listitem>
<listitem>
<para>
<parameter>ident</parameter> - Memory context ID (if any).
</para>
</listitem>
<listitem>
<para>
<parameter>type</parameter> - The type of memory context, possible
values are: AllocSet, Generation, Slab and Bump.
</para>
</listitem>
<listitem>
<para>
<parameter>path</parameter> - Memory contexts are organized in a
tree model with TopMemoryContext as the root, and all other memory
contexts as nodes in the tree. The <parameter>path</parameter>
displays the path from the root to the current memory context. The
path is limited to 100 children per node, which each node limited
to a max depth of 100, to preserve memory during reporting. The
printed path will also be limited to 100 nodes counting from the
TopMemoryContext.
</para>
</listitem>
<listitem>
<para>
<parameter>level</parameter> - The level in the tree of the current
memory context.
</para>
</listitem>
<listitem>
<para>
<parameter>total_bytes</parameter> - The total number of bytes
allocated to this memory context.
</para>
</listitem>
<listitem>
<para>
<parameter>total_nblocks</parameter> - The total number of blocks
used for the allocated memory.
</para>
</listitem>
<listitem>
<para>
<parameter>free_bytes</parameter> - The amount of free memory in
this memory context.
</para>
</listitem>
<listitem>
<para>
<parameter>free_chunks</parameter> - The number of chunks that
<parameter>free_bytes</parameter> corresponds to.
</para>
</listitem>
<listitem>
<para>
<parameter>used_bytes</parameter> - The total number of bytes
currently occupied.
</para>
</listitem>
<listitem>
<para>
<parameter>num_agg_contexts</parameter> - The number of memory
contexts aggregated in the displayed statistics.
</para>
</listitem>
<listitem>
<para>
<parameter>stats_timestamp</parameter> - When the statistics were
extracted from the process.
</para>
</listitem>
</itemizedlist>
</para>
<para>
When <parameter>summary</parameter> is <literal>true</literal>, statistics
for memory contexts at levels 1 and 2 are displayed, with level 1
representing the root node (i.e., <literal>TopMemoryContext</literal>).
Statistics for contexts on level 2 and below are aggregates of all
child contexts' statistics, where <literal>num_agg_contexts</literal>
indicate the number aggregated child contexts. When
<parameter>summary</parameter> is <literal>false</literal>,
<literal>the num_agg_contexts</literal> value is <literal>1</literal>,
indicating that individual statistics are being displayed. The levels
are limited to the first 100 contexts.
</para>
<para>
Busy processes can delay reporting memory context statistics,
<parameter>timeout</parameter> specifies the number of seconds
to wait for updated statistics. <parameter>timeout</parameter> can be
specified in fractions of a second.
</para>
<para>
After receiving memory context statistics from the target process, it
returns the results as one row per context. If all the contexts don't
fit within the pre-determined size limit, the remaining context
statistics are aggregated and a cumulative total is displayed. The
<literal>num_agg_contexts</literal> column indicates the number of
contexts aggregated in the displayed statistics. When
<literal>num_agg_contexts</literal> is <literal>1</literal> is means
that the context statistics are displayed separately.
</para></entry>
</row>
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@ -28802,6 +28940,40 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
<para>
<function>pg_get_process_memory_contexts</function> can be used to request
memory contexts statistics of any <productname>PostgreSQL</productname>
process. For example:
<programlisting>
postgres=# SELECT * FROM pg_get_process_memory_contexts(
(SELECT pid FROM pg_stat_activity
WHERE backend_type = 'checkpointer'),
false, 0.5) LIMIT 1;
-[ RECORD 1 ]----+------------------------------
name | TopMemoryContext
ident |
type | AllocSet
path | {1}
level | 1
total_bytes | 90304
total_nblocks | 3
free_bytes | 2880
free_chunks | 1
used_bytes | 87424
num_agg_contexts | 1
stats_timestamp | 2025-03-24 13:55:47.796698+01
</programlisting>
<note>
<para>
While <function>pg_get_process_memory_contexts</function> can be used to
query memory contexts of the local backend,
<structname>pg_backend_memory_contexts</structname>
(see <xref linkend="view-pg-backend-memory-contexts"/> for more details)
will be less resource intensive when only the local backend is of interest.
</para>
</note>
</para>
</sect2>
<sect2 id="functions-admin-backup">