1
0
mirror of https://github.com/postgres/postgres.git synced 2025-11-28 11:44:57 +03:00
Commit Graph

62713 Commits

Author SHA1 Message Date
Peter Eisentraut
8b3e2c622a Fix pg_isblank()
There was a pg_isblank() function that claimed to be a replacement for
the standard isblank() function, which was thought to be "not very
portable yet".  We can now assume that it's portable (it's in C99).

But pg_isblank() actually diverged from the standard isblank() by also
accepting '\r', while the standard one only accepts space and tab.
This was added to support parsing pg_hba.conf under Windows.  But the
hba parsing code now works completely differently and already handles
line endings before we get to pg_isblank().  The other user of
pg_isblank() is for ident protocol message parsing, which also handles
'\r' separately.  So this behavior is now obsolete and confusing.

To improve clarity, I separated those concerns.  The ident parsing now
gets its own function that hardcodes the whitespace characters
mentioned by the relevant RFC.  pg_isblank() is now static in hba.c
and is a wrapper around the standard isblank(), with some extra logic
to ensure robust treatment of non-ASCII characters.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/170308e6-a7a3-4484-87b2-f960bb564afa%40eisentraut.org
2025-11-28 08:33:07 +01:00
Amit Kapila
e68b6adad9 Add slotsync_skip_reason column to pg_replication_slots view.
Introduce a new column, slotsync_skip_reason, in the pg_replication_slots
view. This column records the reason why the last slot synchronization was
skipped. It is primarily relevant for logical replication slots on standby
servers where the 'synced' field is true. The value is NULL when
synchronization succeeds.

Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
Reviewed-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAE9k0PkhfKrTEAsGz4DjOhEj1nQ+hbQVfvWUxNacD38ibW3a1g@mail.gmail.com
2025-11-28 05:21:35 +00:00
Michael Paquier
9ccc049dfe pg_buffercache: Add pg_buffercache_mark_dirty{,_relation,_all}()
This commit introduces three new functions for marking shared buffers as
dirty by using the functions introduced in 9660906dbd:
* pg_buffercache_mark_dirty() for one shared buffer.
- pg_buffercache_mark_dirt_relation() for all the shared buffers in a
relation.
* pg_buffercache_mark_dirty_all() for all the shared buffers in pool.

The "_all" and "_relation" flavors are designed to address the
inefficiency of repeatedly calling pg_buffercache_mark_dirty() for each
individual buffer, which can be time-consuming when dealing with with
large shared buffers pool.

These functions are intended as developer tools and are available only
to superusers.  There is no need to bump the version of pg_buffercache,
4b203d499c having done this job in this release cycle.

Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Aidar Imamov <a.imamov@postgrespro.ru>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Joseph Koshakow <koshy44@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Yuhang Qiu <iamqyh@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ0h_YoSqqutxV6DES1RW8ig6wcA8CR9rJk358YRMxZFmw@mail.gmail.com
2025-11-28 09:04:04 +09:00
David Rowley
d167c19295 Fix possibly uninitialized HeapScanDesc.rs_startblock
The solution used in 0ca3b1697 to determine the Parallel TID Range
Scan's start location was to modify the signature of
table_block_parallelscan_startblock_init() to allow the startblock
to be passed in as a parameter.  This allows the scan limits to be
adjusted before that function is called so that the limits are picked up
when the parallel scan starts.  The commit made it so the call to
table_block_parallelscan_startblock_init uses the HeapScanDesc's
rs_startblock to pass the startblock to the parallel scan.  That all
works ok for Parallel TID Range scans as the HeapScanDesc rs_startblock
gets set by heap_setscanlimits(), but for Parallel Seq Scans, initscan()
does not initialize rs_startblock, and that results in passing an
uninitialized value to table_block_parallelscan_startblock_init() as
noted by the buildfarm member skink, running Valgrind.

To fix this issue, make it so initscan() sets the rs_startblock for
parallel scans unless we're doing a rescan.  This makes it so
table_block_parallelscan_startblock_init() will be called with the
startblock set to InvalidBlockNumber, and that'll allow the syncscan
code to find the correct start location (when enabled).  For Parallel
TID Range Scans, this InvalidBlockNumber value will be overwritten in
the call to heap_setscanlimits().

initscan() is a bit light on documentation on what's meant to get
initialized where for parallel scans.  From what I can tell, it looks like
it just didn't matter prior to 0ca3b1697 that rs_startblock was left
uninitialized for parallel scans.  To address the light documentation,
I've also added some comments to mention that the syncscan location for
parallel scans is figured out in table_block_parallelscan_startblock_init.
I've also taken the liberty to adjust the if/else if/else code in
initscan() to make it clearer which parts apply to parallel scans and
which parts are for the serial scans.

Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvqALm+k7FyfdQdCw1yF_8HojvR61YRrNhwRQPE=zSmnQA@mail.gmail.com
2025-11-28 12:40:50 +13:00
Michael Paquier
c75bf57a90 doc: Add missing tags in pg_buffercache page
Issue noticed while looking at this area of the documentation, for a
different patch.  This is a matter of style, so no backpatch is done.

Discussion: https://postgr.es/m/CAN55FZ0h_YoSqqutxV6DES1RW8ig6wcA8CR9rJk358YRMxZFmw@mail.gmail.com
2025-11-28 08:00:23 +09:00
Michael Paquier
9660906dbd Add routines for marking buffers dirty efficiently
This commit introduces new internal bufmgr routines for marking shared
buffers as dirty:
* MarkDirtyUnpinnedBuffer()
* MarkDirtyRelUnpinnedBuffers()
* MarkDirtyAllUnpinnedBuffers()

These functions provide an efficient mechanism to respectively mark one
buffer, all the buffers of a relation, or the entire shared buffer pool
as dirty, something that can be useful to force patterns for the
checkpointer.  MarkDirtyUnpinnedBufferInternal(), an extra routine, is
used by these three, to mark as dirty an unpinned buffer.

They are intended as developer tools to manipulate buffer dirtiness in
bulk, and will be used in a follow-up commit.

Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Aidar Imamov <a.imamov@postgrespro.ru>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Joseph Koshakow <koshy44@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Yuhang Qiu <iamqyh@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ0h_YoSqqutxV6DES1RW8ig6wcA8CR9rJk358YRMxZFmw@mail.gmail.com
2025-11-28 07:39:33 +09:00
Tom Lane
5528e8d104 Allow indexscans on partial hash indexes with implied quals.
Normally, if a WHERE clause is implied by the predicate of a partial
index, we drop that clause from the set of quals used with the index,
since it's redundant to test it if we're scanning that index.
However, if it's a hash index (or any !amoptionalkey index), this
could result in dropping all available quals for the index's first
key, preventing us from generating an indexscan.

It's fair to question the practical usefulness of this case.  Since
hash only supports equality quals, the situation could only arise
if the index's predicate is "WHERE indexkey = constant", implying
that the index contains only one hash value, which would make hash
a really poor choice of index type.  However, perhaps there are
other !amoptionalkey index AMs out there with which such cases are
more plausible.

To fix, just don't filter the candidate indexquals this way if
the index is !amoptionalkey.  That's a bit hokey because it may
result in testing quals we didn't need to test, but to do it
more accurately we'd have to redundantly identify which candidate
quals are actually usable with the index, something we don't know
at this early stage of planning.  Doesn't seem worth the effort.

Reported-by: Sergei Glukhov <s.glukhov@postgrespro.ru>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/e200bf38-6b45-446a-83fd-48617211feff@postgrespro.ru
Backpatch-through: 14
2025-11-27 13:09:59 -05:00
Fujii Masao
246ec4a51c doc: Fix misleading synopsis for CREATE/ALTER PUBLICATION.
The documentation for CREATE/ALTER PUBLICATION previously showed:

        [ ONLY ] table_name [ * ] [ ( column_name [, ... ] ) ] [ WHERE ( expression ) ] [, ... ]

to indicate that the table/column specification could be repeated.
However, placing [, ... ] directly after a multi-part construct was
misleading and made it unclear which portion was repeatable.

This commit introduces a new term, table_and_columns, to represent:

        [ ONLY ] table_name [ * ] [ ( column_name [, ... ] ) ] [ WHERE ( expression ) ]

and updates the synopsis to use:

        table_and_columns [, ... ]

which clearly identifies the repeatable element.

Backpatched to v15, where the misleading syntax was introduced.

Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Chao Li <lic@highgo.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAHut+PtsyvYL3KmA6C8f0ZpXQ=7FEqQtETVy-BOF+cm9WPvfMQ@mail.gmail.com
Backpatch-through: 15
2025-11-27 23:29:57 +09:00
Álvaro Herrera
9e8fa05d34 Fix new test for CATCACHE_FORCE_RELEASE builds
Two of the isolation tests introduce by commit bc32a12e0d had a
problem under CATCACHE_FORCE_RELEASE, as evidenced by buildfarm member
prion.  An injection point is hit ahead of what the test spec expects,
so a session goes to sleep and there's no one there to wait it up.  Fix
in the simplest possible way, which is to conditionally wake the process
up if it's waiting.  An alternative output file is necessary to cover
both cases.

This suggests a couple of possible improvements to the injection points
infrastructure: a conditional wakeup (doing nothing if no one is
sleeping, as opposed to throwing an error), as well as a way to attach
to a point in "deactivated" mode, activated later.

Author: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Discussion: https://postgr.es/m/202511261817.fyixgtt3hqdr@alvherre.pgsql
2025-11-27 13:10:56 +01:00
Daniel Gustafsson
e396a18f32 doc: Fix typo in pg_dump documentation
Reported-by: Erik Rijkers <er@xs4all.nl>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/7596672c-43e8-a030-0850-2dd09af98cac@xs4all.nl
2025-11-27 09:25:56 +01:00
Peter Eisentraut
e7075a3405 Use C11 alignas in pg_atomic_uint64 definitions
They were already using pg_attribute_aligned.  This replaces that with
alignas and moves that into the required syntactic position.  This
ends up making these three atomics implementations appear a bit more
consistent, but shouldn't change anything otherwise.

Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/46f05236-d4d4-4b4e-84d4-faa500f14691%40eisentraut.org
2025-11-27 07:53:34 +01:00
Amit Langote
519fa0433b Fix error reporting for SQL/JSON path type mismatches
transformJsonFuncExpr() used exprType()/exprLocation() on the
possibly coerced path expression, which could be NULL when
coercion to jsonpath failed, leading to "cache lookup failed
for type 0" errors.

Preserve the original expression node so that type and location
in the "must be of type jsonpath" error are reported correctly.
Add regression tests to cover these cases.

Reported-by: Jian He <jian.universality@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/CACJufxHunVg81JMuNo8Yvv_hJD0DicgaVN2Wteu8aJbVJPBjZA@mail.gmail.com
Backpatch-through: 17
2025-11-27 12:07:01 +09:00
David Rowley
0ca3b16973 Add parallelism support for TID Range Scans
In v14, bb437f995 added support for scanning for ranges of TIDs using a
dedicated executor node for the purpose.  Here, we allow these scans to
be parallelized.  The range of blocks to scan is divvied up similarly to
how a Parallel Seq Scans does that, where 'chunks' of blocks are
allocated to each worker and the size of those chunks is slowly reduced
down to 1 block per worker by the time we're nearing the end of the
scan.  Doing that means workers finish at roughly the same time.

Allowing TID Range Scans to be parallelized removes the dilemma from the
planner as to whether a Parallel Seq Scan will cost less than a
non-parallel TID Range Scan due to the CPU concurrency of the Seq Scan
(disk costs are not divided by the number of workers).  It was possible
the planner could choose the Parallel Seq Scan which would result in
reading additional blocks during execution than the TID Scan would have.
Allowing Parallel TID Range Scans removes the trade-off the planner
makes when choosing between reduced CPU costs due to parallelism vs
additional I/O from the Parallel Seq Scan due to it scanning blocks from
outside of the required TID range.  There is also, of course, the
traditional parallelism performance benefits to be gained as well, which
likely doesn't need to be explained here.

Author: Cary Huang <cary.huang@highgo.ca>
Author: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
Reviewed-by: Rafia Sabih <rafia.pghackers@gmail.com>
Reviewed-by: Steven Niu <niushiji@gmail.com>
Discussion: https://postgr.es/m/18f2c002a24.11bc2ab825151706.3749144144619388582@highgo.ca
2025-11-27 14:05:04 +13:00
David Rowley
42473b3b31 Have the planner replace COUNT(ANY) with COUNT(*), when possible
This adds SupportRequestSimplifyAggref to allow pg_proc.prosupport
functions to receive an Aggref and allow them to determine if there is a
way that the Aggref call can be optimized.

Also added is a support function to allow transformation of COUNT(ANY)
into COUNT(*).  This is possible to do when the given "ANY" cannot be
NULL and also that there are no ORDER BY / DISTINCT clauses within the
Aggref.  This is a useful transformation to do as it is common that
people write COUNT(1), which until now has added unneeded overhead.
When counting a NOT NULL column.  The overheads can be worse as that
might mean deforming more of the tuple, which for large fact tables may
be many columns in.

It may be possible to add prosupport functions for other aggregates.  We
could consider if ORDER BY could be dropped for some calls, e.g. the
ORDER BY is quite useless in MAX(c ORDER BY c).

There is a little bit of passing fallout from adjusting
expr_is_nonnullable() to handle Const which results in a plan change in
the aggregates.out regression test.  Previously, nothing was able to
determine that "One-Time Filter: (100 IS NOT NULL)" was always true,
therefore useless to include in the plan.

Author: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Discussion: https://postgr.es/m/CAApHDvqGcPTagXpKfH=CrmHBqALpziThJEDs_MrPqjKVeDF9wA@mail.gmail.com
2025-11-27 10:43:28 +13:00
Nathan Bossart
dbdc717ac6 Teach DSM registry to retry entry initialization if needed.
If DSM registry entry initialization fails, backends could try to
use an uninitialized DSM segment, DSA, or dshash table (since the
entry is still added to the registry).  To fix, restructure the
code so that the registry retries initialization as needed.  This
commit also modifies pg_get_dsm_registry_allocations() to leave out
partially-initialized entries, as they shouldn't have any allocated
memory.

DSM registry entry initialization shouldn't fail often in practice,
but retrying was deemed better than leaving entries in a
permanently failed state (as was done by commit 1165a933aa, which
has since been reverted).

Suggested-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/E1vJHUk-006I7r-37%40gemulon.postgresql.org
Backpatch-through: 17
2025-11-26 15:12:25 -06:00
Jeff Davis
1476028225 Allow pg_locale_t APIs to work when ctype_is_c.
Previously, the caller needed to check ctype_is_c first for some
routines and not others. Now, the APIs consistently work, and the
caller can just check ctype_is_c for optimization purposes.

Discussion: https://postgr.es/m/450ceb6260cad30d7afdf155d991a9caafee7c0d.camel@j-davis.com
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
2025-11-26 12:54:37 -08:00
Daniel Gustafsson
1cdb84bb1b Check for correct version of perltidy
pgperltidy requires a particular version of perltidy, but the version
wasn't checked like how pgindent checks the underlying indent binary.
Fix by checking the version of perltidy and error out if an incorrect
version is used.

Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/1209850.1764092152@sss.pgh.pa.us
2025-11-26 20:43:09 +01:00
Jeff Davis
8d299052fe Add #define for UNICODE_CASEMAP_BUFSZ.
Useful for mapping a single codepoint at a time into a
statically-allocated buffer.

Discussion: https://postgr.es/m/450ceb6260cad30d7afdf155d991a9caafee7c0d.camel@j-davis.com
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
2025-11-26 10:05:11 -08:00
Jeff Davis
ec4997a9d7 Inline pg_ascii_tolower() and pg_ascii_toupper().
Discussion: https://postgr.es/m/450ceb6260cad30d7afdf155d991a9caafee7c0d.camel@j-davis.com
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
2025-11-26 10:04:32 -08:00
Nathan Bossart
2dd506b859 Revert "Teach DSM registry to ERROR if attaching to an uninitialized entry."
This reverts commit 1165a933aa (and the corresponding commits on
the back-branches).  In a follow-up commit, we'll teach the
registry to retry entry initialization instead of leaving it in a
permanently failed state.

Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/E1vJHUk-006I7r-37%40gemulon.postgresql.org
Backpatch-through: 17
2025-11-26 11:37:21 -06:00
Melanie Plageman
e135e04457 Split heap_page_prune_and_freeze() into helpers
Refactor the setup and planning phases of pruning and freezing into
helpers. This streamlines heap_page_prune_and_freeze() and makes it more
clear when the examination of tuples ends and page modifications begin.

No code change beyond what was required to extract the code into helper
functions.

Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/mhf4vkmh3j57zx7vuxp4jagtdzwhu3573pgfpmnjwqa6i6yj5y%40sy4ymcdtdklo
2025-11-26 11:00:34 -05:00
Nathan Bossart
9446f918ac Remove a few unused struct members.
Oversights in commits ab9e0e718a, f3049a603a, and 247ce06b88.

Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/aScUuBSawPWogUxs%40ip-10-97-1-34.eu-west-3.compute.internal
2025-11-26 09:50:00 -06:00
Daniel Gustafsson
348020caa7 ssl: Add connection and reload tests for key passphrases
ssl_passphrase_command_supports_reload was not covered by the SSL
testsuite,  and connection tests after unlocking secrets with the
passphrase was also missing.  This adds test coverage for reloads
of passphrase commands as well as connection attempts which tests
the different codepaths for Windows and non-EXEC_BACKEND builds.

Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/5F301096-921A-427D-8EC1-EBAEC2A35082@yesql.se
2025-11-26 14:24:34 +01:00
Daniel Gustafsson
b3fe098d33 Add GUC to show EXEC_BACKEND state
There is no straightforward way to determine if a cluster is running
in EXEC_BACKEND mode or not, which is useful for tests to know. This
adds a GUC debug_exec_backend similar to debug_assertions which will
be true when the server is running in EXEC_BACKEND mode.

Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/5F301096-921A-427D-8EC1-EBAEC2A35082@yesql.se
2025-11-26 14:24:27 +01:00
Daniel Gustafsson
0f4f45772c doc: Clarify passphrase command reloading on Windows
When running on Windows (or EXEC_BACKEND) the SSL configuration will
be reloaded on each backend start, so the passphrase command will be
reloaded along with it.  This implies that passphrase command reload
must be enabled on Windows for connections to work at all.  Document
this since it wasn't mentioned explicitly, and will there add markup
for parameter value to match the rest of the docs.

Backpatch to all supported versions.

Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/5F301096-921A-427D-8EC1-EBAEC2A35082@yesql.se
Backpatch-through: 14
2025-11-26 14:24:04 +01:00
Peter Eisentraut
8fe4aef829 Replace internal C function pg_hypot() by standard hypot()
The code comment said, "It is expected that this routine will
eventually be replaced with the C99 hypot() function.", so let's do
that now.

This function is tested via the geometry regression test, so if it is
faulty on any platform, it will show up there.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/170308e6-a7a3-4484-87b2-f960bb564afa%40eisentraut.org
2025-11-26 07:48:29 +01:00
Jacob Champion
47c7a7ebc8 oauth_validator: Shorten JSON responses in test logs
Response padding from the oauth_validator abuse tests was adding a
couple megabytes to the test logs. We don't need the buildfarm to hold
onto that, and we don't need to read it when debugging; truncate it.

Reported-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/202511251218.zfs4nu2qnh2m%40alvherre.pgsql
Backpatch-through: 18
2025-11-25 20:39:47 -08:00
Amit Kapila
e3e787ca02 Fix test failure caused by commit 76b78721ca.
The test failed because it assumed that a newly created logical
replication slot could be synced to the standby by the slotsync worker.
However, the presence of an existing physical slot caused the new logical
slot to use a non-latest xmin. On the standby, the DDL had already been
replayed, advancing xmin, which led to the slotsync worker failing to sync
the lagging logical slot.

To resolve this, we moved the slot sync statistics tests to run after the
tests that do not require the newly created slot to be sync-ready.

As per buildfarm.

Author: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB14966FE0BFB6C212298BFFEDEF5D1A@OSCPR01MB14966.jpnprd01.prod.outlook.com
2025-11-26 03:26:57 +00:00
Michael Paquier
e1405aa5e3 Add input function for data type pg_dependencies
pg_dependencies is used as data type for the contents of dependencies
extended statistics.  This new input function consumes the format that
has been established by e76defbcf0 for the output function of
pg_dependencies, enforcing some sanity checks for:
- Checks for the input object, which should be a one-dimension array
with correct attributes and values.
- The key names: "attributes", "dependency", "degree".  All are
required, other key names are blocked.
- Value types for each key: "attributes" requires an array of integers,
"dependency" an attribute number, "degree" a float.
- List of attributes.  In this case, it is possible that some
dependencies are not listed in the statistics data, as items with a
degree of 0 are discarded when building the statistics.  This commit
includes checks for simple scenarios, like duplicated attributes, or
overlapping values between the list of "attributes" and the "dependency"
value.  Even if the input function considers the input as valid, a value
still needs to be cross-checked with the attributes defined in a
statistics object at import.
- Based on the discussion, the checks on the values are loose, as there
is also an argument for potentially stats injection.  For example,
"degree" should be defined in [0.0,1.0], but a check is not enforced.

This is required for a follow-up patch that aims to implement the import
of extended statistics.  Some tests are added to check the code paths of
the JSON parser checking the shape of the pg_dependencies inputs, with
91% of code coverage reached.  The tests are located in their own new
test file, for clarity.

Author: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Yuefei Shi <shiyuefei1004@gmail.com>
Discussion: https://postgr.es/m/CADkLM=dpz3KFnqP-dgJ-zvRvtjsa8UZv8wDAQdqho=qN3kX0Zg@mail.gmail.com
2025-11-26 10:53:16 +09:00
Michael Paquier
44eba8f06e Add input function for data type pg_ndistinct
pg_ndistinct is used as data type for the contents of ndistinct extended
statistics.  This new input function consumes the format that has been
established by 1f927cce44 for the output function of pg_ndistinct,
enforcing some sanity checks for:
- Checks for the input object, which should be a one-dimension array
with correct attributes and values.
- The key names: "attributes", "ndistinct".  Both are required, other
key names are blocked.
- Value types for each key: "attributes" requires an array of integers,
and "ndistinct" an integer.
- List of attributes.  Note that this enforces a check so as an
attribute list has to be a subset of the longest attribute list found.
This does not enforce that a full group of attribute sets exist, based
on how the groups are generated when the ndistinct objects are
generated, making the list of ndistinct items a bit loose.  Note a check
would still be required at import to see if the attributes listed match
with the attribute numbers set in the definition of a statistics object.
- Based on the discussion, the checks on the values are loose, as there
is also an argument for potentially stats injection.  The relation and
attribute level stats follow the same line of argument for the values.

This is required for a follow-up patch that aims to implement the import
of extended statistics.  Some tests are added to check the code paths of
the JSON parser checking the shape of the pg_ndistinct inputs, with 90%
of code coverage reached.  The tests are located in their own new test
file, for clarity.

Author: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Yuefei Shi <shiyuefei1004@gmail.com>
Discussion: https://postgr.es/m/CADkLM=dpz3KFnqP-dgJ-zvRvtjsa8UZv8wDAQdqho=qN3kX0Zg@mail.gmail.com
2025-11-26 10:13:18 +09:00
Melanie Plageman
cd38b7e773 Assert that cutoffs are provided if freezing will be attempted
heap_page_prune_and_freeze() requires the caller to initialize
PruneFreezeParams->cutoffs so that the function can correctly evaluate
whether tuples should be frozen. This requirement previously existed
only in comments and was easy to miss, especially after “cutoffs” was
converted from a direct function parameter to a field of the newly
introduced PruneFreezeParams struct (added in 1937ed7062). Adding an
assert makes this requirement explicit and harder to violate.

Also, fix a minor typo while we're at it.

Author: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/0AC177F5-5E26-45EE-B273-357C51212AC5%40gmail.com
2025-11-25 16:41:29 -05:00
Jeff Davis
3b9c118920 Remove a useless length check.
Author: Chao Li <lic@highgo.com>
Discussion: https://postgr.es/m/CAEoWx2mW0P8CByavV58zm3=eb2MQHaKOcDEF5B2UJYRyC2c3ig@mail.gmail.com
2025-11-25 11:39:41 -08:00
Álvaro Herrera
33bb3dc2d8 pg_dump tests: don't put dumps in stdout
This bloats the regression log files for no reason.

Backpatch to 18; no further only because it fails to apply cleanly.
(It's just whitespace change that conflicts, but I don't think this
warrants more effort than this.)

Discussion: https://postgr.es/m/202511251218.zfs4nu2qnh2m@alvherre.pgsql
2025-11-25 19:08:36 +01:00
Álvaro Herrera
417ac9c1ee Improve test case stability
Given unlucky timing, some of the new tests added by commit bc32a12e0d
can fail spuriously.  We haven't seen such failures yet in buildfarm,
but allegedly we can prevent them with this tweak.

While at it, remove an unused injection point I (Álvaro) added.

Author: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Discussion: https://postgr.es/m/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com
Discussion: https://postgr.es/m/CADzfLwV5oQq-Vg_VmG_o4SdL6yHjDoNO4T4pMtgJLzYGmYf74g@mail.gmail.com
2025-11-25 18:20:06 +01:00
Peter Eisentraut
7169c0b96b gen_guc_tables.pl: Validate required GUC fields before code generation
Previously, gen_guc_tables.pl would emit "Use of uninitialized value"
warnings if required fields were missing in guc_parameters.dat (for
example, when an integer or real GUC omitted the 'max' value).  The
resulting error messages were unclear and did not identify which GUC
entry was problematic.

Add explicit validation of required fields depending on the parameter
type, and fail with a clear and specific message such as:

    guc_parameters.dat:1909: error: entry "max_index_keys" of type "int" is missing required field "max"

No changes to generated guc_tables.c.

Author: Chao Li <lic@highgo.com>
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/CAEoWx2%3DoP4LgHi771_OKhPPUS7B-CTqCs%3D%3DuQcNXWrwBoAm5Vg%40mail.gmail.com
2025-11-25 16:50:34 +01:00
Peter Eisentraut
2256af4ba2 backend/nodes cleanup: Move loop variables definitions into for statement
Author: Chao Li (Evan) <lic@highgo.com>
Discussion: https://www.postgresql.org/message-id/flat/CAEoWx2nP12qwAaiJutbn1Kw50atN6FbMJNQ4bh4%2BfP_Ay_u7Eg%40mail.gmail.com
2025-11-25 15:38:23 +01:00
Amit Kapila
3df4df53b0 Fix a BF failure caused by commit 76b78721ca.
The issue occurred because the replication slot was not released in the
slotsync worker when a slot synchronization cycle was skipped. This skip
happened because the required WAL was not received and flushed on the
standby server. As a result, in the next cycle, when attempting to acquire
the slot, an assertion failure was triggered.

Author: Hou Zhijie <houzj.fnst@fujitsu.com>
Discussion: https://postgr.es/m/CAA4eK1KMwYUYy=oAVHu9mam+vX50ixxfhO4_C=kgQC8VCQHEfw@mail.gmail.com
2025-11-25 08:49:46 +00:00
Amit Kapila
76b78721ca Add slotsync skip statistics.
This patch adds two new columns to the pg_stat_replication_slots view:
slotsync_skip_count - the total number of times a slotsync operation was
skipped.
slotsync_skip_at - the timestamp of the most recent skip.

These additions provide better visibility into replication slot
synchronization behavior.

A future patch will introduce the slotsync_skip_reason column in
pg_replication_slots to capture the reason for skip.

Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAE9k0PkhfKrTEAsGz4DjOhEj1nQ+hbQVfvWUxNacD38ibW3a1g@mail.gmail.com
2025-11-25 07:06:02 +00:00
Peter Eisentraut
c581c9a7ac Remove obsolete comment
This comment should probably have been moved to pg_locale_libc.c in
commit 66ac94cdc7 (2024), but upon closer examination it was already
completely obsolete then.

The first part of the comment has been obsolete since commit
85feb77aa0 (2017), which required that the system provides the
required wide-character functions.

The second part has been obsolete since commit e9931bfb75 (2024),
which eliminated code paths depending on the global LC_CTYPE setting.

Discussion: https://www.postgresql.org/message-id/flat/170308e6-a7a3-4484-87b2-f960bb564afa%40eisentraut.org
2025-11-25 06:26:49 +01:00
Michael Paquier
ed823da128 Rename routines for write/read of pgstats file
This commit renames write_chunk and read_chunk to respectively
pgstat_write_chunk() and pgstat_read_chunk(), along with the *_s
convenience macros.

These are made available for plug-ins, so as any code that decides to
write and/or read stats data can rely on a single code path for this
work.

Extracted from a larger patch by the same author.

Author: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/CAA5RZ0s9SDOu+Z6veoJCHWk+kDeTktAtC-KY9fQ9Z6BJdDUirQ@mail.gmail.com
2025-11-25 10:55:40 +09:00
Andres Freund
81f7738953 lwlock: Fix, currently harmless, bug in LWLockWakeup()
Accidentally the code in LWLockWakeup() checked the list of to-be-woken up
processes to see if LW_FLAG_HAS_WAITERS should be unset. That means that
HAS_WAITERS would not get unset immediately, but only during the next,
unnecessary, call to LWLockWakeup().

Luckily, as the code stands, this is just a small efficiency issue.

However, if there were (as in a patch of mine) a case in which LWLockWakeup()
would not find any backend to wake, despite the wait list not being empty,
we'd wrongly unset LW_FLAG_HAS_WAITERS, leading to potentially hanging.

While the consequences in the backbranches are limited, the code as-is
confusing, and it is possible that there are workloads where the additional
wait list lock acquisitions hurt, therefore backpatch.

Discussion: https://postgr.es/m/fvfmkr5kk4nyex56ejgxj3uzi63isfxovp2biecb4bspbjrze7@az2pljabhnff
Backpatch-through: 14
2025-11-24 18:10:48 -05:00
Jeff Davis
f81bf78ce1 Avoid global LC_CTYPE dependency in pg_locale_libc.c.
Call tolower_l() directly instead of through pg_tolower(), because the
latter depends on the global LC_CTYPE.

Discussion: https://postgr.es/m/8186b28a1a39e61a0d833a4c25a8909ebbbabd48.camel@j-davis.com
2025-11-24 14:55:09 -08:00
Tom Lane
698fa924b1 Improve detection of implicitly-temporary views.
We've long had a practice of making views temporary by default if they
reference any temporary tables.  However the implementation was pretty
incomplete, in that it only searched for RangeTblEntry references to
temp relations.  Uses of temporary types, regclass constants, etc
were not detected even though the dependency mechanism considers them
grounds for dropping the view.  Thus a view not believed to be temp
could silently go away at session exit anyhow.

To improve matters, replace the ad-hoc isQueryUsingTempRelation()
logic with use of the dependency-based infrastructure introduced by
commit 572c40ba9.  This is complete by definition, and it's less code
overall.

While we're at it, we can also extend the warning NOTICE (or ERROR
in the case of a materialized view) to mention one of the temp
objects motivating the classification of the view as temp, as was
done for functions in 572c40ba9.

Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Jim Jones <jim.jones@uni-muenster.de>
Discussion: https://postgr.es/m/19cf6ae1-04cd-422c-a760-d7e75fe6cba9@uni-muenster.de
2025-11-24 17:00:16 -05:00
Jacob Champion
0664aa4ff8 Reorganize pqcomm.h a bit
Group the PG_PROTOCOL() codes, add a comment to AuthRequest now that the
AUTH_REQ codes live in a different header, and make some small
adjustments to spacing and comment style for the sake of scannability.

Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CAOYmi%2B%3D6zg4oXXOQtifrVao_YKiujTDa3u6bxnU08r0FsSig4g%40mail.gmail.com
2025-11-24 10:01:30 -08:00
Jacob Champion
e2ceff13d8 postgres: Use pg_{add,mul}_size_overflow()
The backend implementations of add_size() and mul_size() can now make
use of the APIs provided in common/int.h.

Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAOYmi%2B%3D%2BpqUd2MUitvgW1pAJuXgG_TKCVc3_Ek7pe8z9nkf%2BAg%40mail.gmail.com
2025-11-24 09:59:54 -08:00
Jacob Champion
8934f2136c Add pg_add_size_overflow() and friends
Commit 600086f47 added (several bespoke copies of) size_t addition with
overflow checks to libpq. Move this to common/int.h, along with
its subtraction and multiplication counterparts.

pg_neg_size_overflow() is intentionally omitted; I'm not sure we should
add SSIZE_MAX to win32_port.h for the sake of a function with no
callers.

Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAOYmi%2B%3D%2BpqUd2MUitvgW1pAJuXgG_TKCVc3_Ek7pe8z9nkf%2BAg%40mail.gmail.com
2025-11-24 09:59:38 -08:00
Jacob Champion
f1881c7dd3 Make some use of anonymous unions [libpq-oauth]
Make some use of anonymous unions, which are allowed as of C11, as
examples and encouragement for future code, and to test compilers.

This commit changes the json_field struct.

Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/CAOYmi%2BnV25oC5uXFgWodydGrHkfWMDCLUcjbAreM3mNX%3DF2JWw%40mail.gmail.com
2025-11-24 09:55:16 -08:00
Álvaro Herrera
bc32a12e0d Fix infer_arbiter_index during concurrent index operations
Previously, we would only consider indexes marked indisvalid as usable
for INSERT ON CONFLICT.  But that's problematic during CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY, because concurrent transactions
would end up with inconsistents lists of inferred indexes, leading to
deadlocks and spurious errors about unique key violations (because two
transactions are operating on different indexes for the speculative
insertion tokens).  Change this function to return indexes even if
invalid.  This fixes the spurious errors and deadlocks.

Because such indexes might not be complete, we still need uniqueness to
be verified in a different way.  We do that by requiring that at least
one index marked valid is part of the set of indexes returned.  It is
that index that is going to help ensure that the inserted tuple is
indeed unique.

This does not fix similar problems occurring with partitioned tables or
with named constraints.  These problems will be fixed in follow-up
commits.

We have no user report of this problem, even though it exists in all
branches.  Because of that and given that the fix is somewhat tricky, I
decided not to backpatch for now.

Author: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CANtu0ogv+6wqRzPK241jik4U95s1pW3MCZ3rX5ZqbFdUysz7Qw@mail.gmail.com
2025-11-24 17:17:29 +01:00
Michael Paquier
e429c3cecb Move isolation test index-killtuples to src/test/modules/index/
index-killtuples test depends on the contrib modules btree_gin and
btree_gist, which would not be installed in a temporary installation
with an execution of the main isolation test suite like this one:
make -C src/test/isolation/ check

src/test/isolation/ should not depend on contrib/, and EXTRA_INSTALL has
no effect in this case as this test suite uses its own Makefile rules.

This commit moves index-killtuples into its new module, called "index",
whose name looks like the best fit there can be as it depends on more
than one index AM.  btree_gin and btree_gist are now pulled in the
temporary installation with EXTRA_INSTALL.  The test is renamed to
"killtuples", for simplicity.

Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Suggested-by: Andres Freund <andres@anarazel.de>
Suggested-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/aKJsWedftW7UX1WM@paquier.xyz
2025-11-24 19:33:51 +09:00
Peter Eisentraut
d4c0f91f7d C11 alignas instead of unions -- extended alignments
This replaces some uses of pg_attribute_aligned() with the standard
alignas() for cases where extended alignment (larger than max_align_t)
is required.

This patch stipulates that all supported compilers must support
alignments up to PG_IO_ALIGN_SIZE, but that seems pretty likely.

We can then also desupport the case where direct I/O is disabled
because pg_attribute_aligned is not supported.

Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/46f05236-d4d4-4b4e-84d4-faa500f14691%40eisentraut.org
2025-11-24 07:39:37 +01:00