1
0
mirror of https://github.com/postgres/postgres.git synced 2025-12-19 17:02:53 +03:00
Commit Graph

8620 Commits

Author SHA1 Message Date
Daniel Gustafsson
381f5cffae Remove check for NULL in STRICT function
test_bms_make_singleton is defined as STRICT and only takes a single
parameter, so there is no need to check that parameter for NULL as a
NULL input won't ever reach there.

Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/BC483901-9587-4076-B20F-9A414C66AB78@yesql.se
2025-10-02 22:54:37 +02:00
Daniel Gustafsson
a1b064e4b2 Fixes for comments in test_bitmapset
This fixes a typo in the sql/expected test files and removes a
leftover comment from test_bitmapset.c from when the functions
invoked bms_free.

Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/978D21E8-9D3B-40EA-A4B1-F87BABE7868C@yesql.se
2025-10-02 22:41:24 +02:00
Jacob Champion
fd726b8379 test_json_parser: Speed up 002_inline.pl
Some macOS machines are having trouble with 002_inline, which executes
the JSON parser test executables hundreds of times in a nested loop.
Both developer machines and buildfarm critters have shown excessive test
durations, upwards of 20 seconds.

Push the innermost loop of 002_inline, which iterates through differing
chunk sizes, down into the test executable. (I'd eventually like to push
all of the JSON unit tests down into C, but this is an easy win in the
short term.) Testers have reported a speedup between 4-9x.

Reported-by: Robert Haas <robertmhaas@gmail.com>
Suggested-by: Andres Freund <andres@anarazel.de>
Tested-by: Andrew Dunstan <andrew@dunslane.net>
Tested-by: Tom Lane <tgl@sss.pgh.pa.us>
Tested-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA%2BTgmobKoG%2BgKzH9qB7uE4MFo-z1hn7UngqAe9b0UqNbn3_XGQ%40mail.gmail.com
Backpatch-through: 17
2025-10-01 09:48:57 -07:00
David Rowley
3a66158068 Minor fixups of test_bitmapset.c
The macro's comment had become outdated from a prior version and there's
now no longer a need for the do/while loop (or my misplaced semi-colon).

Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvr+P454SP_LDvB=bViPAbDQhV1Jmg5M55GEKz0d3z25NA@mail.gmail.com
2025-10-01 18:50:28 +13:00
Michael Paquier
9952f6c05a test_bitmapset: Simplify code of the module
Two macros are added in this module, to cut duplicated patterns:
- PG_ARG_GETBITMAPSET(), for input argument handling, with knowledge
about NULL.
- PG_RETURN_BITMAPSET_AS_TEXT(), that generates a text result from a
Bitmapset.

These changes limit the code so as the SQL functions are now mostly
wrappers of the equivalent C function.  Functions that use integer input
arguments still need some NULL handling, like bms_make_singleton().

A NULL input is translated to "<>", which is what nodeToString()
generates.  Some of the tests are able to generate this result.

Per discussion, the calls of bms_free() are removed.  These may be
justified if the functions are used in a rather long-lived memory
context, but let's keep the code minimal for now.  These calls used NULL
checks, which were also not necessary as NULL is an input authorized by
bms_free().

Some of the tests existed to cover behaviors related to the SQL
functions for NULL inputs.  Most of them are still relevant, as the
routines of bitmapset.c are able to handle such cases.

The coverage reports of bitmapset.c and test_bitmapset.c remain the
same after these changes, with 300 lines of C code removed.

Author: David Rowley <dgrowleyml@gmail.com>
Co-authored-by: Greg Burd <greg@burd.me>
Discussion: https://postgr.es/m/CAApHDvqghMnm_zgSNefto9oaEJ0S-3Cgb3gdsV7XvLC-hMS02Q@mail.gmail.com
2025-10-01 14:17:54 +09:00
Michael Paquier
3cc689f833 injection_points: Add proper locking when reporting fixed-variable stats
Contrary to its siblings for the archiver, the bgwriter and the
checkpointer stats, pgstat_report_inj_fixed() can be called
concurrently.  This was causing an assertion failure, while messing up
with the stats.

This code is aimed at being a template for extension developers, so it
is not a critical issue, but let's be correct.  This module has also
been useful for some benchmarking, at least for me, and that was how I
have discovered this issue.

Oversight in f68cd847fa.

Author: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Discussion: https://postgr.es/m/aNnXbAXHPFUWPIz2@paquier.xyz
Backpatch-through: 18
2025-09-30 09:02:09 +09:00
Tom Lane
ef38a4d975 Add GROUP BY ALL.
GROUP BY ALL is a form of GROUP BY that adds any TargetExpr that does
not contain an aggregate or window function into the groupClause of
the query, making it exactly equivalent to specifying those same
expressions in an explicit GROUP BY list.

This feature is useful for certain kinds of data exploration.  It's
already present in some other DBMSes, and the SQL committee recently
accepted it into the standard, so we can be reasonably confident in
the syntax being stable.  We do have to invent part of the semantics,
as the standard doesn't allow for expressions in GROUP BY, so they
haven't specified what to do with window functions.  We assume that
those should be treated like aggregates, i.e., left out of the
constructed GROUP BY list.

In passing, wordsmith some existing documentation about GROUP BY,
and update some neglected synopsis entries in select_into.sgml.

Author: David Christensen <david@pgguru.net>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAHM0NXjz0kDwtzoe-fnHAqPB1qA8_VJN0XAmCgUZ+iPnvP5LbA@mail.gmail.com
2025-09-29 16:55:17 -04:00
Noah Misch
a95393ecdb Fix StatisticsObjIsVisibleExt() for pg_temp.
Neighbor get_statistics_object_oid() ignores objects in pg_temp, as has
been the standard for non-relation, non-type namespace searches since
CVE-2007-2138.  Hence, most operations that name a statistics object
correctly decline to map an unqualified name to a statistics object in
pg_temp.  StatisticsObjIsVisibleExt() did not.  Consequently,
pg_statistics_obj_is_visible() wrongly returned true for such objects,
psql \dX wrongly listed them, and getObjectDescription()-based ereport()
and pg_describe_object() wrongly omitted namespace qualification.  Any
malfunction beyond that would depend on how a human or application acts
on those wrong indications.  Commit
d99d58cdc8 introduced this.  Back-patch to
v13 (all supported versions).

Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/20250920162116.2e.nmisch@google.com
Backpatch-through: 13
2025-09-29 11:15:44 -07:00
Michael Paquier
5668fff3c5 test_bitmapset: Expand more the test coverage
This commit expands the set of tests added by 00c3d87a5c, to bring the
coverage of bitmapset.c close to 100% by addressing a lot of corner
cases (most of these relate to word counts and reallocations).

Some of the functions of this module also have their own idea of the
result to return depending on the input values given.  These are
specific to the module, still let's add more coverage for all of them.

Some comments are made more consistent in the tests, while on it.

Author: Greg Burd <greg@burd.me>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/aNR-gsGmLnMaNT5i@paquier.xyz
2025-09-29 15:17:27 +09:00
Michael Paquier
acf0960c23 injection_points: Enable entry count in its variable-sized stats
This serves as coverage for the tracking of entry count added by
7bd2975fa9 as built-in variable-sized stats kinds have no need for it,
at least not yet.

A new function, called injection_points_stats_count(), is added to the
module.  It is able to return the number of entries.  This has been
useful when doing some benchmarking to check the sanity of the counts.

Author: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/aMPKWR81KT5UXvEr@paquier.xyz
2025-09-29 09:06:32 +09:00
Tom Lane
e849bd551c Add minimal sleep to stats isolation test functions.
The functions test_stat_func() and test_stat_func2() had empty
function bodies, so that they took very little time to run.  This made
it possible that on machines with relatively low timer resolution the
functions could return before the clock advanced, making the test fail
(as seen on buildfarm members fruitcrow and hamerkop).

To avoid that, pg_sleep for 10us during the functions.  As far as we
can tell, all current hardware has clock resolution much less than
that.  (The current implementation of pg_sleep will round it up to
1ms anyway, but someday that might get improved.)

Author: Michael Banck <mbanck@gmx.net>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/68d413a3.a70a0220.24c74c.8be9@mx.google.com
Backpatch-through: 15
2025-09-25 13:29:37 -04:00
Daniel Gustafsson
d8f07dbb81 Fix comments in recovery tests
Commit 4464fddf removed the large insertions but missed to remove
all the comments referring to them.  Also remove a superfluous ')'
in another comment.

Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/OSCPR01MB149663A99DAF2826BE691C23DF51FA@OSCPR01MB14966.jpnprd01.prod.outlook.com
2025-09-25 15:24:41 +02:00
John Naylor
5334620eef Update GB18030 encoding from version 2000 to 2022
Mappings for 18 characters have changed, affecting 36 code points. This
is a break in compatibility, but these characters are rarely used.

U+E5E5 (Private Use Area) was previously mapped to \xA3A0. This code
point now maps to \x65356535. Attempting to convert \xA3A0 will now
raise an error.

Separate from the 2022 update, the following mappings were previously
swapped, and subsequently corrected in 2000 and later versions:
 * U+E7C7 (Private Use Area) now maps to \x8135F437
 * U+1E3F (Latin Small Letter M with Acute) now maps to \xA8BC

The 2022 standard mentions the following policy changes, but they
have no effect in our implementation:

66 new ideographs are now required, but these are mapped
algorithmically so were already handled by utf8_and_gb18030.c.

Nine CJK compatibility ideographs are no longer required, but
implementations may retain them, as does the source we use from
the Unicode Consortium.

Release notes: Compatibility section

For further details, see:
https://www.unicode.org/L2/L2022/22274-disruptive-changes.pdf
https://ken-lunde.medium.com/the-gb-18030-2022-standard-3d0ebaeb4132

Author: Chao Li <lic@highgo.com>
Author: Zheng Tao <taoz@highgo.com>
Discussion: https://postgr.es/m/966d9fc.169.198741fe60b.Coremail.jiaoshuntian%40highgo.com
2025-09-24 13:26:05 +07:00
Michael Paquier
f83fe65f3f Fix compiler warnings in test_bitmapset
The macros doing conversions of/from "text" from/to Bitmapset were using
arbitrary casts with Datum, something that is not fine since
2a600a93c7.

These macros do not actually need casts with Datum, as they are given
already "text" and Bitmapset data in input.  They are updated to use
cstring_to_text() and text_to_cstring(), fixing the compiler warnings
reported by the buildfarm.  Note that appending a -m32 to gcc to trigger
32-bit builds was enough to reproduce the warnings here.

While on it, outer parenthesis are added to TEXT_TO_BITMAPSET(), and
inner parenthesis are removed from BITMAPSET_TO_TEXT(), to make these
macros more consistent with the style used in the tree, based on
suggestions by Tom Lane.

Oversights in commit 00c3d87a5c.

Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Author: Greg Burd <greg@burd.me>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/3027069.1758606227@sss.pgh.pa.us
2025-09-24 08:20:23 +09:00
Robert Haas
f2bae51dfd Keep track of what RTIs a Result node is scanning.
Result nodes now include an RTI set, which is only non-NULL when they
have no subplan, and is taken from the relid set of the RelOptInfo that
the Result is generating. ExplainPreScanNode now takes notice of these
RTIs, which means that a few things get schema-qualified in the
regression tests that previously did not. This makes the output more
consistent between cases where some part of the plan tree is replaced by
a Result node and those where this does not happen.

Likewise, pg_overexplain's EXPLAIN (RANGE_TABLE) now displays the RTIs
stored in a Result node just as it already does for other RTI-bearing
node types.

Result nodes also now include a result_reason, which tells us something
about why the Result node was inserted.  Using that information, EXPLAIN
now emits, where relevant, a "Replaces" line describing the origin of
a Result node.

The purpose of these changes is to allow code that inspects a Plan
tree to understand the origin of Result nodes that appear therein.

Discussion: http://postgr.es/m/CA+TgmoYeUZePZWLsSO+1FAN7UPePT_RMEZBKkqYBJVCF1s60=w@mail.gmail.com
Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>
2025-09-23 09:07:55 -04:00
Michael Paquier
00c3d87a5c Add a test module for Bitmapset
Bitmapset has a complex set of APIs, defined in bitmapset.h, and it can
be hard to test edge cases with the backend core code only.

This test module is aimed at closing the gap, and implements a set of
SQL functions that act as wrappers of the low-level C functions of the
same names.  These functions rely on text as data type for the input and
the output as Bitmapset as a node has support for these.  An extra
function, named test_random_operations(), can be used to stress bitmaps
with random member values and a defined number of operations potentially
useful for other purposes than only tests.

The coverage increases from 85.2% to 93.4%.  It should be possible to
cover more code paths, but at least it's a beginning.

Author: Greg Burd <greg@burd.me>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/7BD1ABDB-B03A-464A-9BA9-A73B55AD8A1F@getmailspring.com
2025-09-22 16:53:00 +09:00
Daniel Gustafsson
e1d917182c Add support for base64url encoding and decoding
This adds support for base64url encoding and decoding, a base64
variant which is safe to use in filenames and URLs.  base64url
replaces '+' in the base64 alphabet with '-' and '/' with '_',
thus making it safe for URL addresses and file systems.

Support for base64url was originally suggested by Przemysław Sztoch.

Author: Florents Tselai <florents.tselai@gmail.com>
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Reviewed-by: David E. Wheeler <david@justatheory.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Chao Li (Evan) <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/70f2b6a8-486a-4fdb-a951-84cef35e22ab@sztoch.pl
2025-09-20 23:19:32 +02:00
Tom Lane
1eccb93150 Re-allow using statistics for bool-valued functions in WHERE.
Commit a391ff3c3, which added the ability for a function's support
function to provide a custom selectivity estimate for "WHERE f(...)",
unintentionally removed the possibility of applying expression
statistics after finding there's no applicable support function.
That happened because we no longer fell through to boolvarsel()
as before.  Refactor to do so again, putting the 0.3333333 default
back into boolvarsel() where it had been (cf. commit 39df0f150).

I surely wouldn't have made this error if 39df0f150 had included
a test case, so add one now.  At the time we did not have the
"extended statistics" infrastructure, but we do now, and it is
also unable to work in this scenario because of this error.
So make use of that for the test case.

This is very clearly a bug fix, but I'm afraid to put it into
released branches because of the likelihood of altering plan
choices, which we avoid doing in minor releases.  So, master only.

Reported-by: Frédéric Yhuel <frederic.yhuel@dalibo.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/a8b99dce-1bfb-4d97-af73-54a32b85c916@dalibo.com
2025-09-20 12:44:52 -04:00
Amit Kapila
8aac5923a3 Improve few errdetail messages introduced in commit 0d48d393d4.
Based on suggestions by Tom Lane

Reported-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/20250916.114644.275726106301941878.horikyota.ntt@gmail.com
2025-09-19 04:52:59 +00:00
Amit Langote
8741e48e5d Fix EPQ crash from missing partition pruning state in EState
Commit bb3ec16e14 moved partition pruning metadata into PlannedStmt.
At executor startup this metadata is used to initialize the EState
fields es_part_prune_infos, es_part_prune_states, and
es_part_prune_results.  EvalPlanQualStart() failed to copy those
fields into the child EState, causing NULL dereference when Append
ran partition pruning during a recheck. This can occur with DELETE
or UPDATE on partitioned tables that use runtime pruning, e.g. with
generic plans.

Fix by copying all partition pruning state into the EPQ estate.

Add an isolation test that reproduces the crash with concurrent
UPDATE and DELETE on a partitioned table, where the DELETE session
hits the crash during its EPQ recheck after the UPDATE commits.

Bug: #19056
Reported-by: Fei Changhong <feichanghong@qq.com>
Diagnozed-by: Fei Changhong <feichanghong@qq.com>
Author: David Rowley <dgrowleyml@gmail.com>
Co-authored-by: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/19056-a677cef9b54d76a0%40postgresql.org
2025-09-19 11:38:29 +09:00
Nathan Bossart
16607718c0 Add a test harness for the LWLock tranche code.
This code is heavily used and already has decent test coverage, but
it lacks a dedicated test suite.  This commit changes that.

Author: Sami Imseih <samimseih@gmail.com>
Co-authored-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CAA5RZ0tQ%2BEYSTOd2hQ8RXdsNfGBLAtOe-YmnsTE6ZVg0E-4qew%40mail.gmail.com
Discussion: https://postgr.es/m/CAA5RZ0vpr0P2rbA%3D_K0_SCHM7bmfVX4wEO9FAyopN1eWCYORhA%40mail.gmail.com
2025-09-18 15:23:11 -05:00
Tom Lane
b0cc0a71e0 Calculate agglevelsup correctly when Aggref contains a CTE.
If an aggregate function call contains a sub-select that has
an RTE referencing a CTE outside the aggregate, we must treat
that reference like a Var referencing the CTE's query level
for purposes of determining the aggregate's level.  Otherwise
we might reach the nonsensical conclusion that the aggregate
should be evaluated at some query level higher than the CTE,
ending in a planner error or a broken plan tree that causes
executor failures.

Bug: #19055
Reported-by: BugForge <dllggyx@outlook.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19055-6970cfa8556a394d@postgresql.org
Backpatch-through: 13
2025-09-17 16:32:57 -04:00
Michael Paquier
39f67d9b55 injection_points: Fix incrementation of variable-numbered stats
The pending entry was not used when incrementing its data, directly
manipulating the shared memory pointer, without even locking it.  This
could mean losing statistics under concurrent activity.  The flush
callback was a no-op.

This code serves as a base template for extensions for the custom
cumulative statistics, so let's be clean and use a pending entry for the
incrementations, whose data is then flushed to the corresponding entry
in the shared hashtable when all the stats are reported, in its own
flush callback.

Author: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/CAA5RZ0v0U0yhPbY+bqChomkPbyUrRQ3rQXnZf_SB-svDiQOpgQ@mail.gmail.com
Backpatch-through: 18
2025-09-17 10:15:13 +09:00
David Rowley
ac06ea8f7b Add missing EPQ recheck for TID Range Scan
The EvalPlanQual recheck for TID Range Scan wasn't rechecking the TID qual
still passed after following update chains.  This could result in tuples
being updated or deleted by plans using TID Range Scans where the ctid of
the new (updated) tuple no longer matches the clause of the scan.  This
isn't desired behavior, and isn't consistent with what would happen if the
chosen plan had used an Index or Seq Scan, and that could lead to hard to
predict behavior for scans that contain TID quals and other quals as the
planner has freedom to choose TID Range or some other non-TID scan method
for such queries, and the chosen plan could change at any moment.

Here we fix this by properly implementing the recheck function for TID
Range Scans.

Backpatch to 14, where TID Range Scans were added

Reported-by: Sophie Alpert <pg@sophiebits.com>
Author: Sophie Alpert <pg@sophiebits.com>
Author: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/4a6268ff-3340-453a-9bf5-c98d51a6f729@app.fastmail.com
Backpatch-through: 14
2025-09-17 12:19:15 +12:00
David Rowley
dee21ea6d6 Add missing EPQ recheck for TID Scan
The EvalPlanQual recheck for TID Scan wasn't rechecking the TID qual
still passed after following update chains.  This could result in tuples
being updated or deleted by plans using TID Scans where the ctid of the
new (updated) tuple no longer matches the clause of the scan.  This isn't
desired behavior, and isn't consistent with what would happen if the
chosen plan had used an Index or Seq Scan, and that could lead to hard to
predict behavior for scans that contain TID quals and other quals as the
planner has freedom to choose TID or some other scan method for such
queries, and the chosen plan could change at any moment.

Here we fix this by properly implementing the recheck function for TID
Scans.

Backpatch to 13, oldest supported version

Reported-by: Sophie Alpert <pg@sophiebits.com>
Author: Sophie Alpert <pg@sophiebits.com>
Author: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/4a6268ff-3340-453a-9bf5-c98d51a6f729@app.fastmail.com
Backpatch-through: 13
2025-09-17 11:48:55 +12:00
Tom Lane
8abbbbae61 Revert "Avoid race condition between "GRANT role" and "DROP ROLE"".
This reverts commit 98fc31d649.
That change allowed DROP OWNED BY to drop grants of the target
role to other roles, arguing that nobody would need those
privileges anymore.  But that's not so: if you're not superuser,
you still need admin privilege on the target role so you can
drop it.

It's not clear whether or how the dependency-based approach
to solving the original problem can be adapted to keep these
grants.  Since v18 release is fast approaching, the sanest
thing to do seems to be to revert this patch for now.  The
race-condition problem is low severity and not worth taking
risks for.

I didn't force a catversion bump in 98fc31d64, so I won't do
so here either.

Reported-by: Dipesh Dhameliya <dipeshdhameliya125@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CABgZEgczOFicCJoqtrH9gbYMe_BV3Hq8zzCBRcMgmU6LRsihUA@mail.gmail.com
Backpatch-through: 18
2025-09-16 13:05:53 -04:00
Tom Lane
83a5641945 Provide more-specific error details/hints for function lookup failures.
Up to now we've contented ourselves with a one-size-fits-all error
hint when we fail to find any match to a function or procedure call.
That was mostly okay in the beginning, but it was never great, and
since the introduction of named arguments it's really not adequate.
We at least ought to distinguish "function name doesn't exist" from
"function name exists, but not with those argument names".  And the
rules for named-argument matching are arcane enough that some more
detail seems warranted if we match the argument names but the call
still doesn't work.

This patch creates a framework for dealing with these problems:
FuncnameGetCandidates and related code will now pass back a bitmask of
flags showing how far the match succeeded.  This allows a considerable
amount of granularity in the reports.  The set-bits-in-a-bitmask
approach means that when there are multiple candidate functions, the
report will reflect the match(es) that got the furthest, which seems
correct.  Also, we can avoid mentioning "maybe add casts" unless
failure to match argument types is actually the issue.

Extend the same return-a-bitmask approach to OpernameGetCandidates.
The issues around argument names don't apply to operator syntax,
but it still seems worth distinguishing between "there is no
operator of that name" and "we couldn't match the argument types".

While at it, adjust these messages and related ones to more strictly
separate "detail" from "hint", following our message style guidelines'
distinction between those.

Reported-by: Dominique Devienne <ddevienne@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/1756041.1754616558@sss.pgh.pa.us
2025-09-16 12:17:02 -04:00
Tom Lane
c7b0cb367d Add regression expected-files for older OpenSSL in FIPS mode.
In our previous discussions around making our regression tests
pass in FIPS mode, we concluded that we didn't need to support
the different error message wording observed with pre-3.0 OpenSSL.
However there are still a few LTS distributions soldiering along
with such versions, and now we have some in the buildfarm.
So let's add the variant expected-files needed to make them happy.

This commit only covers the core regression tests.  Previous
discussion suggested that we might need some adjustments in
contrib as well, but it's not totally clear to me what those
would be.  Rather than work it out from first principles,
I'll wait to see what the buildfarm shows.

Back-patch to v17 which is the oldest branch that claims
to support this case.

Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/443709.1757876535@sss.pgh.pa.us
Backpatch-through: 17
2025-09-16 10:09:49 -04:00
Richard Guo
b63a822452 Treat JsonConstructorExpr as non-strict
JsonConstructorExpr can produce non-NULL output with a NULL input, so
it should be treated as a non-strict construct.  Failing to do so can
lead to incorrect query behavior.

For example, in the reported case, when pulling up a subquery that is
under an outer join, if the subquery's target list contains a
JsonConstructorExpr that uses subquery variables and it is mistakenly
treated as strict, it will be pulled up without being wrapped in a
PlaceHolderVar.  As a result, the expression will be evaluated at the
wrong place and will not be forced to null when the outer join should
do so.

Back-patch to v16 where JsonConstructorExpr was introduced.

Bug: #19046
Reported-by: Runyuan He <runyuan@berkeley.edu>
Author: Tender Wang <tndrwang@gmail.com>
Co-authored-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/19046-765b6602b0a8cfdf@postgresql.org
Backpatch-through: 16
2025-09-16 18:42:20 +09:00
Amit Kapila
0f42206531 Fix intermittent BF failures in 035_conflicts.
This commit addresses two sources of instability in the 035_conflicts
test:

Unstable VACUUM usage:
The test previously relied on VACUUM to remove a deleted column, which can
be unreliable due to concurrent background writer or checkpoint activity
that may lock the page containing the deleted tuple. Since the test
already verifies that replication_slot.xmin has advanced sufficiently to
confirm the feature's correctness, so, the VACUUM step is removed to
improve test stability.

Timing-sensitive retention resumption check:
The test includes a check to confirm that retention of conflict-relevant
information resumes after setting max_retention_duration to 0. However, in
some cases, the apply worker resumes retention immediately after the
inactive slot is removed from synchronized_standby_slots, even before
max_retention_duration is updated. This can happen if remote changes are
applied in under 1ms, causing the test to timeout while waiting for a
later log position. To ensure consistent behavior, this commit delays the
removal of synchronized_standby_slots until after max_retention_duration
is set to 0.

Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/TY4PR01MB16907805DE4816E53C54708159414A@TY4PR01MB16907.jpnprd01.prod.outlook.com
2025-09-16 06:25:55 +00:00
Peter Eisentraut
ce71993ae4 Expand virtual generated columns in constraint expressions
Virtual generated columns in constraint expressions need to be
expanded because the optimizer matches these expressions to qual
clauses.  Failing to do so can cause us to miss opportunities for
constraint exclusion.

Author: Richard Guo <guofenglinux@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/204804c0-798f-4c72-bd1f-36116024fda3%40eisentraut.org
2025-09-15 16:27:50 +02:00
Peter Eisentraut
9ec0b29976 CREATE STATISTICS: improve misleading error message
The previous change (commit f225473cba) was still not on target,
because it talked about relation kinds, which are not what is being
checked here.  Provide a more accurate message.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACJufxEZ48toGH0Em_6vdsT57Y3L8pLF=DZCQ_gCii6=C3MeXw@mail.gmail.com
2025-09-15 11:43:34 +02:00
Amit Kapila
0d48d393d4 Resume conflict-relevant data retention automatically.
This commit resumes automatic retention of conflict-relevant data for a
subscription. Previously, retention would stop if the apply process failed
to advance its xmin (oldest_nonremovable_xid) within the configured
max_retention_duration and user needs to manually re-enable
retain_dead_tuples option. With this change, retention will resume
automatically once the apply worker catches up and begins advancing its
xmin (oldest_nonremovable_xid) within the configured threshold.

Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/OS0PR01MB5716BE80DAEB0EE2A6A5D1F5949D2@OS0PR01MB5716.jpnprd01.prod.outlook.com
2025-09-15 08:46:55 +00:00
Peter Eisentraut
bf5da5d6ca Hide duplicate names from extension views
If extensions of equal names were installed in different directories
in the path, the views pg_available_extensions and
pg_available_extension_versions would show all of them, even though
only the first one was actually reachable by CREATE EXTENSION.  To
fix, have those views skip extensions found later in the path if they
have names already found earlier.

Also add a bit of documentation that only the first extension in the
path can be used.

Reported-by: Pierrick <pierrick.chovelon@dalibo.com>
Discussion: https://www.postgresql.org/message-id/flat/8f5a0517-1cb8-4085-ae89-77e7454e27ba%40dalibo.com
2025-09-15 07:30:31 +02:00
Tom Lane
cdf7feb965 Amend recent fix for SIMILAR TO regex conversion.
Commit e3ffc3e91 fixed the translation of character classes in
SIMILAR TO regular expressions.  Unfortunately the fix broke a corner
case: if there is an escape character right after the opening bracket
(for example in "[\q]"), a closing bracket right after the escape
sequence would not be seen as closing the character class.

There were two more oversights: a backslash or a nested opening bracket
right at the beginning of a character class should remove the special
meaning from any following caret or closing bracket.

This bug suggests that this code needs to be more readable, so also
rename the variables "charclass_depth" and "charclass_start" to
something more meaningful, rewrite an "if" cascade to be more
consistent, and improve the commentary.

Reported-by: Dominique Devienne <ddevienne@gmail.com>
Reported-by: Stephan Springl <springl-psql@bfw-online.de>
Author: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAFCRh-8NwJd0jq6P=R3qhHyqU7hw0BTor3W0SvUcii24et+zAw@mail.gmail.com
Backpatch-through: 13
2025-09-13 16:55:51 -04:00
Alexander Korotkov
f6edf403a9 Specify locale provider for pg_regress --no-locale
pg_regress has a --no-locale option that forces the temporary database to
have C locale.  However, currently, locale C only exists in the 'builtin'
locale provider.  This makes 'pg_regress --no-locale' fail when the default
locale provider is not 'builtin'.  This commit makes 'pg_regress --no-locale'
specify both LOCALE='C' and LOCALE_PROVIDER='builtin'.

Discussion: https://postgr.es/m/b54921f95e23b4391b1613e9053a3d58%40postgrespro.ru
Author: Oleg Tselebrovskiy <o.tselebrovskiy@postgrespro.ru>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
2025-09-13 20:39:09 +03:00
Tom Lane
88824e6861 Avoid context dependency in test case added by 9a71989a8.
It's not quite clear to me why this didn't show up in my local
check-world testing, but some of the buildfarm evidently runs
this test with a different database name.  Adjust the test
so that that doesn't affect the reported error messages.
2025-09-12 18:45:06 -04:00
Tom Lane
9a71989a8f Reject "ALTER DATABASE/USER ... RESET foo" with invalid GUC name.
If the database or user had no entry in pg_db_role_setting,
RESET silently did nothing --- including not checking the
validity of the given GUC name.  This is quite inconsistent
and surprising, because you *would* get such an error if there
were any pg_db_role_setting entry, even though it contains
values for unrelated GUCs.

While this is clearly a bug, changing it in stable branches seems
unwise.  The effect will be that some ALTER commands that formerly
were no-ops will now be errors, and people don't like that sort of
thing in minor releases.

Author: Vitaly Davydov <v.davydov@postgrespro.ru>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/30783e-68c28a00-9-41004480@130449754
2025-09-12 18:10:11 -04:00
Tom Lane
f14ea34d6e Fix oversights in pg_event_trigger_dropped_objects() fixes.
Commit a0b99fc12 caused pg_event_trigger_dropped_objects()
to not fill the object_name field for schemas, which it
should have; and caused it to fill the object_name field
for default values, which it should not have.

In addition, triggers and RLS policies really should behave
the same way as we're making column defaults do; that is,
they should have is_temporary = true if they belong to a
temporary table.

Fix those things, and upgrade event_trigger.sql's woefully
inadequate test coverage of these secondary output columns.

As before, back-patch only to v15.

Reported-by: Sergey Shinderuk <s.shinderuk@postgrespro.ru>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/bd7b4651-1c26-4d30-832b-f942fabcb145@postgrespro.ru
Backpatch-through: 15
2025-09-12 17:43:17 -04:00
Noah Misch
4adb0380b9 Replace tests of ALTER DATABASE RESET TABLESPACE.
This unblocks rejection of that syntax.  One copy was a misspelling of
"SET TABLESPACE pg_default" that instead made no persistent changes.
The other copy just needed to populate a DATABASEOID syscache entry.
This slightly raises database.sql test coverage of catcache.c, while
dbcommands.c coverage remains the same.

Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/1802710.1757608564@sss.pgh.pa.us
2025-09-12 12:44:14 -07:00
Tom Lane
a0b99fc122 Report the correct is_temporary flag for column defaults.
pg_event_trigger_dropped_objects() would report a column default
object with is_temporary = false, even if it belongs to a
temporary table.  This seems clearly wrong, so adjust it to
report the table's temp-ness.

While here, refactor EventTriggerSQLDropAddObject to make its
handling of namespace objects less messy and avoid duplication
of the schema-lookup code.  And add some explicit test coverage
of dropped-object reports for dependencies of temp tables.

Back-patch to v15.  The bug exists further back, but the
GetAttrDefaultColumnAddress function this patch depends on does not,
and it doesn't seem worth adjusting it to cope with the older code.

Author: Antoine Violin <violin.antuan@gmail.com>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAFjUV9x3-hv0gihf+CtUc-1it0hh7Skp9iYFhMS7FJjtAeAptA@mail.gmail.com
Backpatch-through: 15
2025-09-11 17:11:57 -04:00
Álvaro Herrera
e8cec3d179 Add test for temporal referential integrity
This commit adds an isolation test showing that temporal foreign keys do
not permit referential integrity violations under concurrency, like
fk-snapshot-2.  You can show that the test fails by passing false for
detectNewRows to ri_PerformCheck in ri_restrict.

Author: Paul Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Rustam ALLAKOV <rustamallakov@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA+renyUp=xja80rBaB6NpY3RRdi750y046x28bo_xg29zKY72Q@mail.gmail.com
2025-09-11 18:16:25 +02:00
Álvaro Herrera
a2b4102a21 Fill testing gap for possible referential integrity violation
This commit adds a missing isolation test for (non-PERIOD) foreign keys.
With REPEATABLE READ, one transaction can insert a referencing row while
another deletes the referenced row, and both see a valid state.  But
after they have committed, the table violates referential integrity.

If the INSERT precedes the DELETE, we use a crosscheck snapshot to see
the just-added row, so that the DELETE can raise a foreign key error.
You can see the table violate referential integrity if you change
ri_restrict to pass false for detectNewRows to ri_PerformCheck.

A crosscheck snapshot is not needed when the DELETE comes first, because
the INSERT's trigger takes a FOR KEY SHARE lock that sees the row now
marked for deletion, waits for that transaction to commit, and raises a
serialization error.  I (Paul) added a test for that too though.

We already have a similar test (in ri-triggers.spec) for SERIALIZABLE
snapshot isolation showing that you can implement foreign keys with just
pl/pgSQL, but that test does nothing to validate ri_triggers.c.  We also
have tests (in fk-snapshot.spec) for other concurrency scenarios, but
not this one: we test concurrently deleting both the referencing and
referenced row, when the constraint activates a cascade/set null action.
But those tests don't exercise ri_restrict, and the consequence of
omitting a crosscheck comparison is different: a serialization failure,
not a referential integrity violation.

Author: Paul Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Rustam ALLAKOV <rustamallakov@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA+renyUp=xja80rBaB6NpY3RRdi750y046x28bo_xg29zKY72Q@mail.gmail.com
2025-09-11 18:11:46 +02:00
Amit Kapila
01d793698f Fix intermittent test failure introduced in 6456c6e2c4.
The test assumes that a backend will execute COMMIT PREPARED on the
publisher and hit the injection point commit-after-delay-checkpoint within
the commit critical section. This should cause the apply worker on the
subscriber to wait for the transaction to complete.

However, the test does not guarantee that the injection point is actually
triggered, creating a race condition where the apply worker may proceed
prematurely during COMMIT PREPARED.

This commit resolves the issue by explicitly waiting for the injection
point to be hit before continuing with the test, ensuring consistent and
reliable behavior.

Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Discussion: https://postgr.es/m/TY4PR01MB1690751D1CA8C128B0770EC6F9409A@TY4PR01MB16907.jpnprd01.prod.outlook.com
2025-09-11 09:33:48 +00:00
Peter Eisentraut
33eec80940 Fix CREATE TABLE LIKE with not-valid check constraint
In CREATE TABLE ... LIKE, any check constraints copied from the source
table should be set to valid if they are ENFORCED (the default).

Bug introduced in commit ca87c415e2.

Author: jian he <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/CACJufxH%3D%2Bod8Wy0P4L3_GpapNwLUP3oAes5UFRJ7yTxrM_M5kg%40mail.gmail.com
2025-09-10 13:25:58 +02:00
Nathan Bossart
530cfa8eb5 test_slru: Fix LWLock tranche allocation in EXEC_BACKEND builds.
Currently, test_slru's shmem_startup_hook unconditionally generates
new LWLock tranche IDs.  This is fine on non-EXEC_BACKEND builds,
where only the postmaster executes this hook, but on EXEC_BACKEND
builds, every backend executes it, too.  To fix, only generate the
tranche IDs in the postmaster process by checking the
IsUnderPostmaster variable.

This is arguably a bug fix and could be back-patched, but since the
damage is limited to some extra unused tranche IDs in a test
module, I'm not going to bother.

Reported-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/CAA5RZ0vaAuonaf12CeDddQJu5xKL%2B6xVyS%2B_q1%2BcH%3D33JXV82w%40mail.gmail.com
2025-09-09 14:09:36 -05:00
Dean Rasheed
faf071b553 Add date and timestamp variants of random(min, max).
This adds 3 new variants of the random() function:

    random(min date, max date) returns date
    random(min timestamp, max timestamp) returns timestamp
    random(min timestamptz, max timestamptz) returns timestamptz

Each returns a random value x in the range min <= x <= max.

Author: Damien Clochard <damien@dalibo.info>
Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Vik Fearing <vik@postgresfriends.org>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/f524d8cab5914613d9e624d9ce177d3d@dalibo.info
2025-09-09 10:39:30 +01:00
Robert Haas
6f79024df3 Don't generate fake "ANY_subquery" aliases, either.
This is just like the previous commit, but for a different invented
alias name.

Author: Robert Haas <rhaas@postgresql.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA+TgmoYSYmDA2GvanzPMci084n+mVucv0bJ0HPbs6uhmMN6HMg@mail.gmail.com
2025-09-08 12:24:02 -04:00
Robert Haas
585e31fcb6 Don't generate fake "*SELECT*" or "*SELECT* %d" subquery aliases.
rte->alias should point only to a user-written alias, but in these
cases that principle was violated. Fixing this causes some regression
test output changes: wherever rte->alias previously had a value and
is now NULL, rte->eref is now set to a generated name rather than to
rte->alias; and the scheme used to generate eref names differs from
what we were doing for aliases.

The upshot is that instead of "*SELECT*" or "*SELECT* %d",
EXPLAIN will now emit "unnamed_subquery" or "unnamed_subquery_%d".
But that's a reasonable descriptor, and we were already producing
that in yet other cases, so this seems not too objectionable.

Author: Tom Lane <tgl@sss.pgh.pa.us>
Co-authored-by: Robert Haas <rhaas@postgresql.org>
Discussion: https://postgr.es/m/CA+TgmoYSYmDA2GvanzPMci084n+mVucv0bJ0HPbs6uhmMN6HMg@mail.gmail.com
2025-09-08 11:50:33 -04:00
Amit Kapila
6456c6e2c4 Add test to prevent premature removal of conflict-relevant data.
A test has been added to ensure that conflict-relevant data is not
prematurely removed when a concurrent prepared transaction is being
committed on the publisher.

This test introduces an injection point that simulates the presence of a
prepared transaction in the commit phase, validating that the system
correctly delays conflict slot advancement until the transaction is fully
committed.

Additionally, the test serves as a safeguard for developers, ensuring that
the acquisition of the commit timestamp does not occur before marking
DELAY_CHKPT_IN_COMMIT in RecordTransactionCommitPrepared.

Reported-by: Robert Haas <robertmhaas@gmail.com>
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/OS9PR01MB16913F67856B0DA2A909788129400A@OS9PR01MB16913.jpnprd01.prod.outlook.com
2025-09-08 12:06:03 +00:00