1
0
mirror of https://github.com/postgres/postgres.git synced 2025-10-19 15:49:24 +03:00
Commit Graph

62003 Commits

Author SHA1 Message Date
Andres Freund
5865150b6d aio: Stop using enum bitfields due to bad code generation
During an investigation into rather odd aio related errors on macos, observed
by Alexander and Konstantin, we started to wonder if bitfield access is
related to the error. At the moment it looks like it is related, we cannot
reproduce the failures when replacing the bitfields. In addition, the problem
can only be reproduced with some compiler [versions] and not everyone has been
able to reproduce the issue.

The observed problem is that, very rarely, PgAioHandle->{state,target} are in
an inconsistent state, after having been checked to be in a valid state not
long before, triggering an assertion failure. Unfortunately, this could be
caused by wrong compiler code generation or somehow of missing memory barriers
- we don't really know. In theory there should not be any concurrent write
access to the handle in the state the bug is triggered, as the handle was idle
and is just being initialized.

Separately from the bug, we observed that at least gcc and clang generate
rather terrible code for the bitfield access. Even if it's not clear if the
observed assertion failure is actually caused by the bitfield somehow, the bad
code generation alone is sufficient reason to stop using bitfields.

Therefore, replace the enum bitfields with uint8s and instead cast in each
switch statement.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reported-by: Konstantin Knizhnik <knizhnik@garret.ru>
Discussion: https://postgr.es/m/1500090.1745443021@sss.pgh.pa.us
Backpatch-through: 18
2025-08-27 19:12:11 -04:00
Peter Eisentraut
310d04169a Put back intra-grant-inplace.spec test coverage
Commit d31bbfb659 lost some test coverage, because the situation
being tested, a concurrent DROP, cannot happen anymore.  Put the test
coverage back with a bit of a trick, by deleting directly from the
catalog table.

Co-authored-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/bf72b82c-124d-4efa-a484-bb928e9494e4@eisentraut.org
2025-08-27 17:46:31 +02:00
Peter Eisentraut
e36fa9319b Improve objectNamesToOids() comment
Commit d31bbfb659 removed the comment at objectNamesToOids() that
there is no locking, because that commit added locking.  But to fix
all the problems, we'd still need a stronger lock.  So put the comment
back with more a detailed explanation.

Co-authored-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/bf72b82c-124d-4efa-a484-bb928e9494e4@eisentraut.org
2025-08-27 17:46:26 +02:00
Peter Eisentraut
990c8db182 Fix: Don't strip $libdir from nested module_pathnames
This patch fixes a bug in how 'load_external_function' handles
'$libdir/ prefixes in module paths.

Previously, 'load_external_function' would unconditionally strip
'$libdir/' from the beginning of the 'filename' string.  This caused
an issue when the path was nested, such as "$libdir/nested/my_lib".
Stripping the prefix resulted in a path of "nested/my_lib", which
would fail to be found by the expand_dynamic_library_name function
because the original '$libdir' macro was removed.

To fix this, the code now checks for the presence of an additional
directory separator ('/' or '\') after the '$libdir/' prefix.  The
prefix is only stripped if the remaining string does not contain a
separator.  This ensures that simple filenames like '"$libdir/my_lib"'
are correctly handled, while nested paths are left intact for
'expand_dynamic_library_name' to process correctly.

Reported-by: Dilip Kumar <dilipbalaut@gmail.com>
Co-authored-by: Matheus Alcantara <matheusssilv97@gmail.com>
Co-authored-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Srinath Reddy Sadipiralla <srinath2133@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAFiTN-uKNzAro4tVwtJhF1UqcygfJ%2BR%2BRL%3Db-_ZMYE3LdHoGhA%40mail.gmail.com
2025-08-27 15:49:58 +02:00
Jeff Davis
ef5b87b970 Check for more Unicode functions during upgrade.
When checking for expression indexes that may be affected by a Unicode
update during upgrade, check for a few more functions. Specifically,
check for documented regexp functions, as well as the new CASEFOLD()
function.

Also, fully-qualify references to pg_catalog.text and
pg_catalog.regtype.

Discussion: https://postgr.es/m/399b656a3abb0c9283538a040f72199c0601525c.camel@j-davis.com
Backpatch-through: 18
2025-08-26 22:55:14 -07:00
Jacob Champion
85b380162c oauth: Explicitly depend on -pthread
Followup to 4e1e41733 and 52ecd05ae. oauth-utils.c uses
pthread_sigmask(), requiring -pthread on Debian bullseye at minimum.

Reported-by: Christoph Berg <myon@debian.org>
Tested-by: Christoph Berg <myon@debian.org>
Discussion: https://postgr.es/m/aK4PZgC0wuwQ5xSK%40msg.df7cb.de
Backpatch-through: 18
2025-08-26 14:16:31 -07:00
Peter Eisentraut
e567e22290 Message style improvements
Mostly adding some quoting.
2025-08-26 22:52:11 +02:00
Nathan Bossart
984d7165dd Document privileges required for vacuumdb --missing-stats-only.
When vacuumdb's --missing-stats-only option is used, the catalog
query for retrieving the list of relations to process must read
pg_statistic and pg_statistic_ext_data.  However, those catalogs
can only be read by superusers by default, so --missing-stats-only
is effectively superuser-only.  This is unfortunate, but since the
option is primarily intended for use by administrators after
running pg_upgrade, let's just live with it for v18.  This commit
adds a note about the aforementioned privilege requirements to the
documentation for --missing-stats-only.

We first tried to improve matters by modifying the query to read
the pg_stats and pg_stats_ext system views instead.  While that is
indeed more lenient from a privilege standpoint, it is also
borderline incomprehensible.  pg_stats shows rows for which the
user has the SELECT privilege on the corresponding column, and
pg_stats_ext shows rows for tables the user owns.  Meanwhile,
ANALYZE requires either MAINTAIN on the table or, for non-shared
relations, ownership of the database.  But even if the privilege
discrepancies were tolerable, the performance impact was not.
Ultimately, the modified query was substantially more expensive, so
we abandoned the idea.

For v19, perhaps we could introduce a simple, inexpensive way to
discover which relations are missing statistics, such as a system
function or view with similar privilege requirements to ANALYZE.
Unfortunately, it is far too late for anything like that in v18.

Reviewed-by: Yugo Nagata <nagata@sraoss.co.jp>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwHh43suEfss1wvBsk7vqiou%3DUY0zcy8HGyE5hBp%2BHZ7SQ%40mail.gmail.com
Backpatch-through: 18
2025-08-26 14:49:01 -05:00
Tom Lane
327b7324d0 Put "excludeOnly" GIN scan keys at the end of the scankey array.
Commit 4b754d6c1 introduced the concept of an excludeOnly scan key,
which cannot select matching index entries but can reject
non-matching tuples, for example a tsquery such as '!term'.  There are
poorly-documented assumptions that such scan keys do not appear as the
first scan key.  ginNewScanKey did nothing to ensure that, however,
with the result that certain GIN index searches could go into an
infinite loop while apparently-equivalent queries with the clauses in
a different order were fine.

Fix by teaching ginNewScanKey to place all excludeOnly scan keys
after all not-excludeOnly ones.  So far as we know at present,
it might be sufficient to avoid the case where the very first
scan key is excludeOnly; but I'm not very convinced that there
aren't other dependencies on the ordering.

Bug: #19031
Reported-by: Tim Wood <washwithcare@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19031-0638148643d25548@postgresql.org
Backpatch-through: 13
2025-08-26 12:08:57 -04:00
Tom Lane
b55068236c Do CHECK_FOR_INTERRUPTS inside, not before, scanGetItem.
The CHECK_FOR_INTERRUPTS call in gingetbitmap turns out to be
inadequate to prevent a long uninterruptible loop, because
we now know a case where looping occurs within scanGetItem.
While the next patch will fix the bug that caused that, it
seems foolish to assume that no similar patterns are possible.
Let's do the CFI within scanGetItem's retry loop, instead.
This demonstrably allows canceling out of the loop exhibited
in bug #19031.

Bug: #19031
Reported-by: Tim Wood <washwithcare@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19031-0638148643d25548@postgresql.org
Backpatch-through: 13
2025-08-26 11:38:41 -04:00
Alexander Korotkov
5f6f951f88 Improve RowMark handling during Self-Join Elimination
The Self-Join Elimination SJE feature messes up keeping and removing RowMark's
in remove_self_joins_one_group().  That didn't lead to user-level error,
because the planned RowMark is only used to reference a rtable entry in later
execution stages.  An RTE entry for keeping and removing relations is
identical and refers to the same relation OID.

To reduce confusion and prevent future issues, this commit cleans up the code
and fixes the incorrect behaviour.  Furthermore, it includes sanity checks in
setrefs.c on existing non-null RTE and RelOptInfo entries for each RowMark.

Discussion: https://postgr.es/m/18c6bd6c-6d2a-419a-b0da-dfedef34b585%40gmail.com
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com>
Backpatch-through: 18
2025-08-26 13:23:18 +03:00
Alexander Korotkov
d713cf9b65 Refactor variable names in remove_self_joins_one_group()
Rename inner and outer to rrel and krel, respectively, to highlight their
connection to r and k indexes.  For the same reason, rename imark and omark
to rmark and kmark.

Discussion: https://postgr.es/m/18c6bd6c-6d2a-419a-b0da-dfedef34b585%40gmail.com
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Greg Sabino Mullane <htamfids@gmail.com>
Backpatch-through: 18
2025-08-26 13:22:43 +03:00
Alexander Korotkov
f8ce9ed220 Further clarify documentation for the initcap function
This is a follow-up for commit c2c2c7e225.  It further clarifies the
following in the initcap function documentation:

 * Document that title case is used for digraphs in specific locales,
 * Reference particular ICU function used,
 * Add note about the purpose of the function.

Discussion: https://postgr.es/m/804cc10ef95d4d3b298e76b181fd9437%40postgrespro.ru
Author: Oleg Tselebrovskiy <o.tselebrovskiy@postgrespro.ru>
Co-authored-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
2025-08-26 13:22:43 +03:00
Peter Eisentraut
f5e0186f86 Raise C requirement to C11
This changes configure and meson.build to require at least C11,
instead of the previous C99.  The installation documentation is
updated accordingly.

configure.ac previously used AC_PROG_CC_C99 to activate C99.  But
there is no AC_PROG_CC_C11 in Autoconf 2.69, because it's too
old.  (Also, post-2.69, the AC_PROG_CC_Cnn macros were deprecated and
AC_PROG_CC activates the last supported C mode.)  We could update the
required Autoconf version, but that might be a separate project that
no one wants to undertake at the moment.  Instead, we open-code the
test for C11 using some inspiration from later Autoconf versions.  But
instead of writing an elaborate test program, we keep it simple and
just check __STDC_VERSION__, which should be good enough in practice.

In meson.build, we update the existing C99 test to C11, but again we
just check for __STDC_VERSION__.

This also removes the separate option for the conforming preprocessor
on MSVC, added by commit 8fd9bb1d96, since that is activated
automatically in C11 mode.

Note, we don't use the "official" way to set the C standard in Meson
using the c_std project option, because that is impossible to use
correctly (see <https://github.com/mesonbuild/meson/issues/14717>).

Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/01a69441-af54-4822-891b-ca28e05b215a@eisentraut.org
2025-08-26 11:50:46 +02:00
Peter Eisentraut
99234e9ddc Message wording improvements
Use "row" instead of "tuple" for user-facing information for
logical replication conflicts.
2025-08-25 23:15:24 +02:00
Nathan Bossart
989b2e4d5c Use PqMsg_* macros in applyparallelworker.c.
Oversight in commit f4b54e1ed9.

Author: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/CAEudQAobFsHaLMypA6C96-9YExvF4AcU1xNPoPuNYRVm3mq4dg%40mail.gmail.com
2025-08-25 14:11:01 -05:00
Jacob Champion
4e1e417330 oauth: Add unit tests for multiplexer handling
To better record the internal behaviors of oauth-curl.c, add a unit test
suite for the socket and timer handling code. This is all based on TAP
and driven by our existing Test::More infrastructure.

This commit is a replay of 1443b6c0e, which was reverted due to
buildfarm failures. Compared with that, this version protects the build
targets in the Makefile with a with_libcurl conditional, and it tweaks
the code style in 001_oauth.pl.

Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/CAOYmi+nDZxJHaWj9_jRSyf8uMToCADAmOfJEggsKW-kY7aUwHA@mail.gmail.com
Discussion: https://postgr.es/m/CAOYmi+m=xY0P_uAzAP_884uF-GhQ3wrineGwc9AEnb6fYxVqVQ@mail.gmail.com
2025-08-25 09:27:45 -07:00
Jacob Champion
52ecd05aee oauth: Always link with -lm for floor()
libpq-oauth uses floor() but did not link against libm. Since libpq
itself uses -lm, nothing in the buildfarm has had problems with
libpq-oauth yet, and it seems difficult to hit a failure in practice.

But commit 1443b6c0e attempted to add an executable based on
libpq-oauth, which ran into link-time failures with Clang due to this
omission. It seems prudent to fix this for both the module and the
executable simultaneously so that no one trips over it in the future.

This is a Makefile-only change. The Meson side already pulls in libm,
through the os_deps dependency.

Discussion: https://postgr.es/m/CAOYmi%2Bn6ORcmV10k%2BdAs%2Bp0b9QJ4bfpk0WuHQaF5ODXxM8Y36A%40mail.gmail.com
Backpatch-through: 18
2025-08-25 09:27:39 -07:00
Nathan Bossart
3ef2b863a3 Use PqMsg_* macros in fe-protocol3.c.
Oversight in commit f4b54e1ed9.

Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Fabrízio de Royes Mello <fabriziomello@gmail.com>
Discussion: https://postgr.es/m/aKx5vEbbP03JNgtp%40nathan
2025-08-25 11:08:26 -05:00
Peter Eisentraut
878656dbde Formatting cleanup of guc_tables.c
This cleans up a few minor formatting inconsistencies.

Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/dae6fe89-1e0c-4c3f-8d92-19d23374fb10%40eisentraut.org
2025-08-25 09:10:27 +02:00
Noah Misch
ad4412480d Rewrite previous commit's test for TestUpgradeXversion compatibility.
v17 introduced the MAINTAIN ON TABLES privilege.  That changed the
applicable "baseacls" reaching buildACLCommands().  That yielded
spurious TestUpgradeXversion diffs.  Change to use a TYPES privilege.
Types have the same one privilege in all supported versions, so they
avoid the problem.  Per buildfarm.  Back-patch to v13, like that commit.

Discussion: https://postgr.es/m/20250823144505.88.nmisch@google.com
Backpatch-through: 13
2025-08-23 16:46:20 -07:00
Noah Misch
b61a5c4bed Sort DO_DEFAULT_ACL dump objects independent of OIDs.
Commit 0decd5e89d missed DO_DEFAULT_ACL,
leading to assertion failures, potential dump order instability, and
spurious schema diffs.  Back-patch to v13, like that commit.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://postgr.es/m/d32aaa8d-df7c-4f94-bcb3-4c85f02bea21@gmail.com
Backpatch-through: 13
2025-08-22 20:50:28 -07:00
Alexander Korotkov
c13070a27b Revert "Get rid of WALBufMappingLock"
This reverts commit bc22dc0e0d.
It appears that conditional variables are not suitable for use inside
critical sections.  If WaitLatch()/WaitEventSetWaitBlock() face postmaster
death, they exit, releasing all locks instead of PANIC.  In certain
situations, this leads to data corruption.

Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/B3C69B86-7F82-4111-B97F-0005497BB745%40yandex-team.ru
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Aleksander Alekseev <aleksander@tigerdata.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Yura Sokolov <y.sokolov@postgrespro.ru>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Backpatch-through: 18
2025-08-22 19:26:38 +03:00
Nathan Bossart
b63952a781 vacuumdb: Fix --missing-stats-only with virtual generated columns.
Statistics aren't created for virtual generated columns, so
"vacuumdb --missing-stats-only" always chooses to analyze tables
that have them.  To fix, modify vacuumdb's query for retrieving
relations that are missing statistics to exclude those columns.

Oversight in commit edba754f05.

Author: Yugo Nagata <nagata@sraoss.co.jp>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/20250820104226.8ba51e43164cd590b863ce41%40sraoss.co.jp
Backpatch-through: 18
2025-08-22 11:11:28 -05:00
Heikki Linnakangas
807ee417e5 Revert unnecessary check for NULL
Jelte pointed out that this was unnecessary, but I failed to remove it
before pushing f6f0542266. Oops.

Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://www.postgresql.org/message-id/CAGECzQT%3DxNV-V%2BvFC7YQwYQMj0wGN61b3p%3DJ1_rL6M0vbjTtrA@mail.gmail.com
Backpatch-through: 18
2025-08-22 14:47:19 +03:00
Heikki Linnakangas
e411a8d25a libpq: Be strict about cancel key lengths
The protocol documentation states that the maximum length of a cancel
key is 256 bytes. This starts checking for that limit in libpq.
Otherwise third party backend implementations will probably start
using more bytes anyway. We also start requiring that a protocol 3.0
connection does not send a longer cancel key, to make sure that
servers don't start breaking old 3.0-only clients by accident. Finally
this also restricts the minimum key length to 4 bytes (both in the
protocol spec and in the libpq implementation).

Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Jacob Champion <jchampion@postgresql.org>
Discussion: https://www.postgresql.org/message-id/df892f9f-5923-4046-9d6f-8c48d8980b50@iki.fi
Backpatch-through: 18
2025-08-22 14:39:29 +03:00
Heikki Linnakangas
f6f0542266 libpq: Handle OOM by disconnecting instead of hanging or skipping msgs
In most cases, if an out-of-memory situation happens, we attach the
error message to the connection and report it at the next
PQgetResult() call. However, there are a few cases, while processing
messages that are not associated with any particular query, where we
handled failed allocations differently and not very nicely:

- If we ran out of memory while processing an async notification,
  getNotify() either returned EOF, which stopped processing any
  further data until more data was received from the server, or
  silently dropped the notification. Returning EOF is problematic
  because if more data never arrives, e.g. because the connection was
  used just to wait for the notification, or because the next
  ReadyForQuery was already received and buffered, it would get stuck
  forever. Silently dropping a notification is not nice either.

- (New in v18) If we ran out of memory while receiving BackendKeyData
  message, getBackendKeyData() returned EOF, which has the same issues
  as in getNotify().

- If we ran out of memory while saving a received a ParameterStatus
  message, we just skipped it. A later call to PQparameterStatus()
  would return NULL, even though the server did send the status.

Change all those cases to terminate the connnection instead. Our
options for reporting those errors are limited, but it seems better to
terminate than try to soldier on. Applications should handle
connection loss gracefully, whereas silently missing a notification,
parameter status, or cancellation key could cause much weirder
problems.

This also changes the error message on OOM while expanding the input
buffer. It used to report "cannot allocate memory for input buffer",
followed by "lost synchronization with server: got message type ...".
The "lost synchronization" message seems unnecessary, so remove that
and report only "cannot allocate memory for input buffer". (The
comment speculated that the out of memory could indeed be caused by
loss of sync, but that seems highly unlikely.)

This evolved from a more narrow patch by Jelte Fennema-Nio, which was
reviewed by Jacob Champion.

Somewhat arbitrarily, backpatch to v18 but no further. These are
long-standing issues, but we haven't received any complaints from the
field. We can backpatch more later, if needed.

Co-authored-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Reviewed-by: Jacob Champion <jchampion@postgresql.org>
Discussion: https://www.postgresql.org/message-id/df892f9f-5923-4046-9d6f-8c48d8980b50@iki.fi
Backpatch-through: 18
2025-08-22 14:39:25 +03:00
Heikki Linnakangas
661f821ef0 Use ereport() rather than elog()
Noah pointed this out before I committed 50f770c3d9, but I
accidentally pushed the old version with elog() anyway. Oops.

Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://www.postgresql.org/message-id/20250820003756.31.nmisch@google.com
2025-08-22 13:35:05 +03:00
Heikki Linnakangas
50f770c3d9 Revert GetTransactionSnapshot() to return historic snapshot during LR
Commit 1585ff7387 changed GetTransactionSnapshot() to throw an error
if it's called during logical decoding, instead of returning the
historic snapshot. I made that change for extra protection, because a
historic snapshot can only be used to access catalog tables while
GetTransactionSnapshot() is usually called when you're executing
arbitrary queries. You might get very subtle visibility problems if
you tried to use the historic snapshot for arbitrary queries.

There's no built-in code in PostgreSQL that calls
GetTransactionSnapshot() during logical decoding, but it turns out
that the pglogical extension does just that, to evaluate row filter
expressions. You would get weird results if the row filter runs
arbitrary queries, but it is sane as long as you don't access any
non-catalog tables. Even though there are no checks to enforce that in
pglogical, a typical row filter expression does not access any tables
and works fine. Accessing tables marked with the user_catalog_table =
true option is also OK.

To fix pglogical with row filters, and any other extensions that might
do similar things, revert GetTransactionSnapshot() to return a
historic snapshot during logical decoding.

To try to still catch the unsafe usage of historic snapshots, add
checks in heap_beginscan() and index_beginscan() to complain if you
try to use a historic snapshot to scan a non-catalog table. We're very
close to the version 18 release however, so add those new checks only
in master.

Backpatch-through: 18
Reported-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://www.postgresql.org/message-id/20250809222338.cc.nmisch@google.com
2025-08-22 13:07:46 +03:00
Peter Eisentraut
16a0039dc0 Reduce lock level for ALTER DOMAIN ... VALIDATE CONSTRAINT
Reduce from ShareLock to ShareUpdateExclusivelock.  Validation during
ALTER DOMAIN ... ADD CONSTRAINT keeps using ShareLock.

Example:

    create domain d1 as int;
    create table t (a d1);
    alter domain d1 add constraint cc10 check (value > 10) not valid;

    begin;
    alter domain d1 validate constraint cc10;

    -- another session
    insert into t values (8);

Now we should still be able to perform DML operations on table t while
the domain constraint is being validated.  The equivalent works
already on table constraints.

Author: jian he <jian.universality@gmail.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CACJufxHz92A88NLRTA2msgE2dpXpE-EoZ2QO61od76-6bfqurA%40mail.gmail.com
2025-08-22 08:56:11 +02:00
Amit Kapila
123e65fdb7 Doc: Fix typo in logicaldecoding.sgml.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Backpatch-through: 17, where it was introduced
Discussion: https://postgr.es/m/OSCPR01MB149662EC5467B4135398E3731F532A@OSCPR01MB14966.jpnprd01.prod.outlook.com
2025-08-22 05:29:36 +00:00
Michael Paquier
13b935cd52 Change dynahash.c and hsearch.h to use int64 instead of long
This code was relying on "long", which is signed 8 bytes everywhere
except on Windows where it is 4 bytes, that could potentially expose it
to overflows, even if the current uses in the code are fine as far as I
know.  This code is now able to rely on the same sizeof() variable
everywhere, with int64.  long was used for sizes, partition counts and
entry counts.

Some callers of the dynahash.c routines used long declarations, that can
be cleaned up to use int64 instead.  There was one shortcut based on
SIZEOF_LONG, that can be removed.  long is entirely removed from
dynahash.c and hsearch.h.

Similar work was done in b1e5c9fa9a.

Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/aKQYp-bKTRtRauZ6@paquier.xyz
2025-08-22 11:59:02 +09:00
Michael Paquier
ef03ea01fe Ignore temporary relations in RelidByRelfilenumber()
Temporary relations may share the same RelFileNumber with a permanent
relation, or other temporary relations associated with other sessions.

Being able to uniquely identify a temporary relation would require
RelidByRelfilenumber() to know about the proc number of the temporary
relation it wants to identify, something it is not designed for since
its introduction in f01d1ae3a1.

There are currently three callers of RelidByRelfilenumber():
- autoprewarm.
- Logical decoding, reorder buffer.
- pg_filenode_relation(), that attempts to find a relation OID based on
a tablespace OID and a RelFileNumber.

This makes the situation problematic particularly for the first two
cases, leading to the possibility of random ERRORs due to
inconsistencies that temporary relations can create in the cache
maintained by RelidByRelfilenumber().  The third case should be less of
an issue, as I suspect that there are few direct callers of
pg_filenode_relation().

The window where the ERRORs are happen is very narrow, requiring an OID
wraparound to create a lookup conflict in RelidByRelfilenumber() with a
temporary table reusing the same OID as another relation already cached.
The problem is easier to reach in workloads with a high OID consumption
rate, especially with a higher number of temporary relations created.

We could get pg_filenode_relation() and RelidByRelfilenumber() to work
with temporary relations if provided the means to identify them with an
optional proc number given in input, but the years have also shown that
we do not have a use case for it, yet.  Note that this could not be
backpatched if pg_filenode_relation() needs changes.  It is simpler to
ignore temporary relations.

Reported-by: Shenhao Wang <wangsh.fnst@fujitsu.com>
Author: Vignesh C <vignesh21@gmail.com>
Reviewed-By: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-By: Robert Haas <robertmhaas@gmail.com>
Reviewed-By: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-By: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Reviewed-By: Michael Paquier <michael@paquier.xyz>
Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com>
Reported-By: Shenhao Wang <wangsh.fnst@fujitsu.com>
Discussion: https://postgr.es/m/bbaaf9f9-ebb2-645f-54bb-34d6efc7ac42@fujitsu.com
Backpatch-through: 13
2025-08-22 09:03:59 +09:00
Peter Eisentraut
47932f3cdc Use consistent type for pgaio_io_get_id() result
The result of pgaio_io_get_id() was being assigned to a mix of int and
uint32 variables.  This fixes it to use int consistently, which seems
the most correct.  Also change the queue empty special value in
method_worker.c to -1 from UINT32_MAX.

Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/70c784b3-f60b-4652-b8a6-75e5f051243e%40eisentraut.org
2025-08-21 19:45:25 +02:00
Fujii Masao
12da45742c Disallow server start with sync_replication_slots = on and wal_level < logical.
Replication slot synchronization (sync_replication_slots = on)
requires wal_level to be logical. This commit prevents the server
from starting if sync_replication_slots is enabled but wal_level
is set to minimal or replica.

Failing early during startup helps users catch invalid configurations
immediately, which is important because changing wal_level requires
a server restart.

Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Shveta Malik <shveta.malik@gmail.com>
Discussion: https://postgr.es/m/CAH0PTU_pc3oHi__XESF9ZigCyzai1Mo3LsOdFyQA4aUDkm01RA@mail.gmail.com
2025-08-21 22:18:11 +09:00
Peter Eisentraut
53eff471c6 PL/Python: Add event trigger support
Allow event triggers to be written in PL/Python.  It provides a TD
dictionary with some information about the event trigger.

Author: Euler Taveira <euler@eulerto.com>
Co-authored-by: Dimitri Fontaine <dimitri@2ndQuadrant.fr>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/03f03515-2068-4f5b-b357-8fb540883c38%40app.fastmail.com
2025-08-21 09:21:11 +02:00
Peter Eisentraut
6e09c960eb PL/Python: Refactor for event trigger support
Change is_trigger type from boolean to enum.  That's a preparation for
adding event trigger support.

Author: Euler Taveira <euler@eulerto.com>
Co-authored-by: Dimitri Fontaine <dimitri@2ndQuadrant.fr>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/03f03515-2068-4f5b-b357-8fb540883c38%40app.fastmail.com
2025-08-21 09:16:29 +02:00
Michael Paquier
e8eb98754b Apply some fat commas to commands of TAP tests
This is similar to 19c6e92b13, in order to keep the style used in the
scripts consistent for the option names and values used in commands.
The places updated in this commit have been added recently in
71ea0d6795.

These changes are cosmetic; there is no need for a backpatch.
2025-08-21 14:17:26 +09:00
Michael Paquier
b5d87a823f doc: Improve description of wal_compression
The description of this GUC provides a list of the situations where
full-page writes are generated.  However, it is not completely exact,
mentioning only the cases where full_page_writes=on or base backups.  It
is possible to generate full-page writes in more situations than these
two, making the description confusing as it implies that no other cases
exist.

The description is slightly reworded to take into account that other
cases are possible, without mentioning them directly to minimize the
maintenance burden should FPWs be generated in more contexts in the
future.

Author: Jingtang Zhang <mrdrivingduck@gmail.com>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/CAPsk3_CtAYa_fy4p6=x7qtoutrdKvg1kGk46D5fsE=sMt2546g@mail.gmail.com
Backpatch-through: 13
2025-08-21 13:25:03 +09:00
Tom Lane
a67d4847a4 Fix re-execution of a failed SQLFunctionCache entry.
If we error out during execution of a SQL-language function, we will
often leave behind non-null pointers in its SQLFunctionCache's cplan
and eslist fields.  This is problematic if the SQLFunctionCache is
re-used, because those pointers will point at resources that were
released during error cleanup.  This problem escaped detection so far
because ordinarily we won't re-use an FmgrInfo+SQLFunctionCache struct
after a query error.  However, in the rather improbable case that
someone implements an opclass support function in SQL language, there
will be long-lived FmgrInfos for it in the relcache, and then the
problem is reachable after the function throws an error.

To fix, add a flag to SQLFunctionCache that tracks whether execution
escapes out of fmgr_sql, and clear out the relevant fields during
init_sql_fcache if so.  (This is going to need more thought if we ever
try to share FMgrInfos across threads; but it's very far from being
the only problem such a project will encounter, since many functions
regard fn_extra as being query-local state.)

This broke at commit 0313c5dc6; before that we did not try to re-use
SQLFunctionCache state across calls.  Hence, back-patch to v18.

Bug: #19026
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19026-90aed5e71d0c8af3@postgresql.org
Backpatch-through: 18
2025-08-20 16:09:18 -04:00
Peter Eisentraut
e9c043a11a Minor error message enhancement
In refuseDupeIndexAttach(), change from

    errdetail("Another index is already attached for partition \"%s\"."...)

to

    errdetail("Another index \"%s\" is already attached for partition \"%s\"."...)

so we can easily understand which index is already attached for
partition \"%s\".

Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/CACJufxGBfykJ_1ztk9T%2BL_gLmkOSOF%2BmL9Mn4ZPydz-rh%3DLccQ%40mail.gmail.com
2025-08-20 18:14:24 +02:00
Michael Paquier
1f2e51e3c7 Fix assertion failure with replication slot release in single-user mode
Some replication slot manipulations (logical decoding via SQL,
advancing) were failing an assertion when releasing a slot in
single-user mode, because active_pid was not set in a ReplicationSlot
when its slot is acquired.

ReplicationSlotAcquire() has some logic to be able to work with the
single-user mode.  This commit sets ReplicationSlot->active_pid to
MyProcPid, to let the slot-related logic fall-through, considering the
single process as the one holding the slot.

Some TAP tests are added for various replication slot functions with the
single-user mode, while on it, for slot creation, drop, advancing, copy
and logical decoding with multiple slot types (temporary, physical vs
logical).  These tests are skipped on Windows, as direct calls of
postgres --single would fail on permission failures.  There is no
platform-specific behavior that needs to be checked, so living with this
restriction should be fine.  The CI is OK with that, now let's see what
the buildfarm tells.

Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Mutaamba Maasha <maasha@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB14966ED588A0328DAEBE8CB25F5FA2@OSCPR01MB14966.jpnprd01.prod.outlook.com
Backpatch-through: 13
2025-08-20 15:00:04 +09:00
Fujii Masao
6429e5b771 vacuumdb: Make vacuumdb --analyze-only process partitioned tables.
vacuumdb should follow the behavior of the underlying VACUUM and ANALYZE
commands. When --analyze-only is used, it ought to analyze regular tables,
materialized views, and partitioned tables, just as ANALYZE (with no explicit
target tables) does. Otherwise, it should only process regular tables and
materialized views, since VACUUM skips partitioned tables when no targets
are given.

Previously, vacuumdb --analyze-only skipped partitioned tables. This was
inconsistent, and also inconvenient after pg_upgrade, where --analyze-only
is typically used to gather missing statistics.

This commit fixes the behavior so that vacuumdb --analyze-only also processes
partitioned tables. As a result, both vacuumdb --analyze-only and
ANALYZE (with no explicit targets) now analyze regular tables,
partitioned tables, and materialized views, but not foreign tables.

Because this is a nontrivial behavior change, it is applied only to master.

Reported-by: Zechman, Derek S <Derek.S.Zechman@snapon.com>
Author: Laurenz Albe <laurenz.albe@cybertec.at>
Co-authored-by: Mircea Cadariu <cadariu.mircea@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CO1PR04MB8281387B9AD9DE30976966BBC045A%40CO1PR04MB8281.namprd04.prod.outlook.com
2025-08-20 13:16:06 +09:00
Nathan Bossart
3eec0e6533 Fix comment for MAX_SIMUL_LWLOCKS.
This comment mentions that pg_buffercache locks all buffer
partitions simultaneously, but it hasn't done so since v10.

Oversight in commit 6e654546fb.

Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/aKTuAHVEuYCUmmIy%40nathan
2025-08-19 16:48:22 -05:00
Masahiko Sawada
eab9e4e27c Add CHECK_FOR_INTERRUPTS in contrib/pg_buffercache functions.
This commit adds CHECK_FOR_INTERRUPTS to loops iterating over shared
buffers in several pg_buffercache functions, allowing them to be
interrupted during long-running operations.

Backpatch to all supported versions. Add CHECK_FOR_INTERRUPTS to the
loop in pg_buffercache_pages() in all supported branches, and to
pg_buffercache_summary() and pg_buffercache_usage_counts() in version
16 and newer.

Author: SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>
Discussion: https://postgr.es/m/CAHg+QDcejeLx7WunFT3DX6XKh1KshvGKa8F5au8xVhqVvvQPRw@mail.gmail.com
Backpatch-through: 13
2025-08-19 12:11:42 -07:00
Nathan Bossart
c6abf24ebf Fix misspelling of "tranche" in dsa.h.
Oversight in commit bb952c8c8b.

Discussion: https://postgr.es/m/aKOWzsCPgrsoEG1Q%40nathan
2025-08-19 10:43:15 -05:00
Fujii Masao
38c5fbd97e doc: Improve pgoutput documentation.
This commit updates the pgoutput documentation with the following changes:

- Specify the data type for each pgoutput option.
- Clarify the relationship between proto_version and options such as
   streaming and two_phase.
- Add a note on the use of pg_logical_slot_peek_changes and
   pg_logical_slot_get_changes with pgoutput.

Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/CAHGQGwFJTbygdhhjR_iP4Oem=Lo1xsptWWOq825uoW+hG_Lfnw@mail.gmail.com
2025-08-19 18:54:27 +09:00
Fujii Masao
34a62c2c7f doc: Improve documentation discoverability for pgoutput.
Previously, the documentation for pgoutput was located in the section on
the logical streaming replication protocol, and there was no index entry
for it. As a result, users had difficulty finding information about pgoutput.

This commit moves the pgoutput documentation under the logical decoding
section and adds an index entry, making it easier for users to locate and
access this information.

Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Discussion: https://postgr.es/m/CAHGQGwFJTbygdhhjR_iP4Oem=Lo1xsptWWOq825uoW+hG_Lfnw@mail.gmail.com
2025-08-19 18:53:56 +09:00
Peter Eisentraut
16d434d53d Add src/include/catalog/README
This just includes a link to the bki documentation, to help people get
started.

Before commit 372728b0d4, there was a README at
src/backend/catalog/README, but then this was moved to the SGML
documentation.  So this effectively puts back a link to what was
moved.  But src/include/catalog/ is probably a better location,
because that's where all the interesting files are.

Co-authored-by: Florents Tselai <florents.tselai@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+v5N400GJFJ9RyXAX7hFKbtF7vVQGvWdFWEfcSQmvVhi9xfrA@mail.gmail.com
2025-08-19 08:41:42 +02:00
Amit Kapila
aa21e49225 Fix self-deadlock during DROP SUBSCRIPTION.
The DROP SUBSCRIPTION command performs several operations: it stops the
subscription workers, removes subscription-related entries from system
catalogs, and deletes the replication slot on the publisher server.
Previously, this command acquired an AccessExclusiveLock on
pg_subscription before initiating these steps.

However, while holding this lock, the command attempts to connect to the
publisher to remove the replication slot. In cases where the connection is
made to a newly created database on the same server as subscriber, the
cache-building process during connection tries to acquire an
AccessShareLock on pg_subscription, resulting in a self-deadlock.

To resolve this issue, we reduce the lock level on pg_subscription during
DROP SUBSCRIPTION from AccessExclusiveLock to RowExclusiveLock. Earlier,
the higher lock level was used to prevent the launcher from starting a new
worker during the drop operation, as a restarted worker could become
orphaned.

Now, instead of relying on a strict lock, we acquire an AccessShareLock on
the specific subscription being dropped and re-validate its existence
after acquiring the lock. If the subscription is no longer valid, the
worker exits gracefully. This approach avoids the deadlock while still
ensuring that orphan workers are not created.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 13
Discussion: https://postgr.es/m/18988-7312c868be2d467f@postgresql.org
2025-08-19 05:33:17 +00:00