1
0
mirror of https://github.com/postgres/postgres.git synced 2025-11-07 19:06:32 +03:00
Commit Graph

62499 Commits

Author SHA1 Message Date
Richard Guo
c1777f2d6d Fix assertion failure in generate_orderedappend_paths()
In generate_orderedappend_paths(), there is an assumption that a child
relation's row estimate is always greater than zero.  There is an
Assert verifying this assumption, and the estimate is also used to
convert an absolute tuple count into a fraction.

However, this assumption is not always valid -- for example, upper
relations can have their row estimates unset, resulting in a value of
zero.  This can cause an assertion failure in debug builds or lead to
the tuple fraction being computed as infinity in production builds.

To fix, use the row estimate from the cheapest_total path to compute
the tuple fraction.  The row estimate in this path should already have
been forced to a valid value.

In passing, update the comment for generate_orderedappend_paths() to
note that the function also considers the cheapest-fractional case
when not all tuples need to be retrieved.  That is, it collects all
the cheapest fractional paths and builds an ordered append path for
each interesting ordering.

Backpatch to v18, where this issue was introduced.

Bug: #19102
Reported-by: Kuntal Ghosh <kuntalghosh.2007@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Kuntal Ghosh <kuntalghosh.2007@gmail.com>
Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>
Discussion: https://postgr.es/m/19102-93480667e1200169@postgresql.org
Backpatch-through: 18
2025-11-05 18:09:21 +09:00
Michael Paquier
a4fd971c6f Fix timing-dependent failure in recovery test 004_timeline_switch
The test introduced by 17b2d5ec75 verifies that a WAL receiver
survives across a timeline jump by searching the server logs for
termination messages.  However, it called restart() before the timeline
switch, which kills the WAL receiver and may log the exact message being
checked, hence failing the test.  As TAP tests reuse the same log file
across restarts, a rotate_logfile() is used before the restart so as the
log matching check is not impacted by log entries generated by a
previous shutdown.

Recent changes to file handle inheritance altered I/O timing enough to
make this fail consistently while testing another patch.

While on it, this adds an extra check based on a PID comparison.  This
test may lead to false positives as it could be possible that the WAL
receiver has processed a timeline jump before the initial PID is
grabbed, but it should be good enough in most cases.

Like 17b2d5ec75, backpatch down to v13.

Author: Bryan Green <dbryan.green@gmail.com>
Co-authored-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/9d00b597-d64a-4f1e-802e-90f9dc394c70@gmail.com
Backpatch-through: 13
2025-11-05 16:48:19 +09:00
Amit Kapila
5509055d69 Add sequence synchronization for logical replication.
This patch introduces sequence synchronization. Sequences that are synced
will have 2 states:
   - INIT (needs [re]synchronizing)
   - READY (is already synchronized)

A new sequencesync worker is launched as needed to synchronize sequences.
A single sequencesync worker is responsible for synchronizing all
sequences. It begins by retrieving the list of sequences that are flagged
for synchronization, i.e., those in the INIT state. These sequences are
then processed in batches, allowing multiple entries to be synchronized
within a single transaction. The worker fetches the current sequence
values and page LSNs from the remote publisher, updates the corresponding
sequences on the local subscriber, and finally marks each sequence as
READY upon successful synchronization.

Sequence synchronization occurs in 3 places:
1) CREATE SUBSCRIPTION
    - The command syntax remains unchanged.
    - The subscriber retrieves sequences associated with publications.
    - Published sequences are added to pg_subscription_rel with INIT
      state.
    - Initiate the sequencesync worker to synchronize all sequences.

2) ALTER SUBSCRIPTION ... REFRESH PUBLICATION
    - The command syntax remains unchanged.
    - Dropped published sequences are removed from pg_subscription_rel.
    - Newly published sequences are added to pg_subscription_rel with INIT
      state.
    - Initiate the sequencesync worker to synchronize only newly added
      sequences.

3) ALTER SUBSCRIPTION ... REFRESH SEQUENCES
    - A new command introduced for PG19 by f0b3573c3a.
    - All sequences in pg_subscription_rel are reset to INIT state.
    - Initiate the sequencesync worker to synchronize all sequences.
    - Unlike "ALTER SUBSCRIPTION ... REFRESH PUBLICATION" command,
      addition and removal of missing sequences will not be done in this
      case.

Author: Vignesh C <vignesh21@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/CAA4eK1LC+KJiAkSrpE_NwvNdidw9F2os7GERUeSxSKv71gXysQ@mail.gmail.com
2025-11-05 05:59:58 +00:00
Michael Paquier
1fd981f053 Drop unnamed portal immediately after execution to completion
Previously, unnamed portals were kept until the next Bind message or the
end of the transaction.  This could cause temporary files to persist
longer than expected and make logging not reflect the actual SQL
responsible for the temporary file.

This patch changes exec_execute_message() to drop unnamed portals
immediately after execution to completion at the end of an Execute
message, making their removal more aggressive.  This forces temporary
file cleanups to happen at the same time as the completion of the portal
execution, with statement logging correctly reflecting to which
statements these temporary files were attached to (see the diffs in the
TAP test updated by this commit for an idea).

The documentation is updated to describe the lifetime of unnamed
portals, and test cases are updated to verify temporary file removal and
proper statement logging after unnamed portal execution.  This changes
how unnamed portals are handled in the protocol, hence no backpatch is
done.

Author: Frédéric Yhuel <frederic.yhuel@dalibo.com>
Co-Authored-by: Sami Imseih <samimseih@gmail.com>
Co-Authored-by: Mircea Cadariu <cadariu.mircea@gmail.com>
Discussion: https://postgr.es/m/CAA5RZ0tTrTUoEr3kDXCuKsvqYGq8OOHiBwoD-dyJocq95uEOTQ%40mail.gmail.com
2025-11-05 14:35:16 +09:00
Richard Guo
59dec6c0b0 Fix comments for ChangeVarNodes() and related functions
The comment for ChangeVarNodes() refers to a parameter named
change_RangeTblRef, which does not exist in the code.

The comment for ChangeVarNodesExtended() contains an extra space,
while the comment for replace_relid_callback() has an awkward line
break and a typo.

This patch fixes these issues and revises some sentences for smoother
wording.

Oversights in commits ab42d643c and fc069a3a6.

Author: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CAMbWs480j16HC1JtjKCgj5WshivT8ZJYkOfTyZAM0POjFomJkg@mail.gmail.com
Backpatch-through: 18
2025-11-05 12:29:31 +09:00
Michael Paquier
2fc3107962 Add assertions checking for the startup process in WAL replay routines
These assertions may prove to become useful to make sure that no process
other than the startup process calls the routines where these checks are
added, as we expect that these do not interfere with a WAL receiver
switched to a "stopping" state by a startup process.

The assumption that only the startup process can use this code has
existed for many years, without a check enforcing it.

Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/aQmGeVLYl51y1m_0@paquier.xyz
2025-11-05 10:41:50 +09:00
Andres Freund
dae00f333b aio: Improve assertions related to io_method
First, the assertions in assign_io_method() were the wrong way round. Second,
the lengthof() assertion checked the length of io_method_options, which is the
wrong array to check and is always longer than pgaio_method_ops_table.

While add it, add a static assert to ensure pgaio_method_ops_table and
io_method_options stay in sync.

Per coverity and Tom Lane.

Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 18
2025-11-04 20:03:53 -05:00
Andres Freund
2d83d729d5 jit: Fix accidentally-harmless type confusion
In 2a0faed9d7, which added JIT compilation support for expressions, I
accidentally used sizeof(LLVMBasicBlockRef *) instead of
sizeof(LLVMBasicBlockRef) as part of computing the size of an allocation. That
turns out to have no real negative consequences due to LLVMBasicBlockRef being
a pointer itself (and thus having the same size). It still is wrong and
confusing, so fix it.

Reported by coverity.

Backpatch-through: 13
2025-11-04 20:03:53 -05:00
Jeff Davis
d115de9d89 Special case C_COLLATION_OID in pg_newlocale_from_collation().
Allow pg_newlocale_from_collation(C_COLLATION_OID) to work even if
there's no catalog access, which some extensions expect.

Not known to be a bug without extensions involved, but backport to 18.

Also corrects an issue in master with dummy_c_locale (introduced in
commit 5a38104b36) where deterministic was not set. That wasn't a bug,
but could have been if that structure was used more widely.

Reported-by: Alexander Kukushkin <cyberdemn@gmail.com>
Reviewed-by: Alexander Kukushkin <cyberdemn@gmail.com>
Discussion: https://postgr.es/m/CAFh8B=nj966ECv5vi_u3RYij12v0j-7NPZCXLYzNwOQp9AcPWQ@mail.gmail.com
Backpatch-through: 18
2025-11-04 16:48:16 -08:00
Masahiko Sawada
8ae0f6a0c3 Add CHECK_FOR_INTERRUPTS in Evict{Rel,All}UnpinnedBuffers.
This commit adds CHECK_FOR_INTERRUPTS to the shared buffer iteration
loops in EvictRelUnpinnedBuffers and EvictAllUnpinnedBuffers. These
functions, used by pg_buffercache's pg_buffercache_evict_relation and
pg_buffercache_evict_all, can now be interrupted during long-running
operations.

Backpatch to version 18, where these functions and their corresponding
pg_buffercache functions were introduced.

Author: Yuhang Qiu <iamqyh@gmail.com>
Discussion: https://postgr.es/m/8DC280D4-94A2-4E7B-BAB9-C345891D0B78%40gmail.com
Backpatch-through: 18
2025-11-04 15:47:25 -08:00
David Rowley
fdda78e361 Fix possible usage of incorrect UPPERREL_SETOP RelOptInfo
03d40e4b5 allowed dummy UNION [ALL] children to be removed from the plan
by checking for is_dummy_rel().  That commit neglected to still account
for the relids from the dummy rel so that the correct UPPERREL_SETOP
RelOptInfo could be found and used for adding the Paths to.

Not doing this could result in processing of subsequent UNIONs using the
same RelOptInfo as a previously processed UNION, which could result in
add_path() freeing old Paths that are needed by the previous UNION.

The same fix was independently submitted (2 mins later) by Richard Guo.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/bee34aec-659c-46f1-9ab7-7bbae0b7616c@gmail.com
2025-11-05 11:48:09 +13:00
Álvaro Herrera
0a3d27bfe0 Fix snapshot handling bug in recent BRIN fix
Commit a95e3d84c0 added ActiveSnapshot push+pop when processing
work-items (BRIN autosummarization), but forgot to handle the case of
a transaction failing during the run, which drops the snapshot untimely.
Fix by making the pop conditional on an element being actually there.

Author: Álvaro Herrera <alvherre@kurilemu.de>
Backpatch-through: 13
Discussion: https://postgr.es/m/202511041648.nofajnuddmwk@alvherre.pgsql
2025-11-04 20:31:43 +01:00
Tomas Vondra
1213cb4753 Trim TIDs during parallel GIN builds more eagerly
The parallel GIN builds perform "freezing" of TID lists when merging
chunks built earlier. This means determining what part of the list can
no longer change, depending on the last received chunk. The frozen part
can be evicted from memory and written out.

The code attempted to freeze items right before merging the old and new
TID list, after already attempting to trim the current buffer. That
means part of the data may get frozen based on the new TID list, but
will be trimmed later (on next loop). This increases memory usage.

This inverts the order, so that we freeze data first (before trimming).
The benefits are likely relatively small, but it's also virtually free
with no other downsides.

Discussion: https://postgr.es/m/CAHLJuCWDwn-PE2BMZE4Kux7x5wWt_6RoWtA0mUQffEDLeZ6sfA@mail.gmail.com
2025-11-04 20:06:01 +01:00
Masahiko Sawada
6d2ff1de4d psql: Add tab completion for COPY ... PROGRAM.
This commit adds tab completion support for COPY TO PROGRAM and COPY
FROM PROGRAM syntax in psql.

Author: Yugo Nagata <nagata@sraoss.co.jp>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/20250605100835.b396f9d656df1018f65a4556@sraoss.co.jp
2025-11-04 10:51:39 -08:00
Masahiko Sawada
02fd47dbfa psql: Improve tab completion for COPY ... STDIN/STDOUT.
This commit enhances tab completion for both COPY FROM and COPY TO
commands to suggest STDIN and STDOUT, respectively.

To make suggesting both file names and keywords easier, it introduces
a new COMPLETE_WITH_FILES_PLUS() macro.

Author: Yugo Nagata <nagata@sraoss.co.jp>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/20250605100835.b396f9d656df1018f65a4556@sraoss.co.jp
2025-11-04 10:40:58 -08:00
Andres Freund
be9efd4929 ci: debian: Switch to Debian Trixie release
Debian Trixie CI images are generated now [1], so use them with the
following changes:

- detect_stack_use_after_return=0 option is added to the ASAN_OPTIONS
  because ASAN uses a "shadow stack" to track stack variable lifetimes
  and this confuses Postgres' stack depth check [2].

- Perl is updated to the newer version (perl5.40-i386-linux-gnu).

- LLVM-14 is no longer default installation, no need to force using
  LLVM-16.

- Switch MinGW CC/CXX to x86_64-w64-mingw32ucrt-* to fix build failure
  from missing _iswctype_l in mingw-w64 v12 headers.

[1] https://github.com/anarazel/pg-vm-images/commit/35a144793f
[2] https://postgr.es/m/20240130212304.q66rquj5es4375ab%40awork3.anarazel.de

Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ1_B1usTskAv+AYt1bA7abVd9YH6XrUUSbr-2Z0d5Wd8w@mail.gmail.com
Backpatch: 15-, where CI support was added
2025-11-04 13:25:22 -05:00
Tomas Vondra
c98dffcb7c Limit the size of TID lists during parallel GIN build
When building intermediate TID lists during parallel GIN builds, split
the sorted lists into smaller chunks, to limit the amount of memory
needed when merging the chunks later.

The leader may need to keep in memory up to one chunk per worker, and
possibly one extra chunk (before evicting some of the data). The code
processing item pointers uses regular palloc/repalloc calls, which means
it's subject to the MaxAllocSize (1GB) limit.

We could fix this by allowing huge allocations, but that'd require
changes in many places without much benefit. Larger chunks do not
actually improve performance, so the memory usage would be wasted.

Fixed by limiting the chunk size to not hit MaxAllocSize. Each worker
gets a fair share.

This requires remembering the number of participating workers, in a
place that can be accessed from the callback. Luckily, the bs_worker_id
field in GinBuildState was unused, so repurpose that.

Report by Greg Smith, investigation and fix by me. Batchpatched to 18,
where parallel GIN builds were introduced.

Reported-by: Gregory Smith <gregsmithpgsql@gmail.com>
Discussion: https://postgr.es/m/CAHLJuCWDwn-PE2BMZE4Kux7x5wWt_6RoWtA0mUQffEDLeZ6sfA@mail.gmail.com
Backpatch-through: 18
2025-11-04 18:51:17 +01:00
Jeff Davis
4bfaea11d2 Remove redundant memset() introduced by a0942f4.
Reported-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/CAEoWx2kAkNaDa01O0nKsQmkfEmxsDvm09SU=f1T0CV8ew3qJEA@mail.gmail.com
2025-11-04 09:46:00 -08:00
Tom Lane
ff4597acd4 Allow "SET list_guc TO NULL" to specify setting the GUC to empty.
We have never had a SET syntax that allows setting a GUC_LIST_INPUT
parameter to be an empty list.  A locution such as
	SET search_path = '';
doesn't mean that; it means setting the GUC to contain a single item
that is an empty string.  (For search_path the net effect is much the
same, because search_path ignores invalid schema names and '' must be
invalid.)  This is confusing, not least because configuration-file
entries and the set_config() function can easily produce empty-list
values.

We considered making the empty-string syntax do this, but that would
foreclose ever allowing empty-string items to be valid in list GUCs.
While there isn't any obvious use-case for that today, it feels like
the kind of restriction that might hurt someday.  Instead, let's
accept the forbidden-up-to-now value NULL and treat that as meaning an
empty list.  (An objection to this could be "what if we someday want
to allow NULL as a GUC value?".  That seems unlikely though, and even
if we did allow it for scalar GUCs, we could continue to treat it as
meaning an empty list for list GUCs.)

Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrei Klychkov <andrew.a.klychkov@gmail.com>
Reviewed-by: Jim Jones <jim.jones@uni-muenster.de>
Discussion: https://postgr.es/m/CA+mfrmwsBmYsJayWjc8bJmicxc3phZcHHY=yW5aYe=P-1d_4bg@mail.gmail.com
2025-11-04 12:37:40 -05:00
Álvaro Herrera
93b7ab5b4b Have psql's "\? variables" show csv_fieldsep
Accidental omission in commit aa2ba50c2c.  There are too many lists of
these variables ...

Discussion: https://postgr.es/m/202511031738.eqaeaedpx5cr@alvherre.pgsql
2025-11-04 17:30:44 +01:00
Peter Eisentraut
040cc5f3c7 Tighten check for generated column in partition key expression
A generated column may end up being part of the partition key
expression, if it's specified as an expression e.g. "(<generated
column name>)" or if the partition key expression contains a whole-row
reference, even though we do not allow a generated column to be part
of partition key expression.  Fix this hole.

Co-authored-by: jian he <jian.universality@gmail.com>
Co-authored-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Discussion: https://www.postgresql.org/message-id/flat/CACJufxF%3DWDGthXSAQr9thYUsfx_1_t9E6N8tE3B8EqXcVoVfQw%40mail.gmail.com
2025-11-04 14:46:58 +01:00
Álvaro Herrera
a95e3d84c0 BRIN autosummarization may need a snapshot
It's possible to define BRIN indexes on functions that require a
snapshot to run, but the autosummarization feature introduced by commit
7526e10224 fails to provide one.  This causes autovacuum to leave a
BRIN placeholder tuple behind after a failed work-item execution, making
such indexes less efficient.  Repair by obtaining a snapshot prior to
running the task, and add a test to verify this behavior.

Author: Álvaro Herrera <alvherre@kurilemu.de>
Reported-by: Giovanni Fabris <giovanni.fabris@icon.it>
Reported-by: Arthur Nascimento <tureba@gmail.com>
Backpatch-through: 13
Discussion: https://postgr.es/m/202511031106.h4fwyuyui6fz@alvherre.pgsql
2025-11-04 13:23:26 +01:00
Peter Eisentraut
c09a06918d Error message stylistic correction
Fixup for commit ef5e60a9d3: The inconsistent use of articles was a
bit awkward.
2025-11-04 12:25:04 +01:00
Michael Paquier
861af92610 libpq: Improve error handling in passwordFromFile()
Previously, passwordFromFile() returned NULL for valid cases (like no
matching password found) and actual errors (two out-of-memory paths).
This made it impossible for its sole caller, pqConnectOptions2(), to
distinguish between these scenarios and fail the connection
appropriately should an out-of-memory error occur.

This patch extends passwordFromFile() to be able to detect both valid
and failure cases, with an error string given back to the caller of the
function.

Out-of-memory failures unlikely happen in the field, so no backpatch is
done.

Author: Joshua Shanks <jjshanks@gmail.com>
Discussion: https://postgr.es/m/CAOxqWDfihFRmhNVdfu8epYTXQRxkCHSOrg+=-ij2c_X3gW=o3g@mail.gmail.com
2025-11-04 20:12:48 +09:00
Álvaro Herrera
ad1581d7fe Use USECS_PER_SEC from datatype/timestamp.h
We had two places defining their own constants for this.

Author: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/202510311750.mxiykx3tp4mx@alvherre.pgsql
2025-11-04 10:07:54 +01:00
Michael Paquier
65f4976189 Add assertion check for WAL receiver state during stream-archive transition
When the startup process switches from streaming to archive as WAL
source, we avoid calling ShutdownWalRcv() if the WAL receiver is not
streaming, based on WalRcvStreaming().  WALRCV_STOPPING is a state set
by ShutdownWalRcv(), called only by the startup process, meaning that it
should not be possible to reach this state while in
WaitForWALToBecomeAvailable().

This commit adds an assertion to make sure that a WAL receiver is never
in a WALRCV_STOPPING state should the startup process attempt to reset
InstallXLogFileSegmentActive.

Idea suggested by Noah Misch.

Author: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/19093-c4fff49a608f82a0@postgresql.org
2025-11-04 13:14:46 +09:00
Michael Paquier
e0ca61e7c4 Add WalRcvGetState() to retrieve the state of a WAL receiver
This has come up as useful as an alternative of WalRcvStreaming(), to be
able to do sanity checks based on the state of a WAL receiver.  This
will be used in a follow-up commit.

Author: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/19093-c4fff49a608f82a0@postgresql.org
2025-11-04 12:57:36 +09:00
Michael Paquier
17b2d5ec75 Fix unconditional WAL receiver shutdown during stream-archive transition
Commit b4f584f9d2 (affecting v15~, later backpatched down to 13 as of
3635a0a35a) introduced an unconditional WAL receiver shutdown when
switching from streaming to archive WAL sources.  This causes problems
during a timeline switch, when a WAL receiver enters WALRCV_WAITING
state but remains alive, waiting for instructions.

The unconditional shutdown can break some monitoring scenarios as the
WAL receiver gets repeatedly terminated and re-spawned, causing
pg_stat_wal_receiver.status to show a "streaming" instead of "waiting"
status, masking the fact that the WAL receiver is waiting for a new TLI
and a new LSN to be able to continue streaming.

This commit changes the WAL receiver behavior so as the shutdown becomes
conditional, with InstallXLogFileSegmentActive being always reset to
prevent the regression fixed by b4f584f9d2: only terminate the WAL
receiver when it is actively streaming (WALRCV_STREAMING,
WALRCV_STARTING, or WALRCV_RESTARTING).  When in WALRCV_WAITING state,
just reset InstallXLogFileSegmentActive flag to allow archive
restoration without killing the process.  WALRCV_STOPPED and
WALRCV_STOPPING are not reachable states in this code path.  For the
latter, the startup process is the one in charge of setting
WALRCV_STOPPING via ShutdownWalRcv(), waiting for the WAL receiver to
reach a WALRCV_STOPPED state after switching walRcvState, so
WaitForWALToBecomeAvailable() cannot be reached while a WAL receiver is
in a WALRCV_STOPPING state.

A regression test is added to check that a WAL receiver is not stopped
on timeline jump, that fails when the fix of this commit is reverted.

Reported-by: Ryan Bird <ryanzxg@gmail.com>
Author: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/19093-c4fff49a608f82a0@postgresql.org
Backpatch-through: 13
2025-11-04 10:47:38 +09:00
Noah Misch
8b18ed6dfb Doc: cover index CONCURRENTLY causing errors in INSERT ... ON CONFLICT.
Author: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CANtu0ojXmqjmEzp-=aJSxjsdE76iAsRgHBoK0QtYHimb_mEfsg@mail.gmail.com
Backpatch-through: 13
2025-11-03 12:57:09 -08:00
Masahiko Sawada
e7ccb247b3 Fix outdated comment of COPY in gram.y.
Author: ChangAo Chen <cca5507@qq.com>
Discussion: https://postgr.es/m/tencent_392C0E92EC52432D0A336B9D52E66426F009@qq.com
2025-11-03 10:34:49 -08:00
Álvaro Herrera
645cb44c54 Add \pset options for boolean value display
New \pset variables display_true and display_false allow the user to
change how true and false values are displayed.

Author: David G. Johnston <David.G.Johnston@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CAKFQuwYts3vnfQ5AoKhEaKMTNMfJ443MW2kFswKwzn7fiofkrw@mail.gmail.com
Discussion: https://postgr.es/m/56308F56.8060908@joh.to
2025-11-03 17:40:39 +01:00
Álvaro Herrera
cf8be02253 Prevent setting a column as identity if its not-null constraint is invalid
We don't allow null values to appear in identity-generated columns in
other ways, so we shouldn't let unvalidated not-null constraints do it
either.  Oversight in commit a379061a22.

Author: jian he <jian.universality@gmail.com>
Backpatch-through: 18
Discussion: https://postgr.es/m/CACJufxGQM_+vZoYJMaRoZfNyV=L2jxosjv_0TLAScbuLJXWRfQ@mail.gmail.com
2025-11-03 15:58:19 +01:00
Álvaro Herrera
f242dbcede Remove WaitPMResult enum in pg_createsubscriber
A simple boolean suffices.  This is cosmetic, so no backpatch.

Author: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/202510311750.mxiykx3tp4mx@alvherre.pgsql
2025-11-03 12:59:32 +01:00
Michael Paquier
ad25744f43 Add wal_fpi_bytes to VACUUM and ANALYZE logs
The new wal_fpi_bytes counter calculates the total amount of full page
images inserted in WAL records, in bytes.  This commit adds this
information to VACUUM and ANALYZE logs alongside the existing counters,
building upon f9a09aa295.

Author: Shinya Kato <shinya11.kato@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/aQMMSSlFXy4Evxn3@paquier.xyz
2025-11-03 19:42:03 +09:00
Peter Eisentraut
fce7c73fba Sort guc_parameters.dat alphabetically by name
The order in this list was previously pretty random and had grown
organically over time.  This made it unnecessarily cumbersome to
maintain these lists, as there was no clear guidelines about where to
put new entries.  Also, after the merger of the type-specific GUC
structs, the list still reflected the previous type-specific
super-order.

By using alphabetical order, the place for new entries becomes clear,
and often related entries will be listed close together.

This patch reorders the existing entries in guc_parameters.dat, and it
also augments the generation script to error if an entry is found at
the wrong place.

Note: The order is actually checked after lower-casing, to handle the
likes of "DateStyle".

Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/8fdfb91e-60fb-44fa-8df6-f5dea47353c9@eisentraut.org
2025-11-03 10:04:14 +01:00
Tom Lane
8f29467c57 Change "long" numGroups fields to be Cardinality (i.e., double).
We've been nibbling away at removing uses of "long" for a long time,
since its width is platform-dependent.  Here's one more: change the
remaining "long" fields in Plan nodes to Cardinality, since the three
surviving examples all represent group-count estimates.  The upstream
planner code was converted to Cardinality some time ago; for example
the corresponding fields in Path nodes are type Cardinality, as are
the arguments of the make_foo_path functions.  Downstream in the
executor, it turns out that these all feed to the table-size argument
of BuildTupleHashTable.  Change that to "double" as well, and fix it
so that it safely clamps out-of-range values to the uint32 limit of
simplehash.h, as was not being done before.

Essentially, this is removing all the artificial datatype-dependent
limitations on these values from upstream processing, and applying
just one clamp at the moment where we're forced to do so by the
datatype choices of simplehash.h.

Also, remove BuildTupleHashTable's misguided attempt to enforce
work_mem/hash_mem_limit.  It doesn't have enough information
(particularly not the expected tuple width) to do that accurately,
and it has no real business second-guessing the caller's choice.
For all these plan types, it's really the planner's responsibility
to not choose a hashed implementation if the hashtable is expected
to exceed hash_mem_limit.  The previous patch improved the
accuracy of those estimates, and even if BuildTupleHashTable had
more information it should arrive at the same conclusions.

Reported-by: Jeff Janes <jeff.janes@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAMkU=1zia0JfW_QR8L5xA2vpa0oqVuiapm78h=WpNsHH13_9uw@mail.gmail.com
2025-11-02 16:57:43 -05:00
Tom Lane
1ea5bdb00b Improve planner's estimates of tuple hash table sizes.
For several types of plan nodes that use TupleHashTables, the
planner estimated the expected size of the table as basically
numEntries * (MAXALIGN(dataWidth) + MAXALIGN(SizeofHeapTupleHeader)).
This is pretty far off, especially for small data widths, because
it doesn't account for the overhead of the simplehash.h hash table
nor for any per-tuple "additional space" the plan node may request.
Jeff Janes noted a case where the estimate was off by about a factor
of three, even though the obvious hazards such as inaccurate estimates
of numEntries or dataWidth didn't apply.

To improve matters, create functions provided by the relevant executor
modules that can estimate the required sizes with reasonable accuracy.
(We're still not accounting for effects like allocator padding, but
this at least gets the first-order effects correct.)

I added functions that can estimate the tuple table sizes for
nodeSetOp and nodeSubplan; these rely on an estimator for
TupleHashTables in general, and that in turn relies on one for
simplehash.h hash tables.  That feels like kind of a lot of mechanism,
but if we take any short-cuts we're violating modularity boundaries.

The other places that use TupleHashTables are nodeAgg, which took
pains to get its numbers right already, and nodeRecursiveunion.
I did not try to improve the situation for nodeRecursiveunion because
there's nothing to improve: we are not making an estimate of the hash
table size, and it wouldn't help us to do so because we have no
non-hashed alternative implementation.  On top of that, our estimate
of the number of entries to be hashed in that module is so suspect
that we'd likely often choose the wrong implementation if we did have
two ways to do it.

Reported-by: Jeff Janes <jeff.janes@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAMkU=1zia0JfW_QR8L5xA2vpa0oqVuiapm78h=WpNsHH13_9uw@mail.gmail.com
2025-11-02 16:57:26 -05:00
Peter Geoghegan
b8f1c62807 Document nbtree row comparison design.
Add comments explaining when and where it is safe for nbtree to treat
row compare keys as if they were simple scalar inequality keys on the
row's most significant column.  This is particularly important within
_bt_advance_array_keys, which deals with required inequality keys in a
general and uniform way, without any special handling for row compares.

Also spell out the implications of _bt_check_rowcompare's approach of
_conditionally_ evaluating lower-order row compare subkeys, particularly
when one of its lower-order subkeys might see NULL index tuple values
(these may or may not affect whether the qual as a whole is satisfied).
The behavior in this area isn't particularly intuitive, so these issues
seem worth going into.

In passing, add a few more defensive/documenting row comparison related
assertions to _bt_first and _bt_check_rowcompare.

Follow-up to commits bd3f59fd and ec986020.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Victor Yegorov <vyegorov@gmail.com>
Reviewed-By: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/CAH2-Wznwkak_K7pcAdv9uH8ZfNo8QO7+tHXOaCUddMeTfaCCFw@mail.gmail.com
Backpatch-through: 18
2025-11-02 15:27:05 -05:00
Peter Geoghegan
4f08586c7a Remove obsolete nbtree equality key comments.
_bt_first reliably uses the same equality key (on each index column) for
initial positioning purposes as the one that _bt_checkkeys can use to
end the scan following commit f09816a0.  _bt_first no longer applies its
own independent rules to determine which initial positioning key to use
on each column (for equality and inequality keys alike).  Preprocessing
is now fully in control of determining which keys start and end each
scan, ensuring that _bt_first and _bt_checkkeys have symmetric behavior.

Remove obsolete comments that described why _bt_first was expected to
use at least one of the available required equality keys for initial
positioning purposes.  The rules in this area are now maximally strict
and uniform, so there's no reason to draw attention to equality keys.
Any column with a required equality key cannot have a redundant required
inequality key (nor can it have a redundant required equality key).

Oversight in commit f09816a0, which removed similar comments from
_bt_first, but missed these comments.

Author: Peter Geoghegan <pg@bowt.ie>
Backpatch-through: 18
2025-11-02 13:34:18 -05:00
Tom Lane
645c1e2752 Avoid mixing void and integer in a conditional expression.
The C standard says that the second and third arguments of a
conditional operator shall be both void type or both not-void
type.  The Windows version of INTERRUPTS_PENDING_CONDITION()
got this wrong.  It's pretty harmless because the result of
the operator is ignored anyway, but apparently recent versions
of MSVC have started issuing a warning about it.  Silence the
warning by casting the dummy zero to void.

Reported-by: Christian Ullrich <chris@chrullrich.net>
Author: Bryan Green <dbryan.green@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/cc4ef8db-f8dc-4347-8a22-e7ebf44c0308@chrullrich.net
Backpatch-through: 13
2025-11-02 12:30:44 -05:00
Tom Lane
b70cafd85f Remove unused variable in recovery/t/006_logical_decoding.pl.
Author: Daniil Davydov <3danissimo@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAJDiXggmZWew8+SY_9o0atpmaJmPTL25wdz07MrDoqCkp4D1ug@mail.gmail.com
2025-11-01 14:01:52 -04:00
Tom Lane
ff8aba65d4 Fix contrib/ltree's subpath() with negative offset.
subpath(ltree,offset,len) now correctly errors when given an offset
less than -n, where n is the number of labels in the given ltree.
There was a duplicate block of code that allowed an offset as low
as -2n.  The documentation says no such thing, so this must have
been a copy-and-paste error in the original ltree patch.

While here, avoid redundant calculation of "end" and write
LTREE_MAX_LEVELS rather than its hard-coded value.

Author: Marcus Gartner <m.a.gartner@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAAUGV_SvBO9gWYbaejb9nhe-mS9FkNP4QADNTdM3wdRhvLobwA@mail.gmail.com
2025-11-01 13:25:42 -04:00
Álvaro Herrera
2648eab377 pg_createsubscriber: reword dry-run log messages
The original messages were confusing in dry-run mode in that they state
that something is being done, when in reality it isn't.  Use alternative
wording in that case, to make the distinction clear.

Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Backpatch-through: 18
Discussion: https://postgr.es/m/CAHut+PsvQJQnQO0KT0S2oegenkvJ8FUuY-QS5syyqmT24R2xFQ@mail.gmail.com
2025-10-31 18:49:50 +01:00
Álvaro Herrera
11144915e1 pg_createsubscriber: Fix error complaining about the wrong thing
The code updates the system identifier, then runs pg_walreset; if the
latter fails, it complains about the former, which makes no sense.
Change the error message to complain about the right thing.

Noticed while reviewing a patch touching nearby code.

Author: Álvaro Herrera <alvherre@kurilemu.de>
Backpatch-through: 17
2025-10-31 17:43:15 +01:00
Peter Eisentraut
8a27d418f8 Mark function arguments of type "Datum *" as "const Datum *" where possible
Several functions in the codebase accept "Datum *" parameters but do
not modify the pointed-to data.  These have been updated to take
"const Datum *" instead, improving type safety and making the
interfaces clearer about their intent.  This change helps the compiler
catch accidental modifications and better documents immutability of
arguments.

Most of "Datum *" parameters have a pairing "bool *isnull" parameter,
they are constified as well.

No functional behavior is changed by this patch.

Author: Chao Li <lic@highgo.com>
Discussion: https://www.postgresql.org/message-id/flat/CAEoWx2msfT0knvzUa72ZBwu9LR_RLY4on85w2a9YpE-o2By5HQ@mail.gmail.com
2025-10-31 10:47:25 +01:00
Peter Eisentraut
aa4535307e formatting.c cleanup: Change fill_str() return type to void
The return value is not used anywhere.

In passing, add a comment explaining the function's arguments.

Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/6dd9d208-a3ed-49b5-b03d-8617261da973%40eisentraut.org
2025-10-31 09:55:12 +01:00
Peter Eisentraut
da2052ab9a formatting.c cleanup: Rename DCH_S_* to DCH_SUFFIX_*
For clarity.  Also rename several related macros and turn them into
inline functions.

Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/6dd9d208-a3ed-49b5-b03d-8617261da973%40eisentraut.org
2025-10-31 08:06:46 +01:00
Peter Eisentraut
378212c68a formatting.c cleanup: Change several int fields to enums
This makes their purpose more self-documenting.

Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/6dd9d208-a3ed-49b5-b03d-8617261da973%40eisentraut.org
2025-10-31 08:06:46 +01:00
Peter Eisentraut
ce5f6817e4 formatting.c cleanup: Change TmFromChar.clock field to bool
This makes the purpose clearer and avoids having two extra symbols,
one of which (CLOCK_24_HOUR) was unused.

Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/6dd9d208-a3ed-49b5-b03d-8617261da973%40eisentraut.org
2025-10-31 08:06:46 +01:00
Michael Paquier
c9e38a569c Add test tracking WAL receiver shutdown for primary_conninfo updates
The test introduced by this commit checks that a reload of
primary_conninfo leads to a WAL receiver restarted, by looking at the
request generated in the server logs.  This is something for what there
was no coverage.

This has come up for a different patch, while discussing a regression
where a WAL receiver should not be stopped while waiting for a new
position to stream, like at the end of a timeline.  In the case of the
other patch, we want to check that this log entry is not generated, but
if the error message is reworded the test would become silently broken.
The test of this commit ensures that we at least keep track the log
message format, for a supported scenario.

Extracted from a larger patch by the same author.

Author: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/aQKlC1v2_MXGV6_9@paquier.xyz
2025-10-31 11:24:24 +09:00