To avoid warnings and lines that extend beyond the right page
margin in the PDF documentation, we have zero-width spaces in
the source at strategic places. They are not visible to the
naked eye, but allow FOP to break the lines in sane places.
However, some of these zero-width spaces were in the middle of
text that people might copy and paste from the HTML documentation,
which leads to mysterious syntax errors.
Scanning the documentation for &zwsp; identified four places
that this patch improves:
- an ISO 8601 interval literal didn't actually need a line break
in either the US letter or A4 format, so simply remove that
zero-width space.
- three v18-era examples for uuid_extract_timestamp() and
uuid_extract_version() had zero-width spaces in the middle of UUID
literals. Instead, add regular spaces around the "::" cast symbols,
allowing FOP to break the lines there. This solution isn't
invisible in the HTML output, but it doesn't look bad.
Reported-by: Y. Saburov <y.saburov@gmail.com>
Author: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/179143326813.1061054.14714458907175925072@wrigleys.postgresql.org
Backpatch-through: 18
This backwards-compatibility hack was introduced twenty years ago (in
8.2), and has surely been of no value for at least the last ten years.
However, we shouldn't just leave it sit, because as was just pointed
out it creates a dump/reload hazard: if it's set in the receiving
environment then array contents will not be restored correctly.
Other applications would likely be broken by it too, so whatever
residual value it may have seems outweighed by the possibility of
misuse.
Our precedent for retiring some other comparable parameters, such as
standard_conforming_strings, has been to leave the parameter in
existence but force it to a fixed value. I judge however that
that's not worth the trouble here, because there's no reason to
believe that a non-negligible number of applications are either
setting or reading this value. Let's just delete it in the master
branch and be done with it.
Bug: #19747
Reported-by: Ke <kehan5800@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/19747-69b01e7fc190cd58@postgresql.org
The page a bit of a mess, with notes interpersed in the list of
parameters, and two "notes" subsections that weren't in the Notes
section.
Limit the commentary under "Description" and "Parameters" to a minimum;
move the existing text to appear in the "Notes" section. Both the
"Notes on Clustering" and "Notes on Resources" subsections, which were
under "Description", are moved to be under "Notes" instead. Add a new
"Notes on Concurrent Operation" subsection there, which now carries some
text that was in "Parameters", and gets some additional text to
(hopefully) explain resource consumption more clearly -- notably, the
hard limit of 104 million tuples that can be concurrently updated/deleted
before running into the palloc size limit.
Add appropriate cross-links.
Author: Radim Marek <radim@boringsql.com>
Author: Álvaro Herrera <alvherre@kurilemu.de>
Author: Shihao Zhong <zhong950419@gmail.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/CAJgoLk+dodrwuwCuXERGYwgjQzSwrKd+xgitYrL32oWDt39zvA@mail.gmail.com
bpchar equality ignores trailing spaces, so equal values in a column
with no length specifier can have different images. Similarly,
oidvector equality intentionally pays no attention to the array lower
bound field. We nevertheless allowed nbtree deduplication to be used
with indexes built with bpchar_ops, bpchar_pattern_ops, or
oidvector_ops. A posting list split could therefore place a TID in an
index tuple whose key was equal to, but not bytewise identical to, the
corresponding table value. This allowed index-only scans to return
subtly wrong results.
To fix, drop support function 4 from the affected opclasses (we can't
change catalog contents on branches 14 through 18, so we just disable
deduplication by adding hard-coding to the relevant support functions).
Deliberately leave the oidvector opclass alone on branches 14 through
18, though. Disallowing oidvector deduplication on these branches would
make pg_amcheck report that pg_proc_proname_args_nsp_index has a corrupt
metapage, needlessly alarming users.
System catalog oidvector values always have a lower bound of 0, so the
issue can't cause harm there; a user can only hit it by indexing an
oidvector column that stores the output of a function like trim_array().
It seems very unlikely that any user would ever do that.
Users should REINDEX any B-Tree index on a bpchar column.
Author: Shihao Zhong <zhong950419@gmail.com>
Co-authored-by: Peter Geoghegan <pg@bowt.ie>
Reported-by: Ke <kehan5800@gmail.com>
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Bug: #19749
Discussion: https://postgr.es/m/19749-d46adf7c7d910b07@postgresql.org
Discussion: https://postgr.es/m/CAGRkXqQQ311Lg=62x5UdZtPuRFgpT6XpPMw4X9E3tgbTu162Mw@mail.gmail.com
Backpatch-through: 14
seg.sgml claims that a seg can store 7 significant digits, but this
is nonsense. The underlying float4 data type is not (quite) that
precise, and seg_out() intentionally rounds off to at most FLT_DIG
decimal places. With IEEE float arithmetic, FLT_DIG will be defined
as 6, or at least that's the case on all platforms I know of.
AFAICT this thinko goes clear back to the introduction of seg
(commit a3694b420), but it wasn't questioned until now.
Bug: #19744
Reported-by: Ke <kehan5800@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19744-8c700582bbb1d075@postgresql.org
Backpatch-through: 14
pg_blocking_pids() assumes that members of the same lock group
never block each other, but that hasn't been true for relation
extension locks since v13. This commit teaches the function to
report lock group members as blockers for relation extension locks.
Since parallel workers are reported with their leader's PID, the
result may now include the blocked session's own PID.
Oversight in commit 85f6b49c2c.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: Shihao Zhong <zhong950419@gmail.com>
Discussion: https://postgr.es/m/apTpV8h%2BCYHf%2B1PJ%40bdtpg
Backpatch-through: 14
This commit addresses the following issues in commit de28140de:
* When the local transaction was DEFERRABLE, begin_remote_xact() started
a remote transaction in the same mode without checking the remote
server version. Since DEFERRABLE is supported only in v9.1 and later,
this would cause a syntax error on older remote servers. Fix this by
propagating the mode only to remote servers that support it.
* pgfdw_xact_callback() committed remote transactions without
synchronizing the read-only mode. If the local transaction had become
READ ONLY since the last remote operation, this would cause deferred
triggers on remote servers to still run in READ WRITE mode. Fix this
by synchronizing the mode at commit time.
* The documentation did not match the actual implementation. Update the
documentation to reflect the implementation.
Reported-by: Fujii Masao <masao.fujii@gmail.com>
Author: Etsuro Fujita <etsuro.fujita@gmail.com>
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwE4UMaWGOZq1Z7YUTUSg-CgL4yAzO5YaGwTZe1-6Pjg2g%40mail.gmail.com
Backpatch-through: 19
pg_stat_progress_vacuum gains three fields, reflecting some activity
related to index cleanups during VACUUM, which is a phase that could
take a long time depending on the quantity of data to treat and the
number of indexes:
- current_index_relid, the index being processed.
- index_blks_total, total number of blocks to process.
- index_blks_done, number of blocks already processed.
There fields are new, and do not reuse the existing fields related to
heap as index cleanups may need multiple passes, where knowing both the
heap activity and the index activity could be relevant. This is
unlikely since tids are accumulated in the heap scanning phase using an
optimized radix tree, still possible. VACUUM can spawn parallel workers
to clean indexes (each worker cleans a different index); these should up
in the progress view in the shape of one extra row for each worker.
Bump catalog version.
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Shihao Zhong <zhong950419@gmail.com>
Discussion: https://postgr.es/m/CALj2ACUgwSchK6jQ2CdKLBWUADTOE_zKdTff2Zg3E6hOuXKv-w@mail.gmail.com
There have been several recent complaints that << and >> don't behave
the same way on all supported platforms. There is a risk that <<
overflows the signed integer type, which is undefined behavior, and there
are platform-dependent behaviors from shifting by numbers of bits >= the
bitwidth of the type, and also from shifting by negative values.
Document this to warn unsuspecting users of these facts.
Author: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAApHDvr0O6ZF8NO=Rq9Q+Sz=FHT=NZ1kkNTXZDcNph-OWuk7TQ@mail.gmail.com
Backpatch-through: 14
The explanation of aggressive scans claimed that if the multixact
members grows to 2 billion entries, we launch autovacuum even if it's
nominally disabled. That's no longer true; anti-wraparound vacuum is
only triggered if we're in danger of running out of multixids, we no
longer launch it just to shrink the 'members' SLRU. There's no
wraparound danger with 'members' anymore, so it's no worse than if a
user table becomes very bloated.
We do still adjust the freeze threshold though, so that if autovacuum
runs, whether on schedule or as an emergency autovacuum to avoid
wraparound, it will freeze more aggressively if the members SLRU is
large.
I removed the mention of the 4 billion member threshold, even though
that still exists, because the explanation felt misleading. The
aggressiveness is scaled proportionally between 2 and 4 billion
members. At 4 billion it's as aggressive as it gets, but there's no
sudden cliff.
In the passing, fix some comments. Remove obsolete mention of safe
threshold for member storage triggering autovacuum. Secondly, the
freezing cutoff is adjusted to avoid the *members* SLRU from growing
too large.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://www.postgresql.org/message-id/20260827231757.78.noahmisch@microsoft.com
Backpatch-through: 19
The "initializing" phase was described as expected to be very brief,
but for REPACK (CONCURRENTLY), it also covers enabling and
initializing logical decoding, either of which can take a long
time. Also, the "catch-up" phase includes waiting for the ACCESS
EXCLUSIVE lock needed to swap the files, which was not mentioned.
Adding separate progress phases for these steps is left for v20; for
v19 we only clarify the description.
Author: Sami Imseih <samimseih.pg@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Manuel Reyes Bravo <manuelreyesbravo@gmail.com>
Discussion: https://postgr.es/m/CAN12%2BYJOeOyPYB5JS28VE_%2Brzg%3DRbzQXfj58TGfMYwv-TX9mJA%40mail.gmail.com
Backpatch-through: 19
Per ISO/IEC 9075-2:2023, 7.11 <JSON table>, Syntax Rules 1)e)iv) and
1)f)xi), a regular or formatted JSON_TABLE column that does not specify
its own ON ERROR clause takes its default error behavior from the
table-level ON ERROR clause: with ERROR ON ERROR on the table, the
column behaves as ERROR ON ERROR; otherwise the column defaults to
NULL ON ERROR.
PostgreSQL instead always defaulted such columns to NULL ON ERROR, so
SELECT * FROM JSON_TABLE(jsonb '"err"', '$'
COLUMNS (a int PATH '$') ERROR ON ERROR) jt;
returned a NULL row where the standard requires an error. The
documentation stated the divergence as if it were a rule, saying that
the table-level clause "does not affect the errors that occur when
evaluating columns".
Implement the standard behavior in the JSON_TABLE syntactic
transformation: when a column lacks its own ON ERROR clause and the
table-level behavior is ERROR ON ERROR, synthesize an implicit ERROR ON
ERROR for the column before it is transformed into a JsonExpr. A column
with its own ON ERROR clause is unaffected. EXISTS columns are not
covered by those syntax rules, so they keep their FALSE ON ERROR
default. A JSON_TABLE stored in a view is now deparsed with the
previously implicit ERROR ON ERROR shown explicitly on the affected
columns, which is a semantically equivalent, round-trip-stable form.
86ab7f4c72 introduced this cascade as part of the PLAN clause and
af6fad879f reverted it, on the grounds that it should be a deliberate
and separately documented change rather than a side effect of an
unrelated feature. This is that change.
Do not back-patch. A query that specifies ERROR ON ERROR at the table
level and relies on its columns still yielding NULL now raises an error
instead of returning rows. Nothing is silently wrong in the released
branches: the table-level clause is ignored for columns consistently,
and that is what the documentation has promised since PostgreSQL 17, so
users could reasonably have depended on it.
Discussion: https://postgr.es/m/CAPpHfdtDXseGhrL14a2asOkwGnFTVrDk8SQc3iWZgWEEhXXMGw%40mail.gmail.com
Reviewed-by: Nikita Malakhov <hukutoc@gmail.com>
SHMEM_ATTACH_UNKNOWN_SIZE can be passed as argument to
ShmemRequestStruct() when the caller wants to attach to an existing
shared memory structure whose size it doesn't know. If the shared
memory structure doesn't exist, the request should fail, but instead,
ProcessShmemRequestsAfterStartup() tried to create the structure with
size = -1. That led to integer overflow in ShmemAllocRaw() and memory
corruption.
Fix by rejecting requests with SHMEM_ATTACH_UNKNOWN_SIZE when the
structure doesn't exist. Also add an integer overflow check in
ShmemAllocRaw(), to protect from this kind of confusion or simply too
large requests. Also document SHMEM_ATTACH_UNKNOWN_SIZE.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/CAExHW5u_fTsOAS85kG981Vu6eR1GV-344rup6zYew7xMjEDREw@mail.gmail.com
Backpatch-through: 19
The reference page said WAIT "cannot be used in contexts where such a
snapshot must remain active, including transactions running at isolation
levels higher than READ COMMITTED", which reads as a blanket ban at those
levels. It is not one. The rule the code enforces is that the current
transaction must hold no snapshot, and a transaction-snapshot-mode
transaction has no snapshot until its first query: WaitStmt is exempt in
PlannedStmtRequiresSnapshot(), so the command itself does not take one.
WAIT therefore works before that first query, which is the order an
application wants anyway, and it is what 447aae13b0 set out to allow by
keeping the wait snapshot-free.
State the rule in terms of the snapshot rather than the isolation level,
say that WAIT acquires none of its own and name the other ways to hold
one, and give the consequence of holding one: it can delay replay, which
standby_replay waits for and which the other standby modes ultimately
depend on as well.
The error for a held snapshot named the internal snapshot states instead
of saying what the user did, and attached the isolation-level
explanation as DETAIL unconditionally, so a READ COMMITTED session with a
cursor was told its isolation level was too high. Report what is held
rather than how it is held, and emit the DETAIL only when the isolation
level is in fact the reason.
Reported-by: Sami Imseih <samimseih.pg@gmail.com>
Reported-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Sami Imseih <samimseih.pg@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/CAN12%2BYJddgAZzA36JZFX%2BYrnNR7Uubp9qWFz6ASs3v0kL30AMA%40mail.gmail.com
Backpatch-through: 19
The reference page and the read-your-writes section both presented
WAIT in standby_replay mode as enough for read-your-writes, without
saying which LSN to wait for. Read as written, an LSN taken after the
last statement but before COMMIT would do, and it would not: the standby
position WAIT compares against is the end of the last replayed record, so
a target that falls inside the COMMIT record is satisfied while that
record is still unapplied.
Say that the target must be at or after the end of the write
transaction's COMMIT record, and note in the example that autocommit is
what puts the UPDATE's commit before the call that reads the position.
Author: Sami Imseih <samimseih.pg@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Discussion: https://postgr.es/m/CAN12%2BYKN7gXb1o9bDWgfA%3D6wjDrqNEgt%2BrXJiAWXgPCXQ5TqpA%40mail.gmail.com
Backpatch-through: 19
The description of NO_THROW covered what the option does but not when an
application would want it. Explain that it suits callers that treat
timeout or not in recovery as expected results to be handled as part of
ordinary control flow, note the obligation to check the returned status,
and give the reason to omit the option.
Be precise about the option's scope: it changes only how those two wait
outcomes are reported. Soften the description's claim that the command
"returns a status string instead of throwing errors" accordingly, and say
that invalid input still raises an error, as does any condition checked
before the wait begins. Name two of those conditions as examples rather
than listing them, since the set grows.
Author: Rithvika Devisetti <devisettirithvika@gmail.com>
Reported-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Kiran Kaki <itskkpg@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Satyanarayana Narlapuram <satyanarlapuram@gmail.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Discussion: https://postgr.es/m/14415b05-16d0-446f-ba7d-c891db8b3406%40eisentraut.org
Backpatch-through: 19
Up until now, pg_plan_advice has had a feature that allows an advice
target to mention a partition name but omit the partition schema;
that is, something like SEQ_SCAN(foo/bar) forces a sequential scan on
every child of table foo whose partition name is bar, regardless of
the schema in which bar appears. However, that feature turns out to
have a nasty design flaw: while advice enforcement handles this case
just fine, the advice feedback code doesn't know about it and will
mark such advice as "matched, failed" even when everything worked
perfectly. Unfortunately, there seems to be no simple code fix for
this problem.
Since the release of PostgreSQL 19 is imminent, take the conservative
course and revert this feature of pg_plan_advice. In other words,
require the partition schema whenever the partition name is present.
You must now write e.g. SEQ_SCAN(foo/public.bar) rather than just
SEQ_SCAN(foo/bar). This doesn't affect any cases where automatically
generated advice is supplied, since generated advice has always
included the partition schema anyway. Manually written advice will
have to conform to the new, stricter rule. For a later release, we
can consider whether to again relax this restriction in some way,
but now is not the time to design new things.
Reported-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/CA+TgmoYmXy-jiP5qDhqNEiYFEBzQsArO6O2d9E8szNZqi1bePQ@mail.gmail.com
Commit 5f3a49abf7 changed the primary status update message to report the
end of the last inserted WAL record instead of the WAL write position.
However, the protocol documentation still described the WAL position field as
the latest WAL write position, which no longer matches the message sent by
walsender.
Update the description of the WAL position field to indicate the end of the
last inserted WAL record, matching the implementation.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwFn2QfpHDR6cWxd98tqNb5vyguEis8rtqnqidFR36p4QQ@mail.gmail.com
Backpatch-through: 19
Commit 0c5d626961 was confused about which indexes are potentially
problematic to rebuild: we thought only !indisready indexes could fail
to rebuild, and thus would attempt to rebuild !indisvalid ones that are
"ready" (that is, being maintained by DML but not yet completed).
However, that's wrong, because it could be that the validation phase of
such an index failed or was interrupted partway through. So, have
REPACK throw an error with all !indisvalid indexes, not just those
!indisready.
(!indisready implies !indisvalid, so this is an expansion of what
happened with the previous commit.)
Reported-by: shihao zhong <zhong950419@gmail.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/CAGRkXqRA4YetT8Nu3+1-45+ffTwi2bd1d1eDFME39Y9OPRMytQ@mail.gmail.com
List of commits reverted:
8e72d914c5 Add UPDATE/DELETE FOR PORTION OF
b6ccd30d8f Add isolation tests for UPDATE/DELETE FOR PORTION OF
33db6c4baf Fix DELETE/UPDATE FOR PORTION OF with rules
7b22f15a01 Add psql tab completion for FOR PORTION OF clause
7ca8c94296 Fix FOR PORTION OF with non-updatable view columns
993a7aa0e4 Fix cross-leftover pollution in FOR PORTION OF insert triggers
7ac030d5b1 Require UPDATE permission on FOR PORTION OF column
f0aedc7cb0 Small terminology fixes in comments
7d13b03a2e Fix FOR PORTION OF for inheritance children
a272a58b94 Move FOR PORTION OF volatile check into planner
a40fdf6588 Reject child partition FDWs in FOR PORTION OF
bc3ae886a7 Forbid FOR PORTION OF with WHERE CURRENT OF
e994f956e4 Forbid generated columns in FOR PORTION OF
dfce19c230 Forbid FOR PORTION OF on views with INSTEAD OF triggers
20fe90bbb9 Fix RLS checks for FOR PORTION OF leftover rows
9170c8b716 Test what BEFORE UPDATE triggers do to FOR PORTION OF
7090c696cc Deparse FOR PORTION OF using the range column's current name.
62d1a5f8be Avoid RETURNING side effects for FOR PORTION OF leftovers.
9924e2d900 Enforce WITH CHECK OPTION on DELETE FOR PORTION OF leftovers
c16a1b4f2c Fix error code for null FOR PORTION OF target
610976420a Fix assertion failures in DELETE FOR PORTION OF tuple routing
438f027238 Resolve untyped parameters in FOR PORTION OF FROM/TO bounds
140fdfcdf1 Don't evaluate the FOR PORTION OF target under EXPLAIN
Discussion: https://www.postgresql.org/message-id/62bf70cc-dc33-4188-8070-a8c177bb61de%40eisentraut.org
A TOAST table whose chunk_id column is oid8 is now fed a 64-bit object
ID thanks to the introduction of a new vartag_external called
VARTAG_ONDISK_OID8, coupled with a new varatt_external_oid8. The only
difference between the existing varatt_external_oid and the new
varatt_external_oid8 is an extra 4 bytes to store 8 bytes worth an
object ID, spread across two uint32 to force alignement as memcmp may be
used for TOAST pointer comparison (line varatt_external_oid).
There is a one-one mapping between the vartag_external and the type of a
chunk_id:
- An oid8 chunk_id implies the use of VARTAG_ONDISK_OID8.
- An oid chunk_id implies the use of VARTAG_ONDISK_OID.
There are a couple of changes worth noting:
- In toast_tuple_init(), for the comparison of old and new external
values with a memcmp over VARSIZE_EXTERNAL(old_value). Now that
multiple types of external on-disk TOAST pointers are possible, we
need to add a check based on the vartag used by an old or new value.
One of the tests added by c68cba09dd cover this case.
- In toast_tuple_find_biggest_attribute(), the initial "biggest_size"
depends on the type of chunk_id. This needs a change in RelationData to
track the chunk_id type; this cached data is filled on demand.
- In detoast.h, toast_external_info_get() and toast_external_data act as
a translation layer to retrieve all the data associated to an Oid or an
Oid8 TOAST pointer, without the need to worry about the vartag dealt
with. The introduction of this function and structure has reduced a lot
of code churn that existed in the initial versions of this patch, as per
a suggestion from Bharath Rupireddy.
- amcheck is updated to check the vartag <-> chunk_id type mapping
requirements, handling both vartags depending on the underlying type of
chunk_id on the TOAST table.
Bump catalog version, due to at least the change of VARTAG_SIZE().
(This counts for the bump forgotten in a625fc570c22..)
Author: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Greg Burd <greg@burd.me>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Yugo Nagata <nagata@sraoss.co.jp>
Discussion: https://postgr.es/m/af19kUjwjhaoUTLn@paquier.xyz
This commit adds the possibility to define TOAST tables with oid8 as
their chunk_id, based on the reloption toast_value_type. All the
external TOAST pointers still rely on varatt_external_oid and a single
vartag. The values inserted in the oid8 TOAST tables are fed from the
OID8 value generator, casted to OID for now, as we do not have a
vartag_external able to store Oid8 values yet.
An upcoming commit will add support for a new vartag_external and its
associated structures, with the code being able to use a different
on-disk external TOAST pointer depending on the attribute type of
chunk_id defined in TOAST relations.
All the changes done here are mechanical, mostly around the TOAST code
that needs to be able to do chunk_id lookups based on the two types now
supported.
Bump catalog version.
Author: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Greg Burd <greg@burd.me>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Yugo Nagata <nagata@sraoss.co.jp>
Discussion: https://postgr.es/m/af19kUjwjhaoUTLn@paquier.xy
A set of documentation updates for the online checksums work, found via
manual as well as LLM-guided review.
* The state diagram was re-done to be improve readability and be
more in line with the look and feel of other diagrams.
* Paragraph about mismatched states moved to its own sect2.
* The documentation for the functions to enable/ disable checksums
failed to mention that they are superuser only.
* The documentation for the checksums progress reporting had
accidentally omitted the work "fork" and for the blocks_* columns.
The values are per fork but the documentation made it seem they
were per relation.
* Add the state image to the meson buildfile
Reported-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/5b08b4d1-0982-4b77-bac5-3bdffc6583f5@iki.fi
Backpatch-through: 19
check_control_files() detected a chain whose backups were taken under
different data checksum states, warned, and proceeded. The output
directory keeps the last backup's control file, so a full backup taken
with checksums off combined with an incremental taken after an offline
enable produces a cluster whose control file says "on" while most of
its blocks carry no checksums; it starts, and then every connection
dies on the first unchecksummed catalog page. An offline enable
between two backups of a chain is all it takes, since it rewrites
every page without logging anything, so the incremental backup does
not re-ship the pages.
Turn the warning into an error, matching what pg_rewind does for the
equivalent combinations. The check stays asymmetric on purpose: when
the last backup was taken with checksums off, stale checksums from an
earlier backup are never verified, and that chain remains usable.
Author: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/anwm6UPxoVS41QA2@bdtpg
Backpatch-through: 19
pg_rewind installs the source's control file on the target, but every
block it does not copy keeps the target's content. With checksums
enabled on the target and disabled on the source, the rewound server
claims enabled checksums while the blocks copied from the source have
none, and fails checksum verification as soon as it reads them; in the
worst case every connection attempt dies on an unverifiable catalog
page.
Comparing the control files is not enough. Replay on the rewound
server resumes from the last common checkpoint and adopts the data
checksum state recorded there, so an offline disable on the source
after the divergence leaves both control files saying "off" while the
rewound server still resumes with verification enabled. Check the
state carried by the divergence checkpoint as well, which pg_rewind
already reads.
Refuse both cases, and an interrupted online transition on either
side, same as pg_checksums does. The opposite mismatch, checksums on
the source only, is allowed with a warning: recovery under the backup
label adopts the divergence state, so the rewound server keeps
checksums disabled (or converges through the replayed WAL if the
source enabled them online), and no page can fail verification.
Author: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/anwm6UPxoVS41QA2@bdtpg
Backpatch-through: 19
An inprogress-on or inprogress-off control file means an online
transition was cut short, running the offline tool on top of it mixes
two procedures. Refuse it in every mode, with a hint pointing at the
start-stop cycle (or, on a standby, at letting replication finish)
that resets the state. Also print that the change applies to one
data directory only, with standby-aware wording, since in a
replication setup the same change must be made on every node.
On a primary, inprogress-on cannot survive a graceful stop: the
checksums launcher resolves it from its exit cleanup, and a crashed
primary is already rejected by the existing "cluster must be shut
down" check. inprogress-off can, however, it is set by the backend
running pg_disable_data_checksums(), and a fast shutdown arriving
between its two barriers leaves it behind in a cleanly shut down
control file, where the next start-stop cycle resolves it at end of
recovery. A standby can be stopped with either state, since it has
no launcher and only carries forward whatever state the last replayed
WAL record left it in, with a restartpoint persisting that as-is.
The standby is also the deterministic way to reach the new guard,
which is why the test coverage uses one; the primary window would
need an injection point between the two barriers.
The pg_checksums page said the tool still processes all relation files
regardless of an interrupted online transition; describe the refusal
instead.
Author: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/anwm6UPxoVS41QA2@bdtpg
Backpatch-through: 19
Offline data checksum changes with pg_checksums are local to one node,
but replay adopted the data checksum state carried by checkpoint record
unconditionally. After enabling checksums offline on only the primary,
a standby thus started verifying checksums it does not have. After an
offline change on a standby, the next replayed checkpoint silently
reverted the state.
To fix, make the control file track this node's state alone, and have
replay cross-check the replayed state against it instead of adopting
it, warning once per divergent value and reporting when the states
agree again. pg_control gains a watermark, normally the end LSN of
the newest XLOG2_CHECKSUMS record the node has written or applied,
and a flag marking that the state was last written by pg_checksums.
Recovery must never overwrite this state with a replayed one. The
control file is updated when all pages are flushed. This requires
a PG_CONTROL_VERSION bump.
Recovery from a base backup may be an exception to not adopting: its
control file was copied at an arbitrary moment, so the state carried
by the starting checkpoint is the one the WAL from there on was
Also document the offline procedure for replication setups.
Author: Zsolt Parragi <zsolt.parragi@percona.com>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reported-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/anwm6UPxoVS41QA2@bdtpg
Backpatch-through: 19
The command tag is WAIT, but the reference page called the command
"WAIT FOR" in its title, index entry, psql help name and throughout its
prose, and so did the error messages. FOR is a noise word that makes the
grammar read like English, exactly as INTO does for INSERT and FROM does
for DELETE; neither of those is part of the command's name. Settle on
WAIT everywhere the command is named, and keep WAIT FOR LSN wherever the
syntax itself is being shown.
Rename the reference page's id and file name to match, following
cb2c696b1d: v19 is not released, so the URL is not yet baked into a
supported version, and every other page's id matches its title.
Reported-by: Masahiko Sawada sawada.mshk@gmail.com
Author: Xuneng Zhou xunengzhou@gmail.com
Reviewed-by: Alexander Korotkov aekorotkov@gmail.com
Discussion: https://postgr.es/m/CAD21AoBdtiPTbm7T_aNeDON9JpFG9g%3DJDUP4WU-Y8t8_xqvR5Q%40mail.gmail.com
Backpatch-through: 19
Commit 447aae13b0 accepted TIMEOUT values up to INT64_MAX
milliseconds, but computing the deadline multiplies by 1000, which
caused an int64 overflow. And the command reported a timeout at once
instead of waiting.
Parse the value as an int instead, which simplifies the code and is
consistent with other timeout values such as the statement_timeout GUC.
The specified values are now rounded to the nearest millisecond, so a
positive timeout below half a millisecond becomes zero and waits
indefinitely.
Backpatch to v19, where the TIMEOUT option was introduced.
Reported-by: ChangAo Chen <cca5507@qq.com>
Author: ChangAo Chen <cca5507@qq.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/tencent_86B83240785807077600A1778566B5C12908@qq.com
Backpatch-through: 19
To import attribute statistics, postgres_fdw sent a query wity COLLATE
"C" to remote servers without checking their version. Since COLLATE is
only supported since v9.1, this could break the import against older
servers; fix by modifying postgres_fdw to check their version and if
older, fall back to sampling. (This commit still allows importing only
relation statistics from such an older server.)
Also, expand the documentation to mention that attribute statistics are
stored into the local server without applying the foreign table columns'
n_distinct option.
Oversight in commit 28972b6fc.
Reported-by: Fujii Masao <masao.fujii@gmail.com>
Reported-by: Osama Abdul Qader <osamaabdulqader.cs@gmail.com>
Author: Etsuro Fujita <etsuro.fujita@gmail.com>
Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwH8%2BuDTwfohpck9v_9L3nwSWACbmC36JYRyqHd5xQN_OA%40mail.gmail.com
Backpatch-through: 19
This is required for a follow-up commit that will add support for 8-byte
TOAST values, with this function being changed so as it is able to
support the largest TOAST value type available.
Most of the changes in this commit are in the regression tests that use
pg_column_toast_chunk_id(), that need to use some oid8-ish changes. The
change is surprisingly not invasive as casts from oid to oid8 are
possible (the opposite cast oid8->oid is not supported).
Bump catalog version.
Author: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Greg Burd <greg@burd.me>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Yugo Nagata <nagata@sraoss.co.jp>
Discussion: https://postgr.es/m/af19kUjwjhaoUTLn@paquier.xy
A backend waiting for WAL replay can retain locks acquired by earlier
statements. If the startup process needs one of those locks, directly or
through another backend, before reaching the target LSN, a deadlock can
arise: startup waits for the backend to release the lock, while the backend
waits for startup to advance replay. The lock manager records the
backend's held locks, but WAIT FOR LSN does not register its dependency on
replay as a lock wait. The deadlock detector therefore cannot see the
complete cycle. With unlimited standby conflict delays and no other
timeout or cancellation, this deadlock can persist indefinitely.
Write and flush waits are restricted as well. Their positions are floored
by the replay position, so without an active walreceiver the startup
process can be their only source of progress. If a held lock blocks
replay, these waits can form the same cycle: the backend waits for replay
to advance, while replay waits for the backend to release the lock.
Streaming does advance them independently, but only while WAL keeps
arriving. If reception stops before the target is reached, a blocked
startup process cannot restart the walreceiver. It also cannot replay
newer checkpoint records needed to advance restartpoints and recycle WAL,
so continued reception can exhaust available space in pg_wal before the
target is reached. An active receiver at the start of the wait therefore
does not guarantee that the wait can finish while replay remains blocked.
Reject an unsatisfied standby_replay, standby_write, or standby_flush wait
while recovery is active when the backend already holds a granted
heavyweight lock. This conservative restriction covers direct
relation-lock cycles and indirect cycles involving advisory locks. It
also rejects some write and flush waits that an active receiver could
satisfy. Requests whose target is observed as already reached are exempt
from this check, as are primary_flush requests and requests issued after
recovery has ended. Existing snapshot and recovery-state checks still
apply.
Report one of the held locks so the user can find it, using the same
description the deadlock report uses.
Add replay-mode tests for relation and advisory locks and for the
already-reached case, and document the restriction along with the
recommended usage pattern.
Author: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/CABPTF7U0gW5%2B-4oL7-qdML-yerZxUb7ku4QXp7JxCYo0qyJ_Tw%40mail.gmail.com
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Backpatch-through: 19
This reverts commit a9c350d9ee.
While that intentionally changed our semantics for CREATE SCHEMA
subcommands, it's being argued that the functionality gain does
not justify potentially-subtle compatibility breakage. The most
useful bit of functionality gain came from commit 404db8f9e
("Execute foreign key constraints in CREATE SCHEMA at the end"),
which we're keeping because it's required by SQL spec and doesn't
seem to create a compatibility hazard by itself.
This is not an exact revert, partly because we're keeping 404db8f9e,
and partly because I kept the API changes that allowed passing down
a ParseState (which allows providing an error cursor for many of the
errors thrown in CREATE SCHEMA).
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20260904180712.cc.noahmisch@microsoft.com
Backpatch-through: 19
This reverts commit d516974840,
along with parts of commit 049b742daa
("psql: Tighten heuristics for BEGIN/END within CREATE SCHEMA").
While there's nothing particularly wrong with d51697484 in itself,
it depends on a9c350d9e ("Don't try to re-order the subcommands of
CREATE SCHEMA"), and concerns have been raised that the compatibility
impact of that outweighs the benefit of allowing more object types
within CREATE SCHEMA.
It's not possible to revert 049b742da verbatim, because the
CVE-2026-6464 patches 3045a25ba ("Teach psql to skip in-line COPY
... FROM STDIN data after a failure") and cf754f741 ("Save/restore
more lexer state when skipping text due to \if") depend on
infrastructure it added. Instead, rip out just the bits specifically
needed to parse CREATE FUNCTION within CREATE SCHEMA. This results in
lexer code that matches v18-and-earlier as modified by CVE-2026-6464.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20260904180712.cc.noahmisch@microsoft.com
Backpatch-through: 19
This relation option gives the possibility to define the attribute type
that can be used for chunk_id in a TOAST table when it is initially
created. An update of this parameter has no effect if a TOAST table
already exists under the relation, even on rewrites. It cannot be set
for TOAST relations.
This new option can be set only to "oid" as of this commit. This
feature is extracted as a patch of its own to ease the introduction of a
second mode in a follow-up change, related to the introduction of 8-byte
OIDs in external TOAST pointers.
Bump catalog version, as StdRdOptions has changed.
Author: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Greg Burd <greg@burd.me>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Yugo Nagata <nagata@sraoss.co.jp>
Discussion: https://postgr.es/m/af19kUjwjhaoUTLn@paquier.xyz
The general nature of this interaction has been understood (at least
by me) since before the original commit, but I failed to explicitly
document it. I also left behind an XXX in the README which I intended
to remove before commit. Fix those things.
Reported-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/CA+TgmoYmXy-jiP5qDhqNEiYFEBzQsArO6O2d9E8szNZqi1bePQ@mail.gmail.com
