The parser accepts any string as an EXTRACT() field name, but
deparsing does not quote and escape it accordingly. To fix, quote
and escape the field name during deparsing as needed. It might be
a good idea to validate the field name during parsing and
deparsing, too, but that is left as a future exercise.
Reported-by: Ben Morris in collaboration with Claude and Anthropic Research
Author: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Etsuro Fujita <etsuro.fujita@gmail.com>
Security: CVE-2026-15741
Backpatch-through: 14
ECPG assumes that any bytea data it receives from a backend starts with
'\x' as its first two bytes, but a check was missed to enforce that. A
rogue server sending some garbage bytea data would be able to crash a
client, resulting in a client-side DoS, in the most common cases.
Reported-by: ylwangtju <ylwangtju@qq.com>
Backpatch-through: 14
Security: CVE-2026-16241
Some of these could overflow on 32-bit systems with the right input.
Convert all cases where we called palloc() with multiplication to fix
them. Not all of them were bugs, but it's better to be safe than
sorry.
Reported-by: Tulya Project, Team Dhiutsa, Bitecope Technologies Private Ltd
Backpatch-through: 14
Security: CVE-2026-14677
The CACHESIGN.sign field is a BITVECP, not a TRGM, so you should not
use GETSIGN() on it. You don't get a compiler warning because the
GETSIGN() macro includes a cast. It resulted in a bogus read beyond
end of buffer, which would cause bad split decisions or a crash if
you're very unlucky.
Reported-by: Mehmet D. INCE <mehmet@mehmetince.net>
Backpatch-through: 14
Security: CVE-2026-14678
When ALTER TABLE ... ALTER COLUMN TYPE (or any subcommand that rebuilds
them) drops and re-creates the extended statistics objects depending on
the altered column, the re-created objects were owned by the role running
ALTER TABLE rather than by the original owner of the statistics.
Remember each object's owner before dropping it, and restore it on
re-creation.
CreateStatistics()'s signature changes and CreateStatsStmt gains a field,
but no known third-party code calls the former or constructs the latter.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 14
Security: CVE-2026-6469
numeric_combine() and several other state-combining functions for
aggregates cheated for the case of both inputs being NULL: they
returned a null pointer without bothering to mark it as a SQL NULL.
This was harmless in the expected usage where the result would be
passed to the same combine function or a related aggregate final
function. But it's bad news from a security standpoint, because
now that value can be passed to an internal-accepting function
even if said function is strict. While a previous patch prevented
such queries from being issued, it seems like good defense-in-depth
to expend the few additional lines of code needed to do this properly.
Comparable functions such as array_agg_combine() already do so.
Reported-by: Amy Burnett (OpenAI Codex Security)
Author: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 14
Security: CVE-2026-14680
Allowing that is a security hole, since there are many different
functions with different ideas of what their "internal" argument or
result is. We already had a defense against the easy case of
"'foo'::internal", but that turns out to be insufficient. Lock down
both function and operator syntax. Also disallow attempts to cast to
or from type internal; those would mostly fail anyway, but we have
created some holes with features such as CoerceViaIO.
Reported-by: Amy Burnett (OpenAI Codex Security)
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Backpatch-through: 14
Security: CVE-2026-14680
The maximum number of arguments allowed for an aggregate function
is FUNC_MAX_ARGS-1 (since the underlying transfn and/or finalfn
will be called with one more argument). parse_func.c failed to
enforce this, allowing construction of calls that would try to
pass FUNC_MAX_ARGS+1 to the underlying functions, resulting in
a memory stomp in the executor. Add correct checking there.
Since it's possible that a bad call has been stored in a view or
SQL function, also add checks in various aggregate-related and
window-function-related code that there are not more than
FUNC_MAX_ARGS arguments. These will also protect us against the
possibility that we're trying to run a stored view that was made
by a server executable with different FUNC_MAX_ARGS. (Arguably,
that scenario does not qualify as a security problem. But let's
just tighten up all of this while we're here, rather than split
hairs over whether an overrun is reachable.)
Likewise check in compute_function_hashkey. Here the hazard is
directly from a pg_proc row, but the scenario is the same.
PL/Tcl has a similar issue with a fixed-size string buffer.
Let's just replace that buffer with a Tcl_DString, removing the
whole issue and making the code look more like what's around it.
There are a lot of other FUNC_MAX_ARGS-sized arrays, but the rest
have nearby guards already, some with comments explicitly pointing
out the hazard of FUNC_MAX_ARGS changing.
I also used palloc_array() in a few related places in funcapi.c.
Those aren't live hazards AFAICS, but nearby code has been
palloc_array-ified already, so it seemed inconsistent to not use
it here.
Reported-by: Masahiko Sawada <sawada.mshk@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Backpatch-through: 14
Security: CVE-2026-14679
The backend doesn't impose any particular limit on the length of this
array, and since there could be entries for both input and output
arguments, it's feasible for the length to exceed FUNC_MAX_ARGS
even without funny business. This could lead to crashes or worse.
Moreover, pg_dump shouldn't rely on hard-coding FUNC_MAX_ARGS in the
first place: it has no business assuming that the backend it's dumping
from was compiled with the same value of FUNC_MAX_ARGS that it is.
So the stanza in dumpFunc() that allocates exactly FUNC_MAX_ARGS space
for the parsed OID array is fundamentally misguided. And it's broken
in another way too: if there are exactly FUNC_MAX_ARGS OIDs, then
parseOidArray won't zero-fill any entries, allowing the subsequent
loop to run off the end of the array. A crash seems unlikely in
this variant, but garbage output is certain.
To fix, redesign parseOidArray's API so that it does the
array-mallocing, which simplifies the callers anyway. While we're
here, tighten and modernize it a bit; in particular, split it into
separate functions for OIDs and integers, as was foreseen long ago.
This lets us get rid of the confusing type-punning involved in
having IndxInfo.indkeys be declared as "Oid *" when it's really
potentially-signed ints. Also, most of the callers expect an exact
number of array entries, so make it verify that not just check for
"too many".
I noted while testing that this dumpFunc() stanza isn't even reached
during check-world. Add a function with transform to the regression
tests to rectify that.
Reported-by: Masahiko Sawada <sawada.mshk@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Backpatch-through: 14
Security: CVE-2026-19385
MatchNamedCall's arggiven array was declared FUNC_MAX_ARGS long,
but we may actually use up to pronallargs elements, and that can
be more than FUNC_MAX_ARGS if the function has OUT arguments
(cf. ProcedureCreate). Convert it to a palloc'd array.
Reported-by: Zheng Yu <zheng@depthfirst.com>
Reported-by: ylwangtju <ylwangtju@qq.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Backpatch-through: 14
Security: CVE-2026-14679
The special case here for estimating conditions involving a ctid
column failed to check that the RHS constant is of type tid.
While that'd always be true for the built-in operators that
reference this selectivity estimator, a maliciously constructed
operator could provide a user-controlled Datum value that would
get interpreted as an ItemPointer pointer. That at least risks
SIGSEGV, and perhaps with a bit of sweat it could be used for
server memory disclosure.
Reported-by: Hcamael <baiyjrh@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 14
Security: CVE-2026-14668
Tied arrays might report different sizes each time they are inspected.
To avoid generating a corrupt result array, fix plperl_array_to_datum()
to read av_len() of each input array only once. If the input does
appear to get shorter, we'll fill nulls for the now-missing entries,
which seems fine. Conversely, if it gets longer, we'll ignore the new
entries.
plperl_to_hstore() assumed that Perl's hv_iterinit() returns the
number of entries in the given Perl hash. Usually that's true,
but per the Perl docs, "the return value is currently only meaningful
for hashes without tie magic". That could potentially end in a memory
stomp. We don't depend on that result value anywhere else, so don't
do so here either.
Reported-by: Hcamael <baiyjrh@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Backpatch-through: 14
Security: CVE-2026-14670
setup_regexp_matches() sizes the buffer used to convert matched
substrings back from pg_wchar form at the smaller of maxlen*eml and
the original string's byte length, on the assumption that such a
conversion cannot produce more bytes than the string it came
from. That assumption holds only for validly encoded input. But
pg_mb2wchar_with_len() silently accepts bytes that are invalid in the
database encoding, turning each such byte into one pg_wchar, and
converting that back can take more bytes than the input did. A string
made of such bytes therefore overruns the conversion buffer by up to
its own length, corrupting the following memory. regexp_match(),
regexp_matches(), regexp_split_to_table() and regexp_split_to_array()
are all affected.
Fix by dropping the tighter bound and always allocating maxlen*eml + 1
bytes.
Reported-by: Francesco Verardi <frevadiscor89@gmail.com>
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 14
Security: CVE-2026-14664
The only overflow hazards I could find in tsquery construction
are in QTN2QT(), which builds a flat tsquery datum from the
QTNode tree representation used by tsquery_or, tsquery_rewrite,
and allied functions. There are two:
1. It seems theoretically possible for the outputs of cntsize() to
overflow an int, so I widened them to size_t. There's no hazard
certainly in tsquery_or and friends, but tsquery_rewrite could expand
the query tree by large multiples (by replacing many identical
subtrees with a large replacement tree), so in a 64-bit machine
with plenty of available memory it should be possible to build a
QTNode tree large enough to cause that. If these counters did
overflow then we'd under-allocate the output tsquery and have a heap
overwrite problem. size_t is sufficient, since it's counting the size
of a subset of an in-memory data structure. We also have to fix the
TSQUERY_TOO_BIG() macro to not get confused if sumlen exceeds
MaxAllocSize.
2. fillQT() neglects to check that the new "distance" value for a
QI_VAL item fits into the available 20-bit field. It's quite easy
to reach this, for example by tsquery_or'ing two near-megabyte-sized
tsquerys. However, the result is only a corrupt tsquery that does
not represent the expected query, so perhaps this doesn't rise to
the level of a security bug. Nonetheless it should be fixed.
Note: I followed the practice used in other tsquery code of checking
each distance value as it's assigned, which means that the last
operand string could extend past the MAXSTRPOS boundary. This is a
bit different from the pattern used for tsvectors, which insist that
the total data length not exceed MAXSTRPOS and thereby avoid making
per-item checks. Perhaps that should be harmonized sometime, but for
now it's okay for the two types to do this differently as long as
each one is self-consistent.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Backpatch-through: 14
Security: CVE-2026-14662
The core of this patch is to prevent array_to_tsvector() from
generating invalid tsvectors. It did not check for overly-long
lexemes (so that WordEntry.len fields could overflow), nor did it
check that the total "datalen" fits within MAXSTRPOS (so that
WordEntry.pos fields could overflow, and the number of entries
in the tsvector could be much more than the normal limit).
While the field overflows couldn't do anything much worse than
produce a corrupted tsvector value, a sufficiently large number
of tsvector entries could cause integer overflows in later
processing, such as tsvectorout.
Another important fix is to prevent tsvectorrecv() from accepting
invalid tsvectors. The main problem there is that it did not
reject empty-string lexemes. Hence, even though it did (mostly)
enforce the MAXSTRPOS limit, it could still produce a result
with an unreasonable number of tsvector entries, if they were
primarily empty strings.
Also, fix tsvectorout's calculation of its required output
buffer size: it was multiplying the string lengths by
pg_database_encoding_max_length() for no reason. That contributed
to the risk of integer overflow there. With valid tsvector input,
there's no risk, but there's still no reason to make the output
buffer several times bigger than needed.
I also tried to make a couple of related routines more robust,
and spent some effort on improving the comments in ts_type.h.
Also, standardize on a single spelling of the "string is too long
for tsvector" message, using %zu instead of an assortment of formats.
These changes aren't security per se but came out of inspecting the
code for problems.
Reported-by: Yuhang Wu <yuhang@depthfirst.com> and Zhenpeng Lin
Reported-by: Zheng Yu <zheng@depthfirst.com>
Reported-by: Hcamael <baiyjrh@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Backpatch-through: 14
Security: CVE-2026-14662
The logic for asynchronous Append assumes that pending requests made for
subplans of an Append are drained during ExecReScanAppend. To ensure
that, commit 9e283fc85 modified postgresReScanForeignScan to drain such
a request if any, but failed to take into account that if such a request
was made for a subplan that is re-scanned with parameter changes or
pruned in the next round by runtime pruning, the postgres_fdw callback
function is called after ExecReScanAppend or never called, respectively.
This would cause such a request to remain even after ExecReScanAppend,
leading to incorrect results, an infinite loop, or an assertion failure.
To fix, modify ExecReScanAppend to, for each of the pending requests,
give the FDW a chance to drain that request using the existing
ForeignAsyncConfigureWait/ForeignAsyncNotify callback functions. This
makes the change made to postgresReScanForeignScan useless, so remove it
as well.
Back-patch to v14 where asynchronous Append was added.
Reported-by: Alexander Korotkov <aekorotkov@gmail.com>
Co-authored-by: Alexander Korotkov <aekorotkov@gmail.com>
Co-authored-by: Gleb Kashkin <g.kashkin@postgrespro.ru>
Co-authored-by: Etsuro Fujita <etsuro.fujita@gmail.com>
Reviewed-by: Alexander Pyhalov <a.pyhalov@postgrespro.ru>
Reviewed-by: Gleb Kashkin <g.kashkin@postgrespro.ru>
Discussion: https://postgr.es/m/CAPpHfduMOTnV5Zj2KGJ7zanL_10QvccZHtPUaDfJvBhsh9axnQ%40mail.gmail.com
Backpatch-through: 14
seq_search_localized() returns the length of the matching prefix in
*len, but because it internally case-folds the inputs, it gets
confused on the length. The caller expects to get the length of the
prefix in the original string, but what it actually returns is the
length of the prefix after case-folding, which can be different if the
case-folded characters have different byte-length than the original,
or with ICU, if the case-folding changes the number of characters
(e.g. "ß", the German double s).
To fix, once we have determined that we have a match, work harder to
find the match's length in the original string. This adds some
overhead, but the strings are expected to be short.
The function does "case-folding" by converting a string to upper-case,
then to lower-case, which is a little ugly given that we have
dedicated functions for case-folding nowadays. But switching to that
doesn't seem appropriate to backpatch in a security fix, and that's
not available in older stable versions, anyway.
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Reported-by: Xint Code
Reviewed-by: Jeff Davis <pgsql@j-davis.com>
heap_force_common() tracked the current position in the caller-supplied
tid[] using OffsetNumber, which is only 16 bits wide, so when the array
held more than 65535 entries, the updated index wrapped around and the
outer loop never reached the exit condition. A SQL call with a
sufficiently large TID array would then run until interrupted.
Fix by tracking the tid[] position using int instead of OffsetNumber.
A regress case based on the report is included.
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reported-by: Yuelin Wang <1217816127@qq.com>
Backpatch-through: 14
Bug: #19607
Discussion: https://postgr.es/m/19607-2f256a66481c514b@postgresql.org
Per commit 8bf6ec3ba3, a column can be GENERATED only if it is such in
the whole inheritance tree.
For this reason, ATPrepDropExpression refuses to be called with ONLY on
a partitioned table. To detect this, the current implementation checks
whether recurse is set to false and the rel has direct children.
Recursion is implemented with ATSimpleRecursion, which calls ATPrepCmd
with recurse = false for every node in the tree. Inner nodes (for
example a partition which itself has subpartitions) then fail the check,
accidentally preventing the command from working on inheritance trees of
depth > 2.
This commit fixes it by also checking that we're at the top level of the
recursive calls using the recursing parameter, which is always true when
called through ATSimpleRecursion, always false when invoked on the root
rel.
Also, remove a comment claiming that DROP EXPRESSION could be
implemented with some effort. It cannot, as the commit message for
8bf6ec3ba3 explains.
Author: Alberto Piai <alberto.piai@gmail.com>
Backpatch-through: 14
Discussion: https://postgr.es/m/DHMT78XOD8BK.341V3H87KZ7NO@gmail.com
Logging connection information, even at DEBUG1, creates unnecessary
risks. Remove the entire log message because it had no other useful
content.
Addresses finding 14 in report from linked discussion.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20260710195902.4f.noahmisch@microsoft.com
Backpatch-through: 14
In the not-too-likely scenario where we successfully created a hash
table entry for a TS dictionary, but then failed to make a small
memory context for it, we left the hash entry in existence but with
a garbage value for dictCtx. This confused the code the next time
through, leading to a crash. Rearrange things so that we leave
the hash entry in a well-defined state with dictCtx == NULL, and
then the next try knows it still needs to make a memory context.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/0f3ddeb5-0dbd-479c-9d0e-ae254758e624@gmail.com
Backpatch-through: 14
Allocate CompoundAffix with room for its terminator, initialize the
old-format flag buffer before NIAddAffix(), and reject incomplete or
missing Hunspell AF aliases. None of these errors would be likely to
trigger on real dictionary files, accounting for the lack of previous
reports; but they're certainly bugs.
Bug: #19595
Reported-by: Michael Malis <michaelmalis2@gmail.com>
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19595-7dc18b4e212c4757@postgresql.org
Backpatch-through: 14
Alberta (America/Edmonton) moved to permanent UTC-06 on
2026-06-18, which will affect their clocks beginning on 2026-11-01.
For lack of any clarity on the point, assume their TZ abbreviation
will be CST from that time forward.
Morocco (Africa/Casablanca) will move to permanent UTC+00,
without daylight saving time transitions, on 2026-09-20.
Backpatch-through: 14
Partition pruning for RANGE-partitioned tables could mistakenly prune
the DEFAULT partition in some cases when it was not valid to do so,
which could lead to rows missing from query results.
The only known cases where this could happen is when combining pruning
steps from an IS NOT NULL clause with other steps that matched to the
DEFAULT partition. This could occur due to RANGE partitioned tables
having two distinct internal representations for marking if the DEFAULT
partition should be scanned. The IS NOT NULL steps would mark the
"scan_default" boolean, but other steps created for different purposes
could mark a bound_offset Bitmapset, which would ultimately translate into
also scanning the default partition. This could all fail after multiple
steps were combined with a combine intersect operator, as that will
intersect the bound_offset bits and only set scan_default if all pruning
steps have that flag set. When both input steps to the intersect operator
had different representations of whether to scan the DEFAULT partition,
the resulting intersect step result would contain neither representation.
Here, we fix this by having the IS NOT NULL pruning result mark the
bound_offsets so that it uses both representations to mark that the
DEFAULT partition must be scanned.
Reported-by: Jacob Brazeal <jacob.brazeal@gmail.com>
Diagnosed-by: Jacob Brazeal <jacob.brazeal@gmail.com>
Author: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CA+COZaDXrfTaBjLE=Z79MTaH6Xun1V4PeKxLvCNv8mXS8wn0rw@mail.gmail.com
Backpatch-through: 14
A cascading standby could fail to reconnect to its upstream standby with
"requested starting point ... is ahead of the WAL flush position" after
falling back to archive recovery. This happened because archive
recovery processes whole segment files, so after replaying a segment the
cascade's next read position lands at the start of the following
segment, which is ahead of the upstream's flush position reported by
GetStandbyFlushRecPtr() (still inside the just-replayed segment).
Fix by having the walreceiver check the upstream's current WAL flush
position via IDENTIFY_SYSTEM before issuing START_REPLICATION.
IDENTIFY_SYSTEM already returns this position (as xlogpos), but
walrcv_identify_system() previously discarded it; now we have a use for
it. If the requested start point exceeds the upstream's flush position
on the same timeline, the walreceiver waits for
wal_retrieve_retry_interval and retries.
The wait is limited to gaps of at most one WAL segment, which is the
expected case from the segment-granularity of archive recovery. Larger
gaps indicate the upstream is genuinely behind, so START_REPLICATION is
allowed to proceed (and fail) normally, letting the startup process fall
back to other WAL sources. The first wait is logged at LOG level;
subsequent waits are demoted to DEBUG1 to avoid log noise. The
walreceiver honors wal_receiver_timeout during the wait, so it will exit
if the upstream doesn't catch up in time.
To preserve ABI compatibility on back branches, the flush position from
IDENTIFY_SYSTEM is communicated via a new global variable
(WalRcvIdentifySystemLsn) rather than changing the signature of
walrcv_identify_system().
The bug was introduced in Postgres 9.3 by commit abfd192b1b, which
added a flush-position check in StartReplication() that rejects requests
ahead of the upstream server's WAL flush position.
Author: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Backpatch-through: 14
Discussion: https://postgr.es/m/CA+nrD2cTuTkkX5WXVZengTYYZbAO6zV8K+Tri-R0fbLFuoyMBA@mail.gmail.com
find_nonnullable_rels and find_nonnullable_vars mistakenly treated a
ScalarArrayOpExpr that could return FALSE as strict, but that's okay
only at top level of a qual expression; further down, we've got to
insist on a guaranteed-NULL result. The result was that we could draw
mistaken conclusions about whether outer joins can be simplified, if
the decision hinged on a non-top-level ScalarArrayOpExpr with a
potentially-empty array argument.
I believe this error dates to commit 72a070a36, which taught
find_nonnullable_rels to descend into non-top-level parts of qual
expressions. is_strict_saop (added earlier by 72153c058) already had
enough intelligence to do the case correctly, but it wasn't passed the
proper flag, ie "top_level" needs to be passed for "falseOK".
e006a24ad copied that mistake into find_nonnullable_vars.
Later, over-eager refactoring in commit 2f153ddfd broke
contain_nonstrict_functions' handling of ScalarArrayOpExpr by treating
it as though it were no different from an OpExpr. It is, because
we must also prove the array is non-empty before concluding that the
expression is strict. This could result in misclassifying an
expression as strict when it is not, leading to assorted planning
mistakes such as inlining a SQL function that shouldn't be inlined.
We can almost fix this by just re-adding the previous handling of
ScalarArrayOpExpr in that function, but doing only that would lead to
also calling check_functions_in_node() and thus redundantly checking
the operator's strictness. Avoid that by turning the if-series into
an else-if chain, as it arguably should have been all along.
The reason these errors have escaped detection for decades is that
they are exposed only in arcane corner cases. ScalarArrayOpExpr with
an empty array isn't typical usage, and even when that's possible
several other conditions apply before the planner can reach a mistaken
conclusion. While it's possible to build test cases demonstrating
these mistakes, I (tgl) judged them too indirect and special-purpose
to justify consuming regression test cycles forevermore.
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAJTYsWV3vqRJmST-gv1NsXEef-zOnjVJpYS910aBaiuMij4nFg@mail.gmail.com
Discussion: https://postgr.es/m/CAJTYsWWcLGmz0f8_QPP_Liq-fc7-geiFSCdqoq3XGeRHPPsWeA@mail.gmail.com
Backpatch-through: 14
A two-phase transaction that is assigned an XID but produces no change
to be decoded -- for example, one that only acquires row locks via
SELECT ... FOR SHARE -- has no base snapshot in the reorder
buffer. ReorderBufferReplay() already skips such a transaction at
PREPARE time and never invokes the begin_prepare/change/prepare
callbacks for it, but ReorderBufferFinishPrepared() still called the
commit_prepared (or rollback_prepared) callback. As a result a
spurious COMMIT/ROLLBACK PREPARED was sent to the output plugin with
no preceding PREPARE. For the built-in subscriber this breaks
replication (the apply worker fails to find the prepared transaction),
and test_decoding could even crash.
Fix this by detecting an empty transaction (base_snapshot == NULL) in
ReorderBufferFinishPrepared() and cleaning it up without invoking the
commit/rollback prepared callbacks, mirroring the existing empty
transaction handling in ReorderBufferReplay().
On v18 and newer versions, commit 072ee847ad changed
ReorderBufferPrepare() to send the prepare whenever it had not already
been sent, which also fires for empty transactions and emits a
spurious PREPARE. On those branches ReorderBufferPrepare() is
therefore additionally guarded with base_snapshot != NULL. This guard
and the Assert(!rbtxn_sent_prepare()) added in
ReorderBufferFinishPrepared(), are not necessary on v17 and older
versions: there ReorderBufferPrepare() only sends a prepare for
concurrently-aborted transactions (which never applies to an empty
transaction) and the RBTXN_SENT_PREPARE flag does not exist.
Back-patch to v14, where decoding of two-phase transactions was
introduced.
Bug: #19556
Reported-by: Alexander Kozhemyakin <a.kozhemyakin@postgrespro.ru>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/19556-daa6d7ea65054d48@postgresql.org
Backpatch-through: 14
Commit fd83c83d09 turned the recursive posting-tree cleanup in
ginVacuumPostingTreeLeaves() into an iterative sweep that follows the
tree's leaf pages via their rightlinks. The recursive version called
vacuum_delay_point() while processing the tree, but that call was removed
and never re-added to the new loop. As that commit only set out to fix a
deadlock, the removal appears to have been unintentional.
Consequently the leaf-page sweep of a single posting tree runs with no
vacuum_delay_point(), and therefore no CHECK_FOR_INTERRUPTS(). A posting
tree stores all the TIDs for one indexed key, so for a frequently
occurring key it can span a large number of leaf pages. While such a
tree is being vacuumed the operation ignores vacuum_cost_delay and does
not respond to query cancellation or statement_timeout; an autovacuum
worker likewise cannot be interrupted mid-sweep when another backend
requests a conflicting lock.
Restore the call, placed after the current page has been unlocked and
released so that no buffer content lock is held across a potential delay
(cf. 21c27af65f). The sibling loops in ginbulkdelete() and
ginvacuumcleanup() already call vacuum_delay_point() once per page.
Author: Paul Kim <mok03127@gmail.com>
Co-authored-by: Alexander Korotkov <aekorotkov@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: solai v <solai.cdac@gmail.com>
Discussion: https://postgr.es/m/178447127453.110.12276981925360691905%40mail.gmail.com
Backpatch-through: 14
index_create_copy is used to create copy definitions of existing indexes.
Currently, it passes 0 as constr_flags to index_create(), which results
in the copied index to always be created as immediate (indimmediate set
to true). For deferrable unique constraints, it means that the
transient index used during the phase 2 of REINDEX CONCURRENTLY forces
immediate constraint checks on concurrent inserts, which can cause
unexpected constraint violations based on the definition of the parent
table, inconsistently set in the copied index.
To fix this without violating the contract of constr_flags (which should
only be used when creating constraints) and without relaxing the strict
assertion in index_create(), this introduces a new index creation flag:
INDEX_CREATE_DEFERRABLE. If set, a copied index's indimmediate is set
to false, meaning that unique constraints are not enforced immediately
on insertion, but at transaction commit time.
An isolation test for REINDEX CONCURRENTLY is added, based on an
injection point waiting after phase 1 of the operation, where an index
copy has been built and is able to accept DMLs for its validation in
phase 2. The test is tentatively backpatched down to v17.
INJECTION_POINT() is outside a transaction context, which should be fine
on HEAD since 8daeaa9b64 but I suspect may cause issues in v19 and
older branches due to the wait facility depending on condition variables
and a DSM setup, but let's see what the buildfarm tells.
Author: Nitin Motiani <nitinmotiani@google.com>
Discussion: https://postgr.es/m/CAH5HC97JmjPpgiQOqW9xm8qXhNiu7zZ1Qh+FfhEESJuDv69kuQ@mail.gmail.com
Backpatch-through: 14
Commit f9b7fc65 fixed a race when predicate-locking completely empty
btrees: without a buffer lock held, a matching key could be inserted
between _bt_search and the PredicateLockRelation call, so the scan would
miss concurrently inserted tuples while the writer wouldn't see the
reader's predicate lock. That commit only fixed _bt_first's _bt_search
path, though. Scans without useful insertion scan keys return early
from _bt_first via _bt_endpoint, which still didn't recheck if the
relation was empty.
To fix, add handling to _bt_endpoint that is analogous to the handling
added to _bt_search by commit f9b7fc65.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WzkNoTn3yXY0iGkSuavJ+sL8EROf+kitW+_2v2tJVWuKmA@mail.gmail.com
Backpatch-through: 14
pg_database_size() allows access to users who have either CONNECT
privilege on the target database or privileges of the pg_read_all_stats
role. However, previously, psql's \l+ checked only for CONNECT,
so users with privileges of pg_read_all_stats still saw "No Access" for
databases they could not connect to.
Fix this by making \l+ also check
pg_has_role('pg_read_all_stats', 'USAGE'), matching
pg_database_size()'s permission rules.
For back branches, emit the pg_read_all_stats check only when
connected to PostgreSQL 10 or later, since earlier releases do not have
that predefined role.
Backpatch to all supported versions.
Author: Christoph Berg <myon@debian.org>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/amCo6qRmnfPVk4-V@msg.df7cb.de
Backpatch-through: 14
It's always been the case that granting these privileges to users that
you don't fully trust was a bad idea, but it hasn't always been
obvious to people reading the documentation that this is the case.
To prevent confusion, and also repeated reports to pgsql-security,
mention it explicitly.
Discussion: http://postgr.es/m/CA+TgmobrjCHBuWHrvX3=2vndUCO2thUOdevrCcMDFW86cqCYvw@mail.gmail.com
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Backpatch-through: 14
An index-only scan filled its result slot from the HeapTuple an index AM
returns in scan->xs_hitup by deforming it with the virtual slot's own
tuple descriptor (during GiST and SP-GiST index-only scans). But index
AMs form that heap tuple using their own descriptor, scan->xs_hitupdesc.
The AM's descriptor may disagree with the IoS virtual slot's descriptor
about each column's precise alignment, leading to "can't happen" errors
in certain rare edge cases. Hard crashes were possible but much less
likely.
To fix, deform the tuple with the descriptor it was formed with. This
is simpler, and makes xs_hitup handling (used by GiST and SP-GiST)
uniform with the nearby existing xs_itup handling (used by nbtree).
In practice this issue was very unlikely to be hit (it was found during
testing of a patch that will change the table AM API used during index
scans). The only currently affected core opclass is GiST's range_ops.
It was only possible for the datum to be accessed at an incorrectly
aligned offset when reading the second or subsequent column from a
multicolumn GiST index. This couldn't happen in the common case where
the datum used an unaligned short varlena header. Moreover, an earlier
column had to leave the range datum at an offset where the two
alignments actually disagree (e.g., an odd-length varlena datum).
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/CAH2-WzkGXa2SKnebdW29RT1hCcQBo_p03v3iqif2u9bjzLB-aQ@mail.gmail.com
Backpatch-through: 14
The previous text claimed that once the slot is disassociated with
ALTER SUBSCRIPTION ... SET (slot_name = NONE), DROP SUBSCRIPTION "will no
longer attempt any actions on a remote host". That is inaccurate:
DROP SUBSCRIPTION may still connect to the publisher to drop
internally-created table synchronization slots when some table
synchronization is left unfinished. Reword to describe this, and note
that if the publisher is unreachable those slots (and the main slot, if
it still exists) must be dropped manually to avoid indefinitely reserving
WAL.
Reported-by: Jeff Davis <pgsql@j-davis.com>
Author: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 14
Discussion: https://postgr.es/m/CAA4eK1+tyYSpPxMBy1974kjivuGeR7YY=yopwRGrK3+vCTysdg@mail.gmail.com
Discussion: https://postgr.es/m/D908370F-2695-4231-851D-17179A6A6F2A@gmail.com
Previously, libpqrcv_create_slot() checked only that
CREATE_REPLICATION_SLOT returned PGRES_TUPLES_OK before reading
values from the first row. If the server unexpectedly returned an
invalid result, such as zero rows, PQgetvalue() could return NULL,
leading to a crash while parsing the LSN.
Other replication commands, such as IDENTIFY_SYSTEM, already validate
the response shape before accessing result values, but
CREATE_REPLICATION_SLOT did not.
Fix this by verifying that CREATE_REPLICATION_SLOT response contains
exactly one row with four fields, and report a protocol violation otherwise.
Backpatch to all supported versions.
Bug: #19547
Reported-by: Yuelin Wang <1217816127@qq.com>
Author: Kenny Chen <kennychen851228@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/19547-f7986f668f71e788@postgresql.org
Discussion: https://postgr.es/m/CAPXstDtW2iqe+DJAOTQTX+rRziJp2UhZSo1+HRj1COAtbu+nKw@mail.gmail.com
Backpatch-through: 14
The documentation for log_parameter_max_length said it affects messages
generated by log_duration. However, log_duration alone does not log bind
parameter values, so this is misleading.
This commit updates the documentation to reference log_min_duration_statement,
which can log bind parameters, to better reflect actual behavior.
Backpatch to all supported versions.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Surya Poondla <suryapoondla4@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwGnCVMVz8-LU9F8Sh57bkQX3jMZzx7age7M0LFEz5=Fog@mail.gmail.com
Backpatch-through: 14
equalPolicy() is used in the relation cache to check if two policy
definitions are equivalent, but missed to check for polpermissive.
ALTER POLICY cannot switch a policy to be PERMISSIVE or RESTRICTIVE, so
this would need a dropped and then re-created policy, which would
trigger a relcache invalidation. Anyway, there is no harm in being
consistent in the check, and if one decides to add an ALTER POLICY to
switch PERMISSIVE or RESTRICTIVE, we would be silently in trouble.
Author: Andreas Lind <andreaslindpetersen@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAMxA3rv1CS6R7JR5ojz-3CmCEnZEFrqu+XXTnGbLRWrjJRH7sA@mail.gmail.com
Backpatch-through: 14
The pg_attribute_always_inline macro name is so long it forces pgindent
to format the code in strange ways. Which may incentivize patch authors
to either structure the code in strange ways (e.g. reorder prototypes),
use shorter names, etc. Neither is very desirable for code readability.
This shortens the name by removing the _attribute_ part. It also makes
it more consistent with pg_noinline, which does not have the _attribute_
part either.
Backpatched to all supported branches, to prevent conflicts when
backpatching other fixes. The backbranches however keep both the old and
new macro name, so that existing code keeps working.
Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Peter Geoghegan <pg@bowt.ie>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/bqqdehahpoa36igpictuqyn2s2mexk3t3ehidh2ffd2slb35e5@rzgksuiszgbg
Backpatch-through: 14
Check explicitly for pqsecure_read() returning an error. It shouldn't
fail, and we would've caught it in the check for a short read, but
better to be explicit so that the error message is more informative.
We also shouldn't update 'inEnd' when the read fails, although that
too is just pro forma as we will bail out and close the connection on
error.
Reported-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/34844e8c-267c-4daf-b1e0-f26059a4a7d3@eisentraut.org
Backpatch-through: 14
Commit aa606b9316 disallowed generated columns in COPY FROM WHERE
expressions, and commit 21c69dc73f disallowed system columns.
However, the COPY reference page still mentions only the restriction
on subqueries.
Update the documentation to also list generated columns and system
columns as unsupported in COPY FROM WHERE expressions.
Backpatch the generated-column documentation change to all supported
versions. Backpatch the system-column documentation change to v19,
where that restriction was introduced.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwEgxErc54yVOAVWCsr1O=8pgw4oKRPuEQ9mfhkoYGR_XA@mail.gmail.com
Backpatch-through: 14
Commit dcb0049523 accidentally changed the final expanded query's
condition to > 2 while rewriting the example into SQL operator notation.
The original query and the preceding rewritten forms all use >= 2,
and view expansion should preserve that qualification. This commit
changes the final condition from > 2 to >= 2.
Backpatch to all supported versions.
Reported-by: Yaroslav Saburov <y.saburov@gmail.com>
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/178248467618.108999.9966122434342474006@wrigleys.postgresql.org
Backpatch-through: 14
The previous commit strengthened a workaround for a hang when large
messages are split across TLS records/GSS tokens. Because that
workaround is implemented in libpq internals, it can only help us when
libpq itself is polling on the socket. In nonblocking situations,
where the client above libpq is expected to poll, the same bugs can
show up.
As a contrived example, consider a large protocol-2.0 error coming
back from a server during PQconnectPoll(), split in an odd way across
two records:
-- TLS record (8192-byte payload) --
EEEE[...repeated a total of 8192 times]
-- TLS record (8193-byte payload) --
EEEE[...repeated a total of 8192 times]\0
The first record will fill the first half of the libpq receive buffer,
which is 16k long by default. The second record completely fills the
last half with its first 8192 bytes, leaving the terminating NULL in
the OpenSSL buffer. Since we still haven't seen the terminator at our
level, PQconnectPoll() will return PGRES_POLLING_READING, expecting to
come back when the server has sent "the rest" of the data. But there
is nothing left to read from the socket; OpenSSL had to pull all of
the data in the 8193-byte record off of the wire to decrypt it.
A real server would probably not split up the records this way, nor
keep the connection open after sending a fatal connection error. But
servers that regularly use larger TLS records can get the libpq
receive buffer into the same state if DataRows are big enough, as
reported on the list. While the PostgreSQL server doesn't use larger
TLS records like that, other non-PostgreSQL servers that implement the
wire protocol are known to do that, as well as proxies that sit
between the server and the client
This is a layering violation. libpq makes decisions based on data in
the application buffer, above the transport buffer (whether SSL or
GSS), but clients are polling the socket below the transport buffer.
One way to fix this in a backportable way, without changing APIs too
much, is to ensure data never stays in the transport buffer. Then
pqReadData's postconditions will look similar for both raw sockets and
SSL/GSS: any available data is either in the application buffer, or
still on the socket.
Building on the prior commit, make pqReadData() to drain all pending
data from the transport layer into conn->inBuffer, expanding the
buffer as necessary. This is not particularly efficient from an
architectural perspective (the pqsecure_read() implementations take
care to fit their packets into the current buffer, and that effort is
now completely discarded), but it's hopefully easier to reason about
than a full rewrite would be for the back branches.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-by: solai v <solai.cdac@gmail.com>
Reported-by: Lars Kanis <lars@greiz-reinsdorf.de>
Discussion: https://postgr.es/m/2039ac58-d3e0-434b-ac1a-2a987f3b4cb1%40greiz-reinsdorf.de
Backpatch-through: 14
An extra check for pending bytes in the SSL layer has been part of
pqReadReady() for a very long time (79ff2e96d). But when GSS transport
encryption was added, it didn't receive the same treatment. (As
79ff2e96d notes, "The bug that I fixed in this patch is exceptionally
hard to reproduce reliably.")
Without that check, it's possible to hit a hang in gssencmode, if the
server splits a large libpq message such that the final message in a
streamed response is part of the same wrapped token as the split
message:
DataRowDataRowDataRowDataRowDataRowData
-- token boundary --
RowDataRowCommandCompleteReadyForQuery
If the split message takes up enough memory to nearly fill libpq's
receive buffer, libpq may return from pqReadData() before the later
messages are pulled out of the PqGSSRecvBuffer. Without additional
socket activity from the server, pqReadReady() (via pqSocketCheck())
will never again return true, hanging the connection.
Pull the pending-bytes check into the pqsecure API layer, where both
SSL and GSS now implement it.
Note that this does not fix the root problem! Third party clients of
libpq have no way to call pqsecure_read_is_pending() in their own
polling. This just brings the GSS implementation up to par with the
existing SSL workaround; a broader fix is left to a subsequent commit.
In preparation for the broader fix, this patch already changes the
*_read_pending() functions to return the number of bytes in the buffer
rather than just a boolean. The current callers don't need that, but
the subsequent fix will.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/CAOYmi%2BmpymrgZ76Jre2dx_PwRniS9YZojwH0rZnTuiGHCsj0rA%40mail.gmail.com
Backpatch-through: 14
ParallelBackupStart() stored _beginthreadex()'s return value as the
worker's thread handle without checking it. On failure that value is 0,
which would later reach WaitForMultipleObjects() as a null handle, caught
only by an Assert. The fork() path already calls pg_fatal() when it
fails; do the same for _beginthreadex(), as pgbench does.
Author: Bryan Green <dbryan.green@gmail.com>
Discussion: https://www.postgresql.org/message-id/8c712d76-ecf7-4749-a6d8-dddc01f298ec@gmail.com
Backpatch-through: 14
