seg.sgml claims that a seg can store 7 significant digits, but this
is nonsense. The underlying float4 data type is not (quite) that
precise, and seg_out() intentionally rounds off to at most FLT_DIG
decimal places. With IEEE float arithmetic, FLT_DIG will be defined
as 6, or at least that's the case on all platforms I know of.
AFAICT this thinko goes clear back to the introduction of seg
(commit a3694b420), but it wasn't questioned until now.
Bug: #19744
Reported-by: Ke <kehan5800@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19744-8c700582bbb1d075@postgresql.org
Backpatch-through: 14
Previously, a backend with an assigned XID deferred the
XLogLogicalInfo update requested by a procsignal barrier, and we
applied it in AbortTransaction() on abort. However, a failed
transaction block keeps its XID until ROLLBACK, so a barrier absorbed
while idle in that state was deferred, and CleanupTransaction() did
not apply it. The next transaction then ran entirely with the stale
value. If logical decoding had just been enabled, that transaction's
changes were written without logical information and silently skipped
by decoding.
Fix by applying the pending update in CleanupTransaction() instead of
AbortTransaction(). We call it after resetting the top-level XID
because, unlike the commit and prepare paths, CleanupTransaction()
isn't always called with interrupts held, so a barrier absorbed before
the reset would be deferred again and carried into the next
transaction.
Patch by Sergei Patiakin, with tests added by me.
Oversight in commit 67c20979ce.
Reported-by: Sergei Patiakin <sergei.patiakin@enterprisedb.com>
Author: Sergei Patiakin <sergei.patiakin@enterprisedb.com>
Reviewed-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CANE55rApeNAFaqxLWrvm-NC0Y5gkrBFZFVaHVYP89AGS2SYMcA@mail.gmail.com
Backpatch-through: 19
COPY FROM runs its WHERE clause through eval_const_expressions(),
which among other things replaces NOT above an operator with the
operator's negator, if it has one (e.g., NOT (a > 0) becomes
a <= 0). The node it builds for the negator has no opfuncid, and
since COPY FROM never calls fix_opfuncids() on the clause, nothing
fills it in before the executor needs it. So
COPY tab FROM stdin WHERE NOT (a > 0)
fails with "cache lookup failed for function 0".
To fix, call fix_opfuncids() on the clause, as the relcache does
for index predicates.
Oversight in commit 31f3817402.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/asPZRTci4oBvxR9A%40nathan
Backpatch-through: 14
When reloading the SSL configuration fails, the existing config is
left in place to keep the server from being inaccessible. The GUC
backing SNI, ssl_sni, was however changed. This could lead to new
connections trying to access non-existing configurations. Fix by
storing the ssl_sni setting at the time of initialization in the
hosts context and only use the stored value during the handshake.
The patch submitted by the reporter has been significantly reworked
and expanded. Additional test cases written by Rui Zhao.
Author: Anthropic OSS program
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
Co-authored-by: Rui Zhao <zhaorui126@gmail.com>
Reported-by: Anthropic OSS program
Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com>
Reviewed-by: Rui Zhao <zhaorui126@gmail.com>
Discussion: https://postgr.es/m/20260920211619.9c.noahmisch@microsoft.com
Backpatch-through: 19
pull_up_simple_subquery() rechecks is_simple_subquery() after pulling
up the subquery's own subqueries, but it flattened join alias Vars in
the subquery's targetlist only after that, and never in its quals. A
join alias Var within the subquery, such as a whole-row Var of a join,
can expand to an expression containing lateral references to the outer
query once the join's inputs have been pulled up, and until it is
flattened is_simple_subquery() cannot see them. We could thus pull up
a LATERAL subquery whose targetlist or quals reference rels outside
the lowest outer join above it. This leads to assertion failures in
distribute_qual_to_rels() or "wrong phnullingrels" errors, and in some
cases to wrong query results, where rows that should have been
preserved by an outer join above the subquery are discarded.
To fix, flatten join alias Vars in the targetlist, and in the jointree
quals if the subquery is LATERAL, before the recheck. If the recheck
fails, the flattened copy is discarded along with the rest of the
modified subquery, so no harm is done.
Back-patch to all supported branches, as this can lead to wrong query
results.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tatsuya Kawata <kawatatatsuya0913@gmail.com>
Discussion: https://postgr.es/m/CAMbWs48GFZ=3Bjc1ug9JRsB0Qupg+CGcb9js5vmTM6kY2OGWOg@mail.gmail.com
Backpatch-through: 14
Commit e5d25959cf added the repack_dropped test to Meson but omitted it
from the Makefile, so make-based isolation runs did not execute it. The
test's teardown also left the injection_points extension installed,
relying on a later test to remove it.
Add repack_dropped to the Makefile's isolation tests and drop the
extension during teardown.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CAHGQGwG9bHxhs4Nj+9avSjSGh0vdYNQgEuz+pmJ-H5be6tEBTw@mail.gmail.com
Backpatch-through: 19
Previously, dropping a NOT NULL column, adding a new column with the same
name, and making it a primary key in a single ALTER TABLE command could
fail with "primary key column ... is not marked NOT NULL". This happened
because ALTER TABLE checked for an existing NOT NULL constraint before
dropping the old column. Finding the constraint caused it to skip adding
one for the new column, even though the old constraint would be removed
with the old column.
Defer this check until after column drops and additions have been
executed, so that it examines the columns that will actually form the
primary key. Add any required NOT NULL constraints before creating the
primary-key index.
Backpatch to v18, where this issue was introduced by commit a379061a22.
Reported-by: Ludvig Janiuk <ludvig.janiuk@proton.me>
Author: Fujii Masao <masao.fujii@gmail.com>
Co-authored-by: Manuel Reyes Bravo <manuelreyesbravo@gmail.com>
Discussion: https://postgr.es/m/0a9b4c44-cdff-4ec8-a8fb-b3a8b59e9ba9@proton.me
Backpatch-through: 18
Since commit e28cf96e9, the copies of a query level's PlaceHolderVars
that were pushed down into its subqueries are preprocessed by that
level before its SubLinks are turned into SubPlans, so such a copy may
already contain SubPlans by the time the subquery is planned. That
commit taught build_subplan not to process the SubLinks of a PHV that
is passed down as a SubPlan argument, but overlooked that such a copy
can also appear within the arguments of an outer-level Aggref,
GroupingFunc, or ReturningExpr. The SubLinks in those arguments are
still processed when build_subplan collects the node, and that
processing descended into the already-preprocessed PHV and tripped the
no-SubPlan assertion in process_sublinks_mutator.
To fix, don't descend into PHVs when processing the SubLinks in such a
node's arguments. Any PHV found there is a pushed-down copy that its
owning level has already preprocessed.
Back-patch to v16, as with commit e28cf96e9.
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CAJTYsWW8THbrnsmYnDqnUgg0PjGrqw2vNOWb8qx7eu191KPW3g@mail.gmail.com
Backpatch-through: 16
ri_FastPathCheck() read riinfo->fpmeta again after
ri_CheckFunctionPermissions(), which looks up catalog entries and so
can process invalidation messages. If one of them reaches
InvalidateConstraintCacheCallBack() for the constraint, as a pg_amop
change or a cache reset would, the callback detaches the metadata and
sets riinfo->fpmeta to NULL, which build_index_scankeys() then
dereferences.
The callback already defers freeing detached metadata until
AtEOXact_RI(), and its comment assumes callers keep their own pointer
to it, so use a local pointer instead of riinfo->fpmeta.
There is no test, as hitting this needs an invalidation to arrive
during those catalog lookups.
Discussion: https://postgr.es/m/CA+HiwqFaipkMsZ8XP-9sMk21h0Q7rih=NpJQXsBAaD39VtnrOA@mail.gmail.com
A Var that is needed by a join but is not a GROUP BY expression
becomes an extra grouping key for partial aggregation. The
EquivalenceClass for sorting on it is created when the grouped paths
are generated, which is after the children of appendrels and
partitionwise joins have received their EC members, so it has no child
members. Sort-based partial aggregation on a child relation then
failed with "could not find pathkey item to sort", as with a
partitionwise join whose join clause includes a non-equality
condition.
To fix, add the missing child members when the grouping pathkeys of a
child relation are built. This is done by the new function
add_child_rel_pathkey_equivalences(), which handles both child base
relations and child joins, including keys that are nullable by an
outer join within the child join.
Reported-by: Robert Haas <robertmhaas@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CA+Tgmob7iSM9YkRM44VjUDuaCchW-fY54MV5njpTZTL9uNyV4w@mail.gmail.com
Backpatch-through: 19
build_grouped_rel() makes a flat copy of the input relation, including
its FDW fields. As a result, when building a grouped join relation
for eager aggregation, add_paths_to_joinrel() called the FDW's
GetForeignJoinPaths for it. postgres_fdw then failed with "Aggref
found where not expected" on the partial Aggrefs in the target list,
and in any case it would have treated a partially aggregated input as
a plain foreign scan.
FDWs are not prepared to handle grouped relations, so clear serverid,
fdwroutine and fdw_private in build_grouped_rel().
Reported-by: Robert Haas <robertmhaas@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CA+Tgmob7iSM9YkRM44VjUDuaCchW-fY54MV5njpTZTL9uNyV4w@mail.gmail.com
Backpatch-through: 19
When a Var of the relation to be partially aggregated is known equal
to a GROUP BY expression through an EC, get_expression_sortgroupref()
let it reuse that expression's SortGroupClause. If the two are of
different types, as with an int2 grouping key joined to an int4
column, the Var was then grouped using the other type's equality
operator, which could put unequal values into the same group and give
wrong results.
To fix, accept only EC members of the same type as the grouping
expression. A cross-type Var is then treated like any other join key
and gets a SortGroupClause built for its own type.
Reported-by: Robert Haas <robertmhaas@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CA+Tgmob7iSM9YkRM44VjUDuaCchW-fY54MV5njpTZTL9uNyV4w@mail.gmail.com
Backpatch-through: 19
Commit 1ff3180ca0 made the leader of a parallel autovacuum propagate
changes to the cost-based delay parameters to its workers, but missed
track_cost_delay_timing. Workers therefore kept the value they had at
launch: turning the parameter on during the vacuum left
pg_stat_progress_vacuum.delay_time counting only the leader's naps,
and turning it off didn't stop workers from adding theirs. Throttling
itself was not affected.
Fix by adding track_cost_delay_timing to PVSharedCostParams. Since a
worker's setting can now change while it runs, we report its residual
delay time at exit whenever any has been accumulated, rather than only
if track_cost_delay_timing is still on.
Backpatch to v19, where parallel autovacuum was introduced.
Reported-by: Sami Imseih <samimseih.pg@gmail.com>
Author: Sami Imseih <samimseih.pg@gmail.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAN12+YJZaALd3zMeKVVRTwcuY5_yDtR_vMPwWc8QNgeYxrn4Zg@mail.gmail.com
Backpatch-through: 19
Commit ce8d5fe0e failed to think hard enough about the case where
we're forced to replan the expression and it becomes no longer simple.
The code produced the right expression result, but its handling of
FOUND and ROW_COUNT was too simplistic since we might have obtained
zero rows. To fix, don't use exec_eval_expr, but go directly to
exec_eval_simple_expr, and if that reports failure then fall through
to the full SPI code path.
Another thinko in ce8d5fe0e is that it added
if (expr->plan == NULL)
exec_prepare_plan(estate, expr, 0);
which is quite useless since we cannot reach here without having
planned the expression at least once. (If the call were reachable
it'd likely be wrong, since it's not passing the same flags as
either the preceding call or the one inside exec_eval_expr.)
Author: shihao zhong <zhong950419@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAGRkXqQB_aqWU9OeVPGvATBVZ7c9EYrNXJyHG0jKXW0FGAy2-g@mail.gmail.com
Backpatch-through: 19
pg_blocking_pids() assumes that members of the same lock group
never block each other, but that hasn't been true for relation
extension locks since v13. This commit teaches the function to
report lock group members as blockers for relation extension locks.
Since parallel workers are reported with their leader's PID, the
result may now include the blocked session's own PID.
Oversight in commit 85f6b49c2c.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: Shihao Zhong <zhong950419@gmail.com>
Discussion: https://postgr.es/m/apTpV8h%2BCYHf%2B1PJ%40bdtpg
Backpatch-through: 14
When we converted these types' input functions to report errors
through the soft-error mechanism, we missed that stack-overflow
errors can be triggered through bad user input. Convert those
to be reported softly, too.
Bug: #19745
Reported-by: Ke <kehan5800@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19745-39cd7b93a505f9a9@postgresql.org
Backpatch-through: 16
create_append_path() has a rather questionable habit of overriding
the caller-supplied pathkeys when it sees that there is a single
child path, and applying the child's pathkeys instead. In most cases
we can get away with that, but it does not work for set-operation
AppendPaths. In set-operation nests, the append's result tlist will
contain "varno 0" Vars, which won't match the child's pathkeys,
leading to failure in create_plan().
We didn't have this problem before 03d40e4b5 allowed eliding
provably-empty child nodes of set-ops; there would never have been
a case with only one surviving child node, so create_append_path()
would always have accepted the specified NIL pathkeys.
As a band-aid fix, force the generated AppendPath's pathkeys to NIL
even if create_append_path() did something else, thus restoring the
status quo ante. This is demonstrably necessary for two of
prepunion.c's three calls; I did it at the third too, although
probably the pathkeys would already be NIL there.
There have been reasons to want to get rid of the "varno 0" hack
for a long time, and this is another one. But that will require
significant surgery in prepunion.c, and likely some changes in
parsetree representation, so we can't tackle it for v19. Hence,
we need a band-aid.
Bug: #19742
Reported-by: Junwen AN <feasiblechart@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Co-authored-by: shihao zhong <zhong950419@gmail.com>
Discussion: https://postgr.es/m/19742-dc403ca277cad1d3@postgresql.org
Backpatch-through: 19
EXPLAIN failed to deal with varno-0 Vars that are made by prepunion.c
and can survive into a finished plan in the case where a provably
empty set operation is replaced by a dummy Result (which is possible
since 03d40e4b5). Commit 928df067d tried to fix this, but it was a
couple bricks shy of a load. First, it only dealt with varno-0 Vars
at the top level of the Result's tlist, but they could be buried
under coercion expressions. Fix that by doing a recursive mutation.
Second, it always replaced varno 0 with varno 1, but that's just
wrong: the Result might represent a group of setop leaf queries that
do not include the leftmost leaf. That led to displaying the wrong
variable(s) as outputs of the Result, risking confusion. Fortunately,
we can get the actual child relids from the recently-added
Result.relids field, and use that to discover the leftmost child
represented by the Result.
This was found while discussing bug #19742, but it's really an
independent issue.
Author: shihao zhong <zhong950419@gmail.com>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAGRkXqTFwygKmjLG_Y=kbXRHsePFmD6k+qrzVLP4_KrG+-=oRg@mail.gmail.com
Backpatch-through: 19
6ed83d5fa made it so tuplesorts could use a bump.c memory allocator for
non-bounded sorts in order to save memory and increase performance.
That commit didn't get the memory accounting correct for datum sorts and
mistakenly passed 0 bytes to tuplesort_puttuple_common(). That was
correct for byval types, but failed to take into account palloc'd memory
for byref types.
Fix by using datumGetSize() when using the bump allocator and by using
GetMemoryChunkSpace() when not.
Backpatch to v17, where 6ed83d5fa was added.
Author: Mario Karuza <mkaruza.pg@icloud.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Joao Detomini <joao.detomini@enterprisedb.com>
Discussion: https://postgr.es/m/e9f9f54337cd86a982837763958518cdbf08863c.camel%40icloud.com
Backpatch-through: 17
It's a bad idea to sleep while holding locks, because you might block
another process that wants to acquire the same lock. Furthermore,
it's not safe to call ProcessConfigFile() in a critical section or
while holding locks. So avoid calling vacuum_delay_point() while
holding locks, and also make vacuum_delay_point() return quickly
without sleeping if it's called while interrupts cannot be processed.
To find the vacuum_delay_point() calls that were made while holding
locks, I added "Assert(INTERRUPTS_CAN_BE_PROCESSED())" in
vacuum_delay_point() and ran the regression tests. I didn't include
that Assert in this commit, because there might be more places that do
that that are not covered by the regression tests, including
extensions.
In ginInsertCleanup(), there was already another vacuum_delay_point()
call later in the loop, while not holding any locks, so just remove
the other call that was made while holding the lock.
In hashbucketcleanup(), move the vacuum_delay_point() call up the
stack to its caller. The call in hashbucketcleanup() was not able to
handle interrupts because it held a lock, and because there were no
vacuum_delay_point() or CHECK_FOR_INTERRUPTS() calls in the caller's
loop, the whole hash index vacuuming phase was uninterruptible by
pending shutdown or query cancel. Now it can be interrupted between
buckets. Unfortunately, hashbucketcleanup() has to process all the
bucket's pages in one go without pausing; fixing that would require
changing how hashbucketcleanup()'s lock chaining works.
In acquire_sample_rows(), move the vacuum_delay_point() call in the
loop to between pages, to a time where we're not holding the buffer
lock.
In master, also remove a misleading CHECK_FOR_INTERRUPTS() call from
pgstat_write_statsfile(). It was a no-op because we're always holding
a lock on the current dshash partition at that point. A call that
does nothing is harmless, but it makes you think that the loop is
interruptible when in reality it's not.
Author: Kevin Rocker <me@kevinrocker.com>
Author: Andrey Borodin <amborodin@acm.org>
Author: Mostafa <mostafa.nabil.nafie@gmail.com>
Reported-by: Greg Burd <greg@burd.me>
Reported-by: Sergei Kornilov <sk@zsrv.org>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: Neil Chen <carpenter.nail.cz@gmail.com>
Tested-by: Greg Burd <greg@burd.me>
Discussion: https://postgr.es/m/19628-c2b17d358181a1ea@postgresql.org
Discussion: https://postgr.es/m/492c6247-43d3-477b-8981-fb0c56767b38%40app.fastmail.com
Backpatch-through: 14
This commit addresses the following issues in commit de28140de:
* When the local transaction was DEFERRABLE, begin_remote_xact() started
a remote transaction in the same mode without checking the remote
server version. Since DEFERRABLE is supported only in v9.1 and later,
this would cause a syntax error on older remote servers. Fix this by
propagating the mode only to remote servers that support it.
* pgfdw_xact_callback() committed remote transactions without
synchronizing the read-only mode. If the local transaction had become
READ ONLY since the last remote operation, this would cause deferred
triggers on remote servers to still run in READ WRITE mode. Fix this
by synchronizing the mode at commit time.
* The documentation did not match the actual implementation. Update the
documentation to reflect the implementation.
Reported-by: Fujii Masao <masao.fujii@gmail.com>
Author: Etsuro Fujita <etsuro.fujita@gmail.com>
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwE4UMaWGOZq1Z7YUTUSg-CgL4yAzO5YaGwTZe1-6Pjg2g%40mail.gmail.com
Backpatch-through: 19
When a LATERAL UNION ALL subquery is flattened into an appendrel, a
child EquivalenceMember's expression can contain lateral references to
other relations. create_join_clause() builds a clause from such a
member with make_restrictinfo(), which computes clause_relids from the
expression, and so includes the relids of the lateral references along
with those of any outer joins that null them. Commit 03107b4ed made
sure that clause_relids include the child's relids, but it only ever
added relids.
This is a problem when a lateral reference is nullable. The child is
parameterized by its lateral_relids, which do not include the outer
joins that null the lateral references, so the clause fails the
join_clause_is_movable_into() assertion that
get_baserel_parampathinfo() applies to EC-derived clauses.
To fix, set a child clause's clause_relids from its members' em_relids
rather than from their expressions. This matches how
adjust_appendrel_attrs() computes clause_relids for a child's other
join clauses. The lateral references need not be included, since
every path for the child is parameterized by at least its
lateral_relids. Also update num_base_rels to match, which commit
03107b4ed neglected to do.
Back-patch to v16, where outer-join relids were added to
clause_relids. Before that, the relids of a lateral reference are
always part of the child's parameterization, so the problem does not
arise.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-cRxpQNyQJW=PkDagSDGQ=3Rp2t_fSAyROKsDStjKHUQ@mail.gmail.com
Backpatch-through: 16
Buildfarm members adder and drongo failed to observe the expected relation
lock wait in the startup deadlock test. In adder's failure, autovacuum
pruned dead rows before the test's explicit VACUUM FREEZE, canceling the
standby backend too early and releasing the relation lock wait before the
test could observe it.
Fix this by disabling autovacuum for the test node, leaving the explicit
VACUUM commands in control of when pruning occurs. This also prevents
auto-analyze workers on other relations from holding back xmin and
interfering with the snapshot conflict test.
Backpatch to v17, where this test is enabled.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/421c0aee-84c8-4c07-b4b9-263095479755%40gmail.com
Discussion: https://postgr.es/m/714729ea-f536-0faf-353b-cbd5c8f241ea%40gmail.com
Backpatch-through: 17
Buildfarm members olingo and adder reported failures in
031_recovery_conflict.pl where a recovery conflict was counted twice.
olingo saw the snapshot conflict count reach 2 instead of 1, while adder
saw the same for the buffer pin conflict count.
The startup process can signal a standby backend again while it is
reporting a recovery-conflict FATAL error. Each processed conflict can
increment the corresponding statistic. Thus, even when the expected
conflict occurs, its counter can exceed 1. The existing poll for exactly 1
can then time out, and the exact check of the aggregate count can fail as
well.
Fix this by polling for a positive count for each conflict type and
accepting an aggregate count of at least the expected number.
Backpatch to v17, where this test was enabled.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/421c0aee-84c8-4c07-b4b9-263095479755%40gmail.com
Backpatch-through: 17
On Windows, a backend exiting resets its connection, and the client
discards any data not yet read, including the FATAL message of a
terminated backend. isolationtester polls one connection at a time, so
when it waits on a different session the message can get lost, failing
the permutation and the tests that run after it.
wait_cleanup in injection_points (down to v17) and slot_creation_error
in test_decoding (down to v15) have this pattern, and have been failing
randomly in the buildfarm. Let's skip them on Windows.
The cause of the instabilities is proving to be tricky to deal with,
pointing for example at 29992a6a50. This will be tackled later,
perhaps with a backpatch (or perhaps with a HEAD-only solution), but it
is difficult to be sure at this stage of the work. In theory,
a8458f508a should make a revert of 29992a6a50 (revert of a revert!)
safe on WIN32.
Note that 278cdea6b9, that has introduced the isolation test
slot_creation_error in test_decoding, has mentioned the potential output
instability.
Author: Zsolt Parragi <zsolt.parragi@percona.com>
Author: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAN4CZFOoiKa9E4Gnh_OzLk3S5Z6RFyg_oY1e_WddCVYhkpTVow@mail.gmail.com
Backpatch-through: 15
preprocess_expression skips flatten_join_alias_vars for TABLESAMPLE
clauses, on the assumption that they cannot contain Vars of the
current query level. That's not true once a LATERAL subquery
containing a sampled relation has been pulled up: the relation's
TABLESAMPLE clause then becomes a lateral reference of this level, and
if it referred to a join alias Var, that Var is left unexpanded.
extract_lateral_references then pulls the join alias Var out as a
lateral reference and fails with "no relation entry for relid N".
To fix, treat such clauses like lateral function and VALUES RTEs:
introduce EXPRKIND_TABLESAMPLE_LATERAL, used for the TABLESAMPLE
clause of a lateral relation RTE, and keep skipping the alias
expansion only for the non-lateral kind.
Back-patch to all supported branches, as this dates back to the
introduction of TABLESAMPLE.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Ewan Young <kdbase.hack@gmail.com>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4834TwOtyyycyvja3MuOBkfpXy3dFAL0zt_vcM1KtqV-Q@mail.gmail.com
Backpatch-through: 14
If a subquery has been flattened into its parent, the joinaliasvars
entries of a join above it can be arbitrary expressions rather than
plain Vars, so expanding a reference to such a join alias may insert a
SubLink into a lower-level subquery. flatten_join_alias_vars_mutator
detects that and sets the subquery's hasSubLinks flag, but only in the
single-column code path; the whole-row path just asserted in a comment
that its recursive call would handle this, which is true only when the
alias entry is itself a Var referencing another join.
Hence a whole-row reference to such a join appearing in a sub-select
left that sub-select's hasSubLinks false, so preprocess_expression
skipped SS_process_sublinks for it, and the unprocessed SubLink
reached code that is not prepared for one: this produced "cannot
handle unplanned sub-select" from cost_qual_eval, an assertion failure
in preprocess_aggrefs, or "unrecognized node type" at execution,
depending on where in the sub-select the SubLink ended up.
To fix, make the same check in the whole-row expansion path.
Back-patch to all supported branches.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49PgeNFhQTZ2gSzwatF0_LXyegMPGt++jSfxUN7NZuQ5Q@mail.gmail.com
Backpatch-through: 14
In an in-tree build, "make coverage-html" invoked lcov with two
"-d ." options, because $(srcdir) is just ".". With lcov 2.0
(and perhaps earlier versions?) that results in scanning every
foo.gcda file twice and producing counts twice what they should
be. With later lcov versions (at least 2.3), it results in
"duplicate file" errors.
In VPATH builds we do need two -d options to cover both regular
and generated .c files. But in in-tree builds it suffices to
specify "-d .", so suppress the second switch.
Author: Narek Galstyan <narek.galstyan@enterprisedb.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAHsn6_xCDQWe8_vVFhtFk27_xTdyVV+Dr0yWzaooBZ6+-VH-5w@mail.gmail.com
Backpatch-through: 14
When repacking a table concurrently, some border conditions can cause
a concurrent update to propagate the original old tuple to the transient
table (such as a trigger that returns OLD). That's essentially just
wasted space, which REPACK is there to remove. Prevent that by setting
the isnull flag to true for any column cataloged as dropped in tuples
being "replayed" as concurrent updates.
Reported-by: Radim Marek <radim@boringsql.com>
Reviewed-by: Antonin Houska <ah@cybertec.at>
Backpatch: 19
Discussion: https://postgr.es/m/CAJgoLkK2UBzB1J9buCsSUbjf7bOqz-o_0=CeiTruU789BhBw-Q@mail.gmail.com
When the index cannot be used to find a recently deleted row,
RelationFindDeletedTupleInfoSeq() scans the table. It looked up the
relation's replica identity or primary key again to determine which
columns to compare, which could differ from the index selected when the
relation was opened. A concurrent DROP INDEX CONCURRENTLY could remove the
replica identity in between, so the search compared the whole row and
reported the conflict as update_deleted instead of update_missing.
Use the index recorded in the relation map instead. The relation map
already records the replica identity index, or the primary key if there is
no replica identity. It also avoids using a deferrable primary key, which
cannot serve as a replica identity; using its columns could incorrectly
report a deleted row with different values as update_deleted instead of
update_missing.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Author: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Zhijie Hou <houzhijie22@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CABdArM5ydwdRrpaZyK1q2p3-vY_+pnBtTmkvg_pcM=gHwmH7Kg@mail.gmail.com
Discussion: https://postgr.es/m/OS7PR01MB1831779ED93F5470CA1D57605F58C2@OS7PR01MB18317.jpnprd01.prod.outlook.com
Backpatch-through: 19
dumpIndex() emits ALTER INDEX .. SET STATISTICS for per-column
statistics targets, but dumpConstraint() never did. Targets set on the
expression columns of an exclusion constraint's index were silently
lost on dump/restore and pg_upgrade.
The same code can be used in the index and constraint dump code to
generate these ALTER INDEX commands. The logic has been refactored so
that both share a new subroutine in charge of generating all these
custom SET STATISTICS, if any.
Oversight in e4fca461ab.
Author: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Andrew Krylosov <krylosov.andrew@gmail.com>
Discussion: https://postgr.es/m/CAN4CZFMd9riOuV5LkM+uV=uF-HYp49C8Fh22xWyMxptGLBCZ3A@mail.gmail.com
Backpatch-through: 14
FetchStatementTargetList() dereferenced the portal named by a
FetchStmt without checking that it still exists. With the extended
query protocol, a client can sent a Parse "FETCH n FROM c" while cursor
"c" is open (which caches a result's TupleDesc), issue a CLOSE on cursor
"c", and then Describe the prepared statement or a portal bound to it.
Describe would re-resolve the target list through the named cursor, and
crash. In a non-assert builds, this causes a NULL pointer dereference.
An assertion was triggered in assert builds slightly before the pointer
dereference.
This commit changes the describe of a FetchStmt to match with the
ExecuteStmt case: if the named cursor's portal is invalid, fail rather
than assume that the portal should always exist.
A test case is added to libpq_pipeline, backpatchable all the way down.
Author: Dirkjan Bussink <d.bussink@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/7DE248EB-7509-4F8A-9F37-4694A40BBAB7@gmail.com
Backpatch-through: 14
Commit 1ff3180ca0 made a parallel autovacuum leader propagate changes
to the cost-based delay parameters to its workers only from
vacuum_delay_point(), which it no longer reaches once it is waiting in
WaitForParallelWorkersToFinish(). While waiting, the leader neither
processed a config reload nor noticed a change in the number of
autovacuum workers sharing the cost limit, so its workers kept running
with stale parameters until they finished.
Fix by refreshing and propagating the parameters on every wakeup of
that wait loop, and by setting the latches of the balanced autovacuum
workers when their count changes so that a waiting leader wakes up.
A regression test will be added in a separate commit.
Backpatch to v19, where parallel autovacuum was introduced.
Reported-by: Nikolay Samokhvalov <nik@postgres.ai>
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Manu <manuelreyesbravo@gmail.com>
Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAM527d-GL=Jp2EJXBSnVBGPK-4XEZwWof5Cv8P0hghS_og6oAg@mail.gmail.com
Backpatch-through: 19
Commit 7164dcb665 added the repack_decode isolation test, whose
s1_decode step counts every change decoded from the slot. A concurrent
autovacuum transaction that modifies only system catalogs is decoded
as an empty BEGIN/COMMIT pair, so the count could come out larger
than expected.
Fix by passing skip-empty-xacts to test_decoding. We pass it, along
with include-xids=0, to s1_decode_updates as well so that both steps
use the same options for consistency, though that step shows only
UPDATE lines and wasn't affected.
Per buildfarm animal skink.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/d704cde4-b1f2-48dc-8c5e-659d1128d286@gmail.com
Backpatch-through: 19
There have been several recent complaints that << and >> don't behave
the same way on all supported platforms. There is a risk that <<
overflows the signed integer type, which is undefined behavior, and there
are platform-dependent behaviors from shifting by numbers of bits >= the
bitwidth of the type, and also from shifting by negative values.
Document this to warn unsuspecting users of these facts.
Author: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAApHDvr0O6ZF8NO=Rq9Q+Sz=FHT=NZ1kkNTXZDcNph-OWuk7TQ@mail.gmail.com
Backpatch-through: 14
Canada's Northwest Territories will move to permanent UTC-06 on
2026-11-01. For now, assume their TZ abbreviation will be CST from
that time forward. This affects only America/Inuvik as the rest of
the territory is covered by America/Edmonton, for which the
equivalent change was released in 2026c.
Manitoba (America/Winnipeg) will move to permanent UTC-05 on
2026-10-31. For now, assume their TZ abbreviation will be EST
from that time forward.
(Upstream notes that more changes may be needed soon if smaller
jurisdictions choose to break away from what these have done,
but for now the plan is wait-and-see.)
Historical corrections for Colombia, Iran, and Ireland.
The backward-compatibility zone names EST5EDT, CST6CDT, MST7MDT, and
PST8PDT now conform better to POSIX. For example, EST5EDT now always
uses the abbreviation "EST" for standard time (now always 5 hours
behind UTC) and "EDT" for daylight saving time, whereas it formerly
had different UT offsets before standard time was introduced and
sometimes used abbreviations like "LMT", "EWT" and "EPT", all contrary
to POSIX. Also, though not specified by POSIX these names now use US
federal rules rather than rules of specific places like New York,
reverting to 2024a behavior. This change affects only timestamps
before 1966-10-30 at 01:00 standard time.
Backpatch-through: 14
Commit 0785d1b8b wrote
jsonapi_shlib.o: override CPPFLAGS += -DJSONAPI_USE_PQEXPBUFFER
forgetting that GNU make treats this change to CPPFLAGS as affecting
not only jsonapi_shlib.o, but anything that gets built in pursuit of
building jsonapi_shlib.o. Because of an even more ancient hack
(cf 7143b3e82), that will include jsonapi.o if it's not been built
already. If we do apply this -D switch to jsonapi.o, it gets built
with the wrong expansible-string infrastructure, which manifests
immediately as a link failure in pg_combinebackup and would misbehave
at runtime even if the build somehow completed.
Perhaps remarkably, we'd not seen field reports of build failures
till now, but it's pretty easy to reproduce a mis-build by trying
to build src/common/ with --shuffle and a high -j factor.
To fix, don't modify CPPFLAGS this way; instead make a dedicated
build rule for jsonapi_shlib.o that injects the additional switches.
There might be other places with similar problems, but some desultory
attempts with -j32 --shuffle on a 32-thread CPU didn't find any more
build failures. Also, it looks like the meson build infrastructure
lacks this problem (but I'm no meson expert).
Bug: #19727
Reported-by: Ross Burton <ross@burtonini.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Ross Burton <ross@burtonini.com>
Discussion: https://postgr.es/m/19727-32d67d9dea410f38@postgresql.org
Backpatch-through: 18
The explanation of aggressive scans claimed that if the multixact
members grows to 2 billion entries, we launch autovacuum even if it's
nominally disabled. That's no longer true; anti-wraparound vacuum is
only triggered if we're in danger of running out of multixids, we no
longer launch it just to shrink the 'members' SLRU. There's no
wraparound danger with 'members' anymore, so it's no worse than if a
user table becomes very bloated.
We do still adjust the freeze threshold though, so that if autovacuum
runs, whether on schedule or as an emergency autovacuum to avoid
wraparound, it will freeze more aggressively if the members SLRU is
large.
I removed the mention of the 4 billion member threshold, even though
that still exists, because the explanation felt misleading. The
aggressiveness is scaled proportionally between 2 and 4 billion
members. At 4 billion it's as aggressive as it gets, but there's no
sudden cliff.
In the passing, fix some comments. Remove obsolete mention of safe
threshold for member storage triggering autovacuum. Secondly, the
freezing cutoff is adjusted to avoid the *members* SLRU from growing
too large.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://www.postgresql.org/message-id/20260827231757.78.noahmisch@microsoft.com
Backpatch-through: 19
ShmemRequestInternal() rejected SHMEM_ATTACH_UNKNOWN_SIZE whenever
IsUnderPostmaster was false. That also rejected an after-startup
attachment in a standalone backend, even if the area exists.
To fix, check whether the shared memory has been initialized
(ShmemIndex == NULL) rather than IsUnderPostmaster.
Refactor the tests, and add more test cases for after-startup
attachment, with SHMEM_ATTACH_UNKNOWN_SIZE and a mismatching size, and
for this single-user case.
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discusssion: https://www.postgresql.org/message-id/CAJTYsWXREGXCnWdZJsXt8pT2UY_Y2GSx+HOUu2X3s8YDCLj0_A@mail.gmail.com
Backpatch-through: 19
ReplicationSlotRelease() copied MyProc->statusFlags unconditionally
when resetting PROC_IN_LOGICAL_DECODING, using its pgxactoff to decide
which area to update.
Slot invalidations can be done by the checkpointer or the startup
process, and ReplicationSlotRelease() ignored the fact that pgxactoff is
only meaningful for processes in the proc array, that auxiliary
processes never join. pgxactoff would remain at 0 for these, meaning
that an incorrect set of status flags would be updated.
The consequences of this issue could be various, depending on the
statsFlags lost in the proc slot incorrectly overwritten. Losing
PROC_AFFECTS_ALL_HORIZONS has been mentioned as the worse case, causing
VACUUM to remove rows a standby may needs even if standby feedback is
enabled. Assertions failures are another possibility.
Let's skip the update of statusFlags unless PROC_IN_LOGICAL_DECODING is
set. Nothing but StartupDecodingContext() sets it, and no auxiliary
processes call it.
Author: Vlad Lesin <vladlesin@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/889a06dd-45fc-423b-9dd2-87d5b5dc60c6@gmail.com
Backpatch-through: 14
SJE didn't consider the possibility that two RTEs referencing the
same table have different securityQuals, and might choose to merge
them anyway, thereby possibly losing quals that need to be enforced.
This is a regression introduced by 2ebf25e7d, since before that we
didn't re-generate baserestrictinfo lists from the RTEs' securityQuals
after performing SJE.
To defend against this, only consider SJE between RTEs with the same
checkAsUser values. That solves the problem because the set of
applicable RLS policies depends only on the role that is considered
to be accessing the table, so that the securityQuals must be equal
if the checkAsUser values are. It might also keep us from creating
similar bugs if we ever invent other features that depend on the
accessing role. And it's a lot cheaper than comparing securityQuals
trees themselves would be, not least because we can't sort them so
the preliminary sort step wouldn't help.
(The first proposed solution was to not perform SJE at all on RTEs
with nonempty securityQuals, but that seems rather sad, especially
since such cases worked before 2ebf25e7d. Making the restriction
depend on checkAsUser seems much less likely to interfere with SJE
unnecessarily, since in most cases that'll be the same for all
potentially-mergeable RTEs.)
The added test cases show that SJE is rejected when necessary, and
also demonstrate two quirks of this implementation. One is that if
a removed security qual includes an InitPlan, we'll still attach the
now-unused InitPlan to the plan. That's because SS_process_sublinks
runs before self-join elimination, so the extra InitPlan has already
been made. The other quirk is that we won't merge RTEs having zero
and nonzero checkAsUser fields, even if the calling user matches the
nonzero checkAsUser value so that there is no difference for this
user. That's intentional so that SJE doesn't require having to mark
the plan as caller-dependent.
Reported-by: Yonghwa Lee <underdog@theori.io>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/20260926174457.13.noahmisch@microsoft.com
Backpatch-through: 18
The help text for -c option documents that zero for either transaction
ID means to leave the corresponding value unchanged. Commit
978cf02bb8 replaced the separate checks for the two values with a
single flag indicating that the option was specified, causing zero to
be written to the control file instead.
Also parse the second transaction ID with strtouint32_strict(), like
the first one. Using strtoul() there was an oversight in commit
636c1914b4, which intended to reject negative and out-of-range
arguments.
Author: Chao Li <lic@highgo.com>
Discussion: https://www.postgresql.org/message-id/F61E486B-746B-42F3-A532-B8D4F1519583@gmail.com
Backpatch-through: 19
