Those changes were intended to avoid dependencies on the global libc
locale settings by basing identifier casefolding on the default
locale. They introduced an unintentional behavior difference for the
builtin provider with locale C and a single-byte encoding.
Instead of fixing that edge case, just revert the changes. The
identifier casefolding behavior has always been based on the global
libc locale, and clients make an effort to use the same rules. Until
we have a plan to change that, provider-specific identifier
casefolding doesn't make sense.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20260826225510.10.noahmisch@microsoft.com
Backpatch-through: 19
The general nature of this interaction has been understood (at least
by me) since before the original commit, but I failed to explicitly
document it. I also left behind an XXX in the README which I intended
to remove before commit. Fix those things.
Reported-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/CA+TgmoYmXy-jiP5qDhqNEiYFEBzQsArO6O2d9E8szNZqi1bePQ@mail.gmail.com
Commit ac58465e06 moved the discussion on clustering from CLUSTER's
refpage to REPACK's, but somehow introduced a mistake regarding when
exactly can the seqscan-and-sort strategy be used with respect to btrees
(spoiler: it was 100% backwards). Fix that, but also restructure and
reword the whole subsection more fully to try and make it more coherent.
Author: Chao Li <li.evan.chao@gmail.com>
Author: Álvaro Herrera <alvherre@kurilemu.de>
Reported-by: Antonin Houska <ah@cybertec.at>
Backpatch-through: 19
Discussion: https://postgr.es/m/69CA7636-0359-42B6-B971-16A036698187@gmail.com
Explain that returning false for TM_SelfModified follows ExecLockRows(),
which skips such tuples. This differs from TM_Updated, which is
unexpected in READ COMMITTED with FIND_LAST_VERSION.
Also remove "fall through to error" from the TM_Updated comment, since
the code calls elog(ERROR) directly.
Reported-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAA4eK1Lk1DfgiFPL-HccZm-Nm+exg+FcsODH8KPw6BOBuzfHtA@mail.gmail.com
Backpatch-through: 19
Commit b7b27eb41a added batching to the direct-index fast path for
foreign key checks introduced by 2da86c1ef9. Instead of probing the
referenced index once per row, it accumulated referencing rows and
checked them in groups, using SK_SEARCHARRAY for single-column foreign
keys.
The batching requires state to survive across trigger invocations and
to be flushed at the end of each trigger-firing cycle. Follow-up work
has had to define how that state interacts with nested trigger firing,
subtransactions, deferred constraints, and SET CONSTRAINTS.
Failure to handle one of those cases can leave a buffered check
unperformed, allowing a transaction to commit a permanent foreign key
violation without reporting an error. With PostgreSQL 19 close to
release, there is not enough time to gain confidence that all relevant
trigger and transaction states have been covered.
Remove the batching and its after-trigger callback infrastructure,
including the per-batch RI cache and associated subtransaction cleanup.
Restore the early exit in AfterTriggerFireDeferred(), since batch
callbacks can no longer queue additional deferred triggers after
afterTriggerInvokeEvents() returns.
Remove tests that exercise only the batching implementation and its
callback and cache lifetime machinery. Retain tests that continue to
exercise the underlying per-row fast path, including validation,
scan-key construction, cross-type rechecks, deferred checks, and
metadata invalidation.
Keep the underlying per-row fast path. It performs each check
synchronously and retains no pending checks across trigger invocations.
Also retain the fixes made to the per-row probe path, including support
for domain-typed referencing columns, restriction to btree referenced
indexes, concurrent index replacement, metadata invalidation, and
nullable referenced keys.
This removal applies only to REL_19_STABLE.
Discussion: https://postgr.es/m/CA+HiwqEDcCSDoYTtoXH4Ta_BpGN+revMZSLLj6GRz2_5ybwE1g@mail.gmail.com
ri_FastPathCheck() acquired its scan snapshot before opening the
referenced relation. If it then waited for the relation lock in READ
COMMITTED mode, a referenced row committed during the wait would not be
visible to the old snapshot. The check could consequently report a
foreign key violation even though the referenced row existed.
The batched path acquires its scan snapshot after opening the relations,
so it does not have the lock-wait problem. The per-row path remains
reachable during constraint validation and checks re-entered from a
batch flush.
Take the snapshot after opening the referenced relation and reloading
the constraint information. This also agrees with the SPI path, which
acquires the referenced-relation lock before selecting the snapshot
used for the check.
Also make the scan snapshot active for the duration of the check, so
STABLE cast and equality functions use the same snapshot as the index
scan. Otherwise, a STABLE cast can miss rows inserted by earlier AFTER
triggers and cause a valid foreign key to be rejected. Add a
regression test comparing this case with the SPI path.
Finally, also advance the command counter after acquiring the
referenced-relation lock and before reloading the constraint
information, so local invalidations are processed under the lock, as in
the SPI path.
Add an injection-point test that pauses a per-row check before locking
the referenced relation, commits the missing referenced row in another
session, and verifies that validation succeeds. Force per-row
validation by using a role without SELECT privilege on the referenced
table, so the test exercises this path even with batching enabled.
Add isolation-test coverage for the visibility of a referenced row
committed after the referencing transaction has executed an earlier
command. A later command can see such a row in READ COMMITTED, but not
in REPEATABLE READ or SERIALIZABLE.
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CA+HiwqEhm+_=bs=2wavAJz-UqC+1KebD31++mapJQQGweE8iQQ@mail.gmail.com
Backpatch-through: 19
Commits 630706ced0 and 9c8de15969 lost some encoding-specific nuances:
in libc with EUC encodings, or in ICU with any non-UTF8 encoding, we
cannot easily classify characters, so we must assume they are
case-varying. Restore these behaviors.
Previously, ILIKE using an index scan could cause wrong results.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20260826221526.89.noahmisch@microsoft.com
Backpatch-through: 19
pg_stat_progress_cluster translates REPACK into VACUUM FULL when
pg_stat_progress_repack.repack_index_relid is zero, and into CLUSTER
otherwise.
Previously, repack_index_relid was set only when an index scan was used.
When REPACK used a sequential scan and sort to order the table by an index,
the field remained zero even though index ordering had been requested. As a
result, pg_stat_progress_cluster incorrectly reported the command as
VACUUM FULL instead of CLUSTER.
Fix this by setting repack_index_relid to the OID of the requested ordering
index when progress reporting starts, regardless of the scan method. This
allows pg_stat_progress_cluster to correctly report REPACK as CLUSTER when
index ordering is requested.
Also update the documentation to clarify the meaning of the index columns
and how REPACK is translated into CLUSTER or VACUUM FULL.
Suggested-by: Fujii Masao <masao.fujii@gmail.com>
Author: Chao Li <lic@highgo.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com>
Discussion: https://postgr.es/m/CAN4CZFMzy2V_wvRCBW5K8=wVCk1-C7nq=8otXO=u+s_1KZHqyA@mail.gmail.com
Backpatch-through: 19
Previously, when another session ran REPACK, CLUSTER, or VACUUM FULL and
the user lacked permission to view its progress details,
pg_stat_progress_cluster incorrectly reported the command as CLUSTER
instead of NULL.
pg_stat_progress_cluster is based on pg_stat_progress_repack and
translates REPACK into CLUSTER or VACUUM FULL depending on whether index
ordering is requested. However, this translation defaulted to CLUSTER,
incorrectly converting NULL commands returned for insufficient privileges
into CLUSTER.
Fix this translation to preserve a NULL command when the user lacks
permission to view the progress details.
Bump catalog version.
Author: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Chao Li <lic@highgo.com>
Discussion: https://postgr.es/m/CAN4CZFMzy2V_wvRCBW5K8=wVCk1-C7nq=8otXO=u+s_1KZHqyA@mail.gmail.com
Backpatch-through: 19
A clause connecting the same two EC members can be requested from the
EC machinery in different contexts. When building parameterized index
paths for a relation, generate_implied_equalities_for_column derives
an indexable clause with parent_ec set, to mark it as redundant with
other join clauses derived from the same EC. Meanwhile, at a join
where the relations of a multi-relation EC member first come together,
generate_join_implied_equalities emits a clause equating the same two
members with parent_ec unset, since that clause is the sole enforcer
of its member there and must not be dropped as redundant. Because
create_join_clause treats parent_ec as part of its cache-lookup key,
such requests can yield two different RestrictInfos for the same
condition, carrying different rinfo_serials.
This breaks the assumption that one condition has exactly one serial
number, which create_nestloop_path relies on to drop join clauses that
are already enforced within a parameterized inner path. As a result,
the same condition could be enforced twice at different plan levels,
wasting execution effort and applying the clause's selectivity twice,
underestimating the join's row count.
To fix, make create_join_clause copy the rinfo_serial from an existing
clause that connects the same two members with the opposite parent_ec
marking, so that the two clauses keep their distinct markings but are
recognized as the same condition.
As a result, a duplicate join filter disappears from one of the
existing regression plans (the "cardinal_number" query in join.sql);
the other plan changes are join-order differences between plans of
essentially equal cost.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAMbWs4-3yZeBMGQDpmYRPdV-sL_=6H2AN=MoeneOT+VUk28PPg@mail.gmail.com
Backpatch-through: 19
Commit f836b688f ensured that at most one clone of a join clause is
chosen for a parameterized path, so long as the clone variants are
parse-tree distinct. Two ways remained to end up enforcing the same
condition more than once, which wastes execution effort and applies
the clause's selectivity multiple times, underestimating the row
count.
If a commuting outer join nulls no Var referenced by a particular
qual, the clone variants of that qual are parse-tree identical, and
the incompatible_relids test cannot tell them apart. To fix, enforce
just the first surviving variant, identified by rinfo_serial.
The other case is in get_joinrel_parampathinfo, where the
clause-recovery pass for dropped EquivalenceClasses queries the EC
machinery with a context that overlaps the preceding loop's, so the
derived-clause cache can hand back a clause that that loop already
accepted. To fix, skip any clause that is already present in the
collected list.
Also add assertions verifying that the clauses to be enforced at a
join or within a parameterized path contain no duplicate rinfo_serial,
to catch any remaining or future violations. These assertions are
added in master only, since hard prevention of duplicate clauses is a
new goal and other unfixed cases may remain.
Back-patch to v19, as with commit f836b688f.
Author: Richard Guo <guofenglinux@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAMbWs4_Ezb1Lnj7BqcTQSysC6B5UFZxfhib8xr6rqpu5uqr8dA@mail.gmail.com
Backpatch-through: 19
Without CHECK_FOR_INTERRUPTS(), backends might not respond to query
cancellations in a timely fashion, especially for JOIN_ORDER()
specifications, where matching against each join can take O(n^2) time
in the length of the join order list in degenerate cases.
Recursive functions should check_stack_depth().
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CA+TgmoYmXy-jiP5qDhqNEiYFEBzQsArO6O2d9E8szNZqi1bePQ@mail.gmail.com
Backpatch-through: 19
The SIMD path for this command reads ahead via CopyLoadInputBuf()
whenever fewer than sizeof(Vector8) bytes remain in the input
buffer, even if those bytes hold a complete end-of-copy marker.
If the input is a pipe whose writer has sent the marker but not
closed the pipe, that read blocks, and COPY waits for data it will
never use. The scalar loop asks only for the bytes it needs, so
it stops at the marker without reading any further.
To fix, teach CopyLoadInputBuf() to decline a speculative load
when the caller's remaining bytes contain a backslash, which in
text mode might begin such a marker. (CSV mode doesn't treat
\. as special, so it is unaffected.) The SIMD path then hands
those bytes to the scalar loop, as it already does when a load
comes up short. Checking for the marker in the SIMD helper itself
would be more direct, but a call there costs the compiler
registers on every line, which measurably slowed COPY of short
lines in testing.
Oversight in commit e0a3a3fd53.
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Greg Burd <greg@burd.me>
Discussion: https://postgr.es/m/CAN55FZ1qFb4Yo3-MWeQfGQnu1_Ksx4xcFC2m3hyw8a--%2BeHQYQ%40mail.gmail.com
Backpatch-through: 19
Commit 86f7c82cf1 reported the user catalog table restriction as a
detail, but the other checks in check_concurrent_repack_requirements()
were still a mix: some were hints, and the details repeated
"REPACK (CONCURRENTLY)", which the primary message already names.
Report them as details with the same "This operation ..." wording. The
one actionable hint, ALTER TABLE ... REPLICA IDENTITY USING INDEX,
stays.
Backpatch to v19, where REPACK (CONCURRENTLY) was introduced.
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Suggested-by: Alvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/apBIFWzHYOaG0auN%40nathan
Backpatch-through: 19
DROP TABLESPACE checked pg_shdepend for dependent objects without
first locking the tablespace. A concurrent command that recorded a
shared dependency on the tablespace right after that check could still
commit, leaving an object whose pg_shdepend entry (or, for a relation,
pg_class.reltablespace) pointed to a tablespace that no longer existed.
Close the race by having DropTableSpace() take an AccessExclusiveLock
on the tablespace before calling checkSharedDependencies(). That
conflicts with the AccessShareLock shdepLockAndCheckObject() takes when
recording a new dependency, so the loser of the race blocks and
rechecks once the winner commits.
That AccessExclusiveLock creates a new deadlock: ALTER TABLESPACE
RENAME/SET and the internal ACL/owner updates in DROP OWNED and
REASSIGN OWNED touched the catalog tuple without locking the
tablespace, risking a lock-order cycle with DROP. Fix by taking an
AccessShareLock first in all four paths, rechecking pg_shdepend after
any wait in the DROP OWNED and REASSIGN OWNED cases. GRANT, REVOKE,
and ALTER TABLESPACE ... OWNER TO already lock the object.
Add isolation tests covering both orderings of the original race
(dependency-first and drop-first) and all four previously-unlocked
update paths.
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/CAJTYsWXjAQFnGKzXsht3XK8KHhyhontwFJw4JAHsUSfs_ptR4g@mail.gmail.com
Backpatch-through: 14
Previously, running REPACK (CONCURRENTLY) on a materialized view failed
with a confusing "has no identity index" error.
REPACK (CONCURRENTLY) replays changes decoded from WAL, but a
materialized view produces none, since it cannot be modified by DML and
REFRESH rewrites it through a transient heap that is not logically
decoded.
Fix this by reporting a clear error for materialized views upfront,
alongside the existing checks for system catalogs and TOAST tables, and
document the limitation.
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Kiran Kaki <itskkpg@gmail.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/CALj2ACXPvy4_LkQ0nHKgbZjAGK9fkDqRYUvXA3hgwxPm_X7Eng@mail.gmail.com
REPACK (CONCURRENTLY) is not an MVCC-safe operation; it doesn't
preserve the visibility information, which logical decoding needs
because it reads user catalog tables under a historic snapshot.
Disallow REPACK (CONCURRENTLY) on user catalog tables. Removing this
check requires making it MVCC-safe and logical rewrite mappings.
Backpatch to v19, where REPACK (CONCURRENTLY) was introduced.
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Antonin Houska <ah@cybertec.at>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/apBIFWzHYOaG0auN%40nathan
Backpatch-through: 19
REPACK (CONCURRENTLY) launches a background worker to decode changes
made while the table is being rewritten. The worker connects as the
table owner but bypassed the LOGIN check only, so CONNECT was still
checked against a role that need not have it, and the command could
fail with "permission denied for database".
Pass BGWORKER_BYPASS_ALLOWCONN as well, as we do for parallel
workers. That is safe because the leader already checked the invoking
user's privileges on the table before starting the worker.
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Author: Matheus Alcantara <mths.dev@pm.me>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/apBbzFd_EYAfHV45@nathan
Backpatch-through: 19
Commit 266543a made initial table synchronization use "COPY table TO"
for a partitioned table, but decided that from the relation kind
alone. A partitioned table is only accepted there since v19, so a v19
subscriber replicating from an older publisher with
publish_via_partition_root failed at table sync.
Check the publisher version, and fall back to the "COPY (SELECT ...)
TO" variant otherwise.
Backpatch to v19, where 266543a changed the initial table
synchronization.
Reported-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20260905040020.57.noahmisch@microsoft.com
Backpatch-through: 19
There's not really any good reason to store an empty advice string
in an advice stash, but it's entirely possible that some people might
do it by accident. If they do that without this fix, restarting the
system will result in failure while reloading pg_stash_advice.tsv.
The root cause is that I (rhaas) did not think clearly enough when
defining pgsa_next_tsv_field(). After returning each field on the
line, it set *cursor to the start of the next field, and after
returning all of them, it set *cursor to the trailing NUL byte. But
when the last field on the line is empty, this is ambiguous: if we're
pointing at the trailing NUL byte, it could be either because it's the
start of the zero-length field or because we're out of fields. To
remove the ambiguity, this commit decides that *cursor will instead
be set to NULL when there are no more fields.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20260827171830.68.noahmisch@microsoft.com
Backpatch-through: 19
ExecInitModifyTable() evaluated ForPortionOfExpr.targetRange
unconditionally at executor start-up, with no EXEC_FLAG_EXPLAIN_ONLY
guard. This caused three problems:
- EXPLAIN (GENERIC_PLAN) failed with "no value found for parameter 1".
- Plain EXPLAIN with FOR PORTION OF (null) raised "FOR PORTION OF
target must not be null".
- Plain EXPLAIN ran user functions appearing in the target a second
time, beyond the one evaluation the planner already does for
selectivity estimation.
Now we skip the evaluation and the accompanying null check in
explain-only mode.
Reported-by: Noah Misch <noah@leadboat.com>
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/flat/CA%2BrenyXci9khYXJ8uSwzF30DVw%2Bzc4QwGb1RdsuOu_YtrFuy1g%40mail.gmail.com
The current implementation doesn't support it correctly, leading to
mysterious warnings and some worse problems. Have the combination throw
an error.
This is intended as a temporary stopgap until we can implement it in a
future release, as it's a useful feature.
Author: Fujii Masao <masao.fujii@gmail.com>
Author: Antonin Houska <ah@cybertec.at>
Author: Osama Abdul Qader <osamaabdulqader.cs@gmail.com>
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/apBCEl4hBHlq0axm@nathan
This reverts commit c06d1a4ba6.
The commit assumed that if the FSM code tolerates torn pages, everything else
does so. Readers that do not tolerate that include RelationCopyStorage(),
used by ALTER TABLE ... SET TABLESPACE for every fork, the read stream in
RelationCopyStorageUsingBuffer() used by CREATE DATABASE ... STRATEGY =
wal_log, and, most awkwardly, the checksum verification in base backups and
pg_checksums.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20260901211837.f6.noahmisch%40microsoft.com
Backpatch-through: 14
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
FOR PORTION OF valid_at FROM $1 TO $2 fails with "could not determine data
type of parameter $1", so using PREPARE requires users to say $1::date
etc. But transformForPortionOfClause() already coerces the bounds, since
it builds a constructor for the targeted range. We just need to store the
coerced expressions in targetFrom and targetTo.
This also improves pg_get_ruledef(), which now renders an untyped NULL
bound as NULL::date instead of NULL::unknown.
Reported-by: Noah Misch <noah@leadboat.com>
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/flat/CA%2BrenyW1uJp1dw%3DiK7MsWfbbZf-dKP_4JLoQ65wj%3Dfry35FjCw%40mail.gmail.com
On slow machines, this test can take long enough to generate WAL that a
time-based checkpoint occurs before the primary is stopped. If the
checkpoint record is written to the tail WAL segment that the test later
removes, a standby initialized from the resulting backup tries to read
the missing checkpoint record and fails with a PANIC during startup.
This caused the test to fail on buildfarm member skink.
Fix this by setting checkpoint_timeout high enough to prevent unrelated
checkpoints during the test. This follows the approach used by other
recovery tests, such as 043_no_contrecord_switch.pl, that depend on a
specific WAL layout.
Backpatch to all supported versions.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwFajfjKhvNSTkTcE-wFn30p_A0_i1cvk-Q=FJhDr_5cXA@mail.gmail.com
Discussion: https://postgr.es/m/9ffdb19a-7a89-424e-925a-dd981c37f0ba@gmail.com
Backpatch-through: 14
Commit 6168c65ddc added a test verifying that the conflict detection
slot's xmin regresses to the horizon pinned by an in-progress transaction
in the database of a newly added subscription that has retain_dead_tuples
enabled. However, it used query_until(), which can return before starting
a transaction, allowing the test to proceed without a pinned xid.
Use query_safe() so the transaction setup completes synchronously and the
xid is assigned before the xmin checks run.
Per buildfarm.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/OS9PR01MB12149257A97EE1BD3296F6AEBF5B12@OS9PR01MB12149.jpnprd01.prod.outlook.com
Backpatch-through: 19
When creating a nestloop plan for a partitionwise child join, the
outer rel's relids are child relids, but PlaceHolderInfo.ph_eval_at is
always expressed in terms of the topmost parent rels. As a result,
replace_nestloop_params() and identify_current_nestloop_params()
failed to recognize that a PlaceHolderVar evaluated at the outer child
rel can be supplied as a nestloop param. Instead, the Vars within the
PHV's expression were replaced with params, but the outer child rel
emits only the PHV, not those bare Vars, leading to "variable not
found in subplan target list" errors from setrefs.c.
To fix, also include the outer rel's top parent relids in the relid
set used for these checks, so that ph_eval_at comparisons are done in
terms of parent rels while Var checks continue to work in terms of
child rels.
On v18 and later, the required-outer set passed to
identify_current_nestloop_params() has the same problem: it is in
terms of child rels once a parameterized child join path has been
reparameterized by an upper child join. With the above fix in place,
a PlaceHolderVar that depends on both the outer rel and the parameter
source becomes a single NestLoopParam, and that param was never
claimed by any nestloop node, leading to "failed to assign all
NestLoopParams to plan nodes" errors. To fix, also include the top
parents of any child rels in that set. Older branches lack this code
path, so they receive only the first change.
Back-patch to all supported branches.
Bug: #19653
Reported-by: Annie <10215501441@stu.ecnu.edu.cn>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19653-9352cc6ba17b662f@postgresql.org
Backpatch-through: 14
List of commits reverted:
2f094e7ac6 SQL Property Graph Queries (SQL/PGQ)
cd8844e7db Make some tests more stable by adding more explicit ordering
182cdf5aea pg_dump: Add appropriate version check
c9babbc881 Dump labels in reproducible order
040a56be4b Cleanup users and roles in graph_table_rls test
5282bf535e Fix some typos and make small stylistic improvements
720f0f89d6 Reject consecutive element patterns of same kind
288ae96872 Add a graph pattern variable only once
c5b3253b8a Property references are preferred over regular column references
a0dd0702e4 Fix cross variable references in graph pattern causing segfault
2ff289d039 Check for stack overflow when rewriting graph queries
9082680c34 Fix typos and grammar in graph table rewrite code
ac3bcc041c Fix collation of expressions in GRAPH_TABLE COLUMNS clause
9d2979dd68 pg_get_viewdef() and lateral references in COLUMNS of GRAPH_TABLE
891a57c739 Do not define type for a property graph
dc9e7c9ed9 Handle nodes that may appear in GraphPattern expression trees
f6edd8ed70 Add ORDER BY to test query to stabilize test
1190f858ea doc: Small synopsis wording change for consistency
d0eac3cafb Make spelling consistent
6827de95ee Simplify code in objectaddress.c for some property graph objects
5778fb3eaf Fix typo in error message
72498a8698 Handle element label and label property objects in object address functions
4cb2a9863d Fix LATERAL references in GRAPH_TABLE with multi-label pattern
9d8cdcbe0c Record dependencies on graph labels and properties
2a7e95b659 Readable identity strings for property graph objects
ac0ad6a7c9 Prevent dropping the last label from a property graph element
36aae3d029 Fix handling of dropping a property not associated with the given label
cc9aa7f3a9 Resolve unknown-type literals in GRAPH_TABLE COLUMNS
d868736830 Make property graph object descriptions better translatable
45d4c917ff Fix properties orphaned by dropping a label
33bfad0f3c Remove apparent support for SECURITY LABEL ON PROPERTY GRAPH
ccca2cd81b Fix pg_dump ACL minimization for PROPERTY GRAPH.
7883ea13ff Fix replace_property_refs() ignoring the root of expression tree
b820c623dd Resolve unknown-type literals in property expressions
01c544e1af Prohibit locking clauses on GRAPH_TABLE
8d2beee027 Restore the ability to use | and -> as prefix operators.
d2ac26eb9b Fix missing space before WHERE in GRAPH_TABLE deparse
4054ec5b4f Prohibit GRANT ... ON TABLE on a property graph
77964322d6 doc: Add PROPERTY GRAPH to the access privilege tables
c5e11de2f3 More tab-completion for DROP PROPERTY GRAPH
8ce749f8f6 Disallow aggregates, window functions, and SRFs in GRAPH_TABLE COLUMNS
22d0eebfef Report duplicate property and label names with a proper error
41a300ae3d Make generate_queries_for_path_pattern_recurse() interruptible
58af213874 Fix inferred property graph keys with INCLUDE columns
63f8e9773a Fix pg_event_trigger_ddl_commands for GRANT ON PROPERTY GRAPH
07df2b2532 doc: reformat GRAPH_TABLE examples
1c8c790ec1 Coerce GRAPH_TABLE pattern WHERE clauses to boolean
plus release notes entries and typedefs.list changes.
Discussion: https://www.postgresql.org/message-id/CAAKRu_bEtjWYWhYxSo0o_t3DaZYRQd5PkC0abA0g9Mp4%2BovH0w%40mail.gmail.com
Since the retain_dead_tuples feature uses a single cluster-wide
replication slot across all subscriptions, adding a new subscription on a
database with an older active transaction can require
conflict_detection_slot.xmin to move backward to avoid missing dead tuples
needed for update_deleted conflict detection.
Previously, if a newly created subscription had an older active
transaction than the slot's current xmin, the slot update occurred during
worker apply, leading to an assertion failure.
Fix this by having the launcher track the set of databases with active
tuple-retaining subscriptions. When a database joins this set (either via
a new subscription or when retention restarts), the launcher resets the
slot's xmin to the cluster-wide safe decoding horizon before starting any
workers. Because this horizon accounts for all active transactions across
the entire cluster, it serves as a safe seed without requiring retroactive
backward movement by individual apply workers.
In passing, update the retain_dead_tuples documentation to clarify two
operational boundary cases:
(a) Conflict data continues to accumulate even when a subscription is
disabled or its apply worker is stopped.
(b) track_commit_timestamp must be enabled for conflict detection to
function properly.
Reported-by: Nisha Moond <nisha.moond412@gmail.com>
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Author: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/TY4PR01MB177182F547A62FC2666EC04EC94B72@TY4PR01MB17718.jpnprd01.prod.outlook.com
Backpatch-through: 19, where it was introduced
When outer-join identity 3 permits a join to commute with lower outer
joins, we generate multiple clone versions of its join clause, of
which only one should be applied in any given plan. When building a
join relation's restriction list, subbuild_joinrel_restrictlist
selects the appropriate clone by checking required_relids and
incompatible_relids, but no such selection was made for movable join
clauses pushed down into a parameterized path. As a result, a
parameterized scan or join could enforce more than one clone of the
same condition, shown by EXPLAIN as a duplicate qual. This wastes
effort evaluating the same condition repeatedly. What is worse, it
applies the clause's selectivity multiple times, underestimating the
result's row count.
To fix, in get_baserel_parampathinfo and get_joinrel_parampathinfo,
skip a clone clause if any outer join listed in its
incompatible_relids has already been computed below the point of
evaluation.
Back-patch to v19, but no further. While this bug goes back to v16,
applying the fix in released branches risks destabilizing plans that
users are currently happy with, and there have been no field
complaints.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-g5TFF5k=jfe3JVWqtv_cCTOMmQRUp0_E8fApgqcqBng@mail.gmail.com
Backpatch-through: 19
When neither BKPIMAGE_HAS_HOLE nor BKPIMAGE_COMPRESSED is set,
DecodeXLogRecord() checks that bimg_len needs to be equal to BLCKSZ.
However, the associated error message reported the data length
associated to a block (DecodedBkpBlock.data_len), and not the length of
the block (DecodedBkpBlock.bimg_len).
Oversight in 57aa5b2bb1, probably due to some copy-paste from the
surroundings.
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/CAJTYsWVz9ymE7aMt2ZiQvs1bxNvhjfTf+XaJEsdFw7MT5dvtYQ@mail.gmail.com
Backpatch-through: 14
When analyzing a foreign table pointing to a partitioned or inherited
table on the remote server, this method just imported relpages and
reltuples for the parent table from the remote pg_class, causing a
discrepancy or an error in results compared to the sampling method.
* In the partitioning case, while the sampling method sets relpages=0,
this method set relpages=-1. (For reltuples, the results of both
methods match.)
* In the inheritance case, while the sampling method sets
reltuples=(# of rows in the parent and child tables), this method set
reltuples=(# of rows in the parent table). (For relpages, the results
of both methods match.)
As relpages and reltuples are used for costing foreign paths, these
would lead to incorrect plans or plan changes depending on the method.
To fix, do the following:
* For the partitioning case, modify postgres_fdw to set relpages=0 to
match the sampling method.
* For the inheritance case, we could also modify it to set the correct
reltuples value by fetching reltuples for the child tables as well,
but 1) the change isn't small, and 2) table inheritance hasn't been
used that much these days, so fix by disabling this method.
While at it, adjust comments and docs slightly.
Oversight in commit 28972b6fc.
Author: Etsuro Fujita <etsuro.fujita@gmail.com>
Discussion: https://postgr.es/m/CAPmGK173b2Xvyoeq%3DY7L09H%2BajgMgxC2fYe%3D7CCunZz8tGTfFA%40mail.gmail.com
Backpatch-through: 19
If a LATERAL subquery references a PlaceHolderVar that must be
evaluated at an outer join, the subquery's lateral_relids include that
outer join's relid, since the lateral parameter is the outer join's
output. join_is_legal() verified that the rels named by a proposed
join's minimum parameterization could still be joined to from outside,
but it did not consider such outer-join relids. As a result, when
identity 3 permitted a commuted join order, we could approve a join
that includes part of that outer join's required input, even though
the outer join could then be completed only above the proposed join,
leaving the lateral parameter forever unsatisfiable.
In assert-enabled builds this tripped the backstop Assert in
try_nestloop_path(). In production builds the bogus join generated
only paths whose parameterization can never be satisfied, so they
could never appear in a complete plan. This not only wasted planning
effort, but also could mislead the clauseless-join heuristics into
thinking that legality of this join means that some other join rel
need not be formed, and that could lead to failure to find any plan at
all.
Such joins used to be rejected by the have_dangerous_phv() check in
join_is_legal(), which refused any join where a PlaceHolderVar's eval
set overlapped the other side of the join without being contained in
it. Commit a16ef313f removed that restriction, exposing this
oversight.
To fix, teach join_is_legal() to reject a proposed join if its minimum
parameterization includes an outer-join relid whose outer join cannot
be formed strictly outside the join, that is, when any rel needed to
form that outer join must become part of the proposed join's own join
tree.
Back-patch to v18, where commit a16ef313f removed the
have_dangerous_phv() restriction.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49ZveAYx9T5b0=gxn0pnvqq=MbN-Z4KLbLAE+kH06mWVA@mail.gmail.com
Backpatch-through: 18
The retain_dead_tuples feature waits for concurrent transactions to be
applied before advancing conflict_detection_slot.xmin. It gets the WAL
write position from the publisher and waits until the apply remote
position passes it. This ensures that dead tuples, commit timestamps, and
origins needed for conflict detection (update_deleted,
update_origin_differs, and delete_origin_differs) are retained while
those transactions are applied.
However, with asynchronous commit enabled on the publisher, a committed
transaction may not have updated the WAL write position yet. As a result,
the reported WAL position can lag behind the latest committed transaction.
This can cause conflict_detection_slot.xmin to advance prematurely,
allowing dead tuples needed for conflict detection to be removed before
subsequent asynchronously committed transactions are applied.
Report the end of the last inserted WAL record instead. A transaction
inserts its commit record before it is marked committed, so the insert
position cannot be behind a transaction that has already committed.
Transactions that have entered the commit phase but have not yet inserted
their commit record are still accounted for by the in-commit transaction
ID reported in the same message.
In addition to above, add missing trailing period to update_deleted
conflict detail messages which are added for the same feature.
Reported-by: Nisha Moond <nisha.moond412@gmail.com>
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/TY4PR01MB177182F547A62FC2666EC04EC94B72@TY4PR01MB17718.jpnprd01.prod.outlook.com
Backpatch-through: 19, where it was introduced
Previously, a session that had created temporary objects could hang during
exit if its default transaction mode was SERIALIZABLE READ ONLY DEFERRABLE
and a concurrent serializable transaction was prepared. During backend
exit, temporary-relation cleanup pushed a regular transaction snapshot,
which honored the session's default settings and could wait for a safe
serializable snapshot. So, if the conflicting transaction was prepared,
this wait could last indefinitely.
The wait occurred while the backend was already exiting and interrupts
were held off, so even pg_terminate_backend() could not cancel it. The
backend therefore remained until the prepared transaction was resolved.
Temporary-relation cleanup only needs an active MVCC snapshot for fetching
TOAST data from catalog tuples while dropping temporary objects. It does
not need a transaction snapshot affected by the user's default isolation
settings. So, use a catalog snapshot instead, which provides the needed
protection without entering the deferrable safe-snapshot wait.
Backpatch to all supported versions.
Bug: #19441
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Andrey Rachitskiy <pl0h0yp1@gmail.com>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/19441-ec29f3b1363b4a68@postgresql.org
Backpatch-through: 14
The buildfarm member akepa reported a failure in the
031_recovery_conflict.pl test.
The test checked pg_stat_database_conflicts immediately after detecting
a recovery conflict in the standby log. However, the conflict counter is
flushed by the canceled backend during backend exit, so WAL replay
completion and the log message did not guarantee that the updated
statistics are visible yet. So, previously, the test could see a conflict
counter of 0 even though the conflict had already occurred, triggering
the test failure.
Fix this by polling for the expected conflict counter instead of reading
it only once, handling the asynchronous pgstats update.
Per buildfarm member akepa.
Backpatch to v17, where this test is enabled and has the same race.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwHmiLNRfvJDAR=PmxQgf7DbzPSO1M-1RoqO8oy=t2G5KA@mail.gmail.com
Backpatch-through: 17
transformGraphElementPattern() and transformGraphPattern() ran their
WHERE clauses through transformExpr() without ever applying
coerce_to_boolean(), so a WHERE clause of any type was accepted and
its bare datum was used as the qual, which resulted in wrong results
without a diagnostic. Route both sites through
transformWhereClause(), like every other WHERE clause, and add
regression tests for the element-level and pattern-level cases.
Author: Ewan Young <kdbase.hack@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAON2xHNZUS8ZwzVEJFeRirsC_-7EJvdbwABSzdGaJ_DF9DvHoQ%40mail.gmail.com
Not many buildfarm members run this test, since it's gated behind
PG_TEST_EXTRA=xid_wraparound, but of those that do, the slower ones
not infrequently fail. The reason seems to be that the test expects
an INSERT command to either succeed or fail, but there's a window
where it can succeed while issuing a warning about impending
wraparound. poll_query_until treats nonempty stderr as a failure,
so it loops an extra time until the INSERT succeeds with no warning.
There seems no reason to treat this behavior as wrong, so adjust
the test to accept it.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Diagnosed-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/7a68bee2-91bd-481d-be44-160e6beebc83@gmail.com
Backpatch-through: 17
Commit f63ae72bbc converted postgresql.conf.sample to spaces so
that it lines up at any tab width, but replace_guc_value() still
pads with tabs when it re-aligns the trailing comment of a
rewritten setting. Fix that.
Oversight in commit f63ae72bbc.
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/apRezMywHRuR5h-6%40nathan
Backpatch-through: 19
Commit 44fb59fc6 taught the grouping-conflict walker to treat the arg
of a simple CASE as a direct operand of each WHEN comparison, but it
only checked the collation, on the assumption that the WHEN operator
is always the type-default "=" and thus matches the grouping eqop.
That assumption fails once the arg is relabeled to another type: the
WHEN then compares under that type's "=", which need not agree with
the grouping equality. For instance, with a DISTINCT over a citext
column, a qual such as "CASE t::text WHEN 'A' THEN ..." was pushed
below the Unique, although the equivalent "t::text = 'A'" is correctly
kept above it.
Instead of special-casing the arg, have the walker bind a Var arg
while walking the WHEN conditions and resolve each CaseTestExpr to it,
so that the arg is checked exactly as each WHEN uses it: with the
opfamily and collation checks of a direct operand when the WHEN is a
comparison, and as a non-operand reference otherwise. A non-Var arg
is walked once as a non-operand, as before. The CaseTestExpr in an
ArrayCoerceExpr's elemexpr and a JsonConstructorExpr's coercion stand
for something else and are left alone.
Reported-by: Tender Wang <tndrwang@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Ewan Young <kdbase.hack@gmail.com>
Discussion: https://postgr.es/m/CAHewXNkvGTOgijRLjmudpg=wz0d-J8ChY2o7ksh3Be+Q_Bxwog@mail.gmail.com
Backpatch-through: 18
When recovery starts from a backup without a signal file, pg_subtrans
is not started at the beginning of recovery and remains unstarted
throughout recovery. However, previously, a restartpoint run by
the checkpointer during recovery nevertheless tried to truncate
pg_subtrans, triggering the assertion failure:
TRAP: failed Assert("TransactionIdIsValid(initial)")
This commit fixes this by tracking whether pg_subtrans has been
started during recovery, and have the checkpointer check this flag
before truncating pg_subtrans at restartpoints.
Backpatch to all supported versions.
Reported-by: Imran Zaheer <imran.zhir@gmail.com>
Author: Imran Zaheer <imran.zhir@gmail.com>
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA+UBfamzfEReT0VOGRUW_=_AecCYQ-CNKLR_T4Q+YEmJAH3fdg@mail.gmail.com
Backpatch-through: 14
Several queries in the foreign_key test sort pg_constraint rows with
"ORDER BY oid::regclass::text". These OIDs are constraint OIDs, not
relation OIDs, so the regclass cast renders them as numeric strings,
and text comparison of numeric strings depends on digit count. If the
OID counter crosses a power of ten while the test runs, the output
order flips, causing spurious test failures. This cannot happen in a
fresh cluster, but installcheck against a long-lived cluster can hit
it.
To fix, sort by constraint name, which is unique within each of these
queries, and use COLLATE "C" so that the order does not depend on the
database's default collation either.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49KaJFGs4rfBWk+YmHPTXZ+pJTMJ12iaz7ruZdSK+sC5w@mail.gmail.com
Backpatch-through: 18
When reduce_outer_joins reduces an outer join to an antijoin, any IS
NULL qual on a Var from the antijoin's nullable side is necessarily
true. Previously, such quals were discarded later in
distribute_qual_to_rels, mainly to avoid bogus selectivity estimates.
But that discard was incomplete: the qual remained in the jointree,
while its Vars were not counted in attr_needed. Since commit
2ebf25e7d, join removal edits the jointree and expects it to contain
no other references to a removed rel, so it could remove a rel that
such a discarded qual still references, and then trip an assertion on
the qual's stale Var.
To fix, move this processing to an earlier phase: such quals are now
removed from the jointree by reduce_outer_joins itself. This way
later phases see a consistent query tree, and
check_redundant_nullability_qual is no longer needed, so remove it.
Back-patch to v16, as with commit 2ebf25e7d.
Reported-by: Tender Wang <tndrwang@gmail.com>
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAHewXNk8b0TsSy4dL=CO7FXL2W3WBm0BcdP-zwNJePa-Qj4HzA@mail.gmail.com
Backpatch-through: 16
Several invocations decoded and printed the full WAL range even though
the test only needs to confirm that a command form works or that
decoding reaches a specific error. Limit those checks to one record,
and start the fall-off-the-end checks near the end of the generated WAL.
This change reduces the IPC overhead overall, particularly on Windows,
without reducing coverage.
I am usually hesitant to backpatch such changes as this is only an
improvement, but the gains are too good in terms of IO and runtime, for
both the CI and the buildfarm. Based on the numbers provided, the CI
takes 30% less time to run the test with this change on Windows (worst
case shown on the lists). These tests have been introduced in
96063e2836.
Author: Sehrope Sarkuni <sehrope@jackdb.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAH7T-araSEdsNpxiKaMOW8kr_ZHhekCLHx5yzm2ksq4FdgULbA@mail.gmail.com
Backpatch-through: 17
