database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-14 08:21:07 +03:00
Code Activity

Still more fixes for planner's handling of LATERAL references.

Browse Source 
More fuzz testing by Andreas Seltenreich exposed that the planner did not
cope well with chains of lateral references.  If relation X references Y
laterally, and Y references Z laterally, then we will have to scan X on the
inside of a nestloop with Z, so for all intents and purposes X is laterally
dependent on Z too.  The planner did not understand this and would generate
intermediate joins that could not be used.  While that was usually harmless
except for wasting some planning cycles, under the right circumstances it
would lead to "failed to build any N-way joins" or "could not devise a
query plan" planner failures.

To fix that, convert the existing per-relation lateral_relids and
lateral_referencers relid sets into their transitive closures; that is,
they now show all relations on which a rel is directly or indirectly
laterally dependent.  This not only fixes the chained-reference problem
but allows some of the relevant tests to be made substantially simpler
and faster, since they can be reduced to simple bitmap manipulations
instead of searches of the LateralJoinInfo list.

Also, when a PlaceHolderVar that is due to be evaluated at a join contains
lateral references, we should treat those references as indirect lateral
dependencies of each of the join's base relations.  This prevents us from
trying to join any individual base relations to the lateral reference
source before the join is formed, which again cannot work.

Andreas' testing also exposed another oversight in the "dangerous
PlaceHolderVar" test added in commit 85e5e222b1.  Simply rejecting
unsafe join paths in joinpath.c is insufficient, because in some cases
we will end up rejecting *all* possible paths for a particular join, again
leading to "could not devise a query plan" failures.  The restriction has
to be known also to join_is_legal and its cohort functions, so that they
will not select a join for which that will happen.  I chose to move the
supporting logic into joinrels.c where the latter functions are.

Back-patch to 9.3 where LATERAL support was introduced.
This commit is contained in:
Tom Lane
2015-12-11 14:22:20 -05:00
parent 69e7235c93
commit acfcd45cac
9 changed files with 528 additions and 187 deletions
Download Patch File Download Diff File Expand all files Collapse all files

129
src/backend/optimizer/plan/initsplan.c
View File

@ -449,9 +449,7 @@ create_lateral_join_info(PlannerInfo *root)
Assert(false);
}
/* We now know all the relids needed for lateral refs in this rel */
if (bms_is_empty(lateral_relids))
continue; /* ensure lateral_relids is NULL if empty */
/* We now have all the direct lateral refs from this rel */
brel->lateral_relids = lateral_relids;
}
@ -459,6 +457,14 @@ create_lateral_join_info(PlannerInfo *root)
* Now check for lateral references within PlaceHolderVars, and make
* LateralJoinInfos describing each such reference. Unlike references in
* unflattened LATERAL RTEs, the referencing location could be a join.
*
* For a PHV that is due to be evaluated at a join, we mark each of the
* join's member baserels as having the PHV's lateral references too. Even
* though the baserels could be scanned without considering those lateral
* refs, we will never be able to form the join except as a path
* parameterized by the lateral refs, so there is no point in considering
* unparameterized paths for the baserels; and we mustn't try to join any
* of those baserels to the lateral refs too soon, either.
*/
foreach(lc, root->placeholder_list)
{
@ -471,6 +477,7 @@ create_lateral_join_info(PlannerInfo *root)
PVC_RECURSE_AGGREGATES,
PVC_INCLUDE_PLACEHOLDERS);
ListCell *lc2;
int ev_at;
foreach(lc2, vars)
{
@ -500,6 +507,15 @@ create_lateral_join_info(PlannerInfo *root)
}
list_free(vars);
ev_at = -1;
while ((ev_at = bms_next_member(eval_at, ev_at)) >= 0)
{
RelOptInfo *brel = find_base_rel(root, ev_at);
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
}
}
@ -508,12 +524,88 @@ create_lateral_join_info(PlannerInfo *root)
return;
/*
* Now that we've identified all lateral references, make a second pass in
* which we mark each baserel with the set of relids of rels that
* reference it laterally (essentially, the inverse mapping of
* lateral_relids). We'll need this for join_clause_is_movable_to().
* At this point the lateral_relids sets represent only direct lateral
* references. Replace them by their transitive closure, so that they
* describe both direct and indirect lateral references. If relation X
* references Y laterally, and Y references Z laterally, then we will have
* to scan X on the inside of a nestloop with Z, so for all intents and
* purposes X is laterally dependent on Z too.
*
* Also, propagate lateral_relids and lateral_referencers from appendrel
* This code is essentially Warshall's algorithm for transitive closure.
* The outer loop considers each baserel, and propagates its lateral
* dependencies to those baserels that have a lateral dependency on it.
*/
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
RelOptInfo *brel = root->simple_rel_array[rti];
Relids outer_lateral_relids;
Index rti2;
if (brel == NULL || brel->reloptkind != RELOPT_BASEREL)
continue;
/* need not consider baserel further if it has no lateral refs */
outer_lateral_relids = brel->lateral_relids;
if (outer_lateral_relids == NULL)
continue;
/* else scan all baserels */
for (rti2 = 1; rti2 < root->simple_rel_array_size; rti2++)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
if (brel2 == NULL || brel2->reloptkind != RELOPT_BASEREL)
continue;
/* if brel2 has lateral ref to brel, propagate brel's refs */
if (bms_is_member(rti, brel2->lateral_relids))
brel2->lateral_relids = bms_add_members(brel2->lateral_relids,
outer_lateral_relids);
}
}
/*
* Now that we've identified all lateral references, mark each baserel
* with the set of relids of rels that reference it laterally (possibly
* indirectly) --- that is, the inverse mapping of lateral_relids.
*/
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
RelOptInfo *brel = root->simple_rel_array[rti];
Relids lateral_relids;
int rti2;
if (brel == NULL || brel->reloptkind != RELOPT_BASEREL)
continue;
/* Nothing to do at rels with no lateral refs */
lateral_relids = brel->lateral_relids;
if (lateral_relids == NULL)
continue;
/*
* We should not have broken the invariant that lateral_relids is
* exactly NULL if empty.
*/
Assert(!bms_is_empty(lateral_relids));
/* Also, no rel should have a lateral dependency on itself */
Assert(!bms_is_member(rti, lateral_relids));
/* Mark this rel's referencees */
rti2 = -1;
while ((rti2 = bms_next_member(lateral_relids, rti2)) >= 0)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
}
/*
* Lastly, propagate lateral_relids and lateral_referencers from appendrel
* parent rels to their child rels. We intentionally give each child rel
* the same minimum parameterization, even though it's quite possible that
* some don't reference all the lateral rels. This is because any append
@ -525,29 +617,10 @@ create_lateral_join_info(PlannerInfo *root)
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
RelOptInfo *brel = root->simple_rel_array[rti];
Relids lateral_referencers;
if (brel == NULL)
continue;
if (brel->reloptkind != RELOPT_BASEREL)
if (brel == NULL || brel->reloptkind != RELOPT_BASEREL)
continue;
/* Compute lateral_referencers using the finished lateral_info_list */
lateral_referencers = NULL;
foreach(lc, root->lateral_info_list)
{
LateralJoinInfo *ljinfo = (LateralJoinInfo *) lfirst(lc);
if (bms_is_member(brel->relid, ljinfo->lateral_lhs))
lateral_referencers = bms_add_members(lateral_referencers,
ljinfo->lateral_rhs);
}
brel->lateral_referencers = lateral_referencers;
/*
* If it's an appendrel parent, copy its lateral_relids and
* lateral_referencers to each child rel.
*/
if (root->simple_rte_array[rti]->inh)
{
foreach(lc, root->append_rel_list)