From 101fd9349eddb7e9ed84a239145d5230a9bc7336 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Mon, 14 Mar 2016 20:04:44 -0400 Subject: [PATCH] Add a GetForeignUpperPaths callback function for FDWs. This is basically like the just-added create_upper_paths_hook, but control is funneled only to the FDW responsible for all the baserels of the current query; so providing such a callback is much less likely to add useless overhead than using the hook function is. The documentation is a bit sketchy. We'll likely want to improve it, and/or adjust the call conventions, when we get some experience with actually using this callback. Hopefully somebody will find time to experiment with it before 9.6 feature freeze. --- doc/src/sgml/fdwhandler.sgml | 43 +++++++++++++++++++++++++-- src/backend/optimizer/README | 5 +++- src/backend/optimizer/plan/planner.c | 16 ++++++---- src/backend/optimizer/util/pathnode.c | 14 ++++----- src/include/foreign/fdwapi.h | 6 ++++ 5 files changed, 67 insertions(+), 17 deletions(-) diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml index bbc9c03721e..15ddc1977ad 100644 --- a/doc/src/sgml/fdwhandler.sgml +++ b/doc/src/sgml/fdwhandler.sgml @@ -176,7 +176,8 @@ GetForeignPlan (PlannerInfo *root, access path. This is called at the end of query planning. The parameters are as for GetForeignRelSize, plus the selected ForeignPath (previously produced by - GetForeignPaths or GetForeignJoinPaths), + GetForeignPaths, GetForeignJoinPaths, + or GetForeignUpperPaths), the target list to be emitted by the plan node, the restriction clauses to be enforced by the plan node, and the outer subplan of the ForeignScan, @@ -344,6 +345,38 @@ GetForeignJoinPaths (PlannerInfo *root, + + FDW Routines For Planning Post-Scan/Join Processing + + + If an FDW supports performing remote post-scan/join processing, such as + remote aggregation, it should provide this callback function: + + + + +void +GetForeignUpperPaths (PlannerInfo *root, + RelOptInfo *scan_join_rel); + + Create possible access paths for upper relation processing, + which is the planner's term for all post-scan/join query processing, such + as aggregation, window functions, sorting, and table updates. This + optional function is called during query planning. Currently, it is + called only if all base relation(s) involved in the query belong to the + same FDW. This function should generate ForeignPath + path(s) for the steps that the FDW knows how to perform remotely, and + call add_path to add these paths to the appropriate upper + relation. As with GetForeignJoinPaths, it is not necessary + that this function succeed in creating any paths, since paths involving + local processing are always possible. + + + + See for additional information. + + + FDW Routines For Updating Foreign Tables @@ -1160,7 +1193,8 @@ GetForeignServerByName(const char *name, bool missing_ok); The FDW callback functions GetForeignRelSize, GetForeignPaths, GetForeignPlan, - PlanForeignModify, and GetForeignJoinPaths + PlanForeignModify, GetForeignJoinPaths, + and GetForeignUpperPaths must fit into the workings of the PostgreSQL planner. Here are some notes about what they must do. @@ -1322,7 +1356,7 @@ GetForeignServerByName(const char *name, bool missing_ok); An FDW might additionally support direct execution of some plan actions that are above the level of scans and joins, such as grouping or aggregation. To offer such options, the FDW should generate paths - (probably ForeignPaths or CustomPaths) and insert them into the + and insert them into the appropriate upper relation. For example, a path representing remote aggregation should be inserted into the relation obtained from fetch_upper_rel(root, UPPERREL_GROUP_AGG, @@ -1332,6 +1366,9 @@ GetForeignServerByName(const char *name, bool missing_ok); else there will be an error at plan time). If the remote-aggregation path wins, which it usually would, it will be converted into a plan in the usual way, by calling GetForeignPlan. + Usually the most convenient place to generate such paths is in + the GetForeignUpperPaths callback function, although + it can be done earlier if that seems appropriate. diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README index 7ecf8c8f0cc..0d973d77e0b 100644 --- a/src/backend/optimizer/README +++ b/src/backend/optimizer/README @@ -916,7 +916,10 @@ remote aggregation into the UPPERREL_GROUP_AGG upperrel, if it notices that the query represents an aggregation that could be done entirely on the foreign server. That Path will then compete with Paths representing local aggregation on a regular scan of the foreign table, once the core -planner reaches the point of considering aggregation. +planner reaches the point of considering aggregation. (In practice, +it will usually be more convenient for FDWs to detect such cases in a +GetForeignUpperPaths callback; but that still represents injecting a +Path before the core code has touched the corresponding upperrel.) Parallel Query and Partial Paths diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index d11f44e64d5..fc0a2d8de35 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -1752,13 +1752,17 @@ grouping_planner(PlannerInfo *root, bool inheritance_update, root->upper_targets[UPPERREL_GROUP_AGG] = grouping_target; /* - * Let extensions, particularly CustomScan providers, consider - * injecting extension Paths into the query's upperrels, where they - * will compete with the Paths we create below. We pass the final - * scan/join rel because that's not so easily findable from the - * PlannerInfo struct; anything else the hook wants to know should be - * obtainable via "root". + * Let extensions, particularly FDWs and CustomScan providers, + * consider injecting extension Paths into the query's upperrels, + * where they will compete with the Paths we create below. We pass + * the final scan/join rel because that's not so easily findable from + * the PlannerInfo struct; anything else the hooks want to know should + * be obtainable via "root". */ + if (current_rel->fdwroutine && + current_rel->fdwroutine->GetForeignUpperPaths) + current_rel->fdwroutine->GetForeignUpperPaths(root, current_rel); + if (create_upper_paths_hook) (*create_upper_paths_hook) (root, current_rel); diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c index 675e47cdd01..b8ea3168a84 100644 --- a/src/backend/optimizer/util/pathnode.c +++ b/src/backend/optimizer/util/pathnode.c @@ -1813,15 +1813,15 @@ create_worktablescan_path(PlannerInfo *root, RelOptInfo *rel, /* * create_foreignscan_path - * Creates a path corresponding to a scan of a foreign table or - * a foreign join, returning the pathnode. + * Creates a path corresponding to a scan of a foreign table, foreign join, + * or foreign upper-relation processing, returning the pathnode. * * This function is never called from core Postgres; rather, it's expected - * to be called by the GetForeignPaths or GetForeignJoinPaths function of - * a foreign data wrapper. We make the FDW supply all fields of the path, - * since we do not have any way to calculate them in core. However, there - * is a sane default for the pathtarget (rel->reltarget), so we let a NULL - * for "target" select that. + * to be called by the GetForeignPaths, GetForeignJoinPaths, or + * GetForeignUpperPaths function of a foreign data wrapper. We make the FDW + * supply all fields of the path, since we do not have any way to calculate + * them in core. However, there is a usually-sane default for the pathtarget + * (rel->reltarget), so we let a NULL for "target" select that. */ ForeignPath * create_foreignscan_path(PlannerInfo *root, RelOptInfo *rel, diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h index ae3c230d667..71643602823 100644 --- a/src/include/foreign/fdwapi.h +++ b/src/include/foreign/fdwapi.h @@ -59,6 +59,9 @@ typedef void (*GetForeignJoinPaths_function) (PlannerInfo *root, JoinType jointype, JoinPathExtraData *extra); +typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root, + RelOptInfo *scan_join_rel); + typedef void (*AddForeignUpdateTargets_function) (Query *parsetree, RangeTblEntry *target_rte, Relation target_relation); @@ -166,6 +169,9 @@ typedef struct FdwRoutine /* Functions for remote-join planning */ GetForeignJoinPaths_function GetForeignJoinPaths; + /* Functions for remote upper-relation (post scan/join) planning */ + GetForeignUpperPaths_function GetForeignUpperPaths; + /* Functions for updating foreign tables */ AddForeignUpdateTargets_function AddForeignUpdateTargets; PlanForeignModify_function PlanForeignModify;