database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-10-28 11:55:03 +03:00
Code Activity

Add a Gather Merge executor node.

Browse Source 
Like Gather, we spawn multiple workers and run the same plan in each
one; however, Gather Merge is used when each worker produces the same
output ordering and we want to preserve that output ordering while
merging together the streams of tuples from various workers.  (In a
way, Gather Merge is like a hybrid of Gather and MergeAppend.)

This works out to a win if it saves us from having to perform an
expensive Sort.  In cases where only a small amount of data would need
to be sorted, it may actually be faster to use a regular Gather node
and then sort the results afterward, because Gather Merge sometimes
needs to wait synchronously for tuples whereas a pure Gather generally
doesn't.  But if this avoids an expensive sort then it's a win.

Rushabh Lathia, reviewed and tested by Amit Kapila, Thomas Munro,
and Neha Sharma, and reviewed and revised by me.

Discussion: http://postgr.es/m/CAGPqQf09oPX-cQRpBKS0Gq49Z+m6KBxgxd_p9gX8CKk_d75HoQ@mail.gmail.com
This commit is contained in:
Robert Haas
2017-03-09 07:40:36 -05:00
parent a72f0365db
commit 355d3993c5
27 changed files with 1355 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
src/backend/optimizer/path/costsize.c
View File

@@ -126,6 +126,7 @@ bool enable_nestloop = true;
bool enable_material = true;
bool enable_mergejoin = true;
bool enable_hashjoin = true;
bool enable_gathermerge = true;
typedef struct
{
@@ -372,6 +373,73 @@ cost_gather(GatherPath *path, PlannerInfo *root,
path->path.total_cost = (startup_cost + run_cost);
}
/*
* cost_gather_merge
* Determines and returns the cost of gather merge path.
*
* GatherMerge merges several pre-sorted input streams, using a heap that at
* any given instant holds the next tuple from each stream. If there are N
* streams, we need about N*log2(N) tuple comparisons to construct the heap at
* startup, and then for each output tuple, about log2(N) comparisons to
* replace the top heap entry with the next tuple from the same stream.
*/
void
cost_gather_merge(GatherMergePath *path, PlannerInfo *root,
RelOptInfo *rel, ParamPathInfo *param_info,
Cost input_startup_cost, Cost input_total_cost,
double *rows)
{
Cost startup_cost = 0;
Cost run_cost = 0;
Cost comparison_cost;
double N;
double logN;
/* Mark the path with the correct row estimate */
if (rows)
path->path.rows = *rows;
else if (param_info)
path->path.rows = param_info->ppi_rows;
else
path->path.rows = rel->rows;
if (!enable_gathermerge)
startup_cost += disable_cost;
/*
* Add one to the number of workers to account for the leader. This might
* be overgenerous since the leader will do less work than other workers
* in typical cases, but we'll go with it for now.
*/
Assert(path->num_workers > 0);
N = (double) path->num_workers + 1;
logN = LOG2(N);
/* Assumed cost per tuple comparison */
comparison_cost = 2.0 * cpu_operator_cost;
/* Heap creation cost */
startup_cost += comparison_cost * N * logN;
/* Per-tuple heap maintenance cost */
run_cost += path->path.rows * comparison_cost * logN;
/* small cost for heap management, like cost_merge_append */
run_cost += cpu_operator_cost * path->path.rows;
/*
* Parallel setup and communication cost. Since Gather Merge, unlike
* Gather, requires us to block until a tuple is available from every
* worker, we bump the IPC cost up a little bit as compared with Gather.
* For lack of a better idea, charge an extra 5%.
*/
startup_cost += parallel_setup_cost;
run_cost += parallel_tuple_cost * path->path.rows * 1.05;
path->path.startup_cost = startup_cost + input_startup_cost;
path->path.total_cost = (startup_cost + run_cost + input_total_cost);
}
/*
* cost_index
* Determines and returns the cost of scanning a relation using an index.