database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-06-29 10:41:53 +03:00
Code Activity

Faster expression evaluation and targetlist projection.

Browse Source 
This replaces the old, recursive tree-walk based evaluation, with
non-recursive, opcode dispatch based, expression evaluation.
Projection is now implemented as part of expression evaluation.

This both leads to significant performance improvements, and makes
future just-in-time compilation of expressions easier.

The speed gains primarily come from:
- non-recursive implementation reduces stack usage / overhead
- simple sub-expressions are implemented with a single jump, without
  function calls
- sharing some state between different sub-expressions
- reduced amount of indirect/hard to predict memory accesses by laying
  out operation metadata sequentially; including the avoidance of
  nearly all of the previously used linked lists
- more code has been moved to expression initialization, avoiding
  constant re-checks at evaluation time

Future just-in-time compilation (JIT) has become easier, as
demonstrated by released patches intended to be merged in a later
release, for primarily two reasons: Firstly, due to a stricter split
between expression initialization and evaluation, less code has to be
handled by the JIT. Secondly, due to the non-recursive nature of the
generated "instructions", less performance-critical code-paths can
easily be shared between interpreted and compiled evaluation.

The new framework allows for significant future optimizations. E.g.:
- basic infrastructure for to later reduce the per executor-startup
  overhead of expression evaluation, by caching state in prepared
  statements.  That'd be helpful in OLTPish scenarios where
  initialization overhead is measurable.
- optimizing the generated "code". A number of proposals for potential
  work has already been made.
- optimizing the interpreter. Similarly a number of proposals have
  been made here too.

The move of logic into the expression initialization step leads to some
backward-incompatible changes:
- Function permission checks are now done during expression
  initialization, whereas previously they were done during
  execution. In edge cases this can lead to errors being raised that
  previously wouldn't have been, e.g. a NULL array being coerced to a
  different array type previously didn't perform checks.
- The set of domain constraints to be checked, is now evaluated once
  during expression initialization, previously it was re-built
  every time a domain check was evaluated. For normal queries this
  doesn't change much, but e.g. for plpgsql functions, which caches
  ExprStates, the old set could stick around longer.  The behavior
  around might still change.

Author: Andres Freund, with significant changes by Tom Lane,
	changes by Heikki Linnakangas
Reviewed-By: Tom Lane, Heikki Linnakangas
Discussion: https://postgr.es/m/20161206034955.bh33paeralxbtluv@alap3.anarazel.de
This commit is contained in:
Andres Freund
2017-03-14 15:45:36 -07:00
parent 7d3957e53e
commit b8d7f053c5
71 changed files with 8868 additions and 6531 deletions
Download Patch File Download Diff File Expand all files Collapse all files

338
src/backend/executor/execUtils.c
View File

@ -31,6 +31,9 @@
* RegisterExprContextCallback Register function shutdown callback
* UnregisterExprContextCallback Deregister function shutdown callback
*
* GetAttributeByName Runtime extraction of columns from tuples.
* GetAttributeByNum
*
* NOTES
* This file has traditionally been the place to stick misc.
* executor support stuff that doesn't really go anyplace else.
@ -44,11 +47,12 @@
#include "nodes/nodeFuncs.h"
#include "parser/parsetree.h"
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/typcache.h"
static bool get_last_attnums(Node *node, ProjectionInfo *projInfo);
static void ShutdownExprContext(ExprContext *econtext, bool isCommit);
@ -464,186 +468,6 @@ ExecGetResultType(PlanState *planstate)
return slot->tts_tupleDescriptor;
}
/* ----------------
* ExecBuildProjectionInfo
*
* Build a ProjectionInfo node for evaluating the given tlist in the given
* econtext, and storing the result into the tuple slot. (Caller must have
* ensured that tuple slot has a descriptor matching the tlist!) Note that
* the given tlist should be a list of ExprState nodes, not Expr nodes.
*
* inputDesc can be NULL, but if it is not, we check to see whether simple
* Vars in the tlist match the descriptor. It is important to provide
* inputDesc for relation-scan plan nodes, as a cross check that the relation
* hasn't been changed since the plan was made. At higher levels of a plan,
* there is no need to recheck.
* ----------------
*/
ProjectionInfo *
ExecBuildProjectionInfo(List *targetList,
ExprContext *econtext,
TupleTableSlot *slot,
TupleDesc inputDesc)
{
ProjectionInfo *projInfo = makeNode(ProjectionInfo);
int len = ExecTargetListLength(targetList);
int *workspace;
int *varSlotOffsets;
int *varNumbers;
int *varOutputCols;
List *exprlist;
int numSimpleVars;
bool directMap;
ListCell *tl;
projInfo->pi_exprContext = econtext;
projInfo->pi_slot = slot;
/* since these are all int arrays, we need do just one palloc */
workspace = (int *) palloc(len * 3 * sizeof(int));
projInfo->pi_varSlotOffsets = varSlotOffsets = workspace;
projInfo->pi_varNumbers = varNumbers = workspace + len;
projInfo->pi_varOutputCols = varOutputCols = workspace + len * 2;
projInfo->pi_lastInnerVar = 0;
projInfo->pi_lastOuterVar = 0;
projInfo->pi_lastScanVar = 0;
/*
* We separate the target list elements into simple Var references and
* expressions which require the full ExecTargetList machinery. To be a
* simple Var, a Var has to be a user attribute and not mismatch the
* inputDesc. (Note: if there is a type mismatch then ExecEvalScalarVar
* will probably throw an error at runtime, but we leave that to it.)
*/
exprlist = NIL;
numSimpleVars = 0;
directMap = true;
foreach(tl, targetList)
{
GenericExprState *gstate = (GenericExprState *) lfirst(tl);
Var *variable = (Var *) gstate->arg->expr;
bool isSimpleVar = false;
if (variable != NULL &&
IsA(variable, Var) &&
variable->varattno > 0)
{
if (!inputDesc)
isSimpleVar = true; /* can't check type, assume OK */
else if (variable->varattno <= inputDesc->natts)
{
Form_pg_attribute attr;
attr = inputDesc->attrs[variable->varattno - 1];
if (!attr->attisdropped && variable->vartype == attr->atttypid)
isSimpleVar = true;
}
}
if (isSimpleVar)
{
TargetEntry *tle = (TargetEntry *) gstate->xprstate.expr;
AttrNumber attnum = variable->varattno;
varNumbers[numSimpleVars] = attnum;
varOutputCols[numSimpleVars] = tle->resno;
if (tle->resno != numSimpleVars + 1)
directMap = false;
switch (variable->varno)
{
case INNER_VAR:
varSlotOffsets[numSimpleVars] = offsetof(ExprContext,
ecxt_innertuple);
if (projInfo->pi_lastInnerVar < attnum)
projInfo->pi_lastInnerVar = attnum;
break;
case OUTER_VAR:
varSlotOffsets[numSimpleVars] = offsetof(ExprContext,
ecxt_outertuple);
if (projInfo->pi_lastOuterVar < attnum)
projInfo->pi_lastOuterVar = attnum;
break;
/* INDEX_VAR is handled by default case */
default:
varSlotOffsets[numSimpleVars] = offsetof(ExprContext,
ecxt_scantuple);
if (projInfo->pi_lastScanVar < attnum)
projInfo->pi_lastScanVar = attnum;
break;
}
numSimpleVars++;
}
else
{
/* Not a simple variable, add it to generic targetlist */
exprlist = lappend(exprlist, gstate);
/* Examine expr to include contained Vars in lastXXXVar counts */
get_last_attnums((Node *) variable, projInfo);
}
}
projInfo->pi_targetlist = exprlist;
projInfo->pi_numSimpleVars = numSimpleVars;
projInfo->pi_directMap = directMap;
return projInfo;
}
/*
* get_last_attnums: expression walker for ExecBuildProjectionInfo
*
* Update the lastXXXVar counts to be at least as large as the largest
* attribute numbers found in the expression
*/
static bool
get_last_attnums(Node *node, ProjectionInfo *projInfo)
{
if (node == NULL)
return false;
if (IsA(node, Var))
{
Var *variable = (Var *) node;
AttrNumber attnum = variable->varattno;
switch (variable->varno)
{
case INNER_VAR:
if (projInfo->pi_lastInnerVar < attnum)
projInfo->pi_lastInnerVar = attnum;
break;
case OUTER_VAR:
if (projInfo->pi_lastOuterVar < attnum)
projInfo->pi_lastOuterVar = attnum;
break;
/* INDEX_VAR is handled by default case */
default:
if (projInfo->pi_lastScanVar < attnum)
projInfo->pi_lastScanVar = attnum;
break;
}
return false;
}
/*
* Don't examine the arguments or filters of Aggrefs or WindowFuncs,
* because those do not represent expressions to be evaluated within the
* overall targetlist's econtext. GroupingFunc arguments are never
* evaluated at all.
*/
if (IsA(node, Aggref))
return false;
if (IsA(node, WindowFunc))
return false;
if (IsA(node, GroupingFunc))
return false;
return expression_tree_walker(node, get_last_attnums,
(void *) projInfo);
}
/* ----------------
* ExecAssignProjectionInfo
@ -659,9 +483,10 @@ ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc)
{
planstate->ps_ProjInfo =
ExecBuildProjectionInfo(planstate->targetlist,
ExecBuildProjectionInfo(planstate->plan->targetlist,
planstate->ps_ExprContext,
planstate->ps_ResultTupleSlot,
planstate,
inputDesc);
}
@ -1009,3 +834,152 @@ ExecLockNonLeafAppendTables(List *partitioned_rels, EState *estate)
}
}
}
/*
* GetAttributeByName
* GetAttributeByNum
*
* These functions return the value of the requested attribute
* out of the given tuple Datum.
* C functions which take a tuple as an argument are expected
* to use these. Ex: overpaid(EMP) might call GetAttributeByNum().
* Note: these are actually rather slow because they do a typcache
* lookup on each call.
*/
Datum
GetAttributeByName(HeapTupleHeader tuple, const char *attname, bool *isNull)
{
AttrNumber attrno;
Datum result;
Oid tupType;
int32 tupTypmod;
TupleDesc tupDesc;
HeapTupleData tmptup;
int i;
if (attname == NULL)
elog(ERROR, "invalid attribute name");
if (isNull == NULL)
elog(ERROR, "a NULL isNull pointer was passed");
if (tuple == NULL)
{
/* Kinda bogus but compatible with old behavior... */
*isNull = true;
return (Datum) 0;
}
tupType = HeapTupleHeaderGetTypeId(tuple);
tupTypmod = HeapTupleHeaderGetTypMod(tuple);
tupDesc = lookup_rowtype_tupdesc(tupType, tupTypmod);
attrno = InvalidAttrNumber;
for (i = 0; i < tupDesc->natts; i++)
{
if (namestrcmp(&(tupDesc->attrs[i]->attname), attname) == 0)
{
attrno = tupDesc->attrs[i]->attnum;
break;
}
}
if (attrno == InvalidAttrNumber)
elog(ERROR, "attribute \"%s\" does not exist", attname);
/*
* heap_getattr needs a HeapTuple not a bare HeapTupleHeader. We set all
* the fields in the struct just in case user tries to inspect system
* columns.
*/
tmptup.t_len = HeapTupleHeaderGetDatumLength(tuple);
ItemPointerSetInvalid(&(tmptup.t_self));
tmptup.t_tableOid = InvalidOid;
tmptup.t_data = tuple;
result = heap_getattr(&tmptup,
attrno,
tupDesc,
isNull);
ReleaseTupleDesc(tupDesc);
return result;
}
Datum
GetAttributeByNum(HeapTupleHeader tuple,
AttrNumber attrno,
bool *isNull)
{
Datum result;
Oid tupType;
int32 tupTypmod;
TupleDesc tupDesc;
HeapTupleData tmptup;
if (!AttributeNumberIsValid(attrno))
elog(ERROR, "invalid attribute number %d", attrno);
if (isNull == NULL)
elog(ERROR, "a NULL isNull pointer was passed");
if (tuple == NULL)
{
/* Kinda bogus but compatible with old behavior... */
*isNull = true;
return (Datum) 0;
}
tupType = HeapTupleHeaderGetTypeId(tuple);
tupTypmod = HeapTupleHeaderGetTypMod(tuple);
tupDesc = lookup_rowtype_tupdesc(tupType, tupTypmod);
/*
* heap_getattr needs a HeapTuple not a bare HeapTupleHeader. We set all
* the fields in the struct just in case user tries to inspect system
* columns.
*/
tmptup.t_len = HeapTupleHeaderGetDatumLength(tuple);
ItemPointerSetInvalid(&(tmptup.t_self));
tmptup.t_tableOid = InvalidOid;
tmptup.t_data = tuple;
result = heap_getattr(&tmptup,
attrno,
tupDesc,
isNull);
ReleaseTupleDesc(tupDesc);
return result;
}
/*
* Number of items in a tlist (including any resjunk items!)
*/
int
ExecTargetListLength(List *targetlist)
{
/* This used to be more complex, but fjoins are dead */
return list_length(targetlist);
}
/*
* Number of items in a tlist, not including any resjunk items
*/
int
ExecCleanTargetListLength(List *targetlist)
{
int len = 0;
ListCell *tl;
foreach(tl, targetlist)
{
TargetEntry *curTle = castNode(TargetEntry, lfirst(tl));
if (!curTle->resjunk)
len++;
}
return len;
}