database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-06-13 07:41:39 +03:00
Code Activity

Virtual generated columns

Browse Source 
This adds a new variant of generated columns that are computed on read
(like a view, unlike the existing stored generated columns, which are
computed on write, like a materialized view).

The syntax for the column definition is

    ... GENERATED ALWAYS AS (...) VIRTUAL

and VIRTUAL is also optional.  VIRTUAL is the default rather than
STORED to match various other SQL products.  (The SQL standard makes
no specification about this, but it also doesn't know about VIRTUAL or
STORED.)  (Also, virtual views are the default, rather than
materialized views.)

Virtual generated columns are stored in tuples as null values.  (A
very early version of this patch had the ambition to not store them at
all.  But so much stuff breaks or gets confused if you have tuples
where a column in the middle is completely missing.  This is a
compromise, and it still saves space over being forced to use stored
generated columns.  If we ever find a way to improve this, a bit of
pg_upgrade cleverness could allow for upgrades to a newer scheme.)

The capabilities and restrictions of virtual generated columns are
mostly the same as for stored generated columns.  In some cases, this
patch keeps virtual generated columns more restricted than they might
technically need to be, to keep the two kinds consistent.  Some of
that could maybe be relaxed later after separate careful
considerations.

Some functionality that is currently not supported, but could possibly
be added as incremental features, some easier than others:

- index on or using a virtual column
- hence also no unique constraints on virtual columns
- extended statistics on virtual columns
- foreign-key constraints on virtual columns
- not-null constraints on virtual columns (check constraints are supported)
- ALTER TABLE / DROP EXPRESSION
- virtual column cannot have domain type
- virtual columns are not supported in logical replication

The tests in generated_virtual.sql have been copied over from
generated_stored.sql with the keyword replaced.  This way we can make
sure the behavior is mostly aligned, and the differences can be
visible.  Some tests for currently not supported features are
currently commented out.

Reviewed-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Tested-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
This commit is contained in:
Peter Eisentraut
2025-02-07 09:09:34 +01:00
parent cbc127917e
commit 83ea6c5402
71 changed files with 3227 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files

200
src/backend/commands/tablecmds.c
View File

@ -3039,6 +3039,15 @@ MergeAttributes(List *columns, const List *supers, char relpersistence,
errhint("A child table column cannot be generated unless its parent column is.")));
}
if (coldef->generated && restdef->generated && coldef->generated != restdef->generated)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_DEFINITION),
errmsg("column \"%s\" inherits from generated column of different kind",
restdef->colname),
errdetail("Parent column is %s, child column is %s.",
coldef->generated == ATTRIBUTE_GENERATED_STORED ? "STORED" : "VIRTUAL",
restdef->generated == ATTRIBUTE_GENERATED_STORED ? "STORED" : "VIRTUAL")));
/*
* Override the parent's default value for this column
* (coldef->cooked_default) with the partition's local
@ -3324,6 +3333,15 @@ MergeChildAttribute(List *inh_columns, int exist_attno, int newcol_attno, const
errhint("A child table column cannot be generated unless its parent column is.")));
}
if (inhdef->generated && newdef->generated && newdef->generated != inhdef->generated)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_DEFINITION),
errmsg("column \"%s\" inherits from generated column of different kind",
inhdef->colname),
errdetail("Parent column is %s, child column is %s.",
inhdef->generated == ATTRIBUTE_GENERATED_STORED ? "STORED" : "VIRTUAL",
newdef->generated == ATTRIBUTE_GENERATED_STORED ? "STORED" : "VIRTUAL")));
/*
* If new def has a default, override previous default
*/
@ -6130,7 +6148,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap)
{
case CONSTR_CHECK:
needscan = true;
con->qualstate = ExecPrepareExpr((Expr *) con->qual, estate);
con->qualstate = ExecPrepareExpr((Expr *) expand_generated_columns_in_expr(con->qual, newrel ? newrel : oldrel, 1), estate);
break;
case CONSTR_FOREIGN:
/* Nothing to do here */
@ -7308,7 +7326,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
* DEFAULT value outside of the heap. This may be disabled inside
* AddRelationNewConstraints if the optimization cannot be applied.
*/
rawEnt->missingMode = (!colDef->generated);
rawEnt->missingMode = (colDef->generated != ATTRIBUTE_GENERATED_STORED);
rawEnt->generated = colDef->generated;
@ -7785,6 +7803,14 @@ ATExecSetNotNull(List **wqueue, Relation rel, char *conName, char *colName,
errmsg("cannot alter system column \"%s\"",
colName)));
/* TODO: see transformColumnDefinition() */
if (TupleDescAttr(RelationGetDescr(rel), attnum - 1)->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("not-null constraints are not supported on virtual generated columns"),
errdetail("Column \"%s\" of relation \"%s\" is a virtual generated column.",
colName, RelationGetRelationName(rel))));
/* See if there's already a constraint */
tuple = findNotNullConstraintAttnum(RelationGetRelid(rel), attnum);
if (HeapTupleIsValid(tuple))
@ -8411,6 +8437,8 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName,
HeapTuple tuple;
Form_pg_attribute attTup;
AttrNumber attnum;
char attgenerated;
bool rewrite;
Oid attrdefoid;
ObjectAddress address;
Expr *defval;
@ -8425,36 +8453,70 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName,
colName, RelationGetRelationName(rel))));
attTup = (Form_pg_attribute) GETSTRUCT(tuple);
attnum = attTup->attnum;
attnum = attTup->attnum;
if (attnum <= 0)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot alter system column \"%s\"",
colName)));
if (attTup->attgenerated != ATTRIBUTE_GENERATED_STORED)
attgenerated = attTup->attgenerated;
if (!attgenerated)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("column \"%s\" of relation \"%s\" is not a generated column",
colName, RelationGetRelationName(rel))));
/*
* TODO: This could be done, just need to recheck any constraints
* afterwards.
*/
if (attgenerated == ATTRIBUTE_GENERATED_VIRTUAL &&
rel->rd_att->constr && rel->rd_att->constr->num_check > 0)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ALTER TABLE / SET EXPRESSION is not supported for virtual generated columns on tables with check constraints"),
errdetail("Column \"%s\" of relation \"%s\" is a virtual generated column.",
colName, RelationGetRelationName(rel))));
/*
* We need to prevent this because a change of expression could affect a
* row filter and inject expressions that are not permitted in a row
* filter. XXX We could try to have a more precise check to catch only
* publications with row filters, or even re-verify the row filter
* expressions.
*/
if (attgenerated == ATTRIBUTE_GENERATED_VIRTUAL &&
GetRelationPublications(RelationGetRelid(rel)) != NIL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ALTER TABLE / SET EXPRESSION is not supported for virtual generated columns on tables that are part of a publication"),
errdetail("Column \"%s\" of relation \"%s\" is a virtual generated column.",
colName, RelationGetRelationName(rel))));
rewrite = (attgenerated == ATTRIBUTE_GENERATED_STORED);
ReleaseSysCache(tuple);
/*
* Clear all the missing values if we're rewriting the table, since this
* renders them pointless.
*/
RelationClearMissing(rel);
if (rewrite)
{
/*
* Clear all the missing values if we're rewriting the table, since
* this renders them pointless.
*/
RelationClearMissing(rel);
/* make sure we don't conflict with later attribute modifications */
CommandCounterIncrement();
/* make sure we don't conflict with later attribute modifications */
CommandCounterIncrement();
/*
* Find everything that depends on the column (constraints, indexes, etc),
* and record enough information to let us recreate the objects after
* rewrite.
*/
RememberAllDependentForRebuilding(tab, AT_SetExpression, rel, attnum, colName);
/*
* Find everything that depends on the column (constraints, indexes,
* etc), and record enough information to let us recreate the objects
* after rewrite.
*/
RememberAllDependentForRebuilding(tab, AT_SetExpression, rel, attnum, colName);
}
/*
* Drop the dependency records of the GENERATED expression, in particular
@ -8483,7 +8545,7 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName,
rawEnt->attnum = attnum;
rawEnt->raw_default = newExpr;
rawEnt->missingMode = false;
rawEnt->generated = ATTRIBUTE_GENERATED_STORED;
rawEnt->generated = attgenerated;
/* Store the generated expression */
AddRelationNewConstraints(rel, list_make1(rawEnt), NIL,
@ -8492,16 +8554,19 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName,
/* Make above new expression visible */
CommandCounterIncrement();
/* Prepare for table rewrite */
defval = (Expr *) build_column_default(rel, attnum);
if (rewrite)
{
/* Prepare for table rewrite */
defval = (Expr *) build_column_default(rel, attnum);
newval = (NewColumnValue *) palloc0(sizeof(NewColumnValue));
newval->attnum = attnum;
newval->expr = expression_planner(defval);
newval->is_generated = true;
newval = (NewColumnValue *) palloc0(sizeof(NewColumnValue));
newval->attnum = attnum;
newval->expr = expression_planner(defval);
newval->is_generated = true;
tab->newvals = lappend(tab->newvals, newval);
tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
tab->newvals = lappend(tab->newvals, newval);
tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
/* Drop any pg_statistic entry for the column */
RemoveStatistics(RelationGetRelid(rel), attnum);
@ -8590,17 +8655,30 @@ ATExecDropExpression(Relation rel, const char *colName, bool missing_ok, LOCKMOD
errmsg("cannot alter system column \"%s\"",
colName)));
if (attTup->attgenerated != ATTRIBUTE_GENERATED_STORED)
/*
* TODO: This could be done, but it would need a table rewrite to
* materialize the generated values. Note that for the time being, we
* still error with missing_ok, so that we don't silently leave the column
* as generated.
*/
if (attTup->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ALTER TABLE / DROP EXPRESSION is not supported for virtual generated columns"),
errdetail("Column \"%s\" of relation \"%s\" is a virtual generated column.",
colName, RelationGetRelationName(rel))));
if (!attTup->attgenerated)
{
if (!missing_ok)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("column \"%s\" of relation \"%s\" is not a stored generated column",
errmsg("column \"%s\" of relation \"%s\" is not a generated column",
colName, RelationGetRelationName(rel))));
else
{
ereport(NOTICE,
(errmsg("column \"%s\" of relation \"%s\" is not a stored generated column, skipping",
(errmsg("column \"%s\" of relation \"%s\" is not a generated column, skipping",
colName, RelationGetRelationName(rel))));
heap_freetuple(tuple);
table_close(attrelation, RowExclusiveLock);
@ -8743,6 +8821,16 @@ ATExecSetStatistics(Relation rel, const char *colName, int16 colNum, Node *newVa
errmsg("cannot alter system column \"%s\"",
colName)));
/*
* Prevent this as long as the ANALYZE code skips virtual generated
* columns.
*/
if (attrtuple->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot alter statistics on virtual generated column \"%s\"",
colName)));
if (rel->rd_rel->relkind == RELKIND_INDEX ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_INDEX)
{
@ -9925,6 +10013,19 @@ ATAddForeignKeyConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
errmsg("invalid %s action for foreign key constraint containing generated column",
"ON DELETE")));
}
/*
* FKs on virtual columns are not supported. This would require
* various additional support in ri_triggers.c, including special
* handling in ri_NullCheck(), ri_KeysEqual(),
* RI_FKey_fk_upd_check_required() (since all virtual columns appear
* as NULL there). Also not really practical as long as you can't
* index virtual columns.
*/
if (attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("foreign key constraints on virtual generated columns are not supported")));
}
/*
@ -12289,7 +12390,7 @@ QueueCheckConstraintValidation(List **wqueue, Relation conrel, Relation rel,
val = SysCacheGetAttrNotNull(CONSTROID, contuple,
Anum_pg_constraint_conbin);
conbin = TextDatumGetCString(val);
newcon->qual = (Node *) stringToNode(conbin);
newcon->qual = expand_generated_columns_in_expr(stringToNode(conbin), rel, 1);
/* Find or create work queue entry for this table */
tab = ATGetQueueEntry(wqueue, rel);
@ -13467,8 +13568,12 @@ ATPrepAlterColumnType(List **wqueue,
list_make1_oid(rel->rd_rel->reltype),
0);
if (tab->relkind == RELKIND_RELATION ||
tab->relkind == RELKIND_PARTITIONED_TABLE)
if (attTup->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
{
/* do nothing */
}
else if (tab->relkind == RELKIND_RELATION ||
tab->relkind == RELKIND_PARTITIONED_TABLE)
{
/*
* Set up an expression to transform the old data value to the new
@ -13541,11 +13646,12 @@ ATPrepAlterColumnType(List **wqueue,
errmsg("\"%s\" is not a table",
RelationGetRelationName(rel))));
if (!RELKIND_HAS_STORAGE(tab->relkind))
if (!RELKIND_HAS_STORAGE(tab->relkind) || attTup->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
{
/*
* For relations without storage, do this check now. Regular tables
* will check it later when the table is being rewritten.
* For relations or columns without storage, do this check now.
* Regular tables will check it later when the table is being
* rewritten.
*/
find_composite_type_dependencies(rel->rd_rel->reltype, rel, NULL);
}
@ -16534,6 +16640,14 @@ MergeAttributesIntoExisting(Relation child_rel, Relation parent_rel, bool ispart
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" in child table must not be a generated column", parent_attname)));
if (parent_att->attgenerated && child_att->attgenerated && child_att->attgenerated != parent_att->attgenerated)
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" inherits from generated column of different kind", parent_attname),
errdetail("Parent column is %s, child column is %s.",
parent_att->attgenerated == ATTRIBUTE_GENERATED_STORED ? "STORED" : "VIRTUAL",
child_att->attgenerated == ATTRIBUTE_GENERATED_STORED ? "STORED" : "VIRTUAL")));
/*
* Regular inheritance children are independent enough not to
* inherit identity columns. But partitions are integral part of
@ -18781,8 +18895,11 @@ ComputePartitionAttrs(ParseState *pstate, Relation rel, List *partParams, AttrNu
parser_errposition(pstate, pelem->location)));
/*
* Generated columns cannot work: They are computed after BEFORE
* triggers, but partition routing is done before all triggers.
* Stored generated columns cannot work: They are computed after
* BEFORE triggers, but partition routing is done before all
* triggers. Maybe virtual generated columns could be made to
* work, but then they would need to be handled as an expression
* below.
*/
if (attform->attgenerated)
ereport(ERROR,
@ -18864,9 +18981,12 @@ ComputePartitionAttrs(ParseState *pstate, Relation rel, List *partParams, AttrNu
}
/*
* Generated columns cannot work: They are computed after
* BEFORE triggers, but partition routing is done before all
* triggers.
* Stored generated columns cannot work: They are computed
* after BEFORE triggers, but partition routing is done before
* all triggers. Virtual generated columns could probably
* work, but it would require more work elsewhere (for example
* SET EXPRESSION would need to check whether the column is
* used in partition keys). Seems safer to prohibit for now.
*/
i = -1;
while ((i = bms_next_member(expr_attrs, i)) >= 0)