database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2026-10-23 13:29:09 +03:00
Code Activity

Release memory allocated by ndistinct_for_combination()

Browse Source 
Building extended statistics of type "ndistinct" calls
ndistinct_for_combination() once for every combination of columns.  Each
call allocates arrays for the whole sample and never frees them, with
memory accumulating until the whole statistics object is built.  The
sample has 300 multiplied by target rows.  With large statistics targets
(default_statistics_target up to 10k), this can add up to gigabytes
worth of memory, useless once a combination is computed.

ndistinct_for_combination() is now called in a dedicated memory context,
reset after each combination is computed.  We only need the estimated
number of distinct values, so there is no need to copy anything out of
the memory context.

Similar fixes have been done in 83772cc78e and ad8a166ca8 for other
areas of the extended stats computations.

Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Discussion: https://postgr.es/m/20d6689e-f490-4ca7-a92f-e92f691ba07e@tantorlabs.com
Backpatch-through: 14
This commit is contained in:
Michael Paquier
2026-10-14 10:14:54 +09:00
parent 6b3f8857aa
commit 6d4607e886
1 changed files with 13 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
src/backend/statistics/mvdistinct.c
View File

@@ -28,6 +28,7 @@
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_statistic_ext_data.h"
#include "statistics/extended_stats_internal.h"
#include "utils/memutils.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
#include "varatt.h"
@@ -89,6 +90,7 @@ statext_ndistinct_build(double totalrows, StatsBuildData *data)
uint32 itemcnt;
int numattrs = data->nattnums;
int numcombs = num_combinations(numattrs);
MemoryContext cxt;
result = palloc(offsetof(MVNDistinct, items) +
numcombs * sizeof(MVNDistinctItem));
@@ -96,6 +98,10 @@ statext_ndistinct_build(double totalrows, StatsBuildData *data)
result->type = STATS_NDISTINCT_TYPE_BASIC;
result->nitems = numcombs;
cxt = AllocSetContextCreate(CurrentMemoryContext,
"ndistinct_for_combination cxt",
ALLOCSET_DEFAULT_SIZES);
itemcnt = 0;
for (k = 2; k <= numattrs; k++)
{
@@ -109,6 +115,7 @@ statext_ndistinct_build(double totalrows, StatsBuildData *data)
{
MVNDistinctItem *item = &result->items[itemcnt];
int j;
MemoryContext oldcxt;
item->attributes = palloc_array(AttrNumber, k);
item->nattributes = k;
@@ -121,8 +128,12 @@ statext_ndistinct_build(double totalrows, StatsBuildData *data)
Assert(AttributeNumberIsValid(item->attributes[j]));
}
/* release memory used by ndistinct calculation */
oldcxt = MemoryContextSwitchTo(cxt);
item->ndistinct =
ndistinct_for_combination(totalrows, data, k, combination);
MemoryContextSwitchTo(oldcxt);
MemoryContextReset(cxt);
itemcnt++;
Assert(itemcnt <= result->nitems);
@@ -131,6 +142,8 @@ statext_ndistinct_build(double totalrows, StatsBuildData *data)
generator_free(generator);
}
MemoryContextDelete(cxt);
/* must consume exactly the whole output array */
Assert(itemcnt == result->nitems);