database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2026-10-23 13:29:09 +03:00
Code Activity

Fix parallel GIN index build with keys larger than 65535 bytes

Browse Source 
During a parallel GIN build, each key is serialized into a GinTuple, a
transient representation used only while sorting.  _gin_build_tuple()
lays out the whole tuple (the palloc size, the key memcpy, and the
offset of the posting list) from a local variable holding the real key
length, but then stored that length in GinTuple.keylen, which was
uint16.  For a key longer than 65535 bytes, the stored length was thus
silently truncated.

On read-back, GinTupleGetFirst() and _gin_parse_tuple_items()
recompute the posting-list offset from the truncated keylen and land
inside the key data, so ginPostingListDecodeAllSegments() decodes
garbage: an assertion failure with assertions enabled, and a read past
the end of the allocation without, which could in turn lead to a crash
or garbage being written into the index.  Only parallel builds are
affected, because only they materialize a GinTuple; a serial build of
the same data succeeds, as index_form_tuple() compresses large keys
before the GinMaxItemSize check.

To fix, widen GinTuple.keylen to Size, which is what VARSIZE_ANY()
returns and what GinBuffer.keylen already uses, and use Size for the
local in _gin_build_tuple() too, which was an int that truncated
VARSIZE_ANY() the same way.

Widening keylen moves GinTuple.data.  The key value is accessed in
place in data, so data must be MAXALIGN'ed; before, that was only true
by accident of the field layout.  Make that explicit with
alignas(MAXIMUM_ALIGNOF) on data, so that the layout stays correct on
32-bit platforms, where Size is 4 bytes and would otherwise leave data
under-aligned when MAXIMUM_ALIGNOF is 8.

Bug: #19545
Author: Ewan Young <kdbase.hack@gmail.com>
Reported-by: Yuelin Wang <1217816127@qq.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/19545-0f25b7e47351e8fc%40postgresql.org
This commit is contained in:
Peter Eisentraut
2026-09-14 15:13:17 +02:00
parent 47f0fa62c0
commit bad8829872
2 changed files with 8 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/backend/access/gin/gininsert.c
View File

@@ -2250,7 +2250,7 @@ _gin_build_tuple(OffsetNumber attrnum, unsigned char category,
char *ptr;
Size tuplen;
int keylen;
Size keylen;
dlist_mutable_iter iter;
dlist_head segments;

9
src/include/access/gin_tuple.h
View File

@@ -23,12 +23,17 @@ typedef struct GinTuple
{
int tuplen; /* length of the whole tuple */
OffsetNumber attrnum; /* attnum of index key */
uint16 keylen; /* bytes in data for key value */
Size keylen; /* bytes in data for key value */
int16 typlen; /* typlen for key */
bool typbyval; /* typbyval for key */
signed char category; /* category: normal or NULL? */
int nitems; /* number of TIDs in the data */
char data[FLEXIBLE_ARRAY_MEMBER];
/*
* The key value is accessed in place, so it must be aligned well enough
* for any key type.
*/
char alignas(MAXIMUM_ALIGNOF) data[FLEXIBLE_ARRAY_MEMBER];
} GinTuple;
static inline ItemPointer