Fix GIN multiple-VACUUM-scans pending list bug.

ginbulkdelete performs pending list cleanup before it searches the entry
tree (and any posting trees) for dead TIDs.  This is necessary to avoid
leaving behind dangling TID references that index vacuuming is required
to remove; nothing prevents recently inserted pending list tuples from
containing TIDs that VACUUM already considers dead.

However, ginbulkdelete neglected to perform pending list cleanup on
VACUUM's second or subsequent call.  It was therefore possible for a
VACUUM that requires multiple rounds of index vacuuming to leave behind
dangling references.

To fix, teach ginbulkdelete to perform pending list cleanup during every
call.  In passing, tweak some related comments in the pending list
cleanup path to make it clear why it's safe for VACUUM to not _fully_
empty an index's pending list.

This was arguably an oversight in commit e2c79e14, which fixed a similar
issue where pending list cleanup by VACUUM could end early, but missed
this closely related problem.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/CAH2-Wzmsa-RPA2Ko8A5LaGOnmbpimJ--71xkiBqwgjk3Fq8YEg@mail.gmail.com
Backpatch-through: 14

src/backend/access/gin/ginfast.c

@@ -777,7 +777,7 @@ processPendingPage(BuildAccumulator *accum, KeyArray *ka,
  * If stats isn't null, we count deleted pending pages into the counts.
  */
 void
-ginInsertCleanup(GinState *ginstate, bool full_clean,
+ginInsertCleanup(GinState *ginstate, bool must_empty_list,
 				 bool fill_fsm, bool forceCleanup,
 				 IndexBulkDeleteResult *stats)
 {
@@ -808,7 +808,9 @@ ginInsertCleanup(GinState *ginstate, bool full_clean,
 	{
 		/*
 		 * We are called from [auto]vacuum/analyze or gin_clean_pending_list()
-		 * and we would like to wait concurrent cleanup to finish.
+		 * and we must wait for concurrent cleanup to finish.  In particular,
+		 * VACUUM must have the opportunity to remove any dead TIDs that are
+		 * now in the pending list.
 		 */
 		LockPage(index, GIN_METAPAGE_BLKNO, ExclusiveLock);
 		workMemory =
@@ -880,11 +882,12 @@ ginInsertCleanup(GinState *ginstate, bool full_clean,
 
 		/*
 		 * Are we walk through the page which as we remember was a tail when
-		 * we start our cleanup?  But if caller asks us to clean up whole
-		 * pending list then ignore old tail, we will work until list becomes
-		 * empty.
+		 * we start our cleanup?  But if caller asks us to fully empty the
+		 * pending list (not just move all items that were in the list when
+		 * blknoFinish was established) then ignore old tail and work until
+		 * the list is fully empty.
 		 */
-		if (blkno == blknoFinish && full_clean == false)
+		if (blkno == blknoFinish && !must_empty_list)
 			cleanupFinish = true;
 
 		/*

src/backend/access/gin/ginvacuum.c

@@ -603,14 +603,20 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	{
 		/* Yes, so initialize stats to zeroes */
 		stats = (IndexBulkDeleteResult *) palloc0(sizeof(IndexBulkDeleteResult));
-
-		/*
-		 * and cleanup any pending inserts
-		 */
-		ginInsertCleanup(&gvs.ginstate, !AmAutoVacuumWorkerProcess(),
-						 false, true, stats);
 	}
 
+	/*
+	 * The pending list might have already-dead TIDs that VACUUM now requires
+	 * us to remove from the index.  We must force cleanup of the pending list
+	 * now, before vacuuming proper begins, to make sure nothing is missed.
+	 *
+	 * When running in an autovacuum worker, we won't necessarily _fully_
+	 * empty the pending list.  This is still safe; concurrent inserters
+	 * cannot insert new tuples whose TIDs VACUUM needs us to remove.
+	 */
+	ginInsertCleanup(&gvs.ginstate, !AmAutoVacuumWorkerProcess(),
+					 false, true, stats);
+
 	/* we'll re-count the tuples each time */
 	stats->num_index_tuples = 0;
 	gvs.result = stats;

src/include/access/gin_private.h

@@ -464,7 +464,7 @@ extern void ginHeapTupleFastCollect(GinState *ginstate,
 									GinTupleCollector *collector,
 									OffsetNumber attnum, Datum value, bool isNull,
 									ItemPointer ht_ctid);
-extern void ginInsertCleanup(GinState *ginstate, bool full_clean,
+extern void ginInsertCleanup(GinState *ginstate, bool must_empty_list,
 							 bool fill_fsm, bool forceCleanup, IndexBulkDeleteResult *stats);
 
 /* ginpostinglist.c */