speed up repetitive failed searches; per pghackers discussion in late
January. inval.c logic substantially simplified, since we can now treat
inserts and deletes alike as far as inval events are concerned. Some
repair work needed in heap_create_with_catalog, which turns out to have
been doing CommandCounterIncrement at a point where the new relation has
non-self-consistent catalog entries. With the new inval code, that
resulted in assert failures during a relcache entry rebuild.
Improve 'pg_internal.init' relcache entry preload mechanism so that it is
safe to use for all system catalogs, and arrange to preload a realistic
set of system-catalog entries instead of only the three nailed-in-cache
indexes that were formerly loaded this way. Fix mechanism for deleting
out-of-date pg_internal.init files: this must be synchronized with transaction
commit, not just done at random times within transactions. Drive it off
relcache invalidation mechanism so that no special-case tests are needed.
Cache additional information in relcache entries for indexes (their pg_index
tuples and index-operator OIDs) to eliminate repeated lookups. Also cache
index opclass info at the per-opclass level to avoid repeated lookups during
relcache load.
Generalize 'systable scan' utilities originally developed by Hiroshi,
move them into genam.c, use in a number of places where there was formerly
ugly code for choosing either heap or index scan. In particular this allows
simplification of the logic that prevents infinite recursion between syscache
and relcache during startup: we can easily switch to heapscans in relcache.c
when and where needed to avoid recursion, so IndexScanOK becomes simpler and
does not need any expensive initialization.
Eliminate useless opening of a heapscan data structure while doing an indexscan
(this saves an mdnblocks call and thus at least one kernel call).
This seems the right thing for most usages, but I notice two places
where it is the wrong thing. One is that the default permissions on
TOAST rels should be no-access, not world-readable; the other is that
PrepareForTupleInvalidation doesn't really need to spend time looking
at tuples of TOAST relations.
SI messages now include the relevant database OID, so that operations
in one database do not cause useless cache flushes in backends attached
to other databases. Declare SI messages properly using a union, to
eliminate the former assumption that Oid is the same size as int or Index.
Rewrite the nearly-unreadable code in inval.c, and document it better.
Arrange for catcache flushes at end of command/transaction to happen before
relcache flushes do --- this avoids loading a new tuple into the catcache
while setting up new relcache entry, only to have it be flushed again
immediately.
CatalogCacheFlushRelation (formerly called SystemCacheRelationFlushed)
how to distinguish tuples it should flush from those it needn't; this
means a relcache flush event now only removes the catcache entries
it ought to, rather than zapping the caches completely as it used to.
Testing with the regression tests indicates that this considerably
improves the lifespan of catcache entries. Also, rearrange catcache
data structures so that the limit on number of cached tuples applies
globally across all the catcaches, rather than being per-catcache.
It was a little silly to have the same size limit on both, say,
pg_attribute caches and pg_am caches (there being only four possible
rows in the latter...). Doing LRU removal across all the caches
instead of locally in each one should reduce cache reload traffic
in the more heavily used caches and improve the efficiency of
cache memory use.
(WAL logging for this is not done yet, however.) Clean up a number of really
crufty things that are no longer needed now that DROP behaves nicely. Make
temp table mapper do the right things when drop or rename affecting a temp
table is rolled back. Also, remove "relation modified while in use" error
check, in favor of locking tables at first reference and holding that lock
throughout the statement.
syscache and relcache flushes). Relcache entry rebuild now preserves
original tupledesc, rewrite rules, and triggers if possible, so that pointers
to these things remain valid --- if these things change while relcache entry
has positive refcount, we elog(ERROR) to avoid later crash. Arrange for
xact-local rels to be rebuilt when an SI inval message is seen for them,
so that they are updated by CommandCounterIncrement the same as regular rels.
(This is useful because of Hiroshi's recent changes to process our own SI
messages at CommandCounterIncrement time.) This allows simplification of
some routines that previously hacked around the lack of an automatic update.
catcache now keeps its own copy of tupledesc for its relation, rather than
depending on the relcache's copy; this avoids needing to reinitialize catcache
during a cache flush, which saves some cycles and eliminates nasty circularity
problems that occur if a cache flush happens while trying to initialize a
catcache.
Eliminate a number of permanent memory leaks that used to happen during
catcache or relcache flush; not least of which was that catcache never
freed any cached tuples! (Rule parsetree storage is still leaked, however;
will fix that separately.)
Nothing done yet about code that uses tuples retrieved by SearchSysCache
for longer than is safe.
we *always* rebuild, rather than deleting, an invalidated relcache entry
that has positive refcount. Otherwise an SI cache overrun leads to
dangling Relation pointers all over the place!
inval.c thought it could safely use the catcache to look up the OIDs of
system relations. Not good, considering that inval.c could be called
during catcache loading, if a shared-inval message arrives. Rip out the
lookup logic and instead use the known OIDs from pg_class.h.
no longer returns buffer pointer, can be gotten from scan;
descriptor; bootstrap can create multi-key indexes;
pg_procname index now is multi-key index; oidint2, oidint4, oidname
are gone (must be removed from regression tests); use System Cache
rather than sequential scan in many places; heap_modifytuple no
longer takes buffer parameter; remove unused buffer parameter in
a few other functions; oid8 is not index-able; remove some use of
single-character variable names; cleanup Buffer variables usage
and scan descriptor looping; cleaned up allocation and freeing of
tuples; 18k lines of diff;
The diff looks so simple and easy. But to find it wasn't fun.
It must have been there for a long time. What happened:
When a tuple in one of some central catalogs was updated, the
referenced relation got flushed, so it would be reopened on
the next access (to reflect new triggers, rules and table
structure changes into the relation cache).
Some data (the tupleDescriptor e.g.) is used in the system
cache too. So when a relation is subject to the system cache,
this must know too that a cached system relation got flushed
because the tupleDesc data gets freed during the flush!
For the GRANT/REVOKE on pg_class it was slightly different.
There is some local data in inval.c that gets initialized on
the first invalidation of a tuple in some central catalogs.
This needs a SysCache lookup in pg_class. But when the first
of all commands is a GRANT on pg_class, exactly the needed
tuple is the one actually invalidated. So I added little code
snippets that the initialization of the local variables in
inval.c will already happen during InitPostgres().
