mirror of
https://github.com/postgres/postgres.git
synced 2025-07-07 00:36:50 +03:00
Improve findoidjoins to cover more cases.
Teach the program and script to deal with OID-array referencing columns, which we now have several of. Also, modify the recommended usage process to specify that the program should be run against the regression database rather than template1. This lets it find numerous joins that cannot be found in template1 because the relevant catalogs are entirely empty. Together these changes add seventeen formerly-missed cases to the oidjoins regression test.
This commit is contained in:
Tom Lane
@ -4,39 +4,41 @@ findoidjoins
|
||||
============
|
||||
|
||||
This program scans a database and prints oid fields (also reg* fields)
|
||||
and the tables they join to. We don't really recommend running it on
|
||||
anything but an empty database, such as template1; else it's likely to
|
||||
be very slow.
|
||||
and the tables they join to. It is normally used to check the system
|
||||
catalog join relationships (shown below for 9.1devel).
|
||||
|
||||
Run on an empty database, it returns the system join relationships (shown
|
||||
below for 9.1devel). Note that unexpected matches may indicate bogus entries
|
||||
in system tables --- don't accept a peculiar match without question.
|
||||
In particular, a field shown as joining to more than one target table is
|
||||
probably messed up. In 9.1devel, the *only* fields that should join to more
|
||||
than one target are pg_description.objoid, pg_depend.objid,
|
||||
pg_depend.refobjid, pg_shdescription.objoid, pg_shdepend.objid, and
|
||||
pg_shdepend.refobjid. (Running make_oidjoins_check is an easy way to spot
|
||||
fields joining to more than one table, BTW.) NOTE: in an empty database,
|
||||
findoidjoins may not report joins for pg_shdescription and pg_shdepend for
|
||||
lack of any entries there.
|
||||
Historically this has been run against an empty database such as template1,
|
||||
but there's a problem with that approach: some of the catalogs are empty
|
||||
and so their joining columns won't show up in the output. Current practice
|
||||
is to run it against the regression-test database, which populates the
|
||||
catalogs in interesting ways.
|
||||
|
||||
Note that unexpected matches may indicate bogus entries in system tables;
|
||||
don't accept a peculiar match without question. In particular, a field
|
||||
shown as joining to more than one target table is probably messed up.
|
||||
In 9.1devel, the *only* fields that should join to more than one target
|
||||
table are pg_description.objoid, pg_depend.objid, pg_depend.refobjid,
|
||||
pg_shdescription.objoid, pg_shdepend.objid, and pg_shdepend.refobjid.
|
||||
(Running make_oidjoins_check is an easy way to spot fields joining to more
|
||||
than one table, BTW.)
|
||||
|
||||
The shell script make_oidjoins_check converts findoidjoins' output
|
||||
into an SQL script that checks for dangling links (entries in an
|
||||
OID or REG* column that don't match any row in the expected table).
|
||||
Note that fields joining to more than one table are NOT processed.
|
||||
Note that fields joining to more than one table are NOT processed,
|
||||
just reported as linking to more than one table.
|
||||
|
||||
The result of make_oidjoins_check should be installed as the "oidjoins"
|
||||
regression test. The oidjoins test should be updated after any
|
||||
revision in the patterns of cross-links between system tables.
|
||||
(Ideally we'd just regenerate the script as part of the regression
|
||||
tests themselves, but that seems too slow...)
|
||||
(Typically we update it at the end of each development cycle.)
|
||||
|
||||
NOTE: in 9.1devel, make_oidjoins_check produces two bogus join checks:
|
||||
NOTE: as of 9.1devel, make_oidjoins_check produces two bogus join checks:
|
||||
Join pg_catalog.pg_class.relfilenode => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_database.datlastsysoid => pg_catalog.pg_database.oid
|
||||
These are artifacts and should not be added to the oidjoins regress test.
|
||||
You might also get output for pg_shdepend.refobjid and pg_shdescription.objoid,
|
||||
neither of which should be added.
|
||||
neither of which should be added to the regress test.
|
||||
|
||||
---------------------------------------------------------------------------
|
||||
|
||||
@ -70,6 +72,7 @@ Join pg_catalog.pg_amproc.amprocfamily => pg_catalog.pg_opfamily.oid
|
||||
Join pg_catalog.pg_amproc.amproclefttype => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_amproc.amprocrighttype => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_amproc.amproc => pg_catalog.pg_proc.oid
|
||||
Join pg_catalog.pg_attrdef.adrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_attribute.attrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_attribute.atttypid => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_attribute.attcollation => pg_catalog.pg_collation.oid
|
||||
@ -78,6 +81,7 @@ Join pg_catalog.pg_cast.casttarget => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_cast.castfunc => pg_catalog.pg_proc.oid
|
||||
Join pg_catalog.pg_class.relnamespace => pg_catalog.pg_namespace.oid
|
||||
Join pg_catalog.pg_class.reltype => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_class.reloftype => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_class.relowner => pg_catalog.pg_authid.oid
|
||||
Join pg_catalog.pg_class.relam => pg_catalog.pg_am.oid
|
||||
Join pg_catalog.pg_class.reltablespace => pg_catalog.pg_tablespace.oid
|
||||
@ -86,7 +90,10 @@ Join pg_catalog.pg_class.reltoastidxid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_collation.collnamespace => pg_catalog.pg_namespace.oid
|
||||
Join pg_catalog.pg_collation.collowner => pg_catalog.pg_authid.oid
|
||||
Join pg_catalog.pg_constraint.connamespace => pg_catalog.pg_namespace.oid
|
||||
Join pg_catalog.pg_constraint.conrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_constraint.contypid => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_constraint.conindid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_constraint.confrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_conversion.connamespace => pg_catalog.pg_namespace.oid
|
||||
Join pg_catalog.pg_conversion.conowner => pg_catalog.pg_authid.oid
|
||||
Join pg_catalog.pg_conversion.conproc => pg_catalog.pg_proc.oid
|
||||
@ -96,10 +103,13 @@ Join pg_catalog.pg_db_role_setting.setdatabase => pg_catalog.pg_database.oid
|
||||
Join pg_catalog.pg_depend.classid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_depend.refclassid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_description.classoid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_enum.enumtypid => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_extension.extowner => pg_catalog.pg_authid.oid
|
||||
Join pg_catalog.pg_extension.extnamespace => pg_catalog.pg_namespace.oid
|
||||
Join pg_catalog.pg_index.indexrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_index.indrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_inherits.inhrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_inherits.inhparent => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_language.lanowner => pg_catalog.pg_authid.oid
|
||||
Join pg_catalog.pg_language.lanplcallfoid => pg_catalog.pg_proc.oid
|
||||
Join pg_catalog.pg_language.laninline => pg_catalog.pg_proc.oid
|
||||
@ -137,6 +147,11 @@ Join pg_catalog.pg_statistic.staop1 => pg_catalog.pg_operator.oid
|
||||
Join pg_catalog.pg_statistic.staop2 => pg_catalog.pg_operator.oid
|
||||
Join pg_catalog.pg_statistic.staop3 => pg_catalog.pg_operator.oid
|
||||
Join pg_catalog.pg_tablespace.spcowner => pg_catalog.pg_authid.oid
|
||||
Join pg_catalog.pg_trigger.tgrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_trigger.tgfoid => pg_catalog.pg_proc.oid
|
||||
Join pg_catalog.pg_trigger.tgconstrrelid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_trigger.tgconstrindid => pg_catalog.pg_class.oid
|
||||
Join pg_catalog.pg_trigger.tgconstraint => pg_catalog.pg_constraint.oid
|
||||
Join pg_catalog.pg_ts_config.cfgnamespace => pg_catalog.pg_namespace.oid
|
||||
Join pg_catalog.pg_ts_config.cfgowner => pg_catalog.pg_authid.oid
|
||||
Join pg_catalog.pg_ts_config.cfgparser => pg_catalog.pg_ts_parser.oid
|
||||
@ -168,6 +183,10 @@ Join pg_catalog.pg_type.typmodout => pg_catalog.pg_proc.oid
|
||||
Join pg_catalog.pg_type.typanalyze => pg_catalog.pg_proc.oid
|
||||
Join pg_catalog.pg_type.typbasetype => pg_catalog.pg_type.oid
|
||||
Join pg_catalog.pg_type.typcollation => pg_catalog.pg_collation.oid
|
||||
Join pg_catalog.pg_constraint.conpfeqop []=> pg_catalog.pg_operator.oid
|
||||
Join pg_catalog.pg_constraint.conppeqop []=> pg_catalog.pg_operator.oid
|
||||
Join pg_catalog.pg_constraint.conffeqop []=> pg_catalog.pg_operator.oid
|
||||
Join pg_catalog.pg_proc.proallargtypes []=> pg_catalog.pg_type.oid
|
||||
|
||||
---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@ -46,9 +46,7 @@ main(int argc, char **argv)
|
||||
|
||||
/* Get a list of relations that have OIDs */
|
||||
|
||||
resetPQExpBuffer(&sql);
|
||||
|
||||
appendPQExpBuffer(&sql, "%s",
|
||||
printfPQExpBuffer(&sql, "%s",
|
||||
"SET search_path = public;"
|
||||
"SELECT c.relname, (SELECT nspname FROM "
|
||||
"pg_catalog.pg_namespace n WHERE n.oid = c.relnamespace) AS nspname "
|
||||
@ -68,9 +66,7 @@ main(int argc, char **argv)
|
||||
|
||||
/* Get a list of columns of OID type (or any OID-alias type) */
|
||||
|
||||
resetPQExpBuffer(&sql);
|
||||
|
||||
appendPQExpBuffer(&sql, "%s",
|
||||
printfPQExpBuffer(&sql, "%s",
|
||||
"SELECT c.relname, "
|
||||
"(SELECT nspname FROM pg_catalog.pg_namespace n WHERE n.oid = c.relnamespace) AS nspname, "
|
||||
"a.attname "
|
||||
@ -113,15 +109,15 @@ main(int argc, char **argv)
|
||||
pk_relname = PQgetvalue(pkrel_res, pk, 0);
|
||||
pk_nspname = PQgetvalue(pkrel_res, pk, 1);
|
||||
|
||||
resetPQExpBuffer(&sql);
|
||||
|
||||
appendPQExpBuffer(&sql,
|
||||
printfPQExpBuffer(&sql,
|
||||
"SELECT 1 "
|
||||
"FROM \"%s\".\"%s\" t1, "
|
||||
"\"%s\".\"%s\" t2 "
|
||||
"WHERE t1.\"%s\"::pg_catalog.oid = t2.oid "
|
||||
"LIMIT 1",
|
||||
fk_nspname, fk_relname, pk_nspname, pk_relname, fk_attname);
|
||||
fk_nspname, fk_relname,
|
||||
pk_nspname, pk_relname,
|
||||
fk_attname);
|
||||
|
||||
res = PQexec(conn, sql.data);
|
||||
if (!res || PQresultStatus(res) != PGRES_TUPLES_OK)
|
||||
@ -139,8 +135,85 @@ main(int argc, char **argv)
|
||||
}
|
||||
}
|
||||
|
||||
PQclear(pkrel_res);
|
||||
PQclear(fkrel_res);
|
||||
|
||||
/* Now, do the same for referencing columns that are arrays */
|
||||
|
||||
/* Get a list of columns of OID-array type (or any OID-alias type) */
|
||||
|
||||
printfPQExpBuffer(&sql, "%s",
|
||||
"SELECT c.relname, "
|
||||
"(SELECT nspname FROM pg_catalog.pg_namespace n WHERE n.oid = c.relnamespace) AS nspname, "
|
||||
"a.attname "
|
||||
"FROM pg_catalog.pg_class c, pg_catalog.pg_attribute a "
|
||||
"WHERE a.attnum > 0 AND c.relkind = 'r' "
|
||||
"AND a.attrelid = c.oid "
|
||||
"AND a.atttypid IN ('pg_catalog.oid[]'::regtype, "
|
||||
" 'pg_catalog.regclass[]'::regtype, "
|
||||
" 'pg_catalog.regoper[]'::regtype, "
|
||||
" 'pg_catalog.regoperator[]'::regtype, "
|
||||
" 'pg_catalog.regproc[]'::regtype, "
|
||||
" 'pg_catalog.regprocedure[]'::regtype, "
|
||||
" 'pg_catalog.regtype[]'::regtype, "
|
||||
" 'pg_catalog.regconfig[]'::regtype, "
|
||||
" 'pg_catalog.regdictionary[]'::regtype) "
|
||||
"ORDER BY nspname, c.relname, a.attnum"
|
||||
);
|
||||
|
||||
res = PQexec(conn, sql.data);
|
||||
if (!res || PQresultStatus(res) != PGRES_TUPLES_OK)
|
||||
{
|
||||
fprintf(stderr, "sql error: %s\n", PQerrorMessage(conn));
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
fkrel_res = res;
|
||||
|
||||
/*
|
||||
* For each column and each relation-having-OIDs, look to see if the
|
||||
* column contains any values matching entries in the relation.
|
||||
*/
|
||||
|
||||
for (fk = 0; fk < PQntuples(fkrel_res); fk++)
|
||||
{
|
||||
fk_relname = PQgetvalue(fkrel_res, fk, 0);
|
||||
fk_nspname = PQgetvalue(fkrel_res, fk, 1);
|
||||
fk_attname = PQgetvalue(fkrel_res, fk, 2);
|
||||
|
||||
for (pk = 0; pk < PQntuples(pkrel_res); pk++)
|
||||
{
|
||||
pk_relname = PQgetvalue(pkrel_res, pk, 0);
|
||||
pk_nspname = PQgetvalue(pkrel_res, pk, 1);
|
||||
|
||||
printfPQExpBuffer(&sql,
|
||||
"SELECT 1 "
|
||||
"FROM \"%s\".\"%s\" t1, "
|
||||
"\"%s\".\"%s\" t2 "
|
||||
"WHERE t2.oid = ANY(t1.\"%s\")"
|
||||
"LIMIT 1",
|
||||
fk_nspname, fk_relname,
|
||||
pk_nspname, pk_relname,
|
||||
fk_attname);
|
||||
|
||||
res = PQexec(conn, sql.data);
|
||||
if (!res || PQresultStatus(res) != PGRES_TUPLES_OK)
|
||||
{
|
||||
fprintf(stderr, "sql error: %s\n", PQerrorMessage(conn));
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
|
||||
if (PQntuples(res) != 0)
|
||||
printf("Join %s.%s.%s []=> %s.%s.oid\n",
|
||||
fk_nspname, fk_relname, fk_attname,
|
||||
pk_nspname, pk_relname);
|
||||
|
||||
PQclear(res);
|
||||
}
|
||||
}
|
||||
|
||||
PQclear(fkrel_res);
|
||||
|
||||
PQclear(pkrel_res);
|
||||
|
||||
PQfinish(conn);
|
||||
|
||||
termPQExpBuffer(&sql);
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
# src/tools/findoidjoins/make_oidjoins_check
|
||||
|
||||
# You first run findoidjoins on the template1 database, and send that
|
||||
# You first run findoidjoins on the regression database, then send that
|
||||
# output into this script to generate a list of SQL statements.
|
||||
|
||||
# NOTE: any field that findoidjoins thinks joins to more than one table
|
||||
@ -12,17 +12,16 @@
|
||||
# Caution: you may need to use GNU awk.
|
||||
AWK=${AWK:-awk}
|
||||
|
||||
TMP="${TMPDIR:-/tmp}/make_oidjoins_check.$$"
|
||||
trap "rm -rf $TMP" 0 1 2 3 15
|
||||
|
||||
# Create a temporary directory with the proper permissions so no one can
|
||||
# intercept our temporary files and cause a security breach.
|
||||
TMP="${TMPDIR:-/tmp}/make_oidjoins_check.$$"
|
||||
OMASK="`umask`"
|
||||
umask 077
|
||||
if ! mkdir $TMP
|
||||
then echo "Can't create temporary directory $TMP." 1>&2
|
||||
exit 1
|
||||
fi
|
||||
trap "rm -rf $TMP" 0 1 2 3 15
|
||||
umask "$OMASK"
|
||||
unset OMASK
|
||||
|
||||
@ -40,7 +39,7 @@ if [ -s $DUPSFILE ] ; then
|
||||
cat $DUPSFILE 1>&2
|
||||
fi
|
||||
|
||||
# Get the non-multiply-referenced fields.
|
||||
# Get the fields without multiple references.
|
||||
cat $INPUTFILE | while read LINE
|
||||
do
|
||||
set -- $LINE
|
||||
@ -49,7 +48,7 @@ done >$NONDUPSFILE
|
||||
|
||||
# Generate the output.
|
||||
cat $NONDUPSFILE |
|
||||
$AWK -F'[ \.]' '\
|
||||
$AWK -F'[ .]' '\
|
||||
BEGIN \
|
||||
{
|
||||
printf "\
|
||||
@ -57,6 +56,7 @@ $AWK -F'[ \.]' '\
|
||||
-- This is created by pgsql/src/tools/findoidjoins/make_oidjoins_check\n\
|
||||
--\n";
|
||||
}
|
||||
$5 == "=>" \
|
||||
{
|
||||
printf "\
|
||||
SELECT ctid, %s\n\
|
||||
@ -65,6 +65,16 @@ WHERE %s != 0 AND\n\
|
||||
NOT EXISTS(SELECT 1 FROM %s.%s pk WHERE pk.oid = fk.%s);\n",
|
||||
$4, $2, $3, $4,
|
||||
$6, $7, $4;
|
||||
}
|
||||
$5 == "[]=>" \
|
||||
{
|
||||
printf "\
|
||||
SELECT ctid, %s\n\
|
||||
FROM (SELECT ctid, unnest(%s) AS %s FROM %s.%s) fk\n\
|
||||
WHERE %s != 0 AND\n\
|
||||
NOT EXISTS(SELECT 1 FROM %s.%s pk WHERE pk.oid = fk.%s);\n",
|
||||
$4, $4, $4, $2, $3, $4,
|
||||
$6, $7, $4;
|
||||
}'
|
||||
|
||||
exit 0
|
||||
|
||||
Reference in New Issue
Block a user