database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-30 11:03:19 +03:00
Code Activity

Add postgres_fdw contrib module.

Browse Source 
There's still a lot of room for improvement, but it basically works,
and we need this to be present before we can do anything much with the
writable-foreign-tables patch.  So let's commit it and get on with testing.

Shigeru Hanada, reviewed by KaiGai Kohei and Tom Lane
This commit is contained in:
Tom Lane
2013-02-21 05:26:23 -05:00
parent f435cd1d38
commit d0d75c4022
28 changed files with 4840 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
doc/src/sgml/client-auth.sgml
View File

@ -1098,7 +1098,7 @@ omicron bryanh guest1
<replaceable>servicename</> can be set on the server side using the
<xref linkend="guc-krb-srvname"> configuration parameter, and on the
client side using the <literal>krbsrvname</> connection parameter. (See
also <xref linkend="libpq-connect">.) The installation default can be
also <xref linkend="libpq-paramkeywords">.) The installation default can be
changed from the default <literal>postgres</literal> at build time using
<literal>./configure --with-krb-srvnam=</><replaceable>whatever</>.
In most environments,

1
doc/src/sgml/contrib.sgml
View File

@ -132,6 +132,7 @@ CREATE EXTENSION <replaceable>module_name</> FROM unpackaged;
&pgstatstatements;
&pgstattuple;
&pgtrgm;
&postgres-fdw;
&seg;
&sepgsql;
&contrib-spi;

26
doc/src/sgml/dblink.sgml
View File

@ -8,11 +8,16 @@
</indexterm>
<para>
<filename>dblink</> is a module which supports connections to
<filename>dblink</> is a module that supports connections to
other <productname>PostgreSQL</> databases from within a database
session.
</para>
<para>
See also <xref linkend="postgres-fdw">, which provides roughly the same
functionality using a more modern and standards-compliant infrastructure.
</para>
<refentry id="CONTRIB-DBLINK-CONNECT">
<refmeta>
<refentrytitle>dblink_connect</refentrytitle>
@ -47,12 +52,10 @@ dblink_connect(text connname, text connstr) returns text
<para>
The connection string may also be the name of an existing foreign
server. It is recommended to use the foreign-data wrapper
<literal>dblink_fdw</literal> when defining the corresponding foreign
server. See the example below, as well as the following:
<simplelist type="inline">
<member><xref linkend="sql-createserver"></member>
<member><xref linkend="sql-createusermapping"></member>
</simplelist>
<literal>dblink_fdw</literal> when defining the foreign
server. See the example below, as well as
<xref linkend="sql-createserver"> and
<xref linkend="sql-createusermapping">.
</para>
</refsect1>
@ -77,8 +80,8 @@ dblink_connect(text connname, text connstr) returns text
<para><application>libpq</>-style connection info string, for example
<literal>hostaddr=127.0.0.1 port=5432 dbname=mydb user=postgres
password=mypasswd</>.
For details see <function>PQconnectdb</> in
<xref linkend="libpq-connect">.
For details see <xref linkend="libpq-connstring">.
Alternatively, the name of a foreign server.
</para>
</listitem>
</varlistentry>
@ -133,9 +136,10 @@ SELECT dblink_connect('myconn', 'dbname=postgres');
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
CREATE USER dblink_regression_test WITH PASSWORD 'secret';
CREATE SERVER fdtest FOREIGN DATA WRAPPER dblink_fdw OPTIONS (hostaddr '127.0.0.1', dbname 'contrib_regression');
CREATE USER dblink_regression_test WITH PASSWORD 'secret';
CREATE USER MAPPING FOR dblink_regression_test SERVER fdtest OPTIONS (user 'dblink_regression_test', password 'secret');
GRANT USAGE ON FOREIGN SERVER fdtest TO dblink_regression_test;
GRANT SELECT ON TABLE foo TO dblink_regression_test;
@ -166,7 +170,7 @@ SELECT * FROM dblink('myconn','SELECT * FROM foo') AS t(a int, b text, c text[])
\c - :ORIGINAL_USER
REVOKE USAGE ON FOREIGN SERVER fdtest FROM dblink_regression_test;
REVOKE SELECT ON TABLE foo FROM dblink_regression_test;
REVOKE SELECT ON TABLE foo FROM dblink_regression_test;
DROP USER MAPPING FOR dblink_regression_test SERVER fdtest;
DROP USER dblink_regression_test;
DROP SERVER fdtest;

1
doc/src/sgml/filelist.sgml
View File

@ -134,6 +134,7 @@
<!ENTITY pgtesttiming SYSTEM "pgtesttiming.sgml">
<!ENTITY pgtrgm SYSTEM "pgtrgm.sgml">
<!ENTITY pgupgrade SYSTEM "pgupgrade.sgml">
<!ENTITY postgres-fdw SYSTEM "postgres-fdw.sgml">
<!ENTITY seg SYSTEM "seg.sgml">
<!ENTITY contrib-spi SYSTEM "contrib-spi.sgml">
<!ENTITY sepgsql SYSTEM "sepgsql.sgml">

2
doc/src/sgml/libpq.sgml
View File

@ -6941,7 +6941,7 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
<para>
The file uses an <quote>INI file</quote> format where the section
name is the service name and the parameters are connection
parameters; see <xref linkend="libpq-connect"> for a list. For
parameters; see <xref linkend="libpq-paramkeywords"> for a list. For
example:
<programlisting>
# comment

325
doc/src/sgml/postgres-fdw.sgml Normal file
View File

@ -0,0 +1,325 @@
<!-- doc/src/sgml/postgres-fdw.sgml -->
<sect1 id="postgres-fdw" xreflabel="postgres_fdw">
<title>postgres_fdw</title>
<indexterm zone="postgres-fdw">
<primary>postgres_fdw</primary>
</indexterm>
<para>
The <filename>postgres_fdw</> module provides the foreign-data wrapper
<literal>postgres_fdw</literal>, which can be used to access data
stored in external <productname>PostgreSQL</productname> servers.
</para>
<para>
The functionality provided by this module overlaps substantially
with the functionality of the older <xref linkend="dblink"> module.
But <filename>postgres_fdw</> provides more transparent and
standards-compliant syntax for accessing remote tables, and can give
better performance in many cases.
</para>
<para>
To prepare for remote access using <filename>postgres_fdw</>:
<orderedlist spacing="compact">
<listitem>
<para>
Install the <filename>postgres_fdw</> extension using <xref
linkend="sql-createextension">.
</para>
</listitem>
<listitem>
<para>
Create a foreign server object, using <xref linkend="sql-createserver">,
to represent each remote database you want to connect to.
Specify connection information, except <literal>user</literal> and
<literal>password</literal>, as options of the server object.
</para>
</listitem>
<listitem>
<para>
Create a user mapping, using <xref linkend="sql-createusermapping">, for
each database user you want to allow to access each foreign server.
Specify the remote user name and password to use as
<literal>user</literal> and <literal>password</literal> options of the
user mapping.
</para>
</listitem>
<listitem>
<para>
Create a foreign table, using <xref linkend="sql-createforeigntable">,
for each remote table you want to access. The columns of the foreign
table must match the referenced remote table. You can, however, use
table and/or column names different from the remote table's, if you
specify the correct remote names as options of the foreign table object.
</para>
</listitem>
</orderedlist>
</para>
<para>
Now you need only <command>SELECT</> from a foreign table to access
the data stored in its underlying remote table.
</para>
<para>
It is generally recommended that the columns of a foreign table be declared
with exactly the same data types, and collations if applicable, as the
referenced columns of the remote table. Although <filename>postgres_fdw</>
is currently rather forgiving about performing data type conversions at
need, surprising semantic anomalies may arise when types or collations do
not match, due to the remote server interpreting <literal>WHERE</> clauses
slightly differently from the local server.
</para>
<para>
Note that a foreign table can be declared with fewer columns, or with a
different column order, than its underlying remote table has. Matching
of columns to the remote table is by name, not position.
</para>
<sect2>
<title>FDW Options of postgres_fdw</title>
<sect3>
<title>Connection Options</title>
<para>
A foreign server using the <filename>postgres_fdw</> foreign data wrapper
can have the same options that <application>libpq</> accepts in
connection strings, as described in <xref linkend="libpq-paramkeywords">,
except that these options are not allowed:
<itemizedlist spacing="compact">
<listitem>
<para>
<literal>user</literal> and <literal>password</literal> (specify these
for a user mapping, instead)
</para>
</listitem>
<listitem>
<para>
<literal>client_encoding</> (this is automatically set from the local
server encoding)
</para>
</listitem>
<listitem>
<para>
<literal>fallback_application_name</> (always set to
<literal>postgres_fdw</>)
</para>
</listitem>
</itemizedlist>
</para>
<para>
Only superusers may connect to foreign servers without password
authentication, so always specify the <literal>password</literal> option
for user mappings belonging to non-superusers.
</para>
</sect3>
<sect3>
<title>Object Name Options</title>
<para>
These options can be used to control the names used in SQL statements
sent to the remote <productname>PostgreSQL</productname> server. These
options are needed when a foreign table is created with names different
from the underlying remote table's names.
</para>
<variablelist>
<varlistentry>
<term><literal>schema_name</literal></term>
<listitem>
<para>
This option, which can be specified for a foreign table, gives the
schema name to use for the foreign table on the remote server. If this
option is omitted, the name of the foreign table's schema is used.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>table_name</literal></term>
<listitem>
<para>
This option, which can be specified for a foreign table, gives the
table name to use for the foreign table on the remote server. If this
option is omitted, the foreign table's name is used.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>column_name</literal></term>
<listitem>
<para>
This option, which can be specified for a column of a foreign table,
gives the column name to use for the column on the remote server.
If this option is omitted, the column's name is used.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect3>
<sect3>
<title>Cost Estimation Options</title>
<para>
<filename>postgres_fdw</> retrieves remote data by executing queries
against remote servers, so ideally the estimated cost of scanning a
foreign table should be whatever it costs to be done on the remote
server, plus some overhead for communication. The most reliable way to
get such an estimate is to ask the remote server and then add something
for overhead &mdash; but for simple queries, it may not be worth the cost
of an additional remote query to get a cost estimate.
So <filename>postgres_fdw</> provides the following options to control
how cost estimation is done:
</para>
<variablelist>
<varlistentry>
<term><literal>use_remote_estimate</literal></term>
<listitem>
<para>
This option, which can be specified for a foreign table or a foreign
server, controls whether <filename>postgres_fdw</> issues remote
<command>EXPLAIN</command> commands to obtain cost estimates.
A setting for a foreign table overrides any setting for its server,
but only for that table.
The default is <literal>false</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>fdw_startup_cost</literal></term>
<listitem>
<para>
This option, which can be specified for a foreign server, is a numeric
value that is added to the estimated startup cost of any foreign-table
scan on that server. This represents the additional overhead of
establishing a connection, parsing and planning the query on the
remote side, etc.
The default value is <literal>100</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>fdw_tuple_cost</literal></term>
<listitem>
<para>
This option, which can be specified for a foreign server, is a numeric
value that is used as extra cost per-tuple for foreign-table
scans on that server. This represents the additional overhead of
data transfer between servers. You might increase or decrease this
number to reflect higher or lower network delay to the remote server.
The default value is <literal>0.01</literal>.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
When <literal>use_remote_estimate</literal> is true,
<filename>postgres_fdw</> obtains rowcount and cost estimates from the
remote server and then adds <literal>fdw_startup_cost</literal> and
<literal>fdw_tuple_cost</literal> to the cost estimates. When
<literal>use_remote_estimate</literal> is false,
<filename>postgres_fdw</> performs local rowcount and cost estimation
and then adds <literal>fdw_startup_cost</literal> and
<literal>fdw_tuple_cost</literal> to the cost estimates. This local
estimation is unlikely to be very accurate unless local copies of the
remote table's statistics are available. Running
<xref linkend="sql-analyze"> on the foreign table is the way to update
the local statistics; this will perform a scan of the remote table and
then calculate and store statistics just as though the table were local.
Keeping local statistics can be a useful way to reduce per-query planning
overhead for a remote table &mdash; but if the remote table is
frequently updated, the local statistics will soon be obsolete.
</para>
</sect3>
</sect2>
<sect2>
<title>Connection Management</title>
<para>
<filename>postgres_fdw</filename> establishes a connection to a
foreign server during the first query that uses a foreign table
associated with the foreign server. This connection is kept and
re-used for subsequent queries in the same session. However, if
multiple user identities (user mappings) are used to access the foreign
server, a connection is established for each user mapping.
</para>
</sect2>
<sect2>
<title>Transaction Management</title>
<para>
During a query that references any remote tables on a foreign server,
<filename>postgres_fdw</filename> opens a transaction on the
remote server if one is not already open corresponding to the current
local transaction. The remote transaction is committed or aborted when
the local transaction commits or aborts. Savepoints are similarly
managed by creating corresponding remote savepoints.
</para>
<para>
The remote transaction uses <literal>SERIALIZABLE</>
isolation level when the local transaction has <literal>SERIALIZABLE</>
isolation level; otherwise it uses <literal>REPEATABLE READ</>
isolation level. This choice ensures that if a query performs multiple
table scans on the remote server, it will get snapshot-consistent results
for all the scans. A consequence is that successive queries within a
single transaction will see the same data from the remote server, even if
concurrent updates are occurring on the remote server due to other
activities. That behavior would be expected anyway if the local
transaction uses <literal>SERIALIZABLE</> or <literal>REPEATABLE READ</>
isolation level, but it might be surprising for a <literal>READ
COMMITTED</> local transaction. A future
<productname>PostgreSQL</productname> release might modify these rules.
</para>
</sect2>
<sect2>
<title>Remote Query Optimization</title>
<para>
<filename>postgres_fdw</> attempts to optimize remote queries to reduce
the amount of data transferred from foreign servers. This is done by
sending query <literal>WHERE</> clauses to the remote server for
execution, and by not retrieving table columns that are not needed for
the current query. To reduce the risk of misexecution of queries,
<literal>WHERE</> clauses are not sent to the remote server unless they use
only built-in data types, operators, and functions. Operators and
functions in the clauses must be <literal>IMMUTABLE</> as well.
</para>
<para>
The query that is actually sent to the remote server for execution can
be examined using <command>EXPLAIN VERBOSE</>.
</para>
</sect2>
<sect2>
<title>Author</title>
<para>
Shigeru Hanada <email>shigeru.hanada@gmail.com</email>
</para>
</sect2>
</sect1>

2
doc/src/sgml/queries.sgml
View File

@ -699,7 +699,7 @@ SELECT *
WHERE proname LIKE 'bytea%';
</programlisting>
The <xref linkend="CONTRIB-DBLINK-FUNCTION"> function
(part of the <xref linkend="dblink"> module>) executes
(part of the <xref linkend="dblink"> module) executes
a remote query. It is declared to return
<type>record</> since it might be used for any kind of query.
The actual column set must be specified in the calling query so

3
doc/src/sgml/recovery-config.sgml
View File

@ -314,8 +314,7 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
<para>
Specifies a connection string to be used for the standby server
to connect with the primary. This string is in the format
accepted by the libpq <function>PQconnectdb</function> function,
described in <xref linkend="libpq-connect">. If any option is
described in <xref linkend="libpq-connstring">. If any option is
unspecified in this string, then the corresponding environment
variable (see <xref linkend="libpq-envars">) is checked. If the
environment variable is not set either, then

8
doc/src/sgml/ref/create_foreign_data_wrapper.sgml
View File

@ -121,14 +121,6 @@ CREATE FOREIGN DATA WRAPPER <replaceable class="parameter">name</replaceable>
There is no support for updating a foreign table, and optimization of
queries is primitive (and mostly left to the wrapper, too).
</para>
<para>
There is one built-in foreign-data wrapper validator function
provided:
<filename>postgresql_fdw_validator</filename>, which accepts
options corresponding to <application>libpq</> connection
parameters.
</para>
</refsect1>
<refsect1>

12
doc/src/sgml/ref/create_foreign_table.sgml
View File

@ -32,7 +32,7 @@ CREATE FOREIGN TABLE [ IF NOT EXISTS ] <replaceable class="PARAMETER">table_name
<title>Description</title>
<para>
<command>CREATE FOREIGN TABLE</command> will create a new foreign table
<command>CREATE FOREIGN TABLE</command> creates a new foreign table
in the current database. The table will be owned by the user issuing the
command.
</para>
@ -54,8 +54,9 @@ CREATE FOREIGN TABLE [ IF NOT EXISTS ] <replaceable class="PARAMETER">table_name
</para>
<para>
To be able to create a table, you must have <literal>USAGE</literal>
privilege on all column types.
To be able to create a foreign table, you must have <literal>USAGE</literal>
privilege on the foreign server, as well as <literal>USAGE</literal>
privilege on all column types used in the table.
</para>
</refsect1>
@ -134,7 +135,7 @@ CREATE FOREIGN TABLE [ IF NOT EXISTS ] <replaceable class="PARAMETER">table_name
<term><replaceable class="PARAMETER">server_name</replaceable></term>
<listitem>
<para>
The name of an existing server for the foreign table.
The name of an existing foreign server to use for the foreign table.
For details on defining a server, see <xref
linkend="SQL-CREATESERVER">.
</para>
@ -164,7 +165,8 @@ CREATE FOREIGN TABLE [ IF NOT EXISTS ] <replaceable class="PARAMETER">table_name
<title>Examples</title>
<para>
Create foreign table <structname>films</> with <structname>film_server</>:
Create foreign table <structname>films</>, which will be accessed through
the server <structname>film_server</>:
<programlisting>
CREATE FOREIGN TABLE films (

24
doc/src/sgml/ref/create_server.sgml
View File

@ -110,11 +110,10 @@ CREATE SERVER <replaceable class="parameter">server_name</replaceable> [ TYPE '<
<title>Notes</title>
<para>
When using the <application>dblink</application> module
(see <xref linkend="dblink">), the foreign server name can be used
When using the <xref linkend="dblink"> module,
a foreign server's name can be used
as an argument of the <xref linkend="contrib-dblink-connect">
function to indicate the connection parameters. See also there for
more examples. It is necessary to have
function to indicate the connection parameters. It is necessary to have
the <literal>USAGE</literal> privilege on the foreign server to be
able to use it in this way.
</para>
@ -123,20 +122,14 @@ CREATE SERVER <replaceable class="parameter">server_name</replaceable> [ TYPE '<
<refsect1>
<title>Examples</title>
<para>
Create a server <literal>foo</> that uses the built-in foreign-data
wrapper <literal>default</>:
<programlisting>
CREATE SERVER foo FOREIGN DATA WRAPPER "default";
</programlisting>
</para>
<para>
Create a server <literal>myserver</> that uses the
foreign-data wrapper <literal>pgsql</>:
foreign-data wrapper <literal>postgres_fdw</>:
<programlisting>
CREATE SERVER myserver FOREIGN DATA WRAPPER pgsql OPTIONS (host 'foo', dbname 'foodb', port '5432');
</programlisting></para>
CREATE SERVER myserver FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host 'foo', dbname 'foodb', port '5432');
</programlisting>
See <xref linkend="postgres-fdw"> for more details.
</para>
</refsect1>
<refsect1>
@ -154,6 +147,7 @@ CREATE SERVER myserver FOREIGN DATA WRAPPER pgsql OPTIONS (host 'foo', dbname 'f
<member><xref linkend="sql-alterserver"></member>
<member><xref linkend="sql-dropserver"></member>
<member><xref linkend="sql-createforeigndatawrapper"></member>
<member><xref linkend="sql-createforeigntable"></member>
<member><xref linkend="sql-createusermapping"></member>
</simplelist>
</refsect1>

7
doc/src/sgml/ref/grant.sgml
View File

@ -357,10 +357,9 @@ GRANT <replaceable class="PARAMETER">role_name</replaceable> [, ...] TO <replace
to create new servers using that foreign-data wrapper.
</para>
<para>
For servers, this privilege enables the grantee to create,
alter, and drop his own user's user mappings associated with
that server. Also, it enables the grantee to query the options
of the server and associated user mappings.
For servers, this privilege enables the grantee to create foreign
tables using the server, and also to create, alter, or drop his own
user's user mappings associated with that server.
</para>
</listitem>
</varlistentry>

3
doc/src/sgml/ref/pg_isready.sgml
View File

@ -54,7 +54,8 @@ PostgreSQL documentation
with a valid <acronym>URI</acronym> prefix
(<literal>postgresql://</literal>
or <literal>postgres://</literal>), it is treated as a
<parameter>conninfo</parameter> string. See <xref linkend="libpq-connect"> for more information.
<parameter>conninfo</parameter> string. See <xref
linkend="libpq-connstring"> for more information.
</para>
</listitem>
</varlistentry>

9
doc/src/sgml/ref/psql-ref.sgml
View File

@ -120,7 +120,8 @@ PostgreSQL documentation
with a valid <acronym>URI</acronym> prefix
(<literal>postgresql://</literal>
or <literal>postgres://</literal>), it is treated as a
<parameter>conninfo</parameter> string. See <xref linkend="libpq-connect"> for more information.
<parameter>conninfo</parameter> string. See <xref
linkend="libpq-connstring"> for more information.
</para>
</listitem>
</varlistentry>
@ -608,9 +609,9 @@ PostgreSQL documentation
$ <userinput>psql "service=myservice sslmode=require"</userinput>
$ <userinput>psql postgresql://dbmaster:5433/mydb?sslmode=require</userinput>
</programlisting>
This way you can also use <acronym>LDAP</acronym> for connection parameter lookup as
described in <xref linkend="libpq-ldap">.
See <xref linkend="libpq-connect"> for more information on all the
This way you can also use <acronym>LDAP</acronym> for connection
parameter lookup as described in <xref linkend="libpq-ldap">.
See <xref linkend="libpq-paramkeywords"> for more information on all the
available connection options.
</para>

2
doc/src/sgml/runtime.sgml
View File

@ -1751,7 +1751,7 @@ pg_dumpall -p 5432 | psql -d postgres -p 5433
(<xref linkend="ssl-tcp">). The TCP client must connect using
<literal>sslmode=verify-ca</> or
<literal>verify-full</> and have the appropriate root certificate
file installed (<xref linkend="libpq-connect">).
file installed (<xref linkend="libq-ssl-certificates">).
</para>
</sect1>