mirror of
https://github.com/postgres/postgres.git
synced 2025-07-30 11:03:19 +03:00
Allow batch insertion during COPY into a foreign table.
Commit3d956d956
allowed the COPY, but it's done by inserting individual rows to the foreign table, so it can be inefficient due to the overhead caused by each round-trip to the foreign server. To improve performance of the COPY in such a case, this patch allows batch insertion, by extending the multi-insert machinery in CopyFrom() to the foreign-table case so that we insert multiple rows to the foreign table at once using the FDW callback routine added by commitb663a4136
. This patch also allows this for postgres_fdw. It is enabled by the "batch_size" option added by commitb663a4136
, which is disabled by default. When doing batch insertion, we update progress of the COPY command after performing the FDW callback routine, to count rows not suppressed by the FDW as well as a BEFORE ROW INSERT trigger. For consistency, this patch changes the timing of updating it for plain tables: previously, we updated it immediately after adding each row to the multi-insert buffer, but we do so only after writing the rows stored in the buffer out to the table using table_multi_insert(), which I think would be consistent even with non-batching mode, because in that mode we update it after writing each row out to the table using table_tuple_insert(). Andrey Lepikhov, heavily revised by me, with review from Ian Barwick, Andrey Lepikhov, and Zhihong Yu. Discussion: https://postgr.es/m/bc489202-9855-7550-d64c-ad2d83c24867%40postgrespro.ru
This commit is contained in:
@ -665,7 +665,9 @@ ExecForeignBatchInsert(EState *estate,
|
||||
|
||||
<para>
|
||||
Note that this function is also called when inserting routed tuples into
|
||||
a foreign-table partition. See the callback functions
|
||||
a foreign-table partition or executing <command>COPY FROM</command> on
|
||||
a foreign table, in which case it is called in a different way than it
|
||||
is in the <command>INSERT</command> case. See the callback functions
|
||||
described below that allow the FDW to support that.
|
||||
</para>
|
||||
|
||||
|
@ -398,6 +398,10 @@ OPTIONS (ADD password_required 'false');
|
||||
exceeds the limit, the <literal>batch_size</literal> will be adjusted to
|
||||
avoid an error.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
This option also applies when copying into foreign tables.
|
||||
</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
|
||||
|
Reference in New Issue
Block a user