1
0
mirror of https://github.com/postgres/postgres.git synced 2025-12-22 17:42:17 +03:00
Commit Graph

3731 Commits

Author SHA1 Message Date
Peter Eisentraut
be2ab58f62 doc: Move DEFAULT parameter on COPY reference page
The DEFAULT parameter seems most similar to the NULL parameter, so
move it next to it, instead of having it at the end of the parameter
list because it was the last one added.
2023-07-09 09:48:16 +02:00
Nathan Bossart
151c22deee Revert MAINTAIN privilege and pg_maintain predefined role.
This reverts the following commits: 4dbdb82513, c2122aae63,
5b1a879943, 9e1e9d6560, ff9618e82a, 60684dd834, 4441fc704d,
and b5d6382496.  A role with the MAINTAIN privilege may be able to
use search_path tricks to escalate privileges to the table owner.
Unfortunately, it is too late in the v16 development cycle to apply
the proposed fix, i.e., restricting search_path when running
maintenance commands.

Bumps catversion.

Reviewed-by: Jeff Davis
Discussion: https://postgr.es/m/E1q7j7Y-000z1H-Hr%40gemulon.postgresql.org
Backpatch-through: 16
2023-07-07 11:25:13 -07:00
Michael Paquier
dd7c60f193 Introduce long options in pg_archivecleanup
This patch is a preliminary refactoring for an upcoming patch aimed at
adding new options to this tool, and using long options for these is
more user-friendly.  The existing short options gain long flavors, as
of:
* -d/--debug
* -n/--dry-run
* -x/--strip-extension

Author: Atsushi Torikoshi
Reviewed-by: Fujii Masao, Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/d660ef741ce3d82f3b4283f1cafd576c@oss.nttdata.com
2023-06-30 15:47:11 +09:00
Michael Paquier
b381d96370 Add timeline ID to file names generated with pg_waldump --save-fullpage
Not including the timeline IDs to the file names generated by pg_waldump
for the individual blocks saved could cause some of these files to be
overwritten when scanning segments across multiple timelines.  Having
this information is also as much useful as the LSNs, to be able to know
from exactly which WAL segment a block is comes from.

While on it, this fixes a few comments in the tests, where the format of
the file was not described as matching with the reality.

Reported-by: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, David Christensen
Discussion: https://postgr.es/m/ZJp921+nITFnvBVS@paquier.xyz
2023-06-28 16:26:55 +09:00
Peter Eisentraut
37a6d81c45 doc: Clean up title case use 2023-06-23 14:14:57 +02:00
Nathan Bossart
c2122aae63 Improve privilege documentation for maintenance commands.
The documentation of the required privileges for maintenance
commands (i.e., VACUUM, ANALYZE, CLUSTER, LOCK TABLE, REFRESH
MATERIALIZED VIEW, and REINDEX) is redundant, inaccurate, and
difficult to read.  This commit fixes and simplifies this
documentation by removing references to ownership, superuser, and
the pg_maintain role.  In addition, this removes notes about
database-wide VACUUM and ANALYZE, clarifies matters for REINDEX on
partitioned indexes and tables, and strengthens the description of
the pg_maintain role.

Reviewed-by: Michael Paquier, Jeff Davis
Discussion: https://postgr.es/m/20230615041044.GA736001%40nathanxps13
2023-06-22 15:48:38 -07:00
Nathan Bossart
4dbdb82513 Fix cache lookup hazards introduced by ff9618e82a.
ff9618e82a introduced has_partition_ancestor_privs(), which is used
to check whether a user has MAINTAIN on any partition ancestors.
This involves syscache lookups, and presently this function does
not take any relation locks, so it is likely subject to the same
kind of cache lookup failures that were fixed by 19de0ab23c.

To fix this problem, this commit partially reverts ff9618e82a.
Specifically, it removes the partition-related changes, including
the has_partition_ancestor_privs() function mentioned above.  This
means that MAINTAIN on a partitioned table is no longer sufficient
to perform maintenance commands on its partitions.  This is more
like how privileges for maintenance commands work on supported
versions.  Privileges are checked for each partition, so a command
that flows down to all partitions might refuse to process them
(e.g., if the current user doesn't have MAINTAIN on the partition).

In passing, adjust a few related comments and error messages, and
add a test for the privilege checks for CLUSTER on a partitioned
table.

Reviewed-by: Michael Paquier, Jeff Davis
Discussion: https://postgr.es/m/20230613211246.GA219055%40nathanxps13
2023-06-22 15:48:20 -07:00
David Rowley
c2d35bb88e Doc: mention that extended stats aren't used for joins
Statistics defined by the CREATE STATISTICS command are only used to
assist with the selectivity estimations of base relations, never for
joins.  Here we mention this fact in the notes section of the CREATE
STATISTICS command.

Discussion: https://postgr.es/m/CAApHDvrMuVgDOrmg_EtFDZ=AOovq6EsJNnHH1ddyZ8EqL4yzMw@mail.gmail.com
Backpatch-through: 11
2023-06-22 12:45:50 +12:00
Jeff Davis
2535c74b1a initdb: change default --locale-provider back to libc.
Reverts 27b62377b4.

Discussion: https://postgr.es/m/eff031036baa07f325de29215371a4c9e69d61f3.camel@j-davis.com
Discussion: https://postgr.es/m/3353947.1682092131@sss.pgh.pa.us
2023-06-21 11:10:03 -07:00
Amit Kapila
a734caa25f Fix the errhint message and docs for drop subscription failure.
The existing errhint message and docs were missing the fact that we can't
disassociate from the slot unless the subscription is disabled.

Author: Robert Sjöblom, Peter Smith
Reviewed-by: Peter Eisentraut, Amit Kapila
Backpatch-through: 11
Discussion: https://postgr.es/m/807bdf85-61ea-88e2-5712-6d9fcd4eabff@fortnox.se
2023-06-21 10:36:09 +05:30
Jeff Davis
a14e75eb0b CREATE DATABASE: make LOCALE apply to all collation providers.
For CREATE DATABASE, make LOCALE parameter apply regardless of the
provider used. Also affects initdb and createdb --locale arguments.

Previously, LOCALE (and --locale) only affected the database default
collation when using the libc provider.

Discussion: https://postgr.es/m/1a63084d-221e-4075-619e-6b3e590f673e@enterprisedb.com
Reviewed-by: Peter Eisentraut
2023-06-16 10:27:32 -07:00
Peter Eisentraut
b93c63d197 doc: Move list entry to proper position 2023-06-13 15:00:52 +02:00
Fujii Masao
378d73ef20 doc: Fix example command for ALTER FOREIGN TABLE ... OPTIONS.
In the documentation, previously the example command for
ALTER FOREIGN TABLE ... OPTIONS incorrectly included both
the option name and value with the DROP operation.
The correct syntax for the DROP operation requires only
the name of the option to be specified. This commit fixes
the example by removing the option value from the DROP operation.

Back-patch to all supported versions.

Author: Mehmet Emin KARAKAS <emin100@gmail.com>
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CANQrdXAHzbcEYhjGoe5A42OmfvdQhHFJzyKj9gJvHuDKyOF5Ng@mail.gmail.com
2023-06-08 20:12:51 +09:00
Nathan Bossart
381d19b3ea Document deprecated createuser option.
2dcd1578c4 left the --role option undocumented, which is
inconsistent with other deprecated options such as pg_dump's
--blobs and --no-blobs.  This change adds --role back to
createuser's documentation and usage output and marks it as
deprecated.

Suggested-by: Peter Eisentraut
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/0e85c9e7-4804-1cdb-5a4a-c72c328f9ad8%40enterprisedb.com
2023-05-23 19:35:05 -07:00
Alvaro Herrera
4042b2e187 pg_dump doc: mention 'long' is valid for --compress
Apparently an oversight in 2820adf775.
2023-05-22 12:26:40 +02:00
Nathan Bossart
2dcd1578c4 Rename some createuser options.
This change renames --admin to --with-admin, --role to --member-of,
and --member to --with-member.  Many people found the previous
names to be confusing.  The --admin and --member options are new in
v16, but --role has been there for a while, so that one has been
kept (but left undocumented) for backward compatibility.

Suggested-by: Peter Eisentraut
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/ZFvVZvQDliIWmOwg%40momjian.us
2023-05-21 20:06:38 -07:00
Bruce Momjian
ce3673dade doc: improve description of adding roles as members
Discussion: https://postgr.es/m/ZFvVZvQDliIWmOwg@momjian.us
2023-05-18 22:22:17 -04:00
Alexander Korotkov
b9a7a82272 Revert "Add USER SET parameter values for pg_db_role_setting"
This reverts commit 096dd80f3c and its fixups beecbe8e50, afdd9f7f0e,
529da086ba, db93e739ac.

Catversion is bumped.

Discussion: https://postgr.es/m/d46f9265-ff3c-6743-2278-6772598233c2%40pgmasters.net
2023-05-17 20:28:57 +03:00
Tom Lane
51b2c08798 Tighten usage of PSQL_WATCH_PAGER.
Don't use PSQL_WATCH_PAGER when stdin/stdout are not a terminal.
This corresponds to the restrictions on when other commands will
use [PSQL_]PAGER.  There isn't a lot of sense in trying to use a
pager in non-interactive cases, and doing so allows an environment
setting to break our tests.

Also, ignore PSQL_WATCH_PAGER if it is set but empty or all-blank,
for the same reasons we ignore such settings of [PSQL_]PAGER (see
commit 18f8f784c).

No documentation change is really needed, since there is nothing
suggesting that these constraints on [PSQL_]PAGER didn't already
apply to PSQL_WATCH_PAGER too.  But I rearranged the text
a little to make it read more naturally (IMHO anyway).

Per report from Pavel Stehule.  Back-patch to v15 where
PSQL_WATCH_PAGER was introduced.

Discussion: https://postgr.es/m/CAFj8pRDTwFzmEWdA-gdAcUh0ZnxUioSfTMre71WyB_wNJy-8gw@mail.gmail.com
2023-05-12 16:11:14 -04:00
Michael Paquier
6fd8ae6888 doc: Fix typo in pg_amcheck for term "schema"
Author: Alexander Lakhin
Discussion: https://postgr.es/m/e8c38840-596a-83d6-bd8d-cebc51111572@gmail.com
Backpatch-through: 14
2023-05-02 11:40:46 +09:00
Peter Eisentraut
2434d60a2a Put new command-line option into sensible order in help output
We have two existing conventions for long options: either alphabetical
among short options, or all long options after all the short options.
But the convention apparently used here, next to a functionally
related option, is not one of them.
2023-04-17 11:09:17 +02:00
David Rowley
fde9c06881 Doc: remove notes about the version vacuumdb options are valid from
Here we remove the notes which mention which version the given vacuumdb
option is available from.  There are now 11 of these notes and they're
both quite untidy and take up far more space than they seem to be worth.
On running a print preview of the compiled HTML, removing these notes
saves about 1 A4 page (~20% less space).

If people need to see which options are available on older versions, then
consulting the documents for that version seems like a good idea.  In any
case, when using newer vacuumdb versions on older servers, the user will
receive an error if they try to use an unsupported option.

Additionally, 3 of the notes are warning about the option only being
available from PostgreSQL 9.6 and later.  That version's support ended 2.5
years ago.  So, it's quite clear that the value of these notes diminishes
over time.

Discussion: https://postgr.es/m/CAApHDvrCQn6tupx2R67VL9RP1Qy4dDuWKRvt4jaB0vk2akQchw@mail.gmail.com
2023-04-17 09:28:34 +12:00
David Rowley
e9f451accb Doc: add note about --buffer-usage-limit being v16 onwards
Other vacuumdb options seem to have notes about which version they're
available from, so let's follow this trend for the newly added
--buffer-usage-limit option.
2023-04-16 21:47:01 +12:00
David Rowley
c0235013c1 Improve VACUUM/ANALYZE BUFFER_USAGE_LIMIT docs
This addresses various deficiencies in the documentation for VACUUM and
ANALYZE's BUFFER_USEAGE_LIMIT docs.

Here we declare "size" in the syntax synopsis for VACUUM and ANALYZE's
BUFFER_USAGE_LIMIT option and then define exactly what values can be
specified for it in the section for that below.

Also, fix the incorrect ordering of vacuumdb options both in the documents
and in vacuumdb's --help output.  These should be in alphabetical order.

In passing also add the minimum/maximum range for the BUFFER_USAGE_LIMIT
option.  These will also serve as example values that can be modified and
used.

Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/16845cb1-b228-e157-f293-5892bced9253@enterprisedb.com
2023-04-16 12:05:34 +12:00
Alvaro Herrera
9ce04b50e1 Revert "Catalog NOT NULL constraints" and fallout
This reverts commit e056c557ae and minor later fixes thereof.

There's a few problems in this new feature -- most notably regarding
pg_upgrade behavior, but others as well.  This new feature is not in any
way critical on its own, so instead of scrambling to fix it we revert it
and try again in early 17 with these issues in mind.

Discussion: https://postgr.es/m/3801207.1681057430@sss.pgh.pa.us
2023-04-12 19:29:21 +02:00
Michael Paquier
644b848f50 doc: Fix some typos and grammar
This is a first batch of the fixes, for the most obvious fixes.  A
little bit more is under discussion.

Author: Thom Brown, Justin Pryzby
Discussion: https://postgr.es/m/CAA-aLv7xCZ0nBJa-NWe0rxBB28TjFjS2JtjiZMoQ+0wsugG+hQ@mail.gmail.com
2023-04-12 13:03:09 +09:00
Tom Lane
26f36fe844 Doc: avoid using pg_get_publication_tables() in an example.
pg_get_publication_tables() is undocumented because it's only meant
as infrastructure for the pg_publication_tables system view.
That being the case, we should use the view not the bare function
in this sample query.

Shi Yu

Discussion: https://postgr.es/m/OSZPR01MB63107E83D07FEDEEABD83A23FD949@OSZPR01MB6310.jpnprd01.prod.outlook.com
2023-04-10 12:22:10 -04:00
Amit Kapila
d73c285af5 Doc: Fix the datatype of the newly added SUBSCRIPTION options.
In docs, the datatype of "password_required" and "run_as_owner" was
incorrectly specified as a string.

Author: Amit Kapila
Reviewed-by: Sawada Masahiko
Discussion: https://postgr.es/m/CAHut+Pu=pnJf=SS1583pknSQ3CbOqLCkWcJCQYt6zxTagHEdmw@mail.gmail.com
2023-04-08 09:42:50 +05:30
Alvaro Herrera
e056c557ae Catalog NOT NULL constraints
We now create pg_constaint rows for NOT NULL constraints with
contype='n'.

We propagate these constraints during operations such as adding
inheritance relationships, creating and attaching partitions, creating
tables LIKE other tables.  We mostly follow the well-known rules of
conislocal and coninhcount that we have for CHECK constraints, with some
adaptations; for example, as opposed to CHECK constraints, we don't
match NOT NULL ones by name when descending a hierarchy to alter it;
instead we match by column number.  This means we don't require the
constraint names to be identical across a hierarchy.

For now, we omit them from system catalogs.  Maybe this is worth
reconsidering.  We don't support NOT VALID nor DEFERRABLE clauses
either; these can be added as separate features later (this patch is
already large and complicated enough.)

This has been very long in the making.  The first patch was written by
Bernd Helmle in 2010 to add a new pg_constraint.contype value ('n'),
which I (Álvaro) then hijacked in 2011 and 2012, until that one was
killed by the realization that we ought to use contype='c' instead:
manufactured CHECK constraints.  However, later SQL standard
development, as well as nonobvious emergent properties of that design
(mostly, failure to distinguish them from "normal" CHECK constraints as
well as the performance implication of having to test the CHECK
expression) led us to reconsider this choice, so now the current
implementation uses contype='n' again.

In 2016 Vitaly Burovoy also worked on this feature[1] but found no
consensus for his proposed approach, which was claimed to be closer to
the letter of the standard, requiring additional pg_attribute columns to
track the OID of the NOT NULL constraint for that column.
[1] https://postgr.es/m/CAKOSWNkN6HSyatuys8xZxzRCR-KL1OkHS5-b9qd9bf1Rad3PLA@mail.gmail.com

Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Author: Bernd Helmle <mailings@oopsware.de>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>

Discussion: https://postgr.es/m/CACA0E642A0267EDA387AF2B%40%5B172.26.14.62%5D
Discussion: https://postgr.es/m/AANLkTinLXMOEMz+0J29tf1POokKi4XDkWJ6-DDR9BKgU@mail.gmail.com
Discussion: https://postgr.es/m/20110707213401.GA27098@alvh.no-ip.org
Discussion: https://postgr.es/m/1343682669-sup-2532@alvh.no-ip.org
Discussion: https://postgr.es/m/CAKOSWNkN6HSyatuys8xZxzRCR-KL1OkHS5-b9qd9bf1Rad3PLA@mail.gmail.com
Discussion: https://postgr.es/m/20220817181249.q7qvj3okywctra3c@alvherre.pgsql
2023-04-07 19:59:57 +02:00
David Rowley
ae78cae3be Add --buffer-usage-limit option to vacuumdb
1cbbee033 added BUFFER_USAGE_LIMIT to the VACUUM and ANALYZE commands, so
here we permit that option to be specified in vacuumdb.

In passing, adjust the documents for vacuum_buffer_usage_limit and the
BUFFER_USAGE_LIMIT VACUUM option to mention "kB" rather than "KB".  Do the
same for the ERROR message in ExecVacuum() and
check_vacuum_buffer_usage_limit().  Without that we might tell a user that
the valid minimum value is 128 KB only to reject that because we accept
only "kB" and not "KB".

Also, add a small reminder comment in vacuum.h to try to trigger the
memory of anyone adding new fields to VacuumParams that they might want to
consider if vacuumdb needs to grow a new option too.

Author: Melanie Plageman
Reviewed-by: Justin Pryzby
Reviewed-by: David Rowley
Discussion: https://postgr.es/m/ZAzTg3iEnubscvbf@telsasoft.com
2023-04-07 12:47:10 +12:00
David Rowley
1cbbee0338 Add VACUUM/ANALYZE BUFFER_USAGE_LIMIT option
Add new options to the VACUUM and ANALYZE commands called
BUFFER_USAGE_LIMIT to allow users more control over how large to make the
buffer access strategy that is used to limit the usage of buffers in
shared buffers.  Larger rings can allow VACUUM to run more quickly but
have the drawback of VACUUM possibly evicting more buffers from shared
buffers that might be useful for other queries running on the database.

Here we also add a new GUC named vacuum_buffer_usage_limit which controls
how large to make the access strategy when it's not specified in the
VACUUM/ANALYZE command.  This defaults to 256KB, which is the same size as
the access strategy was prior to this change.  This setting also
controls how large to make the buffer access strategy for autovacuum.

Per idea by Andres Freund.

Author: Melanie Plageman
Reviewed-by: David Rowley
Reviewed-by: Andres Freund
Reviewed-by: Justin Pryzby
Reviewed-by: Bharath Rupireddy
Discussion: https://postgr.es/m/20230111182720.ejifsclfwymw2reb@awork3.anarazel.de
2023-04-07 11:40:31 +12:00
Tom Lane
31ae2aa9d2 psql: set SHELL_ERROR and SHELL_EXIT_CODE in more places.
Make the \g, \o, \w, and \copy commands set these variables
when closing a pipe.  We missed doing this in commit b0d8f2d98,
but it seems like a good idea.

There are some remaining places in psql that intentionally don't
update these variables after running a child program:
	* pager invocations
	* backtick evaluation within a prompt
	* \e (edit query buffer)

Corey Huinker and Tom Lane

Discussion: https://postgr.es/m/CADkLM=eSKwRGF-rnRqhtBORRtL49QsjcVUCa-kLxKTqxypsakw@mail.gmail.com
2023-04-06 17:33:38 -04:00
Tom Lane
00beecfe83 psql: add an optional execution-count limit to \watch.
\watch can now be told to stop after N executions of the query.

With the idea that we might want to add more options to \watch
in future, this patch generalizes the command's syntax to a list
of name=value options, with the interval allowed to omit the name
for backwards compatibility.

Andrey Borodin, reviewed by Kyotaro Horiguchi, Nathan Bossart,
Michael Paquier, Yugo Nagata, and myself

Discussion: https://postgr.es/m/CAAhFRxiZ2-n_L1ErMm9AZjgmUK=qS6VHb+0SaMn8sqqbhF7How@mail.gmail.com
2023-04-06 13:18:14 -04:00
Tomas Vondra
2820adf775 Support long distance matching for zstd compression
zstd compression supports a special mode for finding matched in distant
past, which may result in better compression ratio, at the expense of
using more memory (the window size is 128MB).

To enable this optional mode, use the "long" keyword when specifying the
compression method (--compress=zstd:long).

Author: Justin Pryzby
Reviewed-by: Tomas Vondra, Jacob Champion
Discussion: https://postgr.es/m/20230224191840.GD1653@telsasoft.com
Discussion: https://postgr.es/m/20220327205020.GM28503@telsasoft.com
2023-04-06 17:18:42 +02:00
Tomas Vondra
84adc8e20f pg_dump: Add support for zstd compression
Allow pg_dump to use the zstd compression, in addition to gzip/lz4. Bulk
of the new compression method is implemented in compress_zstd.{c,h},
covering the pg_dump compression APIs. The rest of the patch adds test
and makes various places aware of the new compression method.

The zstd library (which this patch relies on) supports multithreaded
compression since version 1.5. We however disallow that feature for now,
as it might interfere with parallel backups on platforms that rely on
threads (e.g. Windows). This can be improved / relaxed in the future.

This also fixes a minor issue in InitDiscoverCompressFileHandle(), which
was not updated to check if the file already has the .lz4 extension.

Adding zstd compression was originally proposed in 2020 (see the second
thread), but then was reworked to use the new compression API introduced
in e9960732a9. I've considered both threads when compiling the list of
reviewers.

Author: Justin Pryzby
Reviewed-by: Tomas Vondra, Jacob Champion, Andreas Karlsson
Discussion: https://postgr.es/m/20230224191840.GD1653@telsasoft.com
Discussion: https://postgr.es/m/20201221194924.GI30237@telsasoft.com
2023-04-05 21:39:33 +02:00
Robert Haas
482675987b Add a run_as_owner option to subscriptions.
This option is normally false, but can be set to true to obtain
the legacy behavior where the subscription runs with the permissions
of the subscription owner rather than the permissions of the
table owner. The advantages of this mode are (1) it doesn't require
that the subscription owner have permission to SET ROLE to each
table owner and (2) since no role switching occurs, the
SECURITY_RESTRICTED_OPERATION restrictions do not apply.

On the downside, it allows any table owner to easily usurp
the privileges of the subscription owner - basically, to take
over their account. Because that's generally quite undesirable,
we don't make this mode the default, but we do make it available,
just in case the new behavior causes too many problems for someone.

Discussion: http://postgr.es/m/CA+TgmoZ-WEeG6Z14AfH7KhmpX2eFh+tZ0z+vf0=eMDdbda269g@mail.gmail.com
2023-04-04 12:03:03 -04:00
Amit Kapila
ed94e8563e Add XML ID attributes to create_publication.sgml.
This commit adds XML ID attributes to all varlistentries in
create_publication.sgml. This allows us to include links to refer to
publication options, making documents more readable.

Author: Kuroda Hayato
Reviewed-by: Peter Smith, Amit Kapila
Discussion: https://postgr.es/m/TYAPR01MB58668219FEA4EC231486A433F58E9@TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-03-31 08:59:55 +05:30
Robert Haas
9c487efe35 Fix documentation build for c3afe8cf5a.
This documentation hunk was intended to be part of that commit,
but I goofed.
2023-03-30 12:06:34 -04:00
Robert Haas
c3afe8cf5a Add new predefined role pg_create_subscription.
This role can be granted to non-superusers to allow them to issue
CREATE SUBSCRIPTION. The non-superuser must additionally have CREATE
permissions on the database in which the subscription is to be
created.

Most forms of ALTER SUBSCRIPTION, including ALTER SUBSCRIPTION .. SKIP,
now require only that the role performing the operation own the
subscription, or inherit the privileges of the owner. However, to
use ALTER SUBSCRIPTION ... RENAME or ALTER SUBSCRIPTION ... OWNER TO,
you also need CREATE permission on the database. This is similar to
what we do for schemas. To change the owner of a schema, you must also
have permission to SET ROLE to the new owner, similar to what we do
for other object types.

Non-superusers are required to specify a password for authentication
and the remote side must use the password, similar to what is required
for postgres_fdw and dblink.  A superuser who wants a non-superuser to
own a subscription that does not rely on password authentication may
set the new password_required=false property on that subscription. A
non-superuser may not set password_required=false and may not modify a
subscription that already has password_required=false.

This new password_required subscription property works much like the
eponymous postgres_fdw property.  In both cases, the actual semantics
are that a password is not required if either (1) the property is set
to false or (2) the relevant user is the superuser.

Patch by me, reviewed by Andres Freund, Jeff Davis, Mark Dilger,
and Stephen Frost (but some of those people did not fully endorse
all of the decisions that the patch makes).

Discussion: http://postgr.es/m/CA+TgmoaDH=0Xj7OBiQnsHTKcF2c4L+=gzPBUKSJLh8zed2_+Dg@mail.gmail.com
2023-03-30 11:37:19 -04:00
Amit Kapila
062a844424 Avoid syncing data twice for the 'publish_via_partition_root' option.
When there are multiple publications for a subscription and one of those
publishes via the parent table by using publish_via_partition_root and the
other one directly publishes the child table, we end up copying the same
data twice during initial synchronization. The reason for this was that we
get both the parent and child tables from the publisher and try to copy
the data for both of them.

This patch extends the function pg_get_publication_tables() to take a
publication list as its input parameter. This allows us to exclude a
partition table whose ancestor is published by the same publication list.

This problem does exist in back-branches but we decide to fix it there in
a separate commit if required. The fix for back-branches requires quite
complicated changes to fetch the required table information from the
publisher as we can't update the function pg_get_publication_tables() in
back-branches. We are not sure whether we want to deviate and complicate
the code in back-branches for this problem as there are no field reports
yet.

Author: Wang wei
Reviewed-by: Peter Smith, Jacob Champion, Kuroda Hayato, Vignesh C, Osumi Takamichi, Amit Kapila
Discussion: https://postgr.es/m/OS0PR01MB57167F45D481F78CDC5986F794B99@OS0PR01MB5716.jpnprd01.prod.outlook.com
2023-03-29 10:46:58 +05:30
Amit Kapila
de5a47af2d Add XML ID attributes to create_subscription.sgml.
Commit ecb696527c added an XML ID attribute to one varlistentry in
create_subscription.sgml. Following 78ee60ed84, this commit adds XML ID
attributes to all varlistentries in create_subscription.sgml.
Additionally, links are added to refer to the subscription options,
enhancing the readability of documents.

Author: Kuroda Hayato
Reviewed-by: Peter Smith, Amit Kapila
Discussion: https://postgr.es/m/TYAPR01MB58667AE04D291924671E2051F5879@TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-03-29 09:58:14 +05:30
Tom Lane
3c05284d83 Invent GENERIC_PLAN option for EXPLAIN.
This provides a very simple way to see the generic plan for a
parameterized query.  Without this, it's necessary to define
a prepared statement and temporarily change plan_cache_mode,
which is a bit tedious.

One thing that's a bit of a hack perhaps is that we disable
execution-time partition pruning when the GENERIC_PLAN option
is given.  That's because the pruning code may attempt to
fetch the value of one of the parameters, which would fail.

Laurenz Albe, reviewed by Julien Rouhaud, Christoph Berg,
Michel Pelletier, Jim Jones, and myself

Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel@cybertec.at
2023-03-24 17:07:22 -04:00
Amit Kapila
ecb696527c Allow logical replication to copy tables in binary format.
This patch allows copying tables in the binary format during table
synchronization when the binary option for a subscription is enabled.
Previously, tables are copied in text format even if the subscription is
created with the binary option enabled. Copying tables in binary format
may reduce the time spent depending on column types.

A binary copy for initial table synchronization is supported only when
both publisher and subscriber are v16 or later.

Author: Melih Mutlu
Reviewed-by: Peter Smith, Shi yu, Euler Taveira, Vignesh C,  Kuroda Hayato, Osumi Takamichi, Bharath Rupireddy, Hou Zhijie
Discussion: https://postgr.es/m/CAGPVpCQvAziCLknEnygY0v1-KBtg%2BOm-9JHJYZOnNPKFJPompw%40mail.gmail.com
2023-03-23 08:45:51 +05:30
Tom Lane
3e51b278db Add "-c name=value" switch to initdb.
This option, or its long form --set, sets the GUC "name" to "value".
The setting applies in the bootstrap and standalone servers run by
initdb, and is also written into the generated postgresql.conf.

This can save an extra editing step when creating a new cluster,
but the real use-case is for coping with situations where the
bootstrap server fails to start due to environmental issues;
for example, if it's necessary to force huge_pages to off.

Discussion: https://postgr.es/m/2844176.1674681919@sss.pgh.pa.us
2023-03-22 13:49:05 -04:00
Tom Lane
b0d8f2d983 Add SHELL_ERROR and SHELL_EXIT_CODE magic variables to psql.
These are set after a \! command or a backtick substitution.
SHELL_ERROR is just "true" for error (nonzero exit status) or "false"
for success, while SHELL_EXIT_CODE records the actual exit status
following standard shell/system(3) conventions.

Corey Huinker, reviewed by Maxim Orlov and myself

Discussion: https://postgr.es/m/CADkLM=cWao2x2f+UDw15W1JkVFr_bsxfstw=NGea7r9m4j-7rQ@mail.gmail.com
2023-03-21 13:03:56 -04:00
Peter Eisentraut
4c8044c044 pg_waldump: Allow hexadecimal values for -t/--timeline option
This makes it easier to specify values taken directly from WAL file
names.

The option parsing is arranged in the style of option_parse_int() (but
we need to parse unsigned int), to allow future refactoring in the
same manner.

Reviewed-by: Sébastien Lardière <sebastien@lardiere.net>
Discussion: https://www.postgresql.org/message-id/flat/8fef346e-2541-76c3-d768-6536ae052993@lardiere.net
2023-03-21 08:05:23 +01:00
Tom Lane
bc8cd50fef Fix pg_dump for hash partitioning on enum columns.
Hash partitioning on an enum is problematic because the hash codes are
derived from the OIDs assigned to the enum values, which will almost
certainly be different after a dump-and-reload than they were before.
This means that some rows probably end up in different partitions than
before, causing restore to fail because of partition constraint
violations.  (pg_upgrade dodges this problem by using hacks to force
the enum values to keep the same OIDs, but that's not possible nor
desirable for pg_dump.)

Users can work around that by specifying --load-via-partition-root,
but since that's a dump-time not restore-time decision, one might
find out the need for it far too late.  Instead, teach pg_dump to
apply that option automatically when dealing with a partitioned
table that has hash-on-enum partitioning.

Also deal with a pre-existing issue for --load-via-partition-root
mode: in a parallel restore, we try to TRUNCATE target tables just
before loading them, in order to enable some backend optimizations.
This is bad when using --load-via-partition-root because (a) we're
likely to suffer deadlocks from restore jobs trying to restore rows
into other partitions than they came from, and (b) if we miss getting
a deadlock we might still lose data due to a TRUNCATE removing rows
from some already-completed restore job.

The fix for this is conceptually simple: just don't TRUNCATE if we're
dealing with a --load-via-partition-root case.  The tricky bit is for
pg_restore to identify those cases.  In dumps using COPY commands we
can inspect each COPY command to see if it targets the nominal target
table or some ancestor.  However, in dumps using INSERT commands it's
pretty impractical to examine the INSERTs in advance.  To provide a
solution for that going forward, modify pg_dump to mark TABLE DATA
items that are using --load-via-partition-root with a comment.
(This change also responds to a complaint from Robert Haas that
the dump output for --load-via-partition-root is pretty confusing.)
pg_restore checks for the special comment as well as checking the
COPY command if present.  This will fail to identify the combination
of --load-via-partition-root and --inserts in pre-existing dump files,
but that should be a pretty rare case in the field.  If it does
happen you will probably get a deadlock failure that you can work
around by not using parallel restore, which is the same as before
this bug fix.

Having done this, there seems no remaining reason for the alarmism
in the pg_dump man page about combining --load-via-partition-root
with parallel restore, so remove that warning.

Patch by me; thanks to Julien Rouhaud for review.  Back-patch to
v11 where hash partitioning was introduced.

Discussion: https://postgr.es/m/1376149.1675268279@sss.pgh.pa.us
2023-03-17 13:31:40 -04:00
Tom Lane
f170b572d2 Doc: mention CREATE+ATTACH PARTITION with CREATE TABLE...PARTITION OF.
Clarify that ATTACH/DETACH PARTITION can be used to perform partition
maintenance with less locking than straight CREATE TABLE/DROP TABLE.
This was already stated in some places, but not emphasized.

Back-patch to v14 where DETACH PARTITION CONCURRENTLY was added.
(We had lower lock levels for ATTACH PARTITION before that, but
this wording wouldn't apply.)

Justin Pryzby, reviewed by Robert Treat and Jakub Wartak;
a little further wordsmithing by me

Discussion: https://postgr.es/m/20220718143304.GC18011@telsasoft.com
2023-03-16 16:50:56 -04:00
Tom Lane
a563c24c95 Allow pg_dump to include/exclude child tables automatically.
This patch adds new pg_dump switches
    --table-and-children=pattern
    --exclude-table-and-children=pattern
    --exclude-table-data-and-children=pattern
which act the same as the existing --table, --exclude-table, and
--exclude-table-data switches, except that any partitions or
inheritance child tables of the table(s) matching the pattern
are also included or excluded.

Gilles Darold, reviewed by Stéphane Tachoires

Discussion: https://postgr.es/m/5aa393b5-5f67-8447-b83e-544516990ee2@migops.com
2023-03-14 16:09:03 -04:00
Andrew Dunstan
9f8377f7a2 Add a DEFAULT option to COPY FROM
This allows for a string which if an input field matches causes the
column's default value to be inserted. The advantage of this is that
the default can be inserted in some rows and not others, for which
non-default data is available.

The file_fdw extension is also modified to take allow use of this
option.

Israel Barth Rubio

Discussion: https://postgr.es/m/CAO_rXXAcqesk6DsvioOZ5zmeEmpUN5ktZf-9=9yu+DTr0Xr8Uw@mail.gmail.com
2023-03-13 10:01:56 -04:00