1
0
mirror of https://github.com/postgres/postgres.git synced 2025-04-24 10:47:04 +03:00

16661 Commits

Author SHA1 Message Date
Michael Paquier
af720b4c50 Change custom wait events to use dynamic shared hash tables
Currently, the names of the custom wait event must be registered for
each backend, requiring all these to link to the shared memory area of
an extension, even if these are not loaded with
shared_preload_libraries.

This patch relaxes the constraints related to this infrastructure by
storing the wait events and their names in two dynamic hash tables in
shared memory.  This has the advantage to simplify the registration of
custom wait events to a single routine call that returns an event ID
ready for consumption:
uint32 WaitEventExtensionNew(const char *wait_event_name);

The caller of this routine can then cache locally the ID returned, to be
used for pgstat_report_wait_start(), WaitLatch() or a similar routine.

The implementation uses two hash tables: one with a key based on the
event name to avoid duplicates and a second using the event ID as key
for event lookups, like on pg_stat_activity.  These tables can hold a
minimum of 16 entries, and a maximum of 128 entries, which should be plenty
enough.

The code changes done in worker_spi show how things are simplified (most
of the code removed in this commit comes from there):
- worker_spi_init() is gone.
- No more shared memory hooks required (size requested and
initialization).
- The custom wait event ID is cached in the process that needs to set
it, with one single call to WaitEventExtensionNew() to retrieve it.

Per suggestion from Andres Freund.

Author: Masahiro Ikeda, with a few tweaks from me.
Discussion: https://postgr.es/m/20230801032349.aaiuvhtrcvvcwzcx@awork3.anarazel.de
2023-08-14 14:47:27 +09:00
Etsuro Fujita
74a2dfee22 Doc: update documentation for creating custom scan paths.
Commit f49842d1e added a new callback for custom scan paths, but missed
updating the documentation.

Back-patch to all supported branches.

Discussion: https://postgr.es/m/CAPmGK15ODkN%2B%3DhkBCufj1HBW0x5OTb65Xuy7ryXchMdiCMpx_g%40mail.gmail.com
2023-08-03 17:45:00 +09:00
Michael Paquier
c9af054653 Support custom wait events for wait event type "Extension"
Two backend routines are added to allow extension to allocate and define
custom wait events, all of these being allocated in the type
"Extension":
* WaitEventExtensionNew(), that allocates a wait event ID computed from
a counter in shared memory.
* WaitEventExtensionRegisterName(), to associate a custom string to the
wait event ID allocated.

Note that this includes an example of how to use this new facility in
worker_spi with tests in TAP for various scenarios, and some
documentation about how to use them.

Any code in the tree that currently uses WAIT_EVENT_EXTENSION could
switch to this new facility to define custom wait events.  This is left
as work for future patches.

Author: Masahiro Ikeda
Reviewed-by: Andres Freund, Michael Paquier, Tristan Partin, Bharath
Rupireddy
Discussion: https://postgr.es/m/b9f5411acda0cf15c8fbb767702ff43e@oss.nttdata.com
2023-07-31 17:09:24 +09:00
Nathan Bossart
6d982e3b3b Harmonize password reuse in vacuumdb, clusterdb, and reindexdb.
Commits 83dec5a712 and ff402ae11b taught vacuumdb to reuse
passwords instead of prompting repeatedly.  However, the docs still
warn about repeated prompts, and this improvement was not applied
to clusterdb and reindexdb.  This commit allows clusterdb and
reindexdb to reuse passwords just like vacuumdb does, and it
expunges the aforementioned warnings from the docs.

Reviewed-by: Gurjeet Singh, Zhang Mingli
Discussion: https://postgr.es/m/20230628045741.GA1813397%40nathanxps13
2023-07-28 10:07:44 -07:00
Amit Langote
e055b6be7e doc: add missing <returnvalue> and whitespace
Missed in commit 03734a7fed.

Author: Shinoda, Noriyoshi <noriyoshi.shinoda@hpe.com>
Discussion: https://postgr.es/m/DM4PR84MB1734E58BB4DC0E1B6E2990EBEE01A%40DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
2023-07-28 16:05:44 +09:00
Amit Langote
03734a7fed Add more SQL/JSON constructor functions
This Patch introduces three SQL standard JSON functions:

JSON()
JSON_SCALAR()
JSON_SERIALIZE()

JSON() produces json values from text, bytea, json or jsonb values,
and has facilitites for handling duplicate keys.

JSON_SCALAR() produces a json value from any scalar sql value,
including json and jsonb.

JSON_SERIALIZE() produces text or bytea from input which containis
or represents json or jsonb;

For the most part these functions don't add any significant new
capabilities, but they will be of use to users wanting standard
compliant JSON handling.

Catversion bumped as this changes ruleutils.c.

Author: Nikita Glukhov <n.gluhov@postgrespro.ru>
Author: Teodor Sigaev <teodor@sigaev.ru>
Author: Oleg Bartunov <obartunov@gmail.com>
Author: Alexander Korotkov <aekorotkov@gmail.com>
Author: Andrew Dunstan <andrew@dunslane.net>
Author: Amit Langote <amitlangote09@gmail.com>

Reviewers have included (in no particular order) Andres Freund, Alexander
Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu,
Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby, Álvaro Herrera,
Peter Eisentraut

Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de
Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org
Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
2023-07-26 17:08:33 +09:00
Michael Paquier
e35cc3b3f2 pgbench: Use COPY for client-side data generation
This commit switches the client-side data generation from INSERT queries
to COPY for the two tables pgbench_branches and pgbench_tellers.
pgbench_accounts was already using COPY.

COPY is a better interface for bulk loading or high latency connections
(this point can be countered with the option for server-side data
generation, still client-side is the default), and measurements have
proved that using it for these two other tables can lead to improvements
during initialization.  I did not notice slowdowns at large scale
numbers on a local setup, either, most of the work happening for the
accounts table.

Previously COPY was only used for the pgbench_accounts table because the
amount of data was much larger than the two other tables.  The code is
refactored so as all three tables use the same code path to execute the
COPY queries, with a callback to build data rows.

Author: Tristan Partin
Discussion: https://postgr.es/m/CSTU5P82ONZ1.19XFUGHMXHBRY@c3po
2023-07-24 13:48:22 +09:00
Daniel Gustafsson
29a0ccbce9 Revert "Add notBefore and notAfter to SSL cert info display"
Due to an oversight in reviewing, this used functionality not
compatible with old versions of OpenSSL.

This reverts commit 75ec5e7bec700577d39d653c316e3ae6c505842c.
2023-07-20 17:18:12 +02:00
Daniel Gustafsson
75ec5e7bec Add notBefore and notAfter to SSL cert info display
This adds the X509 attributes notBefore and notAfter to sslinfo
as well as pg_stat_ssl to allow verifying and identifying the
validity period of the current client certificate.

Author: Cary Huang <cary.huang@highgo.ca>
Discussion: https://postgr.es/m/182b8565486.10af1a86f158715.2387262617218380588@highgo.ca
2023-07-20 17:07:32 +02:00
Nathan Bossart
ab29a7a9c6 Doc: move unparenthesized syntaxes for a few commands.
Move documentation of the unparenthesized syntaxes for VACUUM,
ANALYZE, EXPLAIN, and CLUSTER to the "Compatibility" section of
their documentation to improve readability of the preferred,
parenthesized syntaxes.

Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_bc5uHieG1976kGqJKxyWtyQt9yvktjsVX%2Bi7NOigDjOA%40mail.gmail.com
2023-07-19 15:26:59 -07:00
Nathan Bossart
cdaedfc96d Support parenthesized syntax for CLUSTER without a table name.
b5913f6120 added a parenthesized syntax for CLUSTER, but it
requires specifying a table name.  This is unlike commands such as
VACUUM and ANALYZE, which do not require specifying a table in the
parenthesized syntax.  This change resolves this inconsistency.
This is preparatory work for a follow-up commit that will move the
unparenthesized syntax to the "Compatibility" section of the
CLUSTER documentation.

Reviewed-by: Melanie Plageman, Michael Paquier
Discussion: https://postgr.es/m/CAAKRu_bc5uHieG1976kGqJKxyWtyQt9yvktjsVX%2Bi7NOigDjOA%40mail.gmail.com
2023-07-19 15:26:52 -07:00
Nathan Bossart
018b61f81b Rearrange CLUSTER rules in gram.y.
This change moves the unparenthesized syntax for CLUSTER to the end
of the ClusterStmt rules in preparation for a follow-up commit that
will move this syntax to the "Compatibility" section of the CLUSTER
documentation.  The documentation for the CLUSTER syntaxes has also
been consolidated.

Suggested-by: Melanie Plageman
Discussion https://postgr.es/m/CAAKRu_bc5uHieG1976kGqJKxyWtyQt9yvktjsVX%2Bi7NOigDjOA%40mail.gmail.com
2023-07-19 15:26:43 -07:00
Tom Lane
d65ddaca93 Add psql \drg command to display role grants.
With the addition of INHERIT and SET options for role grants,
the historical display of role memberships in \du/\dg is woefully
inadequate.  Besides those options, there are pre-existing
shortcomings that you can't see the ADMIN option nor the grantor.

To fix this, remove the "Member of" column from \du/\dg altogether
(making that output usefully narrower), and invent a new meta-command
"\drg" that is specifically for displaying role memberships.  It
shows one row for each role granted to the selected role(s), with
the grant options and grantor.

We would not normally back-patch such a feature addition post
feature freeze, but in this case the change is mainly driven by
v16 changes in the server, so it seems appropriate to include it
in v16.

Pavel Luzanov, with bikeshedding and review from a lot of people,
but particularly David Johnston

Discussion: https://postgr.es/m/b9be2d0e-a9bc-0a30-492f-a4f68e4f7740@postgrespro.ru
2023-07-19 12:46:30 -04:00
Tom Lane
15c68cd84a Doc: improve description of IN and row-constructor comparisons.
IN and NOT IN work fine on records and arrays, so just say that
they accept "expressions" not "scalar expressions".  I think that
that phrasing was meant to say that they don't work on set-returning
expressions, but that's not the common meaning of "scalar".

Revise the description of row-constructor comparisons to make it
perhaps a bit less confusing.  (This partially reverts some
dubious wording changes made by commit f56651519.)

Per gripe from Ilya Nenashev.  Back-patch to supported branches.
In HEAD and v16, also drop a NOTE about pre-8.2 behavior, which
is hopefully no longer of interest to anybody.

Discussion: https://postgr.es/m/168968062460.632.14303906825812821399@wrigleys.postgresql.org
2023-07-19 11:00:34 -04:00
Michael Paquier
3f8c98d0b6 pg_archivecleanup: Add --clean-backup-history
By default, pg_archivecleanup does not remove backup history files.
These are just few bytes useful for debugging purposes, still keeping
them around can bloat an archive path history files mixed with the WAL
segments if the path has a long history.

This patch adds a new option to control if backup history files are
removed, depending on the oldest segment name to keep around.

While on it, the TAP tests are refactored so as these are now able to
handle lists of files.  Each file has a flag to track if it should still
exist or not depending on the oldest segment defined with the command
run.

Author: Atsushi Torikoshi
Reviewed-by: Kyotaro Horiguchi, Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/d660ef741ce3d82f3b4283f1cafd576c@oss.nttdata.com
2023-07-19 13:41:22 +09:00
Amit Kapila
961cf5c905 Doc: Update the logical replication restriction w.r.t Replica Identity Full.
Author: Kuroda Hayato
Reviewed-by: Peter Smith, Onder Kalaci, Sergei Kornilov, Amit Kapila
Discussion: https://postgr.es/m/TYAPR01MB58662174ED62648E0D611194F530A@TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-07-19 08:11:44 +05:30
Tom Lane
137b131d6f Doc: fix out-of-date example of SPI usage.
The "count" argument of SPI_exec() only limits execution when
the query is actually returning rows.  This was not the case
before PG 9.0, so this example was correct when written; but
we missed updating it in commit 2ddc600f8.  Extend the example
to show the behavior both with and without RETURNING.

While here, improve the commentary and markup for the rest
of the example.

David G. Johnston and Tom Lane, per report from Curt Kolovson.
Back-patch to all supported branches.

Discussion: https://postgr.es/m/CANhYJV6HWtgz_qjx_APfK0PAgLUzY-2vjLuj7i_o=TZF1LAQew@mail.gmail.com
2023-07-18 11:59:39 -04:00
Nathan Bossart
884eee5bfb Remove db_user_namespace.
This feature was intended to be a temporary measure to support
per-database user names.  A better one hasn't materialized in the
~21 years since it was added, and nobody claims to be using it, so
let's just remove it.

Reviewed-by: Michael Paquier, Magnus Hagander
Discussion: https://postgr.es/m/20230630200509.GA2830328%40nathanxps13
Discussion: https://postgr.es/m/20230630215608.GD2941194%40nathanxps13
2023-07-17 11:44:59 -07:00
Nathan Bossart
a0363ab7aa Fix privilege check for SET SESSION AUTHORIZATION.
Presently, the privilege check for SET SESSION AUTHORIZATION checks
whether the original authenticated role was a superuser at
connection start time.  Even if the role loses the superuser
attribute, its existing sessions are permitted to change session
authorization to any role.

This commit modifies this privilege check to verify the original
authenticated role currently has superuser.  In the event that the
authenticated role loses superuser within a session authorization
change, the authorization change will remain in effect, which means
the user can still take advantage of the target role's privileges.
However, [RE]SET SESSION AUTHORIZATION will only permit switching
to the original authenticated role.

Author: Joseph Koshakow
Discussion: https://postgr.es/m/CAAvxfHc-HHzONQ2oXdvhFF9ayRnidPwK%2BfVBhRzaBWYYLVQL-g%40mail.gmail.com
2023-07-13 21:13:45 -07:00
Amit Kapila
edca342434 Allow the use of a hash index on the subscriber during replication.
Commit 89e46da5e5 allowed using BTREE indexes that are neither
PRIMARY KEY nor REPLICA IDENTITY on the subscriber during apply of
update/delete. This patch extends that functionality to also allow HASH
indexes.

We explored supporting other index access methods as well but they don't
have a fixed strategy for equality operation which is required by the
current infrastructure in logical replication to scan the indexes.

Author: Kuroda Hayato
Reviewed-by: Peter Smith, Onder Kalaci, Amit Kapila
Discussion: https://postgr.es/m/TYAPR01MB58669D7414E59664E17A5827F522A@TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-07-14 08:21:54 +05:30
Thomas Munro
d0c28601ef Remove wal_sync_method=fsync_writethrough on Windows.
The "fsync" level already flushes drive write caches on Windows (as does
"fdatasync"), so it only confuses matters to have an apparently higher
level that isn't actually different at all.

That leaves "fsync_writethrough" only for macOS, where it actually does
something different.

Reviewed-by: Magnus Hagander <magnus@hagander.net>
Discussion: https://postgr.es/m/CA%2BhUKGJ2CG2SouPv2mca2WCTOJxYumvBARRcKPraFMB6GSEMcA%40mail.gmail.com
2023-07-14 12:30:13 +12:00
Andres Freund
c66a7d75e6 Handle DROP DATABASE getting interrupted
Until now, when DROP DATABASE got interrupted in the wrong moment, the removal
of the pg_database row would also roll back, even though some irreversible
steps have already been taken. E.g. DropDatabaseBuffers() might have thrown
out dirty buffers, or files could have been unlinked. But we continued to
allow connections to such a corrupted database.

To fix this, mark databases invalid with an in-place update, just before
starting to perform irreversible steps. As we can't add a new column in the
back branches, we use pg_database.datconnlimit = -2 for this purpose.

An invalid database cannot be connected to anymore, but can still be
dropped.

Unfortunately we can't easily add output to psql's \l to indicate that some
database is invalid, it doesn't fit in any of the existing columns.

Add tests verifying that a interrupted DROP DATABASE is handled correctly in
the backend and in various tools.

Reported-by: Evgeny Morozov <postgresql3@realityexists.net>
Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20230509004637.cgvmfwrbht7xm7p6@awork3.anarazel.de
Discussion: https://postgr.es/m/20230314174521.74jl6ffqsee5mtug@awork3.anarazel.de
Backpatch: 11-, bug present in all supported versions
2023-07-13 13:03:28 -07:00
Masahiko Sawada
fd48a86c62 Doc: clarify the conditions of usable indexes for REPLICA IDENTITY FULL tables.
Commit 89e46da5e allowed REPLICA IDENTITY FULL tables to use an index
on the subscriber during apply of update/delete. This commit clarifies
in the documentation that the leftmost field of candidate indexes must
be a column (not an expression) that references the published relation
column.

The source code comments are also updated accordingly.

Reviewed-by: Peter Smith, Amit Kapila
Discussion: https://postgr.es/m/CAD21AoDJjffEvUFKXT27Q5U8-UU9JHv4rrJ9Ke8Zkc5UPWHLvA@mail.gmail.com
Backpatch-through: 16
2023-07-13 15:03:17 +09:00
Daniel Gustafsson
e68dd9e907 doc: Clarify the errhint example
The errhint example wasn't using the mandated style for errhint
which was needlessly confusing.  Fix by rewriting it using the
mandated style.

Reported-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Gurjeet Singh <gurjeet@singh.im>
Reviewed-by: Jonathan S. Katz <jkatz@postgresql.org>
Discussion: https://postgr.es/m/F02F4B4F-7B7E-406A-B25A-FF2AA300AB9A@yesql.se
2023-07-12 19:38:51 +02:00
Peter Eisentraut
8c852ba9a4 Allow some exclusion constraints on partitions
Previously we only allowed unique B-tree constraints on partitions
(and only if the constraint included all the partition keys).  But we
could allow exclusion constraints with the same restriction.  We also
require that those columns be compared for equality, not something
like &&.

Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/ec8b1d9b-502e-d1f8-e909-1bf9dffe6fa5@illuminatedcomputing.com
2023-07-12 09:25:17 +02:00
Thomas Munro
ce0b0fa3e7 Doc: Adjust libpq docs about thread safety.
Describe the situation now that --disable-thread-safety is gone.

Author: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
2023-07-12 08:57:55 +12:00
Thomas Munro
68a4b58eca Remove --disable-thread-safety and related code.
All supported computers have either POSIX or Windows threads, and we no
longer have any automated testing of --disable-thread-safety.  We define
a vestigial ENABLE_THREAD_SAFETY macro to 1 in ecpg_config.h in case it
is useful, but we no longer test it anywhere in PostgreSQL code, and
associated dead code paths are removed.

The Meson and perl-based Windows build scripts never had an equivalent
build option.

Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
2023-07-12 08:20:43 +12:00
Masahiko Sawada
46ebdfe164 Report index vacuum progress.
This commit adds two columns: indexes_total and indexes_processed, to
pg_stat_progress_vacuum system view to show the index vacuum
progress. These numbers are reported in the "vacuuming indexes" and
"cleaning up indexes" phases.

This uses the new parallel message type for progress reporting added
by be06506e7.

Bump catversion because this changes the definition of
pg_stat_progress_vacuum.

Author: Sami Imseih
Reviewed by: Masahiko Sawada, Michael Paquier, Nathan Bossart, Andres Freund
Discussion: https://www.postgresql.org/message-id/flat/5478DFCD-2333-401A-B2F0-0D186AB09228@amazon.com
2023-07-11 12:34:01 +09:00
Peter Eisentraut
3c963d33ea doc: Use proper markup for emphasis 2023-07-09 10:02:15 +02:00
Peter Eisentraut
be2ab58f62 doc: Move DEFAULT parameter on COPY reference page
The DEFAULT parameter seems most similar to the NULL parameter, so
move it next to it, instead of having it at the end of the parameter
list because it was the last one added.
2023-07-09 09:48:16 +02:00
Peter Eisentraut
f8d03ea727 Remove unnecessary unbind in LDAP search+bind mode
Comments in src/backend/libpq/auth.c say: (after successfully finding
the final DN to check the user-supplied password against)

/* Unbind and disconnect from the LDAP server */

and later

/*
 * Need to re-initialize the LDAP connection, so that we can bind to
 * it with a different username.
 */

But the protocol actually permits multiple subsequent authentications
("binds") over a single connection.

So, it seems like the whole connection re-initialization thing was
just a confusion and can be safely removed, thus saving quite a few
network round-trips, especially for the case of ldaps/starttls.

Author: Anatoly Zaretsky <anatoly.zaretsky@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CALbq6kmJ-1+58df4B51ctPfTOSyPbY8Qi2=ct8oR=i4TamkUoQ@mail.gmail.com
2023-07-09 08:51:46 +02:00
David Rowley
c23e7ea4d6 Doc: update old reference to "result cache"
During the PostgreSQL 14 cycle, the Memoize executor node was briefly
called "Result Cache" until it was renamed in 83f4fcc65.  That commit
missed one reference.

Reported-by: Paul A Jungwirth
Packpatch-through: 14, where Memoize was added
Discussion: https://postgr.es/m/CA+renyX=40YXhsfPTzn13oNOPO3TJ12CK9GX-2P2pvnQiScefA@mail.gmail.com
2023-07-09 16:14:00 +12:00
Nathan Bossart
151c22deee Revert MAINTAIN privilege and pg_maintain predefined role.
This reverts the following commits: 4dbdb82513, c2122aae63,
5b1a879943, 9e1e9d6560, ff9618e82a, 60684dd834, 4441fc704d,
and b5d6382496.  A role with the MAINTAIN privilege may be able to
use search_path tricks to escalate privileges to the table owner.
Unfortunately, it is too late in the v16 development cycle to apply
the proposed fix, i.e., restricting search_path when running
maintenance commands.

Bumps catversion.

Reviewed-by: Jeff Davis
Discussion: https://postgr.es/m/E1q7j7Y-000z1H-Hr%40gemulon.postgresql.org
Backpatch-through: 16
2023-07-07 11:25:13 -07:00
Michael Paquier
a14354cac0 Add GUC parameter "huge_pages_status"
This is useful to show the allocation state of huge pages when setting
up a server with "huge_pages = try", where allocating huge pages would
be attempted but the server would continue its startup sequence even if
the allocation fails.  The effective status of huge pages is not easily
visible without OS-level tools (or for instance, a lookup at
/proc/N/smaps), and the environments where Postgres runs may not
authorize that.  Like the other GUCs related to huge pages, this works
for Linux and Windows.

This GUC can report as values:
- "on", if huge pages were allocated.
- "off", if huge pages were not allocated.
- "unknown", a special state that could only be seen when using for
example postgres -C because it is only possible to know if the shared
memory allocation worked after we can check for the GUC values, even if
checking a runtime-computed GUC.  This value should never be seen when
querying for the GUC on a running server.  An assertion is added to
check that.

The discussion has also turned around having a new function to grab this
status, but this would have required more tricks for -DEXEC_BACKEND,
something that GUCs already handle.

Noriyoshi Shinoda has initiated the thread that has led to the result of
this commit.

Author: Justin Pryzby
Reviewed-by: Nathan Bossart, Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/TU4PR8401MB1152EBB0D271F827E2E37A01EECC9@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
2023-07-06 14:42:36 +09:00
Michael Paquier
fa88928470 Generate automatically code and documentation related to wait events
The documentation and the code is generated automatically from a new
file called wait_event_names.txt, formatted in sections dedicated to
each wait event class (Timeout, Lock, IO, etc.) with three tab-separated
fields:
- C symbol in enums
- Format in the system views
- Description in the docs

Using this approach has several advantages, as we have proved to be
rather bad in maintaining this area of the tree across the years:
- The order of each item in the documentation and the code, which should
be alphabetical, has become incorrect multiple times, and the script
generating the code and documentation has a few rules to enforce that,
making the maintenance a no-brainer.
- Some wait events were added to the code, but not documented, so this
cannot be missed now.
- The order of the tables for each wait event class is enforced in the
documentation (the input .txt file does so as well for clarity, though
this is not mandatory).
- Less code, shaving 1.2k lines from the tree, with 1/3 of the savings
coming from the code, the rest from the documentation.

The wait event types "Lock" and "LWLock" still have their own code path
for their code, hence only the documentation is created for them.  These
classes are listed with a special marker called WAIT_EVENT_DOCONLY in
the input file.

Adding a new wait event now requires only an update of
wait_event_names.txt, with "Lock" and "LWLock" treated as exceptions.

This commit has been tested with configure/Makefile, the CI and VPATH
build.  clean, distclean and maintainer-clean were working fine.

Author: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/77a86b3a-c4a8-5f5d-69b9-d70bbf2e9b98@gmail.com
2023-07-05 10:53:11 +09:00
Michael Paquier
28b5726561 libpq: Add support for Close on portals and statements
The following routines are added to libpq:
PGresult *PQclosePrepared(PGconn *conn, const char *stmt);
PGresult *PQclosePortal(PGconn *conn, const char *portal);
int PQsendClosePrepared(PGconn *conn, const char *stmt);
int PQsendClosePortal(PGconn *conn, const char *portal);

The "send" routines are non-blocking versions of the two others.

Close messages are part of the protocol but they did not have a libpq
implementation.  And, having these routines is for instance useful with
connection poolers as these can detect more easily Close messages
than DEALLOCATE queries.

The implementation takes advantage of what the Describe routines rely on
for portals and statements.  Some regression tests are added in
libpq_pipeline, for the four new routines, by closing portals and
statements created already by the tests.

Author: Jelte Fennema
Reviewed-by: Jian He, Michael Paquier
Discussion: https://postgr.es/m/CAGECzQTb4xFAopAVokudB+L62Kt44mNAL4Z9zZ7UTrs1TRFvWA@mail.gmail.com
2023-07-04 14:48:10 +09:00
Nathan Bossart
957845789b Increase size of bgw_library_name.
This commit increases the size of the bgw_library_name member of
the BackgroundWorker struct from BGW_MAXLEN (96) bytes to MAXPGPATH
(default of 1024) bytes so that it can store longer file names
(e.g., absolute paths).

Author: Yurii Rashkovskii
Reviewed-by: Daniel Gustafsson, Aleksander Alekseev
Discussion: https://postgr.es/m/CA%2BRLCQyjFV5Y8tG5QgUb6gjteL4S3p%2B1gcyqWTqigyM93WZ9Pg%40mail.gmail.com
2023-07-03 15:02:16 -07:00
Peter Eisentraut
3ee2f25d21 Change type of pg_statistic_ext.stxstattarget
Change from int32 to int16, to match attstattarget (changed in
90189eefc1).

Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/d6069765-5971-04d3-c10d-e4f7b2e9c459%40eisentraut.org
2023-07-03 07:18:57 +02:00
Michael Paquier
8e278b6576 Remove support for OpenSSL 1.0.1
Here are some notes about this change:
- As X509_get_signature_nid() should always exist (OpenSSL and
LibreSSL), hence HAVE_X509_GET_SIGNATURE_NID is now gone.
- OPENSSL_API_COMPAT is bumped to 0x10002000L.
- One comment related to 1.0.1e introduced by 74242c2 is removed.

Upstream OpenSSL still provides long-term support for 1.0.2 in a closed
fashion, so removing it is out of scope for a few years, at least.

Reviewed-by: Jacob Champion, Daniel Gustafsson
Discussion: https://postgr.es/m/ZG3JNursG69dz1lr@paquier.xyz
2023-07-03 13:20:27 +09:00
Michael Paquier
8c12838001 Make PG_TEST_NOCLEAN work for temporary directories in TAP tests
When set, this environment variable was only effective for data
directories but not for all the other temporary files created by
PostgreSQL::Test::Utils.  Keeping the temporary files after a successful
run can be useful for debugging purposes.

The documentation is updated to reflect the new behavior, with contents
available in doc/ since v16 and in src/test/perl/README since v15.

Author: Jacob Champion
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/CAAWbhmgHtDH1SGZ+Fw05CsXtE0mzTmjbuUxLB9mY9iPKgM6cUw@mail.gmail.com
Discussion: https://postgr.es/m/YyPd9unV14SX2bLF@paquier.xyz
Backpatch-through: 11
2023-07-03 10:06:04 +09:00
Michael Paquier
dd7c60f193 Introduce long options in pg_archivecleanup
This patch is a preliminary refactoring for an upcoming patch aimed at
adding new options to this tool, and using long options for these is
more user-friendly.  The existing short options gain long flavors, as
of:
* -d/--debug
* -n/--dry-run
* -x/--strip-extension

Author: Atsushi Torikoshi
Reviewed-by: Fujii Masao, Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/d660ef741ce3d82f3b4283f1cafd576c@oss.nttdata.com
2023-06-30 15:47:11 +09:00
Tom Lane
5bcc7e6dc8 Stamp HEAD as 17devel.
Let the hacking begin ...
2023-06-29 19:05:23 -04:00
Tom Lane
ac1e974221 Doc: minor wording adjustments in transaction isolation discussion.
Re-word for more clarity, per gripe from Anton Sidyakin.

Discussion: https://postgr.es/m/168745911769.2239590.6062411529242609290@wrigleys.postgresql.org
2023-06-28 12:48:14 -04:00
Michael Paquier
b381d96370 Add timeline ID to file names generated with pg_waldump --save-fullpage
Not including the timeline IDs to the file names generated by pg_waldump
for the individual blocks saved could cause some of these files to be
overwritten when scanning segments across multiple timelines.  Having
this information is also as much useful as the LSNs, to be able to know
from exactly which WAL segment a block is comes from.

While on it, this fixes a few comments in the tests, where the format of
the file was not described as matching with the reality.

Reported-by: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, David Christensen
Discussion: https://postgr.es/m/ZJp921+nITFnvBVS@paquier.xyz
2023-06-28 16:26:55 +09:00
Bruce Momjian
228a2fab9c doc: PG 16 relnotes, update for beta 2 2023-06-26 13:44:40 -04:00
Bruce Momjian
239fd118b0 doc: rename "decades" to be more generic
Reported-by: Michael Paquier

Discussion: https://postgr.es/m/ZJTzwD2rTbHWWQ9g@paquier.xyz

Backpatch-through: 11
2023-06-23 22:50:55 -04:00
Peter Eisentraut
b63cda34e2 doc: Improve punctuation 2023-06-23 14:48:38 +02:00
Peter Eisentraut
37a6d81c45 doc: Clean up title case use 2023-06-23 14:14:57 +02:00
Nathan Bossart
c2122aae63 Improve privilege documentation for maintenance commands.
The documentation of the required privileges for maintenance
commands (i.e., VACUUM, ANALYZE, CLUSTER, LOCK TABLE, REFRESH
MATERIALIZED VIEW, and REINDEX) is redundant, inaccurate, and
difficult to read.  This commit fixes and simplifies this
documentation by removing references to ownership, superuser, and
the pg_maintain role.  In addition, this removes notes about
database-wide VACUUM and ANALYZE, clarifies matters for REINDEX on
partitioned indexes and tables, and strengthens the description of
the pg_maintain role.

Reviewed-by: Michael Paquier, Jeff Davis
Discussion: https://postgr.es/m/20230615041044.GA736001%40nathanxps13
2023-06-22 15:48:38 -07:00
Nathan Bossart
4dbdb82513 Fix cache lookup hazards introduced by ff9618e82a.
ff9618e82a introduced has_partition_ancestor_privs(), which is used
to check whether a user has MAINTAIN on any partition ancestors.
This involves syscache lookups, and presently this function does
not take any relation locks, so it is likely subject to the same
kind of cache lookup failures that were fixed by 19de0ab23c.

To fix this problem, this commit partially reverts ff9618e82a.
Specifically, it removes the partition-related changes, including
the has_partition_ancestor_privs() function mentioned above.  This
means that MAINTAIN on a partitioned table is no longer sufficient
to perform maintenance commands on its partitions.  This is more
like how privileges for maintenance commands work on supported
versions.  Privileges are checked for each partition, so a command
that flows down to all partitions might refuse to process them
(e.g., if the current user doesn't have MAINTAIN on the partition).

In passing, adjust a few related comments and error messages, and
add a test for the privilege checks for CLUSTER on a partitioned
table.

Reviewed-by: Michael Paquier, Jeff Davis
Discussion: https://postgr.es/m/20230613211246.GA219055%40nathanxps13
2023-06-22 15:48:20 -07:00