database/postgres
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2025-07-31 22:04:40 +03:00
Code Activity

Add key management system

Browse Source 
This adds a key management system that stores (currently) two data
encryption keys of length 128, 192, or 256 bits.  The data keys are
AES256 encrypted using a key encryption key, and validated via GCM
cipher mode.  A command to obtain the key encryption key must be
specified at initdb time, and will be run at every database server
start.  New parameters allow a file descriptor open to the terminal to
be passed.  pg_upgrade support has also been added.

Discussion: https://postgr.es/m/CA+fd4k7q5o6Nc_AaX6BcYM9yqTbC6_pnH-6nSD=54Zp6NBQTCQ@mail.gmail.com
Discussion: https://postgr.es/m/20201202213814.GG20285@momjian.us

Author: Masahiko Sawada, me, Stephen Frost
This commit is contained in:
Bruce Momjian
2020-12-25 10:19:44 -05:00
parent 5c31afc49d
commit 978f869b99
49 changed files with 2091 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
doc/src/sgml/config.sgml
View File

@ -7816,6 +7816,52 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</variablelist>
</sect1>
<sect1 id="runtime-config-encryption">
<title>Cluster File Encryption</title>
<variablelist>
<varlistentry id="guc-cluster-key-command" xreflabel="cluster_key_command">
<term><varname>cluster_key_command</varname> (<type>string</type>)
<indexterm>
<primary><varname>cluster_key_command</varname> configuration parameter</primary>
</indexterm>
</term>
<listitem>
<para>
This option specifies an external command to obtain the cluster-level
key for cluster file encryption during server initialization and
server start.
</para>
<para>
The command must print the cluster key to the standard output as
64 hexadecimal characters, and exit with code 0. The command
can prompt for the passphrase or PIN from the terminal if
<option>--authprompt</option> is used. In the parameter value,
<literal>%R</literal> represents the file descriptor number opened
to the terminal that started the server. A file descriptor is only
available if enabled at server start. If <literal>%R</literal>
is used and no file descriptor is available, the server will not
start. Value <literal>%p</literal> is replaced by a pre-defined
prompt string. Value <literal>%d</literal> is replaced by the
directory containing the keys; this is useful if the command
must create files with the keys, e.g., to store a cluster-level
key encryped by a key stored in a hardware security module.
(Write <literal>%%</literal> for a literal <literal>%</literal>.)
Note that the prompt string will probably contain whitespace,
so be sure to quote its use adequately. Newlines are stripped
from the end of the output if present.
</para>
<para>
This parameter can only be set by
<application>initdb</application>, in the
<filename>postgresql.conf</filename> file, or on the server
command line.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
<sect1 id="runtime-config-client">
<title>Client Connection Defaults</title>
@ -9637,6 +9683,22 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
</listitem>
</varlistentry>
<varlistentry id="guc-file-encryption-keylen" xreflabel="file_encryption_keylen">
<term><varname>file_encryption_keylen</varname> (<type>boolean</type>)
<indexterm>
<primary>Cluster file encryption key length</primary>
</indexterm>
</term>
<listitem>
<para>
Reports the bit length of the cluster file
encryption key, or zero if disabled. See <xref
linkend="app-initdb-cluster-key-command"/> for more
information.
</para>
</listitem>
</varlistentry>
<varlistentry id="guc-data-directory-mode" xreflabel="data_directory_mode">
<term><varname>data_directory_mode</varname> (<type>integer</type>)
<indexterm>

97
doc/src/sgml/database-encryption.sgml Normal file
View File

@ -0,0 +1,97 @@
<!-- doc/src/sgml/database-encryption.sgml -->
<chapter id="database-file-encryption">
<title>Cluster File Encryption</title>
<indexterm zone="database-file-encryption">
<primary>Cluster File Encryption</primary>
</indexterm>
<para>
The purpose of cluster file encryption is to prevent users with read
access to the directories used to store database files and write-ahead
log from being able to access the data stored in those files.
For example, when using cluster file encryption, users who have read
access to the cluster directories for backup purposes will not be able
to decrypt the data stored in the these files.
</para>
<para>
Cluster file encryption uses two levels of encryption. The first level
is data encryption keys, specifically keys zero and one. Key zero is
the key used to encrypt database heap and index files which are stored in
the file system, plus temporary files created during database operation.
Key one is used to encrypt write-ahead log (WAL) files. Two different
keys are used so that primary and standby servers can use different zero
(heap/index/temp) keys, but the same one (WAL) key, so that these keys
can eventually be rotated by switching the primary to the standby as
and then changing the WAL key.
</para>
<para>
The second level of encryption is a key used to encrypt first-level
keys. This type of key is often referred to as a Key Encryption Key
(<acronym>KEK</acronym>). This key is <emphasis>not</emphasis> stored
in the file system, but provided at <command>initdb</command> time and
each time the server is started. This key prevents anyone with access
to the database directories from decrypting the data because they do
not know the second-level key which encrypted the first-level keys
which encrypted the database cluster files.
</para>
<sect1 id="encryption-file-encryption">
<title>Initialization</title>
<para>
Cluster file encryption is enabled when
<productname>PostgreSQL</productname> is built
with <literal>--with-openssl</literal> and <xref
linkend="app-initdb-cluster-key-command"/> is specified
during <command>initdb</command>. The cluster key
provided by the <option>--cluster-key-command</option>
option during <command>initdb</command> and the one generated
by <xref linkend="guc-cluster-key-command"/> in the
<filename>postgresql.conf</filename> must match for the database
cluster to start. Note that the cluster key command
passed to <command>initdb</command> must return a key of
64 hexadecimal characters. For example.
<programlisting>
initdb -D dbname --cluster-key-command='ckey_passphrase.sh'
</programlisting>
</para>
</sect1>
<sect1 id="key-encryption-key">
<title>Internals</title>
<para>
During the <command>initdb</command> process, if
<option>--cluster-key-command</option> is specified, two data-level
encryption keys are created. These two keys are then encrypted with
the key enryption key (KEK) supplied by the cluster key command before
being stored in the database directory. The key or passphrase that
derives the key must be supplied from the terminal or stored in a
trusted key store, such as key vault software, hardware security module.
</para>
<para>
If the <productname>PostgreSQL</productname> server has
been initialized to require a cluster key, each time the
server starts the <filename>postgresql.conf</filename>
<varname>cluster_key_command</varname> command will be executed
and the cluster key retrieved. The data encryption keys in the
<filename>pg_cryptokeys</filename> directory will then be decrypted
using the supplied key and integrity-checked to ensure it
matches the initdb-supplied key. If this check fails, the
server will refuse to start.
</para>
<para>
The data encryption keys are randomly generated and are of 128, 192,
or 256-bits in length. They are encrypted by the key encryption key
(KEK) using Advanced Encryption Standard (<acronym>AES256</acronym>)
encryption in Galois/Counter Mode (<acronym>GCM</acronym>), which also
provides KEK authentication.
</para>
</sect1>
</chapter>

1
doc/src/sgml/filelist.sgml
View File

@ -49,6 +49,7 @@
<!ENTITY wal SYSTEM "wal.sgml">
<!ENTITY logical-replication SYSTEM "logical-replication.sgml">
<!ENTITY jit SYSTEM "jit.sgml">
<!ENTITY database-encryption SYSTEM "database-encryption.sgml">
<!-- programmer's guide -->
<!ENTITY bgworker SYSTEM "bgworker.sgml">

5
doc/src/sgml/installation.sgml
View File

@ -976,8 +976,9 @@ build-postgresql:
<listitem>
<para>
Build with support for <acronym>SSL</acronym> (encrypted)
connections. This requires the <productname>OpenSSL</productname>
package to be installed. <filename>configure</filename> will check
connections and cluster file encryption. This requires the
<productname>OpenSSL</productname> package to be installed.
<filename>configure</filename> will check
for the required header files and libraries to make sure that
your <productname>OpenSSL</productname> installation is sufficient
before proceeding.

1
doc/src/sgml/postgres.sgml
View File

@ -171,6 +171,7 @@ break is not needed in a wider output rendering.
&wal;
&logical-replication;
&jit;
&database-encryption;
&regress;
</part>

46
doc/src/sgml/ref/initdb.sgml
View File

@ -163,6 +163,17 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry id="app-initdb-cluster-key-command" xreflabel="cluster key command">
<term><option>--cluster-key-command=<replaceable class="parameter">command</replaceable></option></term>
<listitem>
<para>
This option specifies an external command to obtain the cluster-level
key for cluster file encryption during server initialization and
server start; see <xref linkend="guc-cluster-key-command"/> for details.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
<term><option>--pgdata=<replaceable class="parameter">directory</replaceable></option></term>
@ -223,6 +234,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry id="app-initdb-file-encryption-keylen"
xreflabel="file encryption">
<term><option>-K</option></term>
<term><option>--file-encryption-keylen</option></term>
<listitem>
<para>
Specifies the number of bits for the file encryption keys. The
default is 128 bits.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--locale=<replaceable>locale</replaceable></option></term>
<listitem>
@ -285,6 +308,17 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry>
<term><option>-R</option></term>
<term><option>--authprompt</option></term>
<listitem>
<para>
Allows the <option>--cluster-key-command</option> command
to prompt for a passphrase or PIN.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>-S</option></term>
<term><option>--sync-only</option></term>
@ -307,6 +341,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry>
<term><option>-u <replaceable>datadir</replaceable></option></term>
<term><option>--copy-encryption-keys=<replaceable>datadir</replaceable></option></term>
<listitem>
<para>
Copies cluster file encryption keys from another cluster; required
when using <application>pg_upgrade</application> on a cluster
with cluster file encryption enabled.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>-U <replaceable class="parameter">username</replaceable></option></term>
<term><option>--username=<replaceable class="parameter">username</replaceable></option></term>

13
doc/src/sgml/ref/pg_ctl-ref.sgml
View File

@ -38,6 +38,7 @@ PostgreSQL documentation
<arg choice="opt"><option>-s</option></arg>
<arg choice="opt"><option>-o</option> <replaceable>options</replaceable></arg>
<arg choice="opt"><option>-p</option> <replaceable>path</replaceable></arg>
<arg choice="opt"><option>-R</option></arg>
<arg choice="opt"><option>-c</option></arg>
</cmdsynopsis>
@ -72,6 +73,7 @@ PostgreSQL documentation
<arg choice="opt"><option>-t</option> <replaceable>seconds</replaceable></arg>
<arg choice="opt"><option>-s</option></arg>
<arg choice="opt"><option>-o</option> <replaceable>options</replaceable></arg>
<arg choice="opt"><option>-R</option></arg>
<arg choice="opt"><option>-c</option></arg>
</cmdsynopsis>
@ -373,6 +375,17 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry>
<term><option>-R</option></term>
<term><option>--authprompt</option></term>
<listitem>
<para>
Allows the <option>--cluster-key-command</option> command
to prompt for a passphrase or PIN.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>-s</option></term>
<term><option>--silent</option></term>

18
doc/src/sgml/ref/pgupgrade.sgml
View File

@ -167,6 +167,13 @@ PostgreSQL documentation
</para></listitem>
</varlistentry>
<varlistentry>
<term><option>-R</option></term>
<term><option>--authprompt</option></term>
<listitem><para>allows prompting for a passphrase or PIN
</para></listitem>
</varlistentry>
<varlistentry>
<term><option>-s</option> <replaceable>dir</replaceable></term>
<term><option>--socketdir=</option><replaceable>dir</replaceable></term>
@ -309,7 +316,9 @@ make prefix=/usr/local/pgsql.new install
Again, use compatible <command>initdb</command>
flags that match the old cluster. Many
prebuilt installers do this step automatically. There is no need to
start the new cluster.
start the new cluster. If upgrading a cluster that uses
cluster file encryption, the <command>initdb</command> option
<option>--copy-encryption-keys</option> must be specified.
</para>
</step>
@ -838,6 +847,13 @@ psql --username=postgres --file=script.sql postgres
is down.
</para>
<para>
If the old cluster uses file encryption, the new cluster must use
the same keys, so <command>pg_upgrade</command> copies them to the
new cluster. It is necessary to initialize the new cluster with
the same <varname>cluster_key_command</varname> and the same
file encryption key length.
</para>
</refsect1>
<refsect1>

13
doc/src/sgml/ref/postgres-ref.sgml
View File

@ -297,6 +297,19 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry>
<term><option>-R <replaceable class="parameter">file-descriptor</replaceable></option></term>
<listitem>
<para>
Makes <command>postgres</command> prompt for a passphrase or PIN
from the specified open numeric file descriptor. The descriptor
is closed after the key is read. The file descriptor number
<literal>-1</literal> duplicates standard error for the terminal;
this is useful for single-user mode.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>-s</option></term>
<listitem>

5
doc/src/sgml/storage.sgml
View File

@ -77,6 +77,11 @@ Item
<entry>Subdirectory containing transaction commit timestamp data</entry>
</row>
<row>
<entry><filename>pg_cryptokeys</filename></entry>
<entry>Subdirectory containing file encryption keys</entry>
</row>
<row>
<entry><filename>pg_dynshmem</filename></entry>
<entry>Subdirectory containing files used by the dynamic shared memory