From a551350f226b0635bd328ccadf953055b29e9971 Mon Sep 17 00:00:00 2001 From: ravindk89 Date: Tue, 10 Nov 2020 19:54:09 -0500 Subject: [PATCH] Andreas Review, Big Reconfig, Thales Tutorial --- requirements.txt | 4 +- source/includes/common-minio-kes.rst | 11 + source/minio-cli/kes/kes-identity.rst | 139 ++++- source/minio-cli/kes/kes-key.rst | 123 +++-- source/minio-cli/kes/kes-policy.rst | 245 ++++++--- source/minio-cli/kes/kes-server.rst | 22 +- source/minio-cli/kes/kes-tool-identity.rst | 68 ++- source/minio-cli/minio-kes-reference.rst | 378 ++++++++++++- source/minio-server/minio-server.rst | 35 +- source/security/encryption/minio-kes.rst | 172 +----- source/security/encryption/sse-s3-thales.rst | 548 ++++++++++++++++++- 11 files changed, 1353 insertions(+), 392 deletions(-) create mode 100644 source/includes/common-minio-kes.rst diff --git a/requirements.txt b/requirements.txt index 2ac67f23..8556e716 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,5 @@ sphinx == 3.1.2 sphinx-copybutton == 0.2.12 -git+https://github.com/ravindk89/sphinx-tabs recommonmark == 0.6.0 -sphinx-markdown-tables == 0.0.15 \ No newline at end of file +sphinx-markdown-tables == 0.0.15 +sphinx-tabs == 1.3.0 \ No newline at end of file diff --git a/source/includes/common-minio-kes.rst b/source/includes/common-minio-kes.rst new file mode 100644 index 00000000..bc7c29f9 --- /dev/null +++ b/source/includes/common-minio-kes.rst @@ -0,0 +1,11 @@ +.. start-kes-insecure + +Directs the command skip x.509 certificate validation during the TLS +handshake with the KES server. This allows connections to KES servers +using untrusted certificates (i.e. self-signed or issued by an unknown +Certificate Authority). + +MinIO strongly recommends *against* using this option in production +environments. + +.. end-kes-insecure \ No newline at end of file diff --git a/source/minio-cli/kes/kes-identity.rst b/source/minio-cli/kes/kes-identity.rst index 5118ffc4..19d3d261 100644 --- a/source/minio-cli/kes/kes-identity.rst +++ b/source/minio-cli/kes/kes-identity.rst @@ -10,25 +10,22 @@ .. mc:: kes identity -The :mc:`kes identity` command performs non-persistent association or removal -of x.509 identities to :ref:`policies ` on a KES server. -All changes made by :mc:`kes identity` are lost when the KES server restarts. +The :mc:`kes identity` command temporarily adds or removes :ref:`identities +` to or from :ref:`policies ` on a +KES server. The command can also list available identities on the command line. -To make persistent changes to KES policies, modify the -:kesconf:`policy` section of the KES -:ref:`configuration file `. Specifically, for each -:kesconf:`policy.policyname` to modify, you must add/remove the -identities to/from the :kesconf:`policy.policyname.identities` array. +All changes made by :mc:`kes identity` are lost when the KES server restarts. To +make persistent changes to KES policies, modify the :kesconf:`policy` section of +the KES :ref:`configuration file `. Specifically, for each +:kesconf:`policy.policyname` to modify, you must add/remove the identities +to/from the :kesconf:`policy.policyname.identities` array. -Use the :mc:`kes tool identity of` command line tool to compute the -identity hash for the x.509 certificate to add or remove from a policy. +This page provides reference information for the :mc:`kes identity` command. -This page provides reference information for the :mc:`kes policy` command. +- For more information on KES Access Control, see + :ref:`minio-kes-access-control`. -- For more complete information on KES policies, see - :ref:`minio-kes-policy`. - -- For more complete conceptual information on KES, see :ref:`minio-kes`. +- For conceptual information on KES, see :ref:`minio-kes`. Examples -------- @@ -48,17 +45,119 @@ Syntax .. mc-cmd:: assign :fullpath: - Assigns a new x.509 identity to a policy on the KES server. + Assigns a new temporary :ref:`identity ` to a + :ref:`policy ` on the KES server. KES grants clients + authenticating with that identity access to the operations explicitly allowed + by the associated policy. - Use the :mc:`kes tool identity of` command to compute the identity hash - for the x.509 certificate. + All changes made by :mc:`kes identity assign` are lost when the KES server + restarts. To permanently assign an identity to a policy, modify the + :kesconf:`policy` section of the KES + :ref:`configuration document ` to include the new + association. + + The command has the following syntax: + + .. code-block:: shell + + kes identity assign [OPTIONS] IDENTITY POLICY + + The command accepts the following arguments: + + .. mc-cmd:: IDENTITY + + *Required* + + The SHA-256 hash of an x.509 certificate to use as a KES + :ref:`identity `. Use the + :mc:`kes tool identity of` command to compute the identity for the + x.509 certificate. + + .. mc-cmd:: POLICY + + *Required* + + The name of the policy to which to associate the identity. + + A given identity can associate to at most *one* policy on a + KES server. + + .. mc-cmd:: insecure, k + :option: + + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure .. mc-cmd:: list :fullpath: - Lists the x.509 identities on the KES server. + List all :ref:`identities ` configured on the KES + server. + + The command has the following syntax: + + .. code-block:: shell + + kes identity [OPTIONS] list [PATTERN] + + The command accepts the following arguments: + + .. mc-cmd:: PATTERN + + *Optional* + + The `glob pattern `__ + used to filter identities on the KES server. + + Defaults to ``*`` or all identities. + + .. mc-cmd:: insecure, k + :option: + + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure .. mc-cmd:: forget :fullpath: - Removes an x.509 identity from the KES server. \ No newline at end of file + Temporarily removes an :ref:`identity ` from the KES + server. Removing an identity prevents clients authenticating with that + identity from performing any operation on the KES server. + + All changes made by :mc:`kes identity forget` are lost when the KES server + restarts. To permanently remove an identity-policy association, modify the + :kesconf:`policy` section of the KES + :ref:`configuration document ` to remove the + association. + + The command has the following syntax: + + .. code-block:: shell + + kes identity forget [OPTIONS] IDENTITY + + The command accepts the following arguments: + + .. mc-cmd:: IDENTITY + + *Required* + + The identity to remove from the KES server. + + Use the :mc:`kes tool identity of` command line tool to compute the + identity hash for the x.509 certificate to forget. + + .. mc-cmd:: insecure, k + :option: + + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure \ No newline at end of file diff --git a/source/minio-cli/kes/kes-key.rst b/source/minio-cli/kes/kes-key.rst index 4b3fcd0d..7c8b51b9 100644 --- a/source/minio-cli/kes/kes-key.rst +++ b/source/minio-cli/kes/kes-key.rst @@ -10,8 +10,10 @@ .. mc:: kes key -The :mc:`kes key` command manages creating cryptographic keys for use with -Server-Side Object Encryption (SSE). +The :mc:`kes key` command creates, utilizes, and deletes cryptographic keys +(Secrets) through the MinIO Key Encryption Service (KES). KES stores +created secrets on the configured :ref:`Key Management System (KMS) +`. This page provides reference information for the :mc:`kes key` command. @@ -21,17 +23,24 @@ command. - For an example of using :mc:`kes key` to generate secret keys using Thales CipherTrust as the KMS, see -- Other specific tutorials to follow. - Syntax ------ .. mc-cmd:: create :fullpath: - Creates a new secret key on the KES server. + Creates a new Secret cryptographic key using the MinIO Key Encryption + Service (KES). The Secret key supports generation of + Data Encryption Keys (DEK) via the :mc:`kes key derive` command. - Creates a new CMK. + The created Secret key also supports enabling MinIO Server-Side Encryption + (SSE-S3). Specify the created key name to the + :envvar:`MINIO_KMS_KES_KEY_NAME` environment variable before starting the + :mc:`minio` server. + + KES stores the Secret key on the configured + :ref:`Key Management System (KMS) `. + KES *never* returns the Secret key to clients. The command has the following syntax: @@ -44,31 +53,51 @@ Syntax .. mc-cmd:: NAME - The name of the secret key. + The name of the Secret key. + + - If the :mc-cmd:`kes key create NAME` command *includes* + :mc-cmd:`~kes key create KEY`, the command imports the ``KEY`` + and labels it using the specified :mc-cmd:`~kes key create NAME`. + + - If the :mc-cmd:`kes key create NAME` command *omits* + :mc-cmd:`~kes key create KEY`, the command creates a new Secret on the + configured KMS and labels it using the + specified :mc-cmd:`~kes key create NAME`. .. mc-cmd:: KEY *Optional* - The value to use as the secret key. Specify a base64-encoded string. - - Omit :mc-cmd:`~kes key create KEY` to direct KES to automatically - generate a random value for the key. + The Secret key to import into the configured KMS. Specify a base64-encoded + string. + Specifying this option directs :mc-cmd:`kes key create` to + use the ``v1/key/import`` API endpoint, which has distinct + :ref:`policy ` requirements compared to + key creation. + .. mc-cmd:: insecure, k :option: - Skips x.509 Certificate Validation during TLS handshakes. This option - is required if using self-signed certificates. + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure .. mc-cmd:: delete :fullpath: - Deletes a master key on the KES server. Deleting a master key prevents - decrypting any cryptographic keys derived using that master key, which + Deletes a Secret key on the KES server. Deleting a Secret key prevents + decrypting any cryptographic keys derived using that Secret key, which in turn prevents decrypting any objects encrypted with those cryptographic keys. + .. warning:: + + Deleting a Secret key renders all data encrypted using that key + permanently unreadable. + The command has the following syntax: .. code-block:: shell @@ -82,19 +111,31 @@ Syntax *Required* - The name of the master key to delete. + The name of the Secret key to delete. .. mc-cmd:: insecure, k :option: - Skips x.509 Certificate Validation during TLS handshakes. This option - is required if using self-signed certificates. + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure .. mc-cmd:: derive :fullpath: - Derives a new cryptographic key using a master key on the KES server. The - cryptographic key can support Server-Side Object Encryption (SSE-S3). + Derives a new cryptographic Data Encryption Key (DEK) using a Secret key on + the KES server. + + The command returns the plaintext and ciphertext representations of the DEK. + To encrypt or decrypt data using the DEK, use the following procedure: + + 1. Use :mc-cmd:`kes key decrypt` on the ciphertext to extract the plaintext. + 2. Encrypt or decrypt data using the plaintext. + + Avoid storing the plaintext on disk, as it allows decryption of data + without requiring access to the Secret key used to generate the DEK. The command has the following syntax: @@ -110,34 +151,41 @@ Syntax *Required* - The name of the master key on the KES server to use to generate the - cryptographic key. + The name of the Secret key to use to generate the DEK. .. mc-cmd:: CONTEXT *Optional* - A base64-encoded string to use with the master key for deriving the - cryptographic key. + A base64-encoded string to use with the Secret key for deriving the + DEK. If specified, the ``CONTEXT`` is *required* to decrypt the DEK + ciphertext. .. mc-cmd:: insecure, k :option: - Skips x.509 Certificate Validation during TLS handshakes. This option - is required if using self-signed certificates. + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure .. mc-cmd:: decrypt :fullpath: - Decrypt the ciphertext and return the plain cryptographic key produced :mc-cmd:`kes key derive` - :mc-cmd:`kes key derive`. + Decrypt the Data Encryption Key (DEK) ciphertext and return the plaintext + key. + + Use the plaintext value for encrypting or decrypting data using the DEK. + Avoid storing the plaintext on disk, as it allows decryption of data + without requiring access to the Secret key used to generate the DEK. The command has the following syntax: .. code-block:: shell :class: copyable - kes key decrypt [ARGUMENTS] NAME CIPHERTEXT [CONTEXT] + kes key decrypt [OPTIONS] NAME CIPHERTEXT [CONTEXT] The command accepts the following arguments: @@ -145,28 +193,31 @@ Syntax *Required* - The name of master key used to generate the cryptographic key. + The name of Secret key used to generate the DEK key. - :mc-cmd:`kes key decrypt` fails if the specified master key + :mc-cmd:`kes key decrypt` fails if the specified Secret key was not used to encrypt the :mc-cmd:`~kes key decrypt CIPHERTEXT`. .. mc-cmd:: CIPHERTEXT *Required* - The cryptographic key to decrypt using the specified - :mc-cmd:`master key `. + The DEK ciphertext to decrypt using the specified + :mc-cmd:`Secret `. .. mc-cmd:: CONTEXT *Optional* The base64-encoded string specified to - :mc-cmd:`kes key derive CONTEXT` when creating the cryptographic key, if + :mc-cmd:`kes key derive CONTEXT` when creating the DEK key, if any. .. mc-cmd:: insecure, k :option: - Skips x.509 Certificate Validation during TLS handshakes. This option - is required if using self-signed certificates. + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure diff --git a/source/minio-cli/kes/kes-policy.rst b/source/minio-cli/kes/kes-policy.rst index 90a642be..8fbeeacf 100644 --- a/source/minio-cli/kes/kes-policy.rst +++ b/source/minio-cli/kes/kes-policy.rst @@ -10,10 +10,11 @@ .. mc:: kes policy -The :mc:`kes policy` command performs non-persistent creation and modification -of access policies on a MinIO Key Encryption Service (KES). All changes made by -:mc:`kes policy` are lost when the KES server restarts. +The :mc:`kes policy` command temporarily creates or modifies policies on the +MinIO Key Encryption Service (KES). The command can also list available +policies and display their contents on the command line. +All changes made by :mc:`kes policy` are lost when the KES server restarts. To make persistent changes to KES policies, add or modify the policies listed under the :kesconf:`policy` section of the :ref:`KES configuration file `. :mc:`kes policy` supports @@ -26,88 +27,10 @@ command. - For more complete information on KES policies, see :ref:`minio-kes-policy`. -- For more complete conceptual information on KES, see :ref:`minio-kes`. +- For more information on KES access control, see + :ref:`minio-kes-access-control`. -.. _minio-kes-policy-document: - -Policy Document Structure -~~~~~~~~~~~~~~~~~~~~~~~~~ - -KES uses JSON-formatted policy documents to describe the permissions granted -to authenticated clients. - -The KES Policy Document has the following schema: - -.. code-block:: json - :class: copyable - - { - "policyName": { - "paths": [ - "ENDPOINT", - "ENDPOINT" - ], - "identities": [ - "IDENTITY", - "IDENTITY" - ] - } - } - -The following table describes each field in the KES Policy Document: - -.. kespolicy:: policyName - - *Type: object* - - The name of the KES policy. Replace ``"policyName"`` with a unique name - for the KES policy. You can specify multiple :kespolicy:`policyName` - objects in the KES Policy document: - - .. code-block:: json - - { - "keyManagement" : {}, - "encryptDecrypt" : {} - } - -.. kespolicy:: policyName.paths - - *Type: array* - - An array of KES API endpoints for which the - :kespolicy:`~policyName.identities` can access. - - Each endpoint *must* be a glob pattern in the following form: - - .. code-block:: shell - - ///[//] - - You can specify an asterisk ``*`` to create a catch-all pattern for - a given endpoint. For example, the following endpoint pattern - allows complete access to key creation via the ``/v1/key/create`` - endpoint: - - .. code-block:: shell - - /v1/key/create/* - - See :ref:`minio-kes-endpoints` for a list of KES endpoints and the - actions associated to each. - -.. kespolicy:: policyName.identities - - *Type: array* - - An array of x.509 identities associated to the policy. KES grants clients - authenticating with a matching x.509 certificate access to the - endpoints listed in the :kespolicy:`~policyName.paths` for the - policy. - - Use :mc-cmd:`kes tool identity of` to compute the name of each x.509 - certificate you want to associate to the policy and specify that value to the - array. +- For complete conceptual information on KES, see :ref:`minio-kes`. Examples -------- @@ -127,19 +50,165 @@ Syntax .. mc-cmd:: add :fullpath: - This command adds a new policy to the KES server. + Adds a new temporary :ref:`policy ` to the KES + server. Policies support KES :ref:`access control + `. + + The created policy has no associated :ref:`identities + `. Use :mc-cmd:`kes identity assign` to assign + identities to the policy. + + All changes made by :mc:`kes policy` are lost when the KES server restarts. + To create permanent policies, modify the :kesconf:`policy` section of the KES + :ref:`configuration document ` to include the new policy. + + The command has the following syntax: + + .. code-block:: shell + + kes policy add [OPTIONS] POLICY FILE + + The command supports the following arguments: + + .. mc-cmd:: POLICY + + *Required* + + The name of the policy to add to the KES server. The specified name + *must* be unique among all policies configured on the KES server. + + .. mc-cmd:: FILE + + *Required* + + The ``JSON`` formatted file to use for creating the new policy. + The file must has the following schema: + + .. code-block:: json + :class: copyable + + { + "paths": [ + "ENDPOINT", + "ENDPOINT" + ] + } + + Each ``ENDPOINT`` is a KES :ref:`Server API endpoint + ` to which the policy grants access. KES supports + using + `glob patterns `__ in + the following form: + + .. code-block:: shell + + ///[//] + + The following example uses the ``*`` wildcard character to allow + access to any operation using the ``/v1/key/create`` endpoint: + + .. code-block:: json + + { + "paths" : [ + "/v1/key/create/*" + ] + } + + See :ref:`minio-kes-endpoints` for a list of KES endpoints and the + actions associated to each. + + .. mc-cmd:: insecure, k + :option: + + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure .. mc-cmd:: show :fullpath: - This command outputs the policy document to ``STDOUT``. + Outputs the specified + :ref:`minio-kes-policy` contents to ``STDOUT``. + + The command has the following syntax: + + .. code-block:: shell + + kes policy show [OPTIONS] POLICY + + The command accepts the following arguments: + + .. mc-cmd:: POLICY + + *Required* + + The name of the policy to show. + + .. mc-cmd:: insecure, k + :option: + + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure + + .. mc-cmd:: list :fullpath: - This command lists policies on the KES server. + Lists :ref:`policies ` on the KES server. + + The command has the following syntax: + + .. code-block:: shell + + kes policy list [OPTIONS] [PATTERN] + + The command accepts the following arguments: + + .. mc-cmd:: PATTERN + + *Optional* + + The `glob pattern `__ + used to filter policies on the KES server. + + Defaults to ``*`` or all policies. + + .. mc-cmd:: insecure, k + :option: + + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure .. mc-cmd:: delete :fullpath: - This command deletes a policy on the KES server. + Deletes a :ref:`policies ` on the KES server. + Deleting a policy prevents clients authenticating with an identity + associated to that policy from performing any operations on the KES server. + + The command has the following syntax: + + .. code-block:: shell + + kes policy delete [OPTIONS] POLICY + + The command accepts the following arguments: + + .. mc-cmd:: insecure, k + :option: + + *Optional* + + .. include:: /includes/common-minio-kes.rst + :start-after: start-kes-insecure + :end-before: end-kes-insecure \ No newline at end of file diff --git a/source/minio-cli/kes/kes-server.rst b/source/minio-cli/kes/kes-server.rst index 54c0e6ab..130048ba 100644 --- a/source/minio-cli/kes/kes-server.rst +++ b/source/minio-cli/kes/kes-server.rst @@ -20,9 +20,6 @@ This page provides reference information for the :mc:`kes server` command. For more complete conceptual information on KEs, see :ref:`minio-kes`. -Server Configuration File -~~~~~~~~~~~~~~~~~~~~~~~~~ - Examples -------- @@ -53,7 +50,7 @@ The command has the following syntax: .. mc-cmd:: cert :option: - Path to KES server X.509 certificate served by the KES server to clients when a :abbr: `TLS (Transport Layer Encryption)` connection is established + Path to KES server X.509 certificate served by the KES server to clients when a :abbr: `TLS (Transport Layer Encryption)` connection is established enabling :abbr:`TLS (Transport Layer Encryption)`. .. mc-cmd:: config @@ -71,17 +68,14 @@ The command has the following syntax: .. mc-cmd:: root :option: - The identity with root permissions on the KES server. `kes tool identity of CERTIFICATE`` to - compute the X.509 identity of an arbitrary client certificate. - ``kes tool identity of CERTIFICATE`` to retrieve the certificate to use - for the root identity. + The identity with root permissions on the KES server. + + Use the :mc-cmd:`kes tool identity of` command to compute the X.509 identity + of an arbitrary client certificate. + + .. code-block:: shell -.. mc-cmd:: port - :option: - - The port on which the :mc:`kes server` listens. - - Defaults to ``7373``. + kes tool identity of CERTIFICATE .. mc-cmd:: auth :option: diff --git a/source/minio-cli/kes/kes-tool-identity.rst b/source/minio-cli/kes/kes-tool-identity.rst index bccddc9a..032444b8 100644 --- a/source/minio-cli/kes/kes-tool-identity.rst +++ b/source/minio-cli/kes/kes-tool-identity.rst @@ -22,8 +22,8 @@ command. - For more complete conceptual information on KES, see :ref:`minio-kes`. -:mc:`kes tool identity new` only supports generating self-signed certificates and -is best suited for supporting early development and evaluation of KES. To +:mc:`kes tool identity new` only supports generating self-signed certificates +and is best suited for supporting early development and evaluation of KES. To configure a KES server to use an x.509 certificate generated through other means, including ACME-based tools like CertBot, use :mc:`kes identity assign` instead. @@ -34,27 +34,49 @@ Syntax .. mc-cmd:: new :fullpath: - Creates a new x.509 certificate identity for use with accessing the KES - server. + Creates a new self-signed x.509 certificate for use with accessing the KES + server. These certificates are best used during early development and + evaluation, and require the KES :mc:`~kes server` run with x.509 validation + disabled (:mc-cmd-option:`kes server auth=off `). + + - Use :mc-cmd:`kes tool identity of` to derive the + :ref:`identity hash ` of the new certificate. + + - Use :mc-cmd:`kes identity assign` to temporarily assign the new identity + to a :ref:`policy ` on the KES server. + + - Modify the :kesconf:`policy` section of the KES + :ref:`configuration document ` to permanently assign + the new identity to a :ref:`policy ` on the KES server. + + KES clients can authenticate using the generated certificate by specifying + the certificate ``*.cert`` and private key ``*.key`` to the + ``KES_CLIENT_CERT`` and ``KES_CLIENT_KEY`` environment variables + respectively. The command has the following syntax: .. code-block:: shell :class: copyable - kes tool identity new [OPTIONS] [SUBJECT-NAME] + kes tool identity new [OPTIONS] [SUBJECT] The command accepts the following arguments: - .. mc-cmd:: NAME + .. mc-cmd:: SUBJECT - The x.509 ``commonName`` to associate to the generated x.509 certificates. + *Optional* + + The x.509 ``Subject.commonName`` to associate to the generated x.509 + certificates. Defaults to ``""`` if unspecified. .. mc-cmd:: key :option: + *Optional* + The path to the private key to create for the identity. Defaults to ``./private.key``. @@ -62,20 +84,27 @@ Syntax .. mc-cmd:: cert :option: - The path to the public key certificate to create for the identity. + *Optional* + + The path to the self-signed public key certificate to create for the + identity. Defaults to ``./public.cert``. .. mc-cmd:: time, t :option: + *Optional* + The duration to certificate expiration. - Defaults to ``720h`` or 720 hours. + Defaults to ``720h`` or 720 hours (30 days). .. mc-cmd:: force, f :option: + *Optional* + Directs :mc-cmd:`kes tool identity new` to overwrite the specified :mc-cmd-option:`~kes tool identity new key` or :mc-cmd:`~kes tool identity new cert` if either exists. @@ -83,7 +112,14 @@ Syntax .. mc-cmd:: of :fullpath: - Computes the identity string from a TLS certificate. + Computes a hash from an x.509 certificate. You can use the computed + hash as an :ref:`identity ` for configuring + KES :ref:`access control `. + + Use the computed string when assigning the identity to a :ref:`policy + `. KES grants clients authenticating with the x.509 + certificate access to the operations explicitly allowed by the + associated policy. The command has the following syntax: @@ -96,8 +132,8 @@ Syntax .. mc-cmd:: NAME - The name of the certificate for which the command computes the identity - string. + The name of the x.509 certificate for which the command computes the + identity string. Specify the full path to the certificate. .. mc-cmd:: hash :option: @@ -108,3 +144,11 @@ Syntax - ``SHA-256`` - ``SHA-384`` - ``SHA-512`` + + :mc-cmd:`kes tool identity of` defaults to ``SHA-256`` if this option is + omitted. + + The KES server uses ``SHA-256`` to compute a KES client's identity using + the client x.509 certificate. Omit this option *or* explicitly + specify ``SHA-256`` if computing an identity hash for use with + configuring :ref:`KES access control `. diff --git a/source/minio-cli/minio-kes-reference.rst b/source/minio-cli/minio-kes-reference.rst index 1584e5ae..612520c0 100644 --- a/source/minio-cli/minio-kes-reference.rst +++ b/source/minio-cli/minio-kes-reference.rst @@ -18,13 +18,14 @@ key-management system for high-performance applications. KES provides a bridge between applications running in bare-metal or orchestrated environments to centralised KMS solutions. -This page provides a reference for the :mc:`kes` command line tool. For more -complete conceptual information on KES, see :ref:`minio-kes`. +KES is designed for simplicity, scalability, and security. It requires +minimal configuration to enable full functionality and requires only +basic familiarity with cryptography or key-management concepts. -For tutorials on deploying KES to support Server-Side Object Encryption, see: - -- Server-Side Object Encryption with Thales CipherText -- Other similar tutorials. +This page contains conceptual and reference information on using KES for +performing cryptographic key operations and general data encryption/decryption. +For documentation on deploying KES to support MinIO Server-Side Encryption +(SSE-S3), see :ref:`minio-kes`. .. _minio-kes-installation: @@ -33,6 +34,282 @@ Installation .. include:: /includes/minio-kes-installation.rst +.. _minio-kes-access-control: + +KES Access Control +------------------ + +KES uses mutual TLS authentication (mTLS) for performing both authentication and +authorization of clients making requests to the KES server. + +Authentication +~~~~~~~~~~~~~~ + +Both the client and server present their x.509 certificate and +corresponding private key to their peer. The server only accepts the +connection if the client certificate is valid and authentic: + +- A "valid" certificate is well-formed and current (i.e. not expired). + + mTLS also mTLS also requires the following `Extended Key Usage + `__ extensions for + the client and server. Specifically: + + - The client certificate *must* include Client Authentication + (``extendedKeyUsage = clientAuth``) + + - The server certificate *must* include Server Authentication + (``extendedKeyUsage = serverAuth``) + +- An "authentic" certificate is issued by a trusted Certificate Authority + (CA). Both client and server *must* include the peer certificate CA + in their local system trust store. + + You can start the KES server with the :mc-cmd-option:`kes server auth` + option to perform mTLS with untrusted certificates during testing or early + development. However, untrusted certificates present a security + vulnerability and may open the KES server to access by unknown parties. + MinIO strongly recommends only allowing trusted certificates in production + environments. + +Once authenticated, the KES server proceeds to check if the client +is *authorized* to perform the requested operation. + +.. _minio-kes-authorization: + +Authorization +~~~~~~~~~~~~~ + +KES uses Policy-Based Access Control (PBAC) for determining what operations +a given client has permission to perform. Each policy consists of +one or more identities, where each identity corresponds to the +SHA-256 hash of an x.509 certificate. The server only allows the +client to perform the requested operation if *all* of the following +are true: + +- A policy on the KES server contains the client identity. + +- The policy explicitly allows the requested operation. + +If no such policy exists on the KES server *or* if the policy does not +explicitly allow the requested operation, the KES server denies the +client's request. For more complete documentation on policies, +see :ref:`minio-kes-policy`. + +.. _minio-kes-policy: + +KES Policies +~~~~~~~~~~~~ + +KES uses policy-based access control (PBAC), where a policy describes the +operations which an authenticated client may perform. + +The following ``YAML`` document provides an example of the :kesconf:`policy` +section of the KES server configuration document. The policy ``minio-sse`` +includes the appropriate :ref:`API endpoints ` for +supporting MinIO Server-Side Encryption: + +.. code-block:: yaml + :class: copyable + + policy: + minio-sse: + paths: + - /v1/key/create/* + - /v1/key/generate/* + - /v1/key/decrypt/* + - /v1/key/delete/* + identities: + - + +- Each element in the :kesconf:`policy.policyname.paths` array represents an + :ref:`API endpoint ` to which the policy grants access. + +- Each element in the :kesconf:`policy.policyname.identities` array represents + the SHA-256 hash of an x.509 certificate presented by a client. + + Use the :mc-cmd:`kes tool identity of` command to compute the identity hash + of a client's x.509 certificate. + +Policies and identities have a one-to-many relationship, where one policy can +contain many identities. *However*, a given identity can associate to at most +one policy at a time. + +The KES server provides two methods for configuring policies: + +- The :kesconf:`policy` section of the KES + :ref:`configuration file ` lists the persistent + policies for the KES server. + +- The :mc:`kes policy` command supports creating *ephemeral* policies for the + KES server. The :mc:`kes identity` command supports *ephemeral* modification + of the identities associated to policies on the KES server. + + Policies created or modified using either :mc:`kes policy` or + :mc:`kes identity` disappear after restarting the KES server. + +.. important:: + + Each KES server has its own configuration file from which it derives all + persistent policies. With distributed KES deployments, each server has its + own independent and distinct policy-identity set based on its configuration + file. This may allow for an identity to associate to different policies + depending on which KES server a client connects to. + + Exercise caution in allowing KES servers to contain differing policies, + as it may result in inconsistent client encryption behavior between + servers. MinIO strongly recommends synchronizing changes to configuration + files in distributed KES deployments. + + + +.. _minio-kes-root: + +KES Root Identity +~~~~~~~~~~~~~~~~~ + +The KES ``root`` identity has super-administrator access to all +:ref:`minio-kes-endpoints` and can perform any action on any resource on the KES +server. + +The KES server :mc:`kes` requires specifying a ``root`` identity on startup +via either the :mc-cmd-option:`kes server root` commandline option *or* +the :kesconf:`root` server configuration field. + +KES computes a SHA-256 hash of an authenticated client's x.509 certificate +to determine which policies to assign to the client. For the ``root`` +identity, the client's x.509 certificate *must* match that specified to the +:mc:`kes` server. + +To effectively disable the ``root`` account, specify a value for which the +SHA-256 hash of an x.509 certificate could *never* match. For example: + +.. tabs:: + + .. tab:: Command Line + + - ``kes server --root=_`` + + - ``kes server --root=disabled`` + + .. tab:: Configuration File + + - ``root: _`` + + - ``root: disabled`` + + +Exercise caution when storing or transmitting the ``root`` x.509 certificate and +private key, as any client with access to these credentials can perform +super-administrator actions on the KES server. + +.. _minio-kes-endpoints: + +KES API Endpoints +----------------- + +The following section lists the available KES API endpoints as a quick +reference. For more complete documentation on syntax and usage for each +endpoint, see the :minio-git:`KES Wiki `. + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + :width: 100% + + * - Endpoint + - Description + + * - ``/version`` + - Returns the version of the KES server. + + * - ``/v1/key/create`` + - Creates a cryptographic key on the KES server. + + To restrict access to a specific key prefix, specify that prefix as + an argument to the API. For example, the following endpoint pattern + allows creating keys with the prefix ``myapp``: + + .. code-block:: shell + + /v1/key/create/myapp-* + + * - ``/v1/key/import`` + - Imports a cryptographic key into the KES server. + + * - ``/v1/key/delete`` + - Deletes a cryptographic key on the KES server. + + Deleting a cryptographic key prevents decrypting any data encrypted + with that key, rendering that data permanently unreadable. Consider + restricting access to this endpoint to only those clients which require + it. + + * - ``/v1/key/generate`` + - Generates a Data Encryption Key (DEK) on the KES server. + Client's can use the DEK for performing Server-Side Object Encryption. + + * - ``/v1/key/encrypt`` + - Encrypts a plaintext value using a Data Encryption Key. + + * - ``/v1/key/decrypt`` + - Decrypts a ciphertext value using a Data Encryption Key. + + * - ``/v1/policy/write`` + - Adds a new :ref:`policy ` to the KES server. + + * - ``/v1/policy/read`` + - Retrieves an existing :ref:`policy ` from the KES + server. + + * - ``/v1/policy/list`` + - Lists all :ref:`policies ` on the KES server. + + * - ``/v1/policy/delete`` + - Deletes a :ref:`policy ` from the KES server. + + * - ``/v1/identity/assign`` + - Assigns an x.509 identity to a :ref:`policy ` on the + KES server. + + * - ``/v1/identity/list`` + - Lists all x.509 identities associated to a + :ref:`policy ` on the KES server. + + * - ``/v1/identity/forget`` + - Remove an x.509 identity associated to a :ref:`policy ` + on the KES server. + + * - ``/v1/log/audit/trace`` + - Returns a stream of audit log events produced by the KES server. + + * - ``/v1/log/log/error/trace`` + - Returns a stream of error events produced by the KES server. + +.. _minio-kes-supported-kms: + +Supported Key Management Systems +-------------------------------- + +MinIO Key Encryption Service (KES) stores any Secret keys it generates on +an external Key Management System (KMS). + +The following table lists all supported KMS: + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + :width: 100% + + * - KMS + - Description + + * - Filesystem + - Stores Secret keys directly on the filesystem. + + Use Filesystem KMS only for local development or initial evaluation + of KES. *Never* use Filesystem KMS for production. + .. _minio-kes-config: KES Configuration File @@ -59,7 +336,7 @@ General Configuration .. kesconf:: root - Specify ``disabled`` to disable the :ref:`root ` identity + Specify ``disabled`` to disable the :ref:`root ` identity on the KES server. The ``root`` identity has super administrator access to the KES server. @@ -241,8 +518,8 @@ Policy Configuration ~~~~~~~~~~~~~~~~~~~~ The following section describes policy-related configuration for the KES server. -Policies control :ref:`Identity and Access Management ` for -the KES server. +Policies are a component of :ref:`KES access control +`. Policies listed in this section are *persistent* through KES server restarts. Policies created using :mc:`kes policy` are *ephemeral* and are removed after @@ -278,21 +555,23 @@ KES configuration file: ///[//] - You can specify an asterisk ``*`` to create a catch-all pattern for - a given endpoint. For example, the following endpoint pattern - allows complete access to key creation via the ``/v1/key/create`` - endpoint: + For example, the following endpoint pattern allows complete access to key + creation via the ``/v1/key/create`` endpoint: .. kesconf:: policy.policyname.identities - An array of x.509 identities associated to the policy. KES grants clients - authenticating with a matching x.509 certificate access to the - endpoints listed in the :kespolicy:`~policyName.paths` for the - policy. + An array of identities associated to the policy. An :ref:`identity + ` consists of the SHA-256 hash of an x.509 + certificate. Use :mc-cmd:`kes tool identity of` to compute the identity of + each x.509 certificate you want to associate to the policy and specify that + value to the array. - Use :mc-cmd:`kes tool identity of` to compute the name of each x.509 - certificate you want to associate to the policy and specify that value - to the array. + KES grants clients authenticating with an x.509 certificate identity in the + array access to the KES API endpoints listed in the + :kesconf:`~policy.policyname.paths`. + + A given unique identity can associate to no more than *one* policy on the + KES server. Filesystem KMS Configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -412,6 +691,65 @@ Hashicorp Vault KMS, see . sign the Vault TLS certificates. +Thales CipherTrust / Gemalto KeySecure Configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following section describes Thales CipherTrust (formerly Gemalto +KeySecure)-related configuration for the KES server. + +For more complete documentation on configuring KES for connecting to a +CipherTrust KMS, see :doc:`/security/encryption/sse-s3-thales`. + +.. code-block:: yaml + + keys: + gemalto: + keysecure: + endpoint: : https://[IP|HOSTNAME] + credentials: + token: "" + domain: "" + retry: "" + tls: + ca: + +.. kesconf:: keys.gemalto.keysecure + + The root field for Thales CipherTrust / Gemalto KeySecure configuration. + +.. kesconf:: keys.gemalto.keysecure.endpoint + + *Required* + + The URL endpoint of the CipherTrust/KeySecure server. + +.. kesconf:: keys.gemalto.keysecure.credentials.token + + *Required* + + A refresh token generated by the CipherTrust/KeySecure server. + KES uses refresh tokens for generating short-lived authentication + tokens. + +.. kesconf:: keys.gemalto.keysecure.credentials.domain + + The domain of the CipherTrust/KeySecure server. Omit to default to the + root domain. + +.. kesconf:: keys.gemalto.keysecure.credentials.retry + + The duration KES waits before attempting to reconnect to the + CipherTrust/KeySecure server. Specify a string as ``##h##m##s``. + +.. kesconf:: keys.gemalto.keysecure.tls.ca + + The Certificate Authority which issued the CipherTrust/KeySecure TLS + certificates. + + If you cannot provide the CipherTrust/KeySecure CA, start the + KES server with :mc-cmd:`--auth off ` to disable + x.509 certificate validation. MinIO strongly recommends *against* + starting KES without certificate validation in production environments. .. toctree:: :titlesonly: diff --git a/source/minio-server/minio-server.rst b/source/minio-server/minio-server.rst index ae6c27da..e50d217c 100644 --- a/source/minio-server/minio-server.rst +++ b/source/minio-server/minio-server.rst @@ -335,28 +335,35 @@ MinIO :ref:`Key Encryption Service `: .. envvar:: MINIO_KMS_KES_KEY_FILE - The key file of the x.509 identity to use when authenticating to the - KES server. The specified identity *must* have KES policies attached - such that it can access *at minimum* the following API endpoints: + The private key associated to the the :envvar:`MINIO_KMS_KES_CERT_FILE` + x.509 certificate to use when authenticating to the KES server. + The KES server requires clients to present both their certificate and + private key for performing mutual TLS (mTLS). - - - See :ref:`minio-kes-iam` for more information on KES Identity and Access - Management. + For more complete documentation on KES authentication and authorization, see + :ref:`minio-kes-access-control`. .. envvar:: MINIO_KMS_KES_CERT_FILE - The public certificate of the x.509 identity to use when authenticating - to the KES server. The specified identity *must* have KES policies - attached such that it can access *at minimum* the following API endpoints: + The x.509 certificate to present to the KES server. The KES server computes + an :ref:`identity ` from the certificate and + compares it to the configured :ref:`policies ` on the + server. The KES server grants the :mc:`minio` server access to only those + operations explicitly granted by the policy. + + Ensure the KES server has a policy that contains the identity of the + specified certificate, where the policy grants access to *at minimum* the + following endpoints: - See :ref:`minio-kes-iam` for more information on KES Identity and Access - Management. + For more complete documentation on KES authentication and authorization, see + :ref:`minio-kes-access-control`. .. envvar:: MINIO_KMS_KES_KEY_NAME - Sets the name of the Customer Master Key (CMK) to use for performing - Server-Side Encryption on the MinIO deployment. + The name of an external key (EK) to retrieve from the Key Management System + (KMS) configured on KES. :mc:`minio` uses the + EK to generate additional cryptographic keys for performing + Server Side Encryption of objects (SSE-S3). diff --git a/source/security/encryption/minio-kes.rst b/source/security/encryption/minio-kes.rst index 4991a05a..0dae00a9 100644 --- a/source/security/encryption/minio-kes.rst +++ b/source/security/encryption/minio-kes.rst @@ -27,12 +27,9 @@ KES is designed for simplicity, scalability, and security. It requires minimal configuration to enable full functionality and requires only basic familiarity with cryptography or key-management concepts. -MinIO servers require KES for performing Server-Side Encryption (SSE) of objects -using Key Management Services (KMS). Applications *may* use KES to support -cryptographic key operations independent of MinIO. - -This page documents conceptual information about MinIO KES. For documentation -on the :mc:`kes` command line tool, see the +This page contains information on deploying and configuring KES for supporting +Server-Side Encryption with MinIO. For more complete documentation on the +:mc:`kes` command line tool and the KES :mc:`~kes server` process, see the :doc:`KES reference page `. .. _minio-kes-getting-started: @@ -190,166 +187,5 @@ KES Quick Reference The following table lists the high-level commands of the :mc:`kes` command line tool: -.. _minio-kes-iam: +TODO -KES Identity and Access Management ----------------------------------- - -The KES server configuration includes one or more -:ref:`policies ` that associate x.509 identities -to the API endpoints which those identities can access. Client's *must* -present an x.509 certificate when connecting to the KES server. - -KES Identity -~~~~~~~~~~~~ - -This section to contain information on creating x.509 certificates for use -with KES. - -.. _minio-kes-policy: - -KES Policies -~~~~~~~~~~~~ - -The KES server provides two methods for configuring policies: - -- The :kesconf:`policy` section of the KES - :ref:`configuration file ` lists the persistent - policies for the KES server. - -- The :mc:`kes policy` command supports creating *ephemeral* policies for the - KES server. The :mc:`kes identity` command supports *ephemeral* modification - of the identities associated to policies on the KES server. - - Policies created or modified using either :mc:`kes policy` or - :mc:`kes identity` disappear after restarting the KES server. - -The following ``YAML`` document provides an example of the :kesconf:`policy` -section of the KES server configuration document. The policy ``minio-sse`` -includes the appropriate :ref:`API endpoints ` for -supporting MinIO Server-Side Encryption: - -.. code-block:: yaml - :class: copyable - - policy: - minio-sse: - paths: - - /v1/key/create/* - - /v1/key/generate/* - - /v1/key/decrypt/* - - /v1/key/delete/* - identities: - - HASH - -Each element in the :kesconf:`policy.policyname.paths` array represents an -:ref:`API endpoint ` to which the policy grants access. - -Each element in the :kesconf:`policy.policyname.identities` array represents -the hash of an x.509 certificate generated using the -:mc:`kes tool identity of` command line operation. - -.. _minio-kes-iam-root: - -KES Root Identity -~~~~~~~~~~~~~~~~~ - -The KES ``root`` identity has super-administrator access to all -:ref:`minio-kes-endpoints` and can perform any action on any resource on the KES -server. - -All KES identities consist of an x.509 certificate pair (private ``*.key`` and -public ``*.cert``). Exercise caution when storing or transmitting the ``root`` -x.509 certificates, as any client with access to these certificates can -perform super-administrator actions on the KES server. - -The KES server disables the ``root`` identity by default. To create the -``root`` identity, either: - -- Start the KES server with the :mc-cmd-option:`~kes server root` option, *or* - -- Specify :kesconf:`root : disabled ` in the KES server configuration - document. - -.. _minio-kes-endpoints: - -KES API Endpoints ------------------ - -The following section lists the available KES API endpoints as a quick -reference. For more complete documentation on syntax and usage for each -endpoint, see the :minio-git:`KES Wiki `. - -.. list-table:: - :header-rows: 1 - :widths: 40 60 - :width: 100% - - * - Endpoint - - Description - - * - ``/version`` - - Returns the version of the KES server. - - * - ``/v1/key/create`` - - Creates a cryptographic key on the KES server. - - To restrict access to a specific key prefix, specify that prefix as - an argument to the API. For example, the following endpoint pattern - allows creating keys with the prefix ``myapp``: - - .. code-block:: shell - - /v1/key/create/myapp-* - - * - ``/v1/key/import`` - - Imports a cryptographic key into the KES server. - - * - ``/v1/key/delete`` - - Deletes a cryptographic key on the KES server. - - Deleting a cryptographic key prevents decrypting any data encrypted - with that key, rendering that data permanently unreadable. Consider - restricting access to this endpoint to only those clients which require - it. - - * - ``/v1/key/generate`` - - Generates a Data Encryption Key (DEK) on the KES server. - Client's can use the DEK for performing Server-Side Object Encryption. - - * - ``/v1/key/encrypt`` - - Encrypts a plaintext value using a Data Encryption Key. - - * - ``/v1/key/decrypt`` - - Decrypts a ciphertext value using a Data Encryption Key. - - * - ``/v1/policy/write`` - - Adds a new :ref:`policy ` to the KES server. - - * - ``/v1/policy/read`` - - Retrieves an existing :ref:`policy ` from the KES - server. - - * - ``/v1/policy/list`` - - Lists all :ref:`policies ` on the KES server. - - * - ``/v1/policy/delete`` - - Deletes a :ref:`policy ` from the KES server. - - * - ``/v1/identity/assign`` - - Assigns an x.509 identity to a :ref:`policy ` on the - KES server. - - * - ``/v1/identity/list`` - - Lists all x.509 identities associated to a - :ref:`policy ` on the KES server. - - * - ``/v1/identity/forget`` - - Remove an x.509 identity associated to a :ref:`policy ` - on the KES server. - - * - ``/v1/log/audit/trace`` - - Returns a stream of audit log events produced by the KES server. - - * - ``/v1/log/log/error/trace`` - - Returns a stream of error events produced by the KES server. \ No newline at end of file diff --git a/source/security/encryption/sse-s3-thales.rst b/source/security/encryption/sse-s3-thales.rst index 02312da9..5d495992 100644 --- a/source/security/encryption/sse-s3-thales.rst +++ b/source/security/encryption/sse-s3-thales.rst @@ -11,47 +11,559 @@ Server-Side Encryption with Thales CipherTrust Overview -------- -Paragraph summarizing SSE-S3 and Thales CipherTrust as a KMS. +The MinIO Key Encryption Service (KES) supports using `Thales CipherTrust +`__ (formerly +Gemalto KeySecure) as an external Key Management Service (KMS). MinIO can +therefore use CipherTrust for storing Secret keys associated with +Server-Side Encryption (SSE-S3). -Note that Gemalto KeySecure is now Thales CipherTrust. +This procedure documents + +- Configuring Thales CipherTrust for KES connectivity. + +- Configuring KES for Thales CipherTrust KMS. + +- Configuring MinIO for SSE-S3. + + + +This procedure requires familiarity with Thales CipherTrust deployment and +configuration. Any references made to CipherTrust configuration is made on +a best-effort basis and is not intended as a replacement for the official +CipherTrust documentation or best practices. Prerequisites ------------- -Thales CipherTrust Deployment -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Thales CipherTrust / Gemalto KeySecure +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -High-Level description of CipherTrust requirements: +This procedure assumes a running CipherTrust Manager or KeySecure instance +with network access to the KES server. The procedure has been tested against the +following versions: -- What access will the user need? -- What versions do we support? +- CipherTrust Manager ``k170v`` version ``2.0.0`` +- Gemalto KeySecure ``k170v`` version ``1.9.1`` and version ``1.10.0`` + +This procedure also requires the ``ksctl`` command line tool configured +for access to the CipherTrust Manager or Gemalto KeySecure instance. +Ensure the ``ksctl`` ``config.yaml`` contains at minimum the following +fields: + +.. code-block:: yaml + :class: copyable + + KSCTL_URL: + KSCTL_USERNAME: + KSCTL_PASSWORD: + KSCTL_VERBOSITY: false + KSCTL_RESP: json + KSCTL_NOSSLVERIFY: true + KSCTL_TIMEOUT: 30 + +The specified ``KSCTL_USERNAME`` must correspond to a user on the +CipherTrust/KeySecure deployment with administrative privileges to +perform the following commands: + +- Group creation (``ksctl groups create``) +- User creation (``ksctl users create``) +- Group management (``ksctl groups adduser``) +- Policy management (``ksctl policy create`` and ``ksctl polattach create``) +- Token management (``ksctl tokens create``) + +For CipherTrust/KeySecure deployments configured using a TLS x.509 certificate +issued by a trusted Certificate Authority, you can set +``KSCTL_NOSSLVERIFY: false``. The KES server includes a configuration +setting :kesconf:`keys.gemalto.keysecure.tls.ca` for specifying the CA +used by the CipherTrust/KeySecure. MinIO Key Encryption Service ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -High-level description of KES requirements: +The KES server *must* have network access to: -- A host for deploying at least one KES server -- For Kubernetes, at least one node with enough resources to run the server +- The CipherTrust/KeySecure deployment, *and* +- All :mc:`minio` servers in the MinIO deployment. + +KES uses mutual TLS for authentication and authorization and *requires* +specifying an x.509 certificate and corresponding private key. For production +environments, MinIO strongly recommends using a globally trusted Certificate +Authority for issuing the KES TLS certificate. + +If using an internally trusted Certificate Authority, consider adding the +CA to the system trust store of the :mc:`minio` server hosts and the +CipherTrust/KeySecure deployment to allow for complete validation of the +KES certificates. + +For self-signed certificates *or* certificates issued by a CA which cannot be +added to other hosts, you may need to disable TLS validation on the +:mc:`minio` and CipherTrust/KeySecure deployments to allow for +successful connectivity. MinIO Server ~~~~~~~~~~~~ -High-level description of MinIO server requirements: +The MinIO server *must* have network access to the KES server. -- ? +KES uses mutual TLS (mTLS) for authentication and authorization and *requires* +each :mc:`minio` server present an x.509 certificate and corresponding +private key. Specifically, each :mc:`minio` server *must* enable +:ref:`TLS connectivity `. MinIO strongly recommends using a +globally trusted Certificate Authority for issuing the MinIO TLS certificate. + +If using an internally trusted Certificate Authority, consider adding the +CA to the system trust store of KES host to allow for complete validation of the +MinIO certificates. + +For self-signed certificates *or* certificates issued by a CA which cannot be +added to other hosts, you may need to disable TLS validation on the KES +deployment to allow for successful connectivity. Procedure --------- -1) Configure CipherTrust Manager for MinIO Access +1) Configure CipherTrust/KeySecure for KES Access ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Substeps: +This section assumes a running CipherTrust/KeySecure deployment accessible using +the ``ksctl`` utility. -1. Foo +a. Create a new group for KES +````````````````````````````` -2. Bar +.. container:: indent -2) Configure KES... -~~~~~~~~~~~~~~~~~~~ + Issue the following ``ksctl`` command to create a new group for KES: + + .. code-block:: shell + :class: copyable + + ksctl groups create --name KES-Service + +b. Create a new user in the KES-Service group +````````````````````````````````````````````` + +.. container:: indent + + *Optional:* + + Issue the following ``ksctl`` command to create a new user for KES: + + .. code-block:: shell + :class: copyable + + ksctl users create --name KESServiceUser --pword '' + + Replace the ```` with a unique, long, and random string. Defer + to industry best practices and/or your organization's specific password + generation requirements. + + The command returns a JSON object containing a field ``user_id``. + The next step requires specifying this ID. + + You can skip this step if you already have an existing user + which you want to assign to the ``KES-Service`` group. Use + + ``ksctl users lists`` + + to list users and their corresponding ID: + + ```local|8791ce13-2766-4948-a828-71bac67131c9`` + +c. Assign the user to the KES-Service group +``````````````````````````````````````````` + +.. container:: indent + + Issue the following ``ksctl`` command to assign the ``KESServiceUser`` to the + ``KES-Service`` group: + + .. code-block:: shell + :class: copyable + + ksctl groups adduser --name KES-Service --userid "" + + Replace ```` with the ID of the user created in the previous step. + If specifying an existing user, replace ```` with the ID of that + user. + +d. Create a policy for the KES-Service group +```````````````````````````````````````````` + +.. container:: indent + + Create a ``kes-policy.json`` document with the following structure: + + .. code-block:: json + :class: copyable + + { + "allow": true, + "name": "kes-policy", + "actions": [ + "CreateKey", + "ExportKey", + "ReadKey", + "DeleteKey" + ], + "resources": [ + "kylo:kylo:vault:secrets:*" + ] + } + + Issue the following ``ksctl`` command to create a new policy using the + ``kes-policy.json`` document: + + .. code-block:: shell + :class: copyable + + ksctl policy create --jsonfile kes-policy.json + + For more complete documentation on CipherTrust/KeySecure policies, + see `CipherTrust Manager Policies + `__ + +e. Attach the policy to the KES-Service group +````````````````````````````````````````````` + +.. container:: indent + + Create a ``kes-attachment.json`` policy with the following structure: + + .. code-block:: json + :class: copyable + + { + "cust" : { + "groups" : ["KES-Service"] + } + } + + Issue the following ``ksctl`` command to attach the ``kes-policy`` policy to + the ``KES-Service`` group. + + .. code-block:: shell + :class: copyable + + ksctl polattach create -p kes-policy -g kes-attachment.json + +f. Create a refresh token for KES +````````````````````````````````` + +.. container:: indent + + Issue the following ``ksctl`` command to create a refresh token for KES + to obtain short-lived authentication tokens: + + .. code-block:: shell + :class: copyable + + ksctl tokens create \ + --user KESServiceUser --password '' \ + --issue-rt | jq -r .refresh_token + + If you skipped step + :guilabel:`b. Create a new user in the KES-Service group`, replace + ``KESServiceUser`` with the name of the user specified to step + :guilabel:`c. Assign the user to the KES-Service group`. + + The command outputs a refresh token similar to the following: + + .. code-block:: shell + + CEvk5cdHLG7si05LReIeDbXE3PKD082YdUFAnxX75md3jzV0BnyHyAmPPJiA0 + + You will need this token when configuring KES. + +2) Configure KES for CipherTrust/KeySecure +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +a. Prepare the TLS x.509 Certificates for KES +````````````````````````````````````````````` + +.. container:: indent + + KES requires mutual TLS (mTLS) for performing authentication and + authorization. Specifically, you must provide an x.509 Certificate and + corresponding private key. + + - For local development and evaluation, you can use a self-signed + certificate. See :ref:`minio-kes-getting-started` for + instructions on creating self-signed certificates for KES. + + - For production environments, use a trusted Certificate Authority (CA) to + generate the x.509 certificate. + + If using an internal CA, ensure the CipherTrust/KeySecure host includes + that CA in the system trust store. + +b. Create the root identity for KES +``````````````````````````````````` + +.. container:: indent + + KES requires specifying a :ref:`root ` identity on startup. + + - For local development and evaluation, you can use the + :mc-cmd:`kes tool identity new` command to create a self-signed + x.509 certificate and private key: + + .. code-block:: shell + :class: copyable + + kes tool identity new --key=root.key --cert=root.cert root + + - For production environments, use a trusted Certificate Authority (CA) to + generate the x.509 certificate. If using an internal CA, ensure the + KES host includes that CA in the system trust store. + + Use the :mc:`kes tool identity of` command to compute the + :ref:`identity ` of the certificate. + + .. code-block:: shell + + kes tool identity of root.cert + + The command outputs a SHA-256 hash. You must specify this value in a + later step. + +c. Create the identity for the KES client +````````````````````````````````````````` + +.. container:: indent + + The :mc:`kes` client supports interfacing with and performing operations + on a KES :mc:`~kes server`. The KES client *must* provide an + x.509 certificate and corresponding private key to connect to the KES + server. + + - For local development and evaluation, you can use the + :mc-cmd:`kes tool identity new` command to create a self-signed + x.509 certificate and private key: + + .. code-block:: shell + :class: copyable + + kes tool identity new --key=kes-client.key --cert=kes-client.cert kes-client + + - For production environments, use a trusted Certificate Authority (CA) to + generate the x.509 certificate. + + If using an internal CA, ensure the KES host includes + that CA in the system trust store. + + Use the :mc:`kes tool identity of` command to compute the + :ref:`identity ` of the certificate. + + .. code-block:: shell + + kes tool identity of kes-client.cert + + The command outputs a SHA-256 hash. You must specify this value in a + later step. + +d. Create an identity for each MinIO server +``````````````````````````````````````````` + +.. container:: indent + + Each :mc:`minio` server which communicates with KES *must* provide an + x.509 certificate and corresponding private key to connect to the KES + server. + + - For local development and evaluation, you can use the + :mc-cmd:`kes tool identity new` command to create a self-signed + x.509 certificate and private key: + + .. code-block:: shell + :class: copyable + + kes tool identity new --key=minio-server-1.key --cert=minio-server-1.cert minio-server-1 + + - For production environments, use a trusted Certificate Authority (CA) to + generate the x.509 certificate. + + If using an internal CA, ensure the KES host includes + that CA in the system trust store. + + Use the :mc:`kes tool identity of` command to compute the + :ref:`identity ` of the certificate. + + .. code-block:: shell + + kes tool identity of minio-server-1.cert + + The command outputs a SHA-256 hash. You must specify this value in a + later step. + +d. Create the KES server configuration +`````````````````````````````````````` + +.. container:: indent + + The KES :mc:`~kes server` process uses a + :ref:`configuration file ` during startup. + The following example configuration file includes the minimum required + fields for starting KES with CipherTrust/KeySecure. You can save this + file as ``kes-config.yaml``. You *must* modify this file according to your + deployment prior to using it to start KES. + + .. code-block:: yaml + :class: copyable + + root: "" + + tls: + key: server.key + cert: server.cert + + policy: + my-app: + paths: + - /v1/key/create/my-app* + - /v1/key/generate/my-app* + - /v1/key/generate/my-app* + identities: + - "" + - "" + + keys: + gemalto: + keysecure: + endpoint: "https://ciphertrust.example.net" + credentials: + token: "" + domain: "" + retry: 15s + tls: + ca: "ciphertrust-ca.cert" + + - Replace the ```` with the hash of the certificate + generated or selected as part of + :guilabel:`2.b. Create the root identity for KES`. + + You *may* disable root access by specifying ``"_"`` or + ``"disabled"`` for the root user. *However*, you must then include + a :kesconf:`policy` which grants administrative access to specific + client x.509 :ref:`identities `. + + - Replace ```` with the hash of the certificate + generated or selected as part of + :guilabel:`2.c. Create the identity for the KES client`. + + - Replace ```` with the hash of the certificate + generated or selected as part of + :guilabel:`2.d. Create an identity for each MinIO server`. + + For distributed clusters, you may need to specify multiple identities. + + - Replace ```` with the refresh token generated + in :guilabel:`1.f. Create a refresh token for KES`. + + - Replace ```` with the domain of the CipherTrust/KeySecure + deployment. Omit to default to "root". + + - Replace ``ciphertrust-ca.cert`` with the Certificate Authority (CA) used + to sign the CipherTrust/KeySecure TLS certificates. Optional if the + CipherTrust/KeySecure certificate was signed by a globally trusted + CA. + +d. Start the KES server +``````````````````````` + +.. container:: indent + + Start the KES :mc:`~kes server` process using the configuration file: + + .. code-block:: shell + :class: copyable + + kes server --config=kes-config.yaml + + If using self-signed certificates for the :kesconf:`root` identity + *or* for any application :kesconf:`identity `, + you *must* specify :mc-cmd-option:`--auth=off ` to + disable TLS certificate validation. + +e. Connect to the KES server and generate a Secret key +`````````````````````````````````````````````````````` + +.. container:: indent + + Set the following environment variables on a host machine with + access to the :mc:`kes server` process: + + .. code-block:: shell + :class: copyable + + export KES_CLIENT_CERT= + export KES_CLIENT_KEY=kes-client.key + + Replace ```` with the hash of the certificate + generated or selected as part of + :guilabel:`2.c. Create the identity for the KES client`. + + Issue the following command to create a new Secret key for use with + enabling Server Side Encryption. KES stores the Secret key on the + CipherTrust/KeySecure server. + + .. code-block:: shell + :class: copyable + + kes key create my-app-sse-key + + If using self-signed certificates, you must include the + :mc-cmd-option:`~kes key create insecure` option. + +3. Configure the MinIO Server +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +a. Set the Environment Variables +```````````````````````````````` + +.. container:: indent + + Set the following environment variables on each :mc:`minio` server host: + + .. code-block:: shell + + export MINIO_KMS_KES_ENDPOINT="https://kes.example.net:7373" + export MINIO_KMS_KES_KEY_FILE="minio-server-1.key" + export MINIO_KMS_KES_CERT_FILE="minio-server-1.cert" + export MINIO_KMS_KES_KEY_NAME="my-app-sse-key" + + - Set :envvar:`MINIO_KMS_KES_ENDPOINT` to the HTTP endpoint of the KES + server. + + - Set :envvar:`MINIO_KMS_KES_CERT_FILE` to the x.509 certificate + generated or selected as part of + :guilabel:`2.d. Create an identity for each MinIO server`. + + - Set :envvar:`MINIO_KMS_KES_KEY_FILE` to the private key + corresponding to the specified :envvar:`MINIO_KMS_KES_CERT_FILE`. + + - Set :envvar:`MINIO_KMS_KES_KEY_NAME` to the name of the Secret key created + in :guilabel:`2.e. Connect to the KES server and generate a Secret key`. + +b. Start or Restart the MinIO server +```````````````````````````````````` + +.. container:: indent + + Restart each :mc:`minio` server in the deployment to load the + KES environment variables. + + To test Server-Side Encryption, create an object on the MinIO deployment + using one of the following methods: + + - Use an :mc:`mc` command which supports SSE-S3 encryption: + + .. code-block:: shell + :class: copyable + + mc cp --encrypt ~/Downloads/data.csv myminio/data/data.csv + + - Issue a request including the ``x-amz-server-side-encryption`` + header. + + - more detail