From 21797d127b3f136bf273288a1e303873b1023045 Mon Sep 17 00:00:00 2001 From: Ravind Kumar Date: Tue, 18 Oct 2022 13:15:05 -0400 Subject: [PATCH] DOCS-610: Update KES procedures for KES 0.21.0, minor cleanups for Vault (#612) Closes #610 Did a QA pass after reports of some issues with the vault setup. Identified an issue where my local testing was done with 0.20.0, resulting in some errors on 0.21.0 due to dropping --mlock. Also did a few other general tune ups as I ran end-to-ends again using Vault + Systemd instead of vault in dev mode. Staging views in #minio-docs channel --- source/extra/kes.service | 2 +- .../common/common-minio-kes-hashicorp.rst | 24 +++- source/includes/common/common-minio-kes.rst | 24 ++-- .../includes/container/common-minio-kes.rst | 12 +- source/includes/k8s/common-minio-kes.rst | 6 + source/includes/linux/common-minio-kes.rst | 82 ++++------- .../linux/steps-configure-minio-kes-aws.rst | 4 +- .../linux/steps-configure-minio-kes-azure.rst | 4 +- .../linux/steps-configure-minio-kes-gcp.rst | 4 +- ...ps-configure-minio-kes-hashicorp-quick.rst | 122 ++++++++++------- .../steps-configure-minio-kes-hashicorp.rst | 128 ++++++++++++------ source/includes/windows/common-minio-kes.rst | 2 - .../configure-minio-kes-hashicorp.rst | 121 ++++++++++------- 13 files changed, 318 insertions(+), 217 deletions(-) diff --git a/source/extra/kes.service b/source/extra/kes.service index 5314d298..e9c54e05 100644 --- a/source/extra/kes.service +++ b/source/extra/kes.service @@ -14,7 +14,7 @@ User=kes Group=kes ProtectProc=invisible -ExecStart=/usr/local/bin/kes server --config=/opt/kes/config.yaml --mlock +ExecStart=/usr/local/bin/kes server --config=/opt/kes/config.yaml # Let systemd restart this service always Restart=always diff --git a/source/includes/common/common-minio-kes-hashicorp.rst b/source/includes/common/common-minio-kes-hashicorp.rst index 9c8c0f43..ed4e8713 100644 --- a/source/includes/common/common-minio-kes-hashicorp.rst +++ b/source/includes/common/common-minio-kes-hashicorp.rst @@ -1,7 +1,8 @@ .. start-kes-configuration-hashicorp-vault-desc |KES| uses a YAML-formatted configuration file. -The following YAML provides the minimum required fields for using Hashicorp Vault as the root |KMS| and is intended for use in this tutorial. +The following YAML provides the minimum required fields for using Hashicorp Vault as the root |KMS|. +You must modify this YAML to reflect your deployment environment. .. code-block:: shell :class: copyable @@ -30,7 +31,7 @@ The following YAML provides the minimum required fields for using Hashicorp Vaul - /v1/key/generate/* # e.g. '/minio-' - /v1/key/decrypt/* identities: - - ${MINIO_IDENTITY_HASH} # Replace with the output of 'kes identity of minio-kes.cert' + - MINIO_IDENTITY_HASH # Replace with the output of 'kes identity of minio-kes.cert' # In production environments, each client connecting to KES must # Have their TLS hash listed under at least one `policy`. @@ -64,5 +65,24 @@ This procedure assumes an existing `Hashicorp Vault `__ provides a sufficient foundation for the purposes of this procedure. Defer to the `Vault Documentation `__ for guidance on deployment and configuration. +.. admonition:: KMS Key Creation Requires Unsealed Vault + :class: important + + KES requires unsealing the Vault instance for creating new External Keys (EK) for supporting SSE operations. + A sealed Vault blocks key creation operations and results in errors on both KES and the KES client (MinIO). + + You can safely seal the Vault after KES completes any key creation operations. + See the Vault documentation on `Seal/Unseal `__ for more information. + .. end-kes-prereq-hashicorp-vault-desc +.. start-kes-vault-seal-unseal-desc + +.. admonition:: KMS Key Creation Requires Unsealed Vault + :class: important + + You must unseal the backing Vault instance to allow KES to create new External Keys (EK) for supporting SSE operations. + This step requires an unsealed Vault to complete successfully. + See the Vault documentation on `Seal/Unseal `__ for more information. + +.. end-kes-vault-seal-unseal-desc \ No newline at end of file diff --git a/source/includes/common/common-minio-kes.rst b/source/includes/common/common-minio-kes.rst index 0339bdf8..5fa83890 100644 --- a/source/includes/common/common-minio-kes.rst +++ b/source/includes/common/common-minio-kes.rst @@ -48,13 +48,13 @@ The following commands create two TLS certificates that expire within 30 days of # These commands output keys to |kescertpath| # and |miniocertpath| respectively - kes identity new \ + kes identity new kes_server \ --key |kescertpath|/kes-server.key \ --cert |kescertpath|/kes-server.cert \ --ip "127.0.0.1" \ --dns localhost - kes identity new \ + kes identity new minio_server \ --key |miniocertpath|/minio-kes.key \ --cert |miniocertpath|/minio-kes.cert \ --ip "127.0.0.1" \ @@ -89,12 +89,13 @@ Run the following commands in a terminal or shell to start the KES server as a f sudo setcap cap_ipc_lock=+ep $(readlink -f $(which kes)) - kes server --mlock \ - --config=|kesconfigpath|kes-config.yaml \ - --auth=off + kes server --auth=off --config=|kesconfigpath|/kes-config.yaml + The first command allows |KES| to use the `mlock `__ system call without running as root. ``mlock`` ensures the OS does not write in-memory data to disk (swap memory) and mitigates the risk of cryptographic operations being written to unsecured disk at any time. +KES 0.21.0 and later automatically detect and enable ``mlock`` if supported by the host OS. +Versions 0.20.0 and earlier required specifying the ``--mlock`` argument to KES. The second command starts the KES server in the foreground using the configuration file created in the last step. The ``--auth=off`` disables strict validation of client TLS certificates. @@ -131,8 +132,6 @@ The following command uses the ``kes key create`` command to add a new External Add the following lines to the MinIO Environment file on each MinIO host. See the tutorials for :ref:`minio-snsd`, :ref:`minio-snmd`, or :ref:`minio-mnmd` for more detailed descriptions of a base MinIO environment file. -This command assumes the ``minio-kes.cert``, ``minio-kes.key``, and ``kes-server.cert`` certificates are accessible at the specified location: - .. code-block:: shell :class: copyable :substitutions: @@ -142,16 +141,19 @@ This command assumes the ``minio-kes.cert``, ``minio-kes.key``, and ``kes-server MINIO_KMS_KES_ENDPOINT=https://HOSTNAME:7373 MINIO_KMS_KES_CERT_FILE=|miniocertpath|/minio-kes.cert MINIO_KMS_KES_KEY_FILE=|miniocertpath|/minio-kes.key - MINIO_KMS_KES_CAPATH=|miniocertpath|/kes-server.cert - MINIO_KMS_KES_KEY_NAME=minio-backend-default-key - minio server [ARGUMENTS] + # Allows validation of the KES Server Certificate (Self-Signed or Third-Party CA) + # Change this path to the location of the KES CA Path + MINIO_KMS_KES_CAPATH=|kescertpath|/kes-server.cert + + # Sets the default KMS key for the backend and SSE-KMS/SSE-S3 Operations) + MINIO_KMS_KES_KEY_NAME=minio-backend-default-key Replace ``HOSTNAME`` with the IP address or hostname of the KES server. If the MinIO server host machines cannot resolve or reach the specified ``HOSTNAME``, the deployment may return errors or fail to start. - If using a single KES server host, specify the IP or hostname of that host -- If using multiple KES server hosts, specify the load balancer or reverse proxy managing connections to those hosts. +- If using multiple KES server hosts, specify a comma-separated list of IPs or hostnames of each host MinIO uses the :envvar:`MINIO_KMS_KES_KEY_NAME` key for the following cryptographic operations: diff --git a/source/includes/container/common-minio-kes.rst b/source/includes/container/common-minio-kes.rst index 9d71a82a..bcc08a1c 100644 --- a/source/includes/container/common-minio-kes.rst +++ b/source/includes/container/common-minio-kes.rst @@ -25,7 +25,7 @@ The commands in this section create the following resources: -e KES_CLIENT_KEY=/certs/kes-server.key \ -e KES_CLIENT_CERT=/certs/kes-server.cert \ quay.io/minio/kes:|kes-stable| server \ - --mlock --auth \ + --auth \ --config=/etc/default/kes-config.yaml \ sudo podman run -dt \ @@ -73,14 +73,14 @@ The following commands create two TLS certificates that expire within 30 days of podman run --rm \ -v |kescertpath|:/certs \ - quay.io/minio/kes:|kes-stable| identity new \ + quay.io/minio/kes:|kes-stable| identity new kes_server \ --key /certs/kes-server.key \ --cert /certs/kes-server.cert \ kes-server podman run --rm \ -v |miniocertpath|:/certs \ - quay.io/minio/kes:|kes-stable| identity new \ + quay.io/minio/kes:|kes-stable| identity new minio_server \ --key /certs/minio-kes.key \ --cert /certs/minio-kes.cert \ minio-server @@ -123,6 +123,12 @@ KES automatically creates this key if it does not already exist on the root KMS. .. start-kes-generate-key-desc +.. admonition:: Unseal Vault Before Creating Key + :class: important + + You must unseal the backing Vault instance before creating new encryption keys. + See the Vault documentation on `Seal/Unseal `__ for more information. + MinIO requires that the |EK| exist on the root KMS *before* performing |SSE| operations using that key. Use ``kes key create`` *or* :mc-cmd:`mc admin kms key create` to create a new |EK| for use with |SSE|. diff --git a/source/includes/k8s/common-minio-kes.rst b/source/includes/k8s/common-minio-kes.rst index ce7d9c03..acdcb1e6 100644 --- a/source/includes/k8s/common-minio-kes.rst +++ b/source/includes/k8s/common-minio-kes.rst @@ -80,6 +80,12 @@ You can use either the MinIO Tenant Console or the MinIO :mc:`mc` CLI to enable .. start-kes-generate-key-desc +.. admonition:: Unseal Vault Before Creating Key + :class: important + + You must unseal the backing Vault instance before creating new encryption keys. + See the Vault documentation on `Seal/Unseal `__ for more information. + MinIO requires that the |EK| for a given bucket or object exist on the root KMS *before* performing |SSE| operations using that key. You can use the :mc-cmd:`mc admin kms key create` command against the MinIO Tenant. diff --git a/source/includes/linux/common-minio-kes.rst b/source/includes/linux/common-minio-kes.rst index a343ebc2..c3c8cafd 100644 --- a/source/includes/linux/common-minio-kes.rst +++ b/source/includes/linux/common-minio-kes.rst @@ -11,14 +11,15 @@ The following example code downloads the latest Linux AMD64-compatible binary an :class: copyable :substitutions: - wget https://github.com/minio/kes/releases/download/|kes-stable|/kes-linux-amd64 -O /tmp/kes + curl --retry 10 https://github.com/minio/kes/releases/download/|kes-stable|/kes-linux-amd64 -o /tmp/kes chmod +x /tmp/kes sudo mv /tmp/kes /usr/local/bin kes --version For distributed KES topologies, repeat this step and all following KES-specific instructions for each host on which you want to deploy KES. -MinIO strongly recommends configuring a load balancer with a "Least Connections" configuration to manage connections to distributed KES hosts. +MinIO uses a round-robin approach by default for routing connections to multiple configured KES servers. +For more granular controls, deploy a dedicated load balancer to manage connections to distributed KES hosts. .. end-kes-download-desc @@ -92,66 +93,39 @@ Depending on your Vault configuration, you may also need to create a dedicated s Defer to your organizations best practices around generating production-ready TLS certificates. Place the certificates and corresponding private keys an appropriate directory such that the MinIO and KES service users can access and read their contents. -This procedure assumes a structure similar to the following: +The following example structure uses the folder hierarchy suggested in the beginning of this procedure: - .. code-block:: shell - :substitutions: +.. tab-set:: - # For the MinIO Hosts - -rw-r--r-- 1 minio-user:minio-user |miniocertpath|/minio-kes.cert - -rw-r--r-- 1 minio-user:minio-user |miniocertpath|/minio-kes.key + .. tab-item:: KES Hosts - # If KES certs are self-signed or use a non-global CA - # Include the CA certs as well - -rw-r--r-- 1 minio-user:minio-user |miniocertpath|/kes-server.cert + .. code-block:: shell + :substitutions: - # For the KES Hosts - -rw-r--r-- 1 kes:kes |kescertpath|/kes-server.cert - -rw-r--r-- 1 kes:kes |kescertpath|/kes-server.key + -rw-r--r-- 1 kes:kes |kescertpath|/kes-server.cert + -rw-r--r-- 1 kes:kes |kescertpath|/kes-server.key -If the KES certificates are self-signed *or* signed by Certificate Authority (CA) that is *not* globally trusted, you **must** add the CA certificate to the |miniocertpath| directory such that each MinIO server can properly validate the KES certificates. + # If the Vault certs are self-signed or use a non-global CA + # Include those CA certs as well + + -rw-r--r-- 1 kes:kes |kescertpath|/vault-CA.cert + + .. tab-item:: MinIO Hosts + + .. code-block:: shell + :substitutions: + + -rw-r--r-- 1 minio-user:minio-user |miniocertpath|/minio-kes.cert + -rw-r--r-- 1 minio-user:minio-user |miniocertpath|/minio-kes.key + + # If KES certs are self-signed or use a non-global CA + # Include the CA certs as well + -rw-r--r-- 1 minio-user:minio-user |miniocertpath|/kes-server.cert + +The general strategy for cert management is to ensure that each process (MinIO, KES, and Vault) have their own mTLS certificates *and* the Certificate Authority (CA) used to sign each client certificate. .. end-kes-generate-kes-certs-prod-desc -.. start-kes-configuration-minio-desc - -Add the following lines to the MinIO Environment file on each MinIO host. -See the tutorials for :ref:`minio-snsd`, :ref:`minio-snmd`, or :ref:`minio-mnmd` for more detailed descriptions of a base MinIO environment file. - -This command assumes the ``minio-kes.cert``, ``minio-kes.key``, and ``kes-server.cert`` certificates are accessible at the specified location: - -.. code-block:: shell - :class: copyable - :substitutions: - - # Add these environment variables to the existing environment file - - MINIO_KMS_KES_ENDPOINT=https://HOSTNAME:7373 - MINIO_KMS_KES_CERT_FILE=|miniocertpath|/minio-kes.cert - MINIO_KMS_KES_KEY_FILE=|miniocertpath|/minio-kes.key - MINIO_KMS_KES_CAPATH=|kescertpath|/kes-server.cert - MINIO_KMS_KES_KEY_NAME=minio-backend-default-key - - minio server [ARGUMENTS] - -Replace ``HOSTNAME`` with the IP address or hostname of the KES server. -If the MinIO server host machines cannot resolve or reach the specified ``HOSTNAME``, the deployment may return errors or fail to start. - -- If using a single KES server host, specify the IP or hostname of that host -- If using multiple KES server hosts, specify the load balancer or reverse proxy managing connections to those hosts. - -MinIO uses the :envvar:`MINIO_KMS_KES_KEY_NAME` key for the following cryptographic operations: - -- Encrypting the MinIO backend (IAM, configuration, etc.) -- Encrypting objects using :ref:`SSE-KMS ` if the request does not - include a specific |EK|. -- Encrypting objects using :ref:`SSE-S3 `. - -The ``minio-kes`` certificates enable mTLS between the MinIO deployment and the KES server *only*. -They do not otherwise enable TLS for other client connections to MinIO. - -.. end-kes-configuration-minio-desc - .. start-kes-generate-key-desc MinIO requires that the |EK| exist on the root KMS *before* performing |SSE| operations using that key. diff --git a/source/includes/linux/steps-configure-minio-kes-aws.rst b/source/includes/linux/steps-configure-minio-kes-aws.rst index bff954de..fd8350e3 100644 --- a/source/includes/linux/steps-configure-minio-kes-aws.rst +++ b/source/includes/linux/steps-configure-minio-kes-aws.rst @@ -72,7 +72,7 @@ b. Configure the MinIO Environment File MinIO defaults to expecting this file at ``/etc/default/minio``. If you modified your deployment to use a different location for the environment file, modify the file at that location. - .. include:: /includes/linux/common-minio-kes.rst + .. include:: /includes/common/common-minio-kes.rst :start-after: start-kes-configuration-minio-desc :end-before: end-kes-configuration-minio-desc @@ -99,7 +99,7 @@ b. Start the MinIO Server 5) Generate a New Encryption Key ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. include:: /includes/linux/common-minio-kes.rst +.. include:: /includes/common/common-minio-kes.rst :start-after: start-kes-generate-key-desc :end-before: end-kes-generate-key-desc diff --git a/source/includes/linux/steps-configure-minio-kes-azure.rst b/source/includes/linux/steps-configure-minio-kes-azure.rst index 6085c91a..db23d49e 100644 --- a/source/includes/linux/steps-configure-minio-kes-azure.rst +++ b/source/includes/linux/steps-configure-minio-kes-azure.rst @@ -71,7 +71,7 @@ b. Configure the MinIO Environment File MinIO defaults to expecting this file at ``/etc/default/minio``. If you modified your deployment to use a different location for the environment file, modify the file at that location. - .. include:: /includes/linux/common-minio-kes.rst + .. include:: /includes/common/common-minio-kes.rst :start-after: start-kes-configuration-minio-desc :end-before: end-kes-configuration-minio-desc @@ -98,7 +98,7 @@ b. Start the MinIO Server 5) Generate a New Encryption Key ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. include:: /includes/linux/common-minio-kes.rst +.. include:: /includes/common/common-minio-kes.rst :start-after: start-kes-generate-key-desc :end-before: end-kes-generate-key-desc diff --git a/source/includes/linux/steps-configure-minio-kes-gcp.rst b/source/includes/linux/steps-configure-minio-kes-gcp.rst index 260744eb..3b8c95fd 100644 --- a/source/includes/linux/steps-configure-minio-kes-gcp.rst +++ b/source/includes/linux/steps-configure-minio-kes-gcp.rst @@ -71,7 +71,7 @@ b. Configure the MinIO Environment File MinIO defaults to expecting this file at ``/etc/default/minio``. If you modified your deployment to use a different location for the environment file, modify the file at that location. - .. include:: /includes/linux/common-minio-kes.rst + .. include:: /includes/common/common-minio-kes.rst :start-after: start-kes-configuration-minio-desc :end-before: end-kes-configuration-minio-desc @@ -98,7 +98,7 @@ b. Start the MinIO Server 5) Generate a New Encryption Key ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. include:: /includes/linux/common-minio-kes.rst +.. include:: /includes/common/common-minio-kes.rst :start-after: start-kes-generate-key-desc :end-before: end-kes-generate-key-desc diff --git a/source/includes/linux/steps-configure-minio-kes-hashicorp-quick.rst b/source/includes/linux/steps-configure-minio-kes-hashicorp-quick.rst index 61faa6f7..b4b37531 100644 --- a/source/includes/linux/steps-configure-minio-kes-hashicorp-quick.rst +++ b/source/includes/linux/steps-configure-minio-kes-hashicorp-quick.rst @@ -1,17 +1,31 @@ -Deploy MinIO and KES with Server-Side Encryption using Hashicorp Vault for Local Development --------------------------------------------------------------------------------------------- +Enable Server-Side Encryption using Hashicorp Vault for Local Development +------------------------------------------------------------------------- -Prior to starting these steps, create the following folders: +This procedure assumes deploying MinIO and KES onto the same host. +You can use and modify the information below to better suit your local development environment. -.. code-block:: shell - :class: copyable - :substitutions: +This procedure assumes the following: - mkdir -P |kescertpath| - mkdir -P |kesconfigpath| - mkdir -P |miniocertpath| - mkdir -P |minioconfigpath| - mkdir -P |miniodatapath| +- An existing Vault deployment +- A single host machine for deploying KES and MinIO + +.. admonition:: Set Up Folder Hierarchy + :class: note + + Create the following folders on the host machine if they do not already exist. + + .. code-block:: shell + :class: copyable + :substitutions: + + mkdir -P |kescertpath| + mkdir -P |kesconfigpath| + mkdir -P |miniocertpath| + mkdir -P |minioconfigpath| + mkdir -P |miniodatapath| + + This procedure uses these paths in the following steps. + If you use different paths, make the necessary modifications for each step to reflect those changes 1) Download the KES Server Binary ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -34,65 +48,73 @@ Defer to the client documentation for instructions on trusting a third-party CA. 3) Create the KES and MinIO Configurations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -a. Create the KES Configuration File +.. container:: procedure - Create the configuration file using your preferred text editor. - The following example uses ``nano``: + a. Create the KES Configuration File - .. code-block:: shell - :substitutions: - - nano |kesconfigpath|/kes-config.yaml - - .. include:: /includes/common/common-minio-kes-hashicorp.rst - :start-after: start-kes-configuration-hashicorp-vault-desc - :end-before: end-kes-configuration-hashicorp-vault-desc - - - Set ``MINIO_IDENTITY_HASH`` to the identity hash of the MinIO mTLS certificate. - - The following command computes the necessary hash: + Create the configuration file using your preferred text editor. + The following example uses ``nano``: .. code-block:: shell - :class: copyable :substitutions: - kes identity of |miniocertpath|/minio-kes.cert + nano |kesconfigpath|/kes-config.yaml - - Replace the ``vault.endpoint`` with the hostname of the Vault server(s). - - Set the ``vault.engine`` and ``vault.version`` to the appropriate values for the Vault K/V Engine configuration - - Replace the ``VAULTAPPID`` and ``VAULTAPPSECRET`` with the appropriate :ref:`Vault AppRole credentials `. + .. include:: /includes/common/common-minio-kes-hashicorp.rst + :start-after: start-kes-configuration-hashicorp-vault-desc + :end-before: end-kes-configuration-hashicorp-vault-desc -b. Create the MinIO Environment File + - Set ``MINIO_IDENTITY_HASH`` to the identity hash of the MinIO mTLS certificate. - Create or modify the environment file for the MinIO deployment using your preferred text editor. - The following example uses ``nano``: + The following command computes the necessary hash: - .. code-block:: shell - :substitutions: + .. code-block:: shell + :class: copyable + :substitutions: - nano |minioconfigpath|/minio + kes identity of |miniocertpath|/minio-kes.cert - .. include:: /includes/common/common-minio-kes.rst - :start-after: start-kes-configuration-minio-desc - :end-before: end-kes-configuration-minio-desc + - Replace the ``vault.endpoint`` with the hostname of the Vault server(s). + - Set the ``vault.engine`` and ``vault.version`` to the appropriate values for the Vault K/V Engine configuration + - Replace the ``VAULTAPPID`` and ``VAULTAPPSECRET`` with the appropriate :ref:`Vault AppRole credentials `. + + b. Create the MinIO Environment File + + Create or modify the environment file for the MinIO deployment using your preferred text editor. + The following example uses ``nano``: + + .. code-block:: shell + :substitutions: + + nano |minioconfigpath|/minio + + .. include:: /includes/common/common-minio-kes.rst + :start-after: start-kes-configuration-minio-desc + :end-before: end-kes-configuration-minio-desc 4) Start KES and MinIO ~~~~~~~~~~~~~~~~~~~~~~ +.. include:: /includes/common/common-minio-kes-hashicorp.rst + :start-after: start-kes-vault-seal-unseal-desc + :end-before: end-kes-vault-seal-unseal-desc + You must start KES *before* starting MinIO. The MinIO deployment requires access to KES as part of its startup. -a. Start the KES Server +.. container:: procedure - .. include:: /includes/common/common-minio-kes.rst - :start-after: start-kes-start-server-desc - :end-before: end-kes-start-server-desc + a. Start the KES Server -b. Start the MinIO Server + .. include:: /includes/common/common-minio-kes.rst + :start-after: start-kes-start-server-desc + :end-before: end-kes-start-server-desc - .. include:: /includes/common/common-minio-kes.rst - :start-after: start-kes-minio-start-server-desc - :end-before: end-kes-minio-start-server-desc + b. Start the MinIO Server + + .. include:: /includes/common/common-minio-kes.rst + :start-after: start-kes-minio-start-server-desc + :end-before: end-kes-minio-start-server-desc Foreground processes depend on the shell or terminal in which they run. Exiting or terminating the shell/terminal instance also kills the attached process. @@ -101,6 +123,10 @@ Defer to your operating system best practices for running processes in the backg 5) Generate a New Encryption Key ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. include:: /includes/common/common-minio-kes-hashicorp.rst + :start-after: start-kes-vault-seal-unseal-desc + :end-before: end-kes-vault-seal-unseal-desc + .. include:: /includes/common/common-minio-kes.rst :start-after: start-kes-generate-key-desc :end-before: end-kes-generate-key-desc diff --git a/source/includes/linux/steps-configure-minio-kes-hashicorp.rst b/source/includes/linux/steps-configure-minio-kes-hashicorp.rst index 9250a120..a5d13ce2 100644 --- a/source/includes/linux/steps-configure-minio-kes-hashicorp.rst +++ b/source/includes/linux/steps-configure-minio-kes-hashicorp.rst @@ -1,30 +1,56 @@ -Deploy MinIO and KES with Server-Side Encryption using Hashicorp Vault for Production -------------------------------------------------------------------------------------- +Enable Server-Side Encryption using Hashicorp Vault in Production Environments +------------------------------------------------------------------------------ -Prior to starting these steps, create the following folders if they do not already exist: +This procedure provides instructions for configuring and enabling Server-Side Encryption using Hashicorp Vault in production environments. +Specifically, this procedure assumes the following: -.. code-block:: shell - :class: copyable - :substitutions: +- An existing production-grade Vault deployment +- One or more hosts for deploying KES +- One or more hosts for a new or existing MinIO deployment - mkdir -P |kescertpath| - mkdir -P |kesconfigpath| - mkdir -P |miniocertpath| +.. admonition:: Set Up Folder Hierarchy + :class: note + + Create the following folders on the KES and MinIO host machines if they do not already exist: + + .. tab-set:: + + .. tab-item:: KES Hosts + + .. code-block:: shell + :class: copyable + :substitutions: + + mkdir -P |kescertpath| + mkdir -P |kesconfigpath| + + .. tab-item:: MinIO Hosts + + .. code-block:: shell + :class: copyable + :substitutions: + + mkdir -P |miniocertpath| + + This procedure uses these paths in the following steps. + If you use different paths, make the necessary modifications for each step to reflect those changes 1) Download KES and Create the Service File ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -a. Download KES +.. container:: procedure - .. include:: /includes/linux/common-minio-kes.rst - :start-after: start-kes-download-desc - :end-before: end-kes-download-desc + a. Download KES -b. Create the Service File + .. include:: /includes/linux/common-minio-kes.rst + :start-after: start-kes-download-desc + :end-before: end-kes-download-desc - .. include:: /includes/linux/common-minio-kes.rst - :start-after: start-kes-service-file-desc - :end-before: end-kes-service-file-desc + b. Create the Service File + + .. include:: /includes/linux/common-minio-kes.rst + :start-after: start-kes-service-file-desc + :end-before: end-kes-service-file-desc 2) Generate TLS Certificates for KES and MinIO ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -40,48 +66,58 @@ Defer to the client documentation for instructions on trusting a third-party CA. 3) Create the KES and MinIO Configurations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -a. Create the KES Configuration File +.. container:: procedure - Create the configuration file using your preferred text editor. - The following example uses ``nano``: + a. Create the KES Configuration File - .. code-block:: shell - :substitutions: - - nano /opt/kes/config.yaml - - .. include:: /includes/common/common-minio-kes-hashicorp.rst - :start-after: start-kes-configuration-hashicorp-vault-desc - :end-before: end-kes-configuration-hashicorp-vault-desc - - - Set ``MINIO_IDENTITY_HASH`` to the identity hash of the MinIO mTLS certificate. - - The following command computes the necessary hash: + Create the configuration file using your preferred text editor. + The following example uses ``nano``: .. code-block:: shell - :class: copyable :substitutions: - kes identity of |miniocertpath|/minio-kes.cert + nano /opt/kes/config.yaml - - Replace the ``vault.endpoint`` with the hostname of the Vault server(s). + .. include:: /includes/common/common-minio-kes-hashicorp.rst + :start-after: start-kes-configuration-hashicorp-vault-desc + :end-before: end-kes-configuration-hashicorp-vault-desc - - Replace the ``VAULTAPPID`` and ``VAULTAPPSECRET`` with the appropriate :ref:`Vault AppRole credentials `. + - Set ``MINIO_IDENTITY_HASH`` to the identity hash of the MinIO mTLS certificate. -b. Configure the MinIO Environment File + The following command computes the necessary hash: - Modify the MinIO Server environment file for all hosts in the target deployment to include the following environment variables. + .. code-block:: shell + :class: copyable + :substitutions: - MinIO defaults to expecting this file at ``/etc/default/minio``. - If you modified your deployment to use a different location for the environment file, modify the file at that location. + kes identity of |miniocertpath|/minio-kes.cert - .. include:: /includes/linux/common-minio-kes.rst - :start-after: start-kes-configuration-minio-desc - :end-before: end-kes-configuration-minio-desc + - Replace the ``keystore.vault.endpoint`` with the hostname of the Vault server(s). + + - Replace ``keystore.vault.engine`` and ``keystore.vault.version`` with the path and version of the KV engine used for storing secrets. + + - Replace the ``VAULTAPPID`` and ``VAULTAPPSECRET`` with the appropriate :ref:`Vault AppRole credentials `. + + - Modify the ``keystore.vault.tls.ca`` value to correspond to the path to the Vault :abbr:`CA (Certificate Authority)` certificate used to sign the Vault TLS keys. + + b. Configure the MinIO Environment File + + Modify the MinIO Server environment file for all hosts in the target deployment to include the following environment variables. + + MinIO defaults to expecting this file at ``/etc/default/minio``. + If you modified your deployment to use a different location for the environment file, modify the file at that location. + + .. include:: /includes/common/common-minio-kes.rst + :start-after: start-kes-configuration-minio-desc + :end-before: end-kes-configuration-minio-desc 4) Start KES and MinIO ~~~~~~~~~~~~~~~~~~~~~~ +.. include:: /includes/common/common-minio-kes-hashicorp.rst + :start-after: start-kes-vault-seal-unseal-desc + :end-before: end-kes-vault-seal-unseal-desc + You must start KES *before* starting MinIO. The MinIO deployment requires access to KES as part of its startup. @@ -102,7 +138,11 @@ b. Start the MinIO Server 5) Generate a New Encryption Key ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. include:: /includes/linux/common-minio-kes.rst +.. include:: /includes/common/common-minio-kes-hashicorp.rst + :start-after: start-kes-vault-seal-unseal-desc + :end-before: end-kes-vault-seal-unseal-desc + +.. include:: /includes/common/common-minio-kes.rst :start-after: start-kes-generate-key-desc :end-before: end-kes-generate-key-desc diff --git a/source/includes/windows/common-minio-kes.rst b/source/includes/windows/common-minio-kes.rst index 595eedd1..8c3d8075 100644 --- a/source/includes/windows/common-minio-kes.rst +++ b/source/includes/windows/common-minio-kes.rst @@ -134,8 +134,6 @@ This command assumes the ``minio-kes.cert``, ``minio-kes.key``, and ``kes-server MINIO_KMS_KES_CAPATH=|miniocertpath|\kes-server.cert MINIO_KMS_KES_KEY_NAME=minio-backend-default-key - minio.exe server [ARGUMENTS] - MinIO uses the :envvar:`MINIO_KMS_KES_KEY_NAME` key for the following cryptographic operations: - Encrypting the MinIO backend (IAM, configuration, etc.) diff --git a/source/operations/server-side-encryption/configure-minio-kes-hashicorp.rst b/source/operations/server-side-encryption/configure-minio-kes-hashicorp.rst index b6e37090..73295c36 100644 --- a/source/operations/server-side-encryption/configure-minio-kes-hashicorp.rst +++ b/source/operations/server-side-encryption/configure-minio-kes-hashicorp.rst @@ -128,59 +128,73 @@ Deploy or Ensure Access to a Hashicorp Vault Service :start-after: start-kes-prereq-hashicorp-vault-desc :end-before: end-kes-prereq-hashicorp-vault-desc -MinIO |KES| supports both the V1 and V2 Vault engines. -Select the corresponding tab to the engine used by your Vault deployment for instructions on configuring the necessary permissions: +MinIO |KES| supports either the V1 or V2 Vault `K/V engines `__. -.. tab-set:: +MinIO KES requires using AppRole authentication to the Vault server. +You must create an AppRole, assign it a policy that the necessary permissions, and retrieve the AppRole ID and Secret for use in configuring KES. - .. tab-item:: Vault Engine V1 +You can use the following steps to enable AppRole authentication and create the necessary policies to support core KES functionality against Vault: - Create an access policy ``kes-policy.hcl`` with a configuration similar to the following: - - .. code-block:: shell - :class: copyable +1. Enable AppRole Authentication - path "kv/*" { - capabilities = [ "create", "read", "delete" ] - } + .. code-block:: shell + :class: copyable - Write the policy to Vault using ``vault policy write kes-policy kes-policy.hcl``. + vault auth enable approle - .. tab-item:: Vault Engine V2 +#. Create a Policy for KES - Create an access policy ``kes-policy.hcl`` with a configuration similar to the following: + Create a `policy with necessary capabilities `__ for KES to use when accessing Vault. + Select the tab corresponding to the KV engine used for storing KES secrets: + + .. tab-set:: - .. code-block:: shell - :class: copyable + .. tab-item:: Vault Engine V1 - path "kv/data/*" { - capabilities = [ "create", "read"] - } + Create an access policy ``kes-policy.hcl`` with a configuration similar to the following: + + .. code-block:: shell + :class: copyable - path "kv/metadata/*" { - capabilities = [ "list", "delete"] - } - - Write the policy to Vault using ``vault policy write kes-policy kes-policy.hcl`` + path "kv/*" { + capabilities = [ "create", "read", "delete" ] + } -MinIO requires using AppRole authentication for secure communication with the Vault server. -The following commands: + Write the policy to Vault using ``vault policy write kes-policy kes-policy.hcl``. -- Enable AppRole Authentication -- Create an App Role ID for |KES| -- Binds that role to the created KES policy -- Requests a RoleID and SecretID + .. tab-item:: Vault Engine V2 - .. code-block:: shell - :class: copyable + Create an access policy ``kes-policy.hcl`` with a configuration similar to the following: - vault auth enable approle - vault write auth/approle/role/kes-role token_num_uses=0 secret_id_num_uses=0 period=5m - vault write auth/approle/role/kes-role policies=kes-policy - vault read auth/approle/role/kes-role/role-id - vault write -f auth/approle/role/kes-role/secret-id + .. code-block:: shell + :class: copyable + + path "kv/data/*" { + capabilities = [ "create", "read"] + } + + path "kv/metadata/*" { + capabilities = [ "list", "delete"] + } + + Write the policy to Vault using ``vault policy write kes-policy kes-policy.hcl`` + +#. Create an AppRole for KES and assign it the created policy + + .. code-block:: shell + :class: copyable + + vault write auth/approle/role/kes-role token_num_uses=0 secret_id_num_uses=0 period=5m + vault write auth/approle/role/kes-role policies=kes-policy + +#. Retrieve the AppRole ID and Secret + + .. code-block:: shell + :class: copyable + + vault read auth/approle/role/kes-role/role-id + vault write -f auth/approle/role/kes-role/secret-id -You must specify both RoleID and SecretID as part of this procedure. .. cond:: linux or macos or windows @@ -232,7 +246,7 @@ You must specify both RoleID and SecretID as part of this procedure. .. cond:: macos .. |kescertpath| replace:: ~/minio-kes-vault/certs - .. |kesconfigpath| replace:: ~/minio-kes-vault/config/ + .. |kesconfigpath| replace:: ~/minio-kes-vault/config .. |kesconfigcertpath| replace:: ~/minio-kes-vault/certs .. |miniocertpath| replace:: ~/minio-kes-vault/certs .. |minioconfigpath| replace:: ~/minio-kes-vault/config @@ -300,6 +314,10 @@ using Hashicorp Vault as the root Key Management Service (KMS) for |SSE|: keystore: vault: endpoint: https://vault.example.net:8200 + engine: "kv" + version: "v1" + namespace: "minio" + prefix: "keys" approle: id: ${KES_APPROLE_ID} secret: ${KES_APPROLE_SECRET} @@ -307,6 +325,8 @@ using Hashicorp Vault as the root Key Management Service (KMS) for |SSE|: status: ping: 10s tls: + key: "kes-mtls.key" + cert: "kes-mtls.cert" ca: vault-tls.cert .. tab-item:: Reference @@ -348,15 +368,24 @@ using Hashicorp Vault as the root Key Management Service (KMS) for |SSE|: - The configuration for the Hashicorp Vault keystore. The following fields are *required*: - - ``endpoint`` - The hostname for the vault server(s). The - hostname *must* be resolvable by the KES server host. + - ``endpoint`` - The hostname for the vault server(s). + The hostname *must* be resolvable by the KES server host. - - ``approle`` - The `AppRole - `__ used by - KES for performing authenticated operations against Vault. + - ``engine`` - The path to the K/V engine to use. + Defaults to ``kv`` - The specified AppRole must have the - appropriate :ref:`permissions ` + - ``version`` - The version of the K/V engine to use. + + Specify either ``v1`` or ``v2``. + Defaults to ``v1``. + + - ``namespace`` - The Vault namespace to use for secret storage. + + - ``prefix`` - The prefix to use for secret storage. + + - ``approle`` - The `AppRole `__ used by KES for performing authenticated operations against Vault. + + The specified AppRole must have the appropriate :ref:`permissions ` - ``tls.ca`` - The Certificate Authority used to sign the Vault TLS certificates. Typically required if the Vault