minio/docs
1
0
Fork 0
mirror of https://github.com/minio/docs.git synced 2025-07-28 19:42:10 +03:00
Code Activity

DOCS-912: Part 1: Cleaning up Vault (#949)

Browse Source 
Staged:
http://192.241.195.202:9000/staging/DOCS-912/linux/operations/server-side-encryption/configure-minio-kes-hashicorp.html

---------

Co-authored-by: Andrea Longo <feorlen@users.noreply.github.com>
Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com>
This commit is contained in:
Ravind Kumar
2023-08-11 17:08:36 -04:00
committed by GitHub
parent d2e8899685
commit 20403a7464
14 changed files with 68 additions and 255 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
source/default-conf.py
View File

@ -75,7 +75,7 @@ extlinks = {
'gcs-docs' : ('https://cloud.google.com/storage/docs/%s', None),
'aks-docs' : ('https://learn.microsoft.com/en-us/azure/aks/%s', None),
'azure-docs' : ('https://learn.microsoft.com/en-us/azure/%s', None),
'kes-docs' : ('https://min.io/docs/kes/%s', None),
}
suppress_warnings = [

2
source/includes/common/common-minio-kes-aws.rst
View File

@ -34,7 +34,7 @@ Manager:
- /v1/key/generate/* # e.g. '/minio-'
- /v1/key/decrypt/*
- /v1/key/bulk/decrypt
- /v1/key/list
- /v1/key/list/*
- /v1/status
- /v1/metrics
- /v1/log/audit

2
source/includes/common/common-minio-kes-azure.rst
View File

@ -35,7 +35,7 @@ Manager:
- /v1/key/generate/* # e.g. '/minio-'
- /v1/key/decrypt/*
- /v1/key/bulk/decrypt
- /v1/key/list
- /v1/key/list/*
- /v1/status
- /v1/metrics
- /v1/log/audit

2
source/includes/common/common-minio-kes-gcp.rst
View File

@ -34,7 +34,7 @@ Manager:
- /v1/key/generate/* # e.g. '/minio-'
- /v1/key/decrypt/*
- /v1/key/bulk/decrypt
- /v1/key/list
- /v1/key/list/*
- /v1/status
- /v1/metrics
- /v1/log/audit

15
source/includes/common/common-minio-kes-hashicorp.rst
View File

@ -16,12 +16,17 @@ You must modify this YAML to reflect your deployment environment.
identity: disabled
# Specify the TLS keys generated in the previous step here
# For production environments, use keys signed by a known and trusted
# Certificate Authority (CA).
# For production environments, use keys signed by a known and trusted Certificate Authority (CA).
tls:
key: |kesconfigcertpath|kes-server.key
cert: |kesconfigcertpath|kes-server.cert
# Specify the path to CAs used by KES for validating client certificates
# This can alternatively be a single CA
# KES uses these CAs in addition to the system trust store for validating client certificates.
ca: |kesconfigcertpath|CAs/
# Sets access policies for KES
# The `minio` policy grants access to the listed APIs.
policy:
@ -31,15 +36,13 @@ You must modify this YAML to reflect your deployment environment.
- /v1/key/generate/* # e.g. '/minio-'
- /v1/key/decrypt/*
- /v1/key/bulk/decrypt
- /v1/key/list
- /v1/key/list/*
- /v1/status
- /v1/metrics
- /v1/log/audit
- /v1/log/error
identities:
- MINIO_IDENTITY_HASH # Replace with the output of 'kes identity of minio-kes.cert'
# In production environments, each client connecting to KES must
# Have their TLS hash listed under at least one `policy`.
- MINIO_API_KEY_HASH # Replace with the hash output returned from kes identity new
# Specify the connection information for the Vault server.
# The endpoint should be resolvable from the host.

19
source/includes/common/common-minio-kes.rst
View File

@ -113,17 +113,12 @@ If you run |KES| without tying it to the current shell session (e.g. with ``nohu
MinIO requires that the |EK| exist on the root KMS *before* performing |SSE| operations using that key.
Use ``kes key create`` *or* :mc-cmd:`mc admin kms key create` to add a new |EK| for use with |SSE|.
The following command uses the ``kes key create`` command to add a new External Key (EK) stored on the root KMS server for use with encrypting the MinIO backend.
The following command uses the :mc-cmd:`mc admin kms key create` command to add a new External Key (EK) stored on the root KMS server for use with encrypting the MinIO backend.
.. code-block:: shell
:class: copyable
:substitutions:
export KES_SERVER=https://127.0.0.1:7373
export KES_CLIENT_KEY=|miniocertpath|/minio-kes.key
export KES_CLIENT_CERT=|miniocertpath|/minio-kes.cert
kes key create -k encrypted-bucket-key
mc admin kms key create ALIAS KEYNAME
.. end-kes-generate-key-desc
@ -139,8 +134,7 @@ See the tutorials for :ref:`minio-snsd`, :ref:`minio-snmd`, or :ref:`minio-mnmd`
# Add these environment variables to the existing environment file
MINIO_KMS_KES_ENDPOINT=https://HOSTNAME:7373
MINIO_KMS_KES_CERT_FILE=|miniocertpath|/minio-kes.cert
MINIO_KMS_KES_KEY_FILE=|miniocertpath|/minio-kes.key
MINIO_KMS_KES_API_KEY="kes:v1:ACTpAsNoaGf2Ow9o5gU8OmcaG6Af/VcZ1Mt7ysuKoBjv"
# Allows validation of the KES Server Certificate (Self-Signed or Third-Party CA)
# Change this path to the location of the KES CA Path
@ -150,7 +144,7 @@ See the tutorials for :ref:`minio-snsd`, :ref:`minio-snmd`, or :ref:`minio-mnmd`
MINIO_KMS_KES_KEY_NAME=minio-backend-default-key
# Optional, defines the name for the KES server enclave to use.
MINIO_KMS_KES_ENCLAVE=<name>
# MINIO_KMS_KES_ENCLAVE=<name>
Replace ``HOSTNAME`` with the IP address or hostname of the KES server.
If the MinIO server host machines cannot resolve or reach the specified ``HOSTNAME``, the deployment may return errors or fail to start.
@ -168,14 +162,11 @@ MinIO uses the :envvar:`MINIO_KMS_KES_KEY_NAME` key for the following cryptograp
MinIO uses the :envvar:`MINIO_KMS_KES_ENCLAVE` key to define the name of the KES enclave to use.
- Replace ``<name>`` with the name of the :term:`enclave` to use.
- If not defined, MinIO does not send any enclave information.
If not defined, MinIO does not send any enclave information.
This may result in using the default enclave for stateful KES servers.
A KES :term:`enclave` provides an isolated space for its associated keys separate from other enclaves on a stateful KES server.
The ``minio-kes`` certificates enable mTLS between the MinIO deployment and the KES server *only*.
They do not otherwise enable TLS for other client connections to MinIO.
.. end-kes-configuration-minio-desc
.. start-kes-enable-sse-kms-desc

38
source/includes/linux/common-minio-kes.rst
View File

@ -25,7 +25,7 @@ For more granular controls, deploy a dedicated load balancer to manage connectio
.. start-kes-service-file-desc
Create the ``/etc/systemd/system/kes.service`` file on all KES hosts:
Create the ``/lib/systemd/system/kes.service`` file on all KES hosts:
.. literalinclude:: /extra/kes.service
:language: shell
@ -88,18 +88,17 @@ For existing MinIO deployments, run the following command on each MinIO host to
.. start-kes-generate-kes-certs-prod-desc
Enabling connectivity between MinIO and KES requires at minimum one TLS certificate for performing mutual TLS (mTLS) authentication.
KES requires TLS connectivity for all client connections, including those originating from MinIO.
See :ref:`minio-tls` for more information on enabling TLS for the MinIO deployment.
Depending on your Vault configuration, you may also need to create a dedicated set of TLS certificates for KES to connect and authenticate to Vault.
Defer to your organizations best practices around generating production-ready TLS certificates.
Place the certificates and corresponding private keys an appropriate directory such that the MinIO and KES service users can access and read their contents.
The following example structure uses the folder hierarchy suggested in the beginning of this procedure:
Defer to your organization's best practices around generating production-ready TLS certificates.
.. tab-set::
Place the certificates and corresponding private keys in a directory that the KES service user has permissions to access and read the directory's contents.
For example:
.. tab-item:: KES Hosts
.. code-block:: shell
.. code-block:: shell
:substitutions:
-rw-r--r-- 1 kes:kes |kescertpath|/kes-server.cert
@ -110,20 +109,6 @@ The following example structure uses the folder hierarchy suggested in the begin
-rw-r--r-- 1 kes:kes |kescertpath|/vault-CA.cert
.. tab-item:: MinIO Hosts
.. code-block:: shell
:substitutions:
-rw-r--r-- 1 minio-user:minio-user |miniocertpath|/minio-kes.cert
-rw-r--r-- 1 minio-user:minio-user |miniocertpath|/minio-kes.key
# If KES certs are self-signed or use a non-global CA
# Include the CA certs as well
-rw-r--r-- 1 minio-user:minio-user |miniocertpath|/kes-server.cert
The general strategy for cert management is to ensure that each process (MinIO, KES, and Vault) have their own mTLS certificates *and* the Certificate Authority (CA) used to sign each client certificate.
.. end-kes-generate-kes-certs-prod-desc
.. start-kes-generate-key-desc
@ -135,12 +120,7 @@ The following command uses the ``kes key create`` command to add a new External
.. code-block:: shell
:class: copyable
:substitutions:
export KES_SERVER=https://127.0.0.1:7373
export KES_CLIENT_KEY=|miniocertpath|/minio-kes.key
export KES_CLIENT_CERT=|miniocertpath|/minio-kes.cert
kes key create -k encrypted-bucket-key
mc admin kms key create ALIAS KEYNAME
.. end-kes-generate-key-desc

139
source/includes/linux/steps-configure-minio-kes-hashicorp-quick.rst
View File

@ -1,139 +0,0 @@
Enable Server-Side Encryption using Hashicorp Vault for Local Development
-------------------------------------------------------------------------
This procedure assumes deploying MinIO and KES onto the same host.
You can use and modify the information below to better suit your local development environment.
This procedure assumes the following:
- An existing Vault deployment
- A single host machine for deploying KES and MinIO
.. admonition:: Set Up Folder Hierarchy
:class: note
Create the following folders on the host machine if they do not already exist.
.. code-block:: shell
:class: copyable
:substitutions:
mkdir -P |kescertpath|
mkdir -P |kesconfigpath|
mkdir -P |miniocertpath|
mkdir -P |minioconfigpath|
mkdir -P |miniodatapath|
This procedure uses these paths in the following steps.
If you use different paths, make the necessary modifications for each step to reflect those changes
1) Download the KES Server Binary
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/linux/common-minio-kes.rst
:start-after: start-kes-download-desc
:end-before: end-kes-download-desc
2) Generate TLS Certificates for KES and MinIO
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common/common-minio-kes.rst
:start-after: start-kes-generate-kes-certs-desc
:end-before: end-kes-generate-kes-certs-desc
Depending on your Vault configuration, you may need to pass the ``kes-server.cert`` certificate as a trusted Certificate Authority.
See the `Hashicorp Server Configuration Documentation <https://www.vaultproject.io/docs/configuration/listener/tcp#tls_client_ca_file>`__ for more information.
Defer to the client documentation for instructions on trusting a third-party CA.
3) Create the KES and MinIO Configurations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. container:: procedure
a. Create the KES Configuration File
Create the configuration file using your preferred text editor.
The following example uses ``nano``:
.. code-block:: shell
:substitutions:
nano |kesconfigpath|/kes-config.yaml
.. include:: /includes/common/common-minio-kes-hashicorp.rst
:start-after: start-kes-configuration-hashicorp-vault-desc
:end-before: end-kes-configuration-hashicorp-vault-desc
- Set ``MINIO_IDENTITY_HASH`` to the identity hash of the MinIO mTLS certificate.
The following command computes the necessary hash:
.. code-block:: shell
:class: copyable
:substitutions:
kes identity of |miniocertpath|/minio-kes.cert
- Replace the ``vault.endpoint`` with the hostname of the Vault server(s).
- Set the ``vault.engine`` and ``vault.version`` to the appropriate values for the Vault K/V Engine configuration
- Replace the ``VAULTAPPID`` and ``VAULTAPPSECRET`` with the appropriate :ref:`Vault AppRole credentials <minio-sse-vault-prereq-vault>`.
b. Create the MinIO Environment File
Create or modify the environment file for the MinIO deployment using your preferred text editor.
The following example uses ``nano``:
.. code-block:: shell
:substitutions:
nano |minioconfigpath|/minio
.. include:: /includes/common/common-minio-kes.rst
:start-after: start-kes-configuration-minio-desc
:end-before: end-kes-configuration-minio-desc
4) Start KES and MinIO
~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common/common-minio-kes-hashicorp.rst
:start-after: start-kes-vault-seal-unseal-desc
:end-before: end-kes-vault-seal-unseal-desc
You must start KES *before* starting MinIO.
The MinIO deployment requires access to KES as part of its startup.
.. container:: procedure
a. Start the KES Server
.. include:: /includes/common/common-minio-kes.rst
:start-after: start-kes-start-server-desc
:end-before: end-kes-start-server-desc
b. Start the MinIO Server
.. include:: /includes/common/common-minio-kes.rst
:start-after: start-kes-minio-start-server-desc
:end-before: end-kes-minio-start-server-desc
Foreground processes depend on the shell or terminal in which they run.
Exiting or terminating the shell/terminal instance also kills the attached process.
Defer to your operating system best practices for running processes in the background.
5) Generate a New Encryption Key
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common/common-minio-kes-hashicorp.rst
:start-after: start-kes-vault-seal-unseal-desc
:end-before: end-kes-vault-seal-unseal-desc
.. include:: /includes/common/common-minio-kes.rst
:start-after: start-kes-generate-key-desc
:end-before: end-kes-generate-key-desc
6) Enable SSE-KMS for a Bucket
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common/common-minio-kes.rst
:start-after: start-kes-enable-sse-kms-desc
:end-before: end-kes-enable-sse-kms-desc

68
source/includes/linux/steps-configure-minio-kes-hashicorp.rst
View File

@ -1,5 +1,5 @@
Enable Server-Side Encryption using Hashicorp Vault in Production Environments
------------------------------------------------------------------------------
Procedure
---------
This procedure provides instructions for configuring and enabling Server-Side Encryption using Hashicorp Vault in production environments.
Specifically, this procedure assumes the following:
@ -8,33 +8,6 @@ Specifically, this procedure assumes the following:
- One or more hosts for deploying KES
- One or more hosts for a new or existing MinIO deployment
.. admonition:: Set Up Folder Hierarchy
:class: note
Create the following folders on the KES and MinIO host machines if they do not already exist:
.. tab-set::
.. tab-item:: KES Hosts
.. code-block:: shell
:class: copyable
:substitutions:
mkdir -P |kescertpath|
mkdir -P |kesconfigpath|
.. tab-item:: MinIO Hosts
.. code-block:: shell
:class: copyable
:substitutions:
mkdir -P |miniocertpath|
This procedure uses these paths in the following steps.
If you use different paths, make the necessary modifications for each step to reflect those changes
1) Download KES and Create the Service File
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -52,8 +25,8 @@ Specifically, this procedure assumes the following:
:start-after: start-kes-service-file-desc
:end-before: end-kes-service-file-desc
2) Generate TLS Certificates for KES and MinIO
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2) Generate TLS Certificates for KES
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/linux/common-minio-kes.rst
:start-after: start-kes-generate-kes-certs-prod-desc
@ -63,7 +36,21 @@ Depending on your Vault configuration, you may also need to specify the CA used
See the `Hashicorp Vault Configuration Docs <https://www.vaultproject.io/docs/configuration/listener/tcp#tls_client_ca_file>`__ for more information.
Defer to the client documentation for instructions on trusting a third-party CA.
3) Create the KES and MinIO Configurations
3) Generate a KES API Key for use by MinIO
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Starting with KES version :minio-git:`2023-02-15T14-54-37Z <kes/releases/tag/2023-02-15T14-54-37Z>`, you can generate an API key to use for authenticating to the KES server.
Use the :kes-docs:`kes identity new <cli/kes-identity/new>` command to generate a new API key for use by the MinIO Server:
.. code-block:: shell
:class: copyable
kes identity new
The output includes both the API Key for use with MinIO and the Identity hash for use with the :kes-docs:`KES Policy configuration <tutorials/configuration/#policy-configuration>`.
4) Create the KES and MinIO Configurations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. important::
@ -87,15 +74,14 @@ Defer to the client documentation for instructions on trusting a third-party CA.
:start-after: start-kes-configuration-hashicorp-vault-desc
:end-before: end-kes-configuration-hashicorp-vault-desc
- Set ``MINIO_IDENTITY_HASH`` to the identity hash of the MinIO mTLS certificate.
- Set ``MINIO_IDENTITY_HASH`` to the identity hash of the API Key generated in the previous step.
The following command computes the necessary hash:
The following command recomputes the necessary hash from the API key:
.. code-block:: shell
:class: copyable
:substitutions:
kes identity of |miniocertpath|/minio-kes.cert
kes identity of kes:v1:KEY/KEY
- Replace the ``keystore.vault.endpoint`` with the hostname of the Vault server(s).
@ -107,16 +93,16 @@ Defer to the client documentation for instructions on trusting a third-party CA.
b. Configure the MinIO Environment File
Modify the MinIO Server environment file for all hosts in the target deployment to include the following environment variables.
MinIO defaults to expecting this file at ``/etc/default/minio``.
If you modified your deployment to use a different location for the environment file, modify the file at that location.
Create or modify the MinIO Server environment file for all hosts in the target deployment to include the following environment variables:
.. include:: /includes/common/common-minio-kes.rst
:start-after: start-kes-configuration-minio-desc
:end-before: end-kes-configuration-minio-desc
4) Start KES and MinIO
MinIO defaults to expecting this file at ``/etc/default/minio``.
If you modified your deployment to use a different location for the environment file, modify the file at that location.
5) Start KES and MinIO
~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common/common-minio-kes-hashicorp.rst

2
source/operations/network-encryption.rst
View File

@ -283,6 +283,8 @@ MinIO supports the following TLS 1.2 and 1.3 cipher suites as supported by `Go <
- ``TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256``
- ``TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384``
.. _minio-TLS-third-party-ca:
Third-Party Certificate Authorities
-----------------------------------

2
source/operations/server-side-encryption/configure-minio-kes-aws.rst
View File

@ -275,7 +275,7 @@ The following section describes each of the |KES-git| configuration settings for
- /v1/key/generate/*
- /v1/key/decrypt/*
- /v1/key/bulk/decrypt
- /v1/key/list
- /v1/key/list/*
- /v1/status
- /v1/metrics
- /v1/log/audit

2
source/operations/server-side-encryption/configure-minio-kes-azure.rst
View File

@ -259,7 +259,7 @@ using Azure Key Vault as the root Key Management Service
- /v1/key/generate/*
- /v1/key/decrypt/*
- /v1/key/bulk/decrypt
- /v1/key/list
- /v1/key/list/*
- /v1/status
- /v1/metrics
- /v1/log/audit

2
source/operations/server-side-encryption/configure-minio-kes-gcp.rst
View File

@ -265,7 +265,7 @@ The following section describes each of the |KES-git| configuration settings for
- /v1/key/generate/*
- /v1/key/decrypt/*
- /v1/key/bulk/decrypt
- /v1/key/list
- /v1/key/list/*
- /v1/status
- /v1/metrics
- /v1/log/audit

16
source/operations/server-side-encryption/configure-minio-kes-hashicorp.rst
View File

@ -18,18 +18,11 @@ Server-Side Object Encryption with Hashicorp Vault Root KMS
.. |rootkms| replace:: `Hashicorp Vault <https://vaultproject.io/>`__
.. |rootkms-short| replace:: Vault
MinIO Server-Side Encryption (SSE) protects objects as part of write operations, allowing clients to take advantage of server processing power to secure objects at the storage layer (encryption-at-rest).
SSE also provides key functionality to regulatory and compliance requirements around secure locking and erasure.
MinIO SSE uses |KES-git| and an external root Key Management Service (KMS) for performing secured cryptographic operations at scale.
The root KMS provides stateful and secured storage of External Keys (EK) while |KES| is stateless and derives additional cryptographic keys from the root-managed |EK|.
.. Conditionals to handle the slight divergences in procedures between platforms.
.. cond:: linux
This procedure provides guidance for deploying and configuring KES at scale for a supporting |SSE| on a production MinIO deployment.
You can also use this procedure for deploying to local environments for testing and evaluation.
This procedure provides guidance for deploying MinIO configured to use KES and enable :ref:`Server Side Encryption <minio-sse-data-encryption>`.
As part of this procedure, you will:
@ -43,8 +36,6 @@ The root KMS provides stateful and secured storage of External Keys (EK) while |
#. Configure automatic bucket-default :ref:`SSE-KMS <minio-encryption-sse-kms>`
For production orchestrated environments, use the MinIO Kubernetes Operator to deploy a tenant with |SSE| enabled and configured for use with Hashicorp Vault.
.. cond:: macos or windows
This procedure assumes a single local host machine running the MinIO and KES processes.
@ -239,8 +230,6 @@ You can use the following steps to enable AppRole authentication and create the
.. |minioconfigpath| replace:: /opt/minio/config
.. |miniodatapath| replace:: ~/minio
.. include:: /includes/linux/steps-configure-minio-kes-hashicorp-quick.rst
.. include:: /includes/linux/steps-configure-minio-kes-hashicorp.rst
.. cond:: macos
@ -296,7 +285,8 @@ The following section describes each of the |KES-git| configuration settings for
.. code-block:: yaml
address: 0.0.0.0:7373
root: ${ROOT_IDENTITY}
admin:
identity: ${ROOT_IDENTITY}
tls:
key: kes-server.key