1
0
mirror of https://github.com/minio/docs.git synced 2025-07-30 07:03:26 +03:00

Add cert-manager documentation (#1317)

Adds cert-manager docs for Kubernetes outputs.

Closes #1245 

Partially addresses #1273
This commit is contained in:
Daryl White
2024-10-01 16:06:37 -04:00
committed by GitHub
parent 3f9a46bcbe
commit ca34aa7e43
12 changed files with 1560 additions and 1 deletions

View File

@ -0,0 +1,113 @@
.. _minio-certmanager:
============
cert-manager
============
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 1
TLS certificate management with cert-manager
--------------------------------------------
This guide shows you how to install cert-manager for TLS certificate management.
The guide assumes a new or fresh MinIO Operator installation.
.. note::
This guide uses a self-signed ``Cluster Issuer``.
You can also use `other Issuers supported by cert-manager <https://cert-manager.io/docs/configuration/issuers/>`__.
The main difference is that you must provide that ``Issuer`` CA certificate to MinIO, instead of the CA's mentioned in this guide.
Refer to the `cert-manager documentation <https://cert-manager.io>`__ and your own organization's certificate requirements for more advanced configurations.
cert-manager manages certificates within Kubernetes clusters.
The MinIO Operator supports using cert-manager for managing and provisioning certificates as an alternative to the MinIO Operator managing certificates for itself and its tenants.
cert-manager obtains valid certificates from an ``Issuer`` or ``ClusterIssuer`` and can automatically renew certificates prior to expiration.
A ``ClusterIssuer`` issues certificates for multiple namespaces.
An ``Issuer`` only mints certificates for its own namespace.
The following graphic depicts how cert-manager provides certificates in namespaces across a Kubernetes cluster.
- A ``ClusterIssuer`` exists at the root level of the Kubernetes cluster, typically the ``default`` namespace, to provide certificates to all other namespaces.
- The ``minio-operator`` namespace receives its own, local ``Issuer``.
- Each tenant's namespace receives its own, local ``Issuer``.
- The certificates issued by each tenant namespace must be made known to and trusted by the MinIO Operator.
.. image:: /images/k8s/cert-manager-graph.png
:width: 600px
:alt: A graph of the namespaces in a Kubernetes cluster showing the relationship between the root level ClusterIssuer and three other namespaces with their own Issuer.
:align: center
Prerequisites
-------------
- A `supported version of Kubernetes <https://kubernetes.io/releases/>`__.
- `kustomize <https://kustomize.io/>`__ installed
- ``kubectl`` access to your ``k8s`` cluster
.. _minio-setup-certmanager:
Setup cert-manager
------------------
Install cert-manager
~~~~~~~~~~~~~~~~~~~~
The following command installs version 1.12.13 using ``kubectl``.
.. code-block:: shell
:class: copyable
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.13/cert-manager.yaml
`Release 1.12.X LTS <https://cert-manager.io/docs/releases/release-notes/release-notes-1.12/>`__ is preferred, but you may install the latest version.
For more details on installing cert-manager, see their `installation instructions <https://cert-manager.io/docs/installation/>`__.
.. _minio-cert-manager-create-cluster-issuer:
Create a self-signed Cluster Issuer for the cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The ``Cluster Issuer`` is the top level Issuer from which all other certificates in the cluster derive.
1. Request cert-manager to generate this by creating a ``ClusterIssuer`` resource.
Create a file called ``selfsigned-root-clusterissuer.yaml`` with the following contents:
.. code-block:: yaml
:class: copyable
# selfsigned-root-clusterissuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned-root
spec:
selfSigned: {}
2. Apply the resource to the cluster:
.. code-block:: shell
:class: copyable
kubectl apply -f selfsigned-root-clusterissuer.yaml
Next steps
----------
Set up :ref:`cert-manager for the MinIO Operator <minio-certmanager-operator>`.
.. toctree::
:titlesonly:
:hidden:
/operations/cert-manager/cert-manager-operator
/operations/cert-manager/cert-manager-tenants

View File

@ -0,0 +1,255 @@
.. _minio-certmanager-operator:
=========================
cert-manager for Operator
=========================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 1
MinIO Operator manages TLS certificate issuing for the services hosted in the ``minio-operator`` namespace.
This page describes how to manage the Operator's TLS certificates with :ref:`cert-manager <minio-certmanager>`.
Prerequisites
-------------
- A `supported version of Kubernetes <https://kubernetes.io/releases/>`__.
- `kustomize <https://kustomize.io/>`__ installed
- ``kubectl`` access to your ``k8s`` cluster
- Completed the steps to :ref:`set up cert-manager <minio-setup-certmanager>`
- The MinIO Operator must not yet be installed.
1) Create a CA Issuer for the ``minio-operator`` namespace
----------------------------------------------------------
This guide **disables** the automatic generation of certificates in MinIO Operator and issues certificates using cert-manager instead.
The ``minio-operator`` namespace must have its own certificate authority (CA), derived from the cluster's ``ClusterIssuer`` certificate created during :ref:`cert-manager setup <minio-certmanager>`.
Create this CA certificate using cert-manager.
.. important::
This CA certificate **must** exist *before* installing MinIO Operator.
1. If it does not exist, create the ``minio-operator`` namespace
.. code-block:: shell
:class: copyable
kubectl create ns minio-operator
2. Request a new Certificate with ``spec.isCA: true`` specified.
This certificate serves as the :abbr:`CA (Certificate Authority)` for the `minio-operator` namespace.
Create a file called ``operator-ca-tls-secret.yaml`` with the following contents:
.. code-block:: yaml
:class: copyable
:emphasize-lines: 7,8
# operator-ca-tls-secret.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: minio-operator-ca-certificate
namespace: minio-operator
spec:
isCA: true
commonName: operator
secretName: operator-ca-tls
duration: 70128h # 8y
privateKey:
algorithm: ECDSA
size: 256
issuerRef:
name: selfsigned-root
kind: ClusterIssuer
group: cert-manager.io
.. important::
The ``spec.issueRef.name`` must match the name of the ``ClusterIssuer`` created when :ref:`setting up cert-manager <minio-cert-manager-create-cluster-issuer>`.
If you specified a different ``ClusterIssuer`` name or are using a different ``Issuer`` from the guide, modify the ``issuerRef`` to match your environment.
3. Apply the resource:
.. code-block:: shell
:class: copyable
kubectl apply -f operator-ca-tls-secret.yaml
Kubernetes creates a new secret with the name ``operator-ca-tls`` in the ``minio-operator`` namespace.
.. important::
Make sure to trust this certificate in any applications that need to interact with the MinIO Operator.
2) Use the secret to create the ``Issuer``
------------------------------------------
Use the ``operator-ca-tls`` secret to add an ``Issuer`` resource for the ``minio-operator`` namespace.
1. Create a file called ``operator-ca-issuer.yaml`` with the following contents:
.. code-block:: yaml
# operator-ca-issuer.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: minio-operator-ca-issuer
namespace: minio-operator
spec:
ca:
secretName: operator-ca-tls
2. Apply the resource:
.. code-block:: shell
kubectl apply -f operator-ca-issuer.yaml
3) Create TLS certificate
-------------------------
Now that the ``Issuer`` exists in the ``minio-operator`` namespace, cert-manager can add a certificate.
The certificate from cert-manager must be valid for the following DNS domains:
- ``sts``
- ``sts.minio-operator.svc.``
- ``sts.minio-operator.svc.<cluster domain>``
.. important::
Replace ``<cluster domain>`` with the actual value for your MinIO tenant.
``cluster domain`` is the internal root DNS domain assigned in your Kubernetes cluster.
Typically, this is ``cluster.local``, but confirm the value by checking your CoreDNS configuration for the correct value for your Kubernetes cluster.
For example:
.. code-block:: shell
:class: copyable
kubectl get configmap coredns -n kube-system -o jsonpath="{.data}"
Different Kubernetes providers manage the root domain differently.
Check with your Kubernetes provider for more information.
1. Create a ``Certificate`` for the specified domains:
Create a file named ``sts-tls-certificate.yaml`` with the following contents:
.. code-block:: yaml
:class: copyable
:emphasize-lines: 7,12
# sts-tls-certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: sts-certmanager-cert
namespace: minio-operator
spec:
dnsNames:
- sts
- sts.minio-operator.svc
- sts.minio-operator.svc.cluster.local # Replace cluster.local with the value for your domain.
secretName: sts-tls
issuerRef:
name: minio-operator-ca-issuer
.. important::
The ``spec.secretName`` is not optional.
The secret name **must** be ``sts-tls``.
Confirm this by setting ``spec.secretName: sts-tls`` as highlighted in the certificate YAML.
2. Apply the resource:
.. code-block:: shell
:class: copyable
kubectl apply -f sts-tls-certificate.yaml
This creates a secret called ``sts-tls`` in the ``minio-operator`` namespace.
.. warning::
The STS service will not start if the ``sts-tls`` secret, containing the TLS certificate, is missing or contains an invalid ``key-value`` pair.
4) Install Operator with Auto TLS disabled
------------------------------------------
You can now :ref:`install the MinIO Operator <minio-operator-installation>`.
When installing the Operator deployment, set the ``OPERATOR_STS_AUTO_TLS_ENABLED`` environment variable to ``off`` in the ``minio-operator`` container.
Disabling this environment variable prevents the MinIO Operator from issuing the certificates.
Instead, Operator relies on cert-manager to issue the TLS certificate.
There are various methods to define an environment variable depending on how you install the Operator.
The following steps define the variable with kustomize.
1. Create a kustomization patch file called ``kustomization.yaml`` with the following contents:
.. code-block:: yaml
:class: copyable
# minio-operator/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- github.com/minio/operator/resources
patches:
- patch: |-
apiVersion: apps/v1
kind: Deployment
metadata:
name: minio-operator
namespace: minio-operator
spec:
template:
spec:
containers:
- name: minio-operator
env:
- name: OPERATOR_STS_AUTO_TLS_ENABLED
value: "off"
- name: OPERATOR_STS_ENABLED
value: "on"
2. Apply the kustomization resource to the cluster:
.. code-block:: shell
:class: copyable
kubectl apply -k minio-operator
Migrate an existing MinIO Operator deployment to cert-manager
-------------------------------------------------------------
To transition an existing MinIO Operator deployment from using AutoCert to cert-manager, complete the following steps:
1. Complete the steps for :ref:`installing cert-manager <minio-certmanager>`, including disabling auto-cert.
2. Complete steps 1-3 on this page to generate the certificate authority for the Operator.
3. When you get to the install step on this page, instead replace the existing Operator TLS certificate with the cert-manager issued certificate.
4. Create new cert-manager certificates for each tenant, similar to the steps described on the :ref:`cert-manager for Tenants <minio-certmanager-tenants>` page.
5. Replace the secrets in the MinIO Operator namespace for the tenants with secrets related to each tenant's cert-manager issued certificate.
Next steps
----------
Set up :ref:`cert-manager for a MinIO Tenant <minio-certmanager-tenants>`.

View File

@ -0,0 +1,272 @@
.. _minio-certmanager-tenants:
========================
cert-manager for Tenants
========================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 1
The following procedures create and apply the resources necessary to use cert-manager for the TLS certificates within a tenant.
.. note::
The procedures use ``tenant-1`` as the name of the tenant.
Replace the string ``tenant-1`` throughout the procedures to reflect the name of your tenant.
Prerequisites
-------------
- `kustomize <https://kustomize.io/>`__ installed
- ``kubectl`` access to your ``k8s`` cluster
- Completed the steps to :ref:`set up cert-manager <minio-setup-certmanager>`
- The MinIO Operator installed and :ref:`set up for cert-manager <minio-certmanager-operator>`.
1) Create the tenant namespace CA Issuer
----------------------------------------
Before deploying a new tenant, create a Certificate Authority and Issuer for the tenant's namespace.
1. If necessary, create the tenant's namespace.
.. code-block:: shell
:class: copyable
kubectl create ns tenant-1
This much match the value of the ``metadata.namespace`` field in the tenant's YAML.
2. Request a Certificate for a new Certificate Authority with ``spec.isCA`` set to ``true``.
Create a file called ``tenant-1-ca-certificate.yaml`` with the following contents:
.. code-block:: yaml
:class: copyable
:emphasize-lines: 7,8
# tenant-1-ca-certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: tenant-1-ca-certificate
namespace: tenant-1
spec:
isCA: true
commonName: tenant-1-ca
secretName: tenant-1-ca-tls
duration: 70128h # 8y
privateKey:
algorithm: ECDSA
size: 256
issuerRef:
name: selfsigned-root
kind: ClusterIssuer
group: cert-manager.io
.. important::
The ``spec.issueRef.name`` must match the name of the ``ClusterIssuer`` created when :ref:`setting up cert-manager <minio-cert-manager-create-cluster-issuer>`.
If you specified a different ``ClusterIssuer`` name or are using a different ``Issuer`` from the guide, modify the ``issuerRef`` to match your environment.
3. Apply the resource:
.. code-block:: shell
:class: copyable
kubectl apply -f tenant-1-ca-certificate.yaml
2) Create the ``Issuer``
------------------------
The ``Issuer`` issues the certificates within the tenant namespace.
1. Generate a resource definition for an ``Issuer``.
Create a file called ``tenant-1-ca-issuer.yaml`` with the following contents:
.. code-block:: yaml
:class: copyable
# tenant-1-ca-issuer.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: tenant-1-ca-issuer
namespace: tenant-1
spec:
ca:
secretName: tenant-1-ca-tls
2. Apply the ``Issuer`` resource definition:
.. code-block:: shell
:class: copyable
kubectl apply -f tenant-1-ca-issuer.yaml
3) Create a certificate for the tenant
--------------------------------------
Request that cert-manager issue a new TLS server certificate for MinIO.
The certificate must be valid for the following DNS domains:
- ``minio.<namespace>``
- ``minio.<namespace>.svc``
- ``minio.<namespace>.svc.<cluster domain>``
- ``*.<tenant-name>-hl.<namespace>.svc.<cluster domain>``
- ``*.<namespace>.svc.<cluster domain>``
- ``*.<tenant-name>.minio.<namespace>.svc.<cluster domain>'``
.. important::
Replace the the placeholder text (marked with the ``<`` and ``>`` characters) with values for your tenant:
- ``<cluster domain>`` is the internal root DNS domain assigned in your Kubernetes cluster.
Typically, this is ``cluster.local``, but confirm the value by checking your CoreDNS configuration for the correct value for your Kubernetes cluster.
For example:
.. code-block:: shell
:class: copyable
kubectl get configmap coredns -n kube-system -o jsonpath="{.data}"
Different Kubernetes providers manage the root domain differently.
Check with your Kubernetes provider for more information.
- ``tenant-name`` is the name provided to your tenant in the ``metadata.name`` of the Tenant YAML.
For this example it is ``myminio``.
- ``namespace`` is the value created earlier where the tenant will be installed.
In the tenant YAML, it is defined in the the ``metadata.namespace`` field.
For this example it is ``tenant-1``.
1. Request a ``Certificate`` for the specified domains
Create a file called ``tenant-1-minio-certificate.yaml``.
The contents of the file should resemble the following, modified to reflect your cluster and tenant configurations:
.. code-block:: yaml
:class: copyable
# tenant-1-minio-certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: tenant-certmanager-cert
namespace: tenant-1
spec:
dnsNames:
- "minio.tenant-1"
- "minio.tenant-1.svc"
- 'minio.tenant-1.svc.cluster.local'
- '*.minio.tenant-1.svc.cluster.local'
- '*.myminio-hl.tenant-1.svc.cluster.local'
- '*.myminio.minio.tenant-1.svc.cluster.local'
secretName: myminio-tls
issuerRef:
name: tenant-1-ca-issuer
.. tip::
For this example, the Tenant name is ``myminio``.
We recommend naming the secret in the field ``spec.secretName`` as ``<tenant-name>-tls`` as a naming convention.
2. Apply the certificate resource:
.. code-block:: shell
:class: copyable
kubectl apply -f tenant-1-minio-certificate.yaml
3. Validate the changes took effect:
.. code-block:: shell
:class: copyable
kubectl describe secret/myminio-tls -n tenant-1
.. note::
- Replace ``tenant-1`` with the namespace for your tenant.
- Replace ``myminio-tls`` with the name of your secret, if different.
4) Deploy the tenant using cert-manager for TLS certificate management
----------------------------------------------------------------------
When deploying a Tenant, you must set the TLS configuration such that:
- The Tenant does not automatically generate its own certificates (``spec.requestAutoCert: false``) *and*
- The Tenant has a valid cert-manager reference (``spec.externalCertSecret``)
This directs the Operator to deploy the Tenant using the cert-manager certificates exclusively.
The following YAML ``spec`` provides a baseline configuration meeting these requirements:
.. code-block:: yaml
:emphasize-lines: 6,9,11
apiVersion: minio.min.io/v2
kind: Tenant
metadata:
name: myminio
namespace: tenant-1
spec:
...
## Disable default tls certificates.
requestAutoCert: false
## Use certificates generated by cert-manager.
externalCertSecret:
- name: myminio-tls
type: cert-manager.io/v1
...
5) Trust the tenant's CA in MinIO Operator
------------------------------------------
The MinIO Operator does not trust the tenant's CA by default.
To trust the tenant's CA, you must pass the certificate to the Operator as a secret.
To do this, create a secret with the prefix ``operator-ca-tls-`` followed by a unique identifier in the `minio-operator` namespace.
MinIO Operator mounts and trusts **all** certificates issued by the provided Certificate Authorities.
This is required because the MinIO Operator performs health checks using the ``/minio/health/cluster`` endpoint.
Create ``operator-ca-tls-tenant-1`` secret
++++++++++++++++++++++++++++++++++++++++++
Copy the tenant's cert-manager generated CA public key (``ca.crt``) into the `minio-operator` namespace.
This allows Operator to trust the cert-manager issued CA and all certificates derived from it.
1. Create a ``ca.crt`` file containing the CA:
.. code-block:: shell
:class: copyable
kubectl get secrets -n tenant-1 tenant-1-ca-tls -o=jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
2. Create the secret:
.. code-block:: shell
:class: copyable
kubectl create secret generic operator-ca-tls-tenant-1 --from-file=ca.crt -n minio-operator
.. tip::
In this example we chose a secret name of ``operator-ca-tls-tenant-1``.
We used the tenant namespace ``tenant-1`` as a suffix for easy identification of which namespace the CA comes from.
Use the name of your tenant namespace for easier linking secrets to the related resources.
6) Deploy the tenant
--------------------
With the Certificate Authority and ``Issuer`` in place for the tenant's namespace, you can now :ref:`deploy the object store tenant <minio-k8s-deploy-minio-tenant>`.
Use the modified baseline tenant YAML to disable AutoCert and reference the secret you generated.

View File

@ -39,6 +39,8 @@ a Prometheus :ref:`alert <minio-metrics-and-alerts>` using the
``minio_cluster_nodes_offline_total`` metric to detect whether one or
more MinIO nodes are offline.
.. _minio-cluster-write-quorum:
Cluster Write Quorum
--------------------

View File

@ -64,6 +64,12 @@ Enabling TLS
If you have a custom Subject Alternative Name (SAN) certificate that is *not* also a wildcard cert, the TLS certificate SAN **must** apply to the hostname for its parent node.
Without a wildcard, the SAN must match exactly to be able to connect to the tenant.
Certificate Management with cert-manager
----------------------------------------
The MinIO Operator supports using `cert-manager <https://cert-manager.io/>`__ as a full replacement for its built-in automatic certificate management *or* user-driven manual certificate management.
For instructions for deploying the MinIO Operator and tenants using cert-manager, refer to the :ref:`cert-manager page <minio-certmanager>`.
.. cond:: linux