minio/docs
1
0
Fork 0
mirror of https://github.com/minio/docs.git synced 2025-04-22 19:02:57 +03:00
Code
docs/source/operations/install-deploy-manage/deploy-operator-helm.rst
Daryl White fa4e28f78c
Splits the helm chart reference page (#1142) 
Internal request to split the Operator and Tenant helm chart into
separate pages.

This adds a replacement link in default-conf to automate updating the
operator version in the URL.
2024-03-06 09:29:08 -05:00

21 KiB
Raw Blame History

Deploy Operator With Helm

minio

Table of Contents

Overview

Helm is a tool for automating the deployment of applications to Kubernetes clusters. A Helm chart is a set of YAML files, templates, and other files that define the deployment details. The following procedure uses a Helm Chart to install the MinIO Kubernetes Operator <minio-operator-installation> to a Kubernetes cluster.

Prerequisites

To install the Operator with Helm you will need the following:

  • An existing Kubernetes cluster.
  • The kubectl CLI tool on your local host, the same version as the cluster.
  • Helm version 3.8 or greater.
  • yq version 4.18.1 or greater.
  • Access to run kubectl commands on the cluster from your local host.

For more about Operator installation requirements, including supported Kubernetes versions and TLS certificates, see the Operator deployment prerequisites <minio-operator-prerequisites>.

This procedure assumes familiarity with the referenced Kubernetes concepts and utilities. While this documentation may provide guidance for configuring or deploying Kubernetes-related resources on a best-effort basis, it is not a replacement for the official Kubernetes Documentation <>.

Install the MinIO Operator using Helm Charts

The following procedure installs the Operator using the MinIO Operator Chart Repository. This method supports a simplified installation path compared to the local chart installation <minio-k8s-deploy-operator-helm-local>. You can modify the Operator deployment after installation.

Important

Do not use kubectl krew or similar methods to update or manage the MinIO Operator installation. If you use Helm charts to install the Operator, you must use Helm to manage that installation.

  1. Add the MinIO Operator Repo to Helm

    MinIO maintains a Helm-compatible repository at https://operator.min.io. Add this repository to Helm:

    helm repo add minio-operator https://operator.min.io

    You can validate the repo contents using helm search:

    helm search repo minio-operator

    The response should resemble the following:

    NAME                            CHART VERSION   APP VERSION     DESCRIPTION                    
minio-operator/minio-operator   4.3.7           v4.3.7          A Helm chart for MinIO Operator
minio-operator/operator         5.0.10          v5.0.10         A Helm chart for MinIO Operator
minio-operator/tenant           5.0.10          v5.0.10         A Helm chart for MinIO Operator

    The minio-operator/minio-operator is a legacy chart and should not be installed under normal circumstances.

  2. Install the Operator

    Run the helm install command to install the Operator:

    helm install \
  --namespace minio-operator \
  --create-namespace \
  operator minio-operator/operator

  3. Verify the Operator installation

    Check the contents of the specified namespace (minio-operator) to ensure all pods and services have started successfully.

    kubectl get all -n minio-operator

    The response should resemble the following:

    NAME                                  READY   STATUS    RESTARTS   AGE
pod/console-68d955874d-vxlzm          1/1     Running   0          25h
pod/minio-operator-699f797b8b-th5bk   1/1     Running   0          25h
pod/minio-operator-699f797b8b-nkrn9   1/1     Running   0          25h

NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/console    ClusterIP   10.43.195.224   <none>        9090/TCP,9443/TCP   25h
service/operator   ClusterIP   10.43.44.204    <none>        4221/TCP            25h
service/sts        ClusterIP   10.43.70.4      <none>        4223/TCP            25h

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/console          1/1     1            1           25h
deployment.apps/minio-operator   2/2     2            2           25h

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/console-68d955874d          1         1         1       25h
replicaset.apps/minio-operator-699f797b8b   2         2         2       25h

  4. (Optional) Enable NodePort Access to the Console

    You can enable Node Port <concepts/services-networking/service/#type-nodeport> access to the service/console service to allow simplified access to the MinIO Operator. You can skip this step if you intend to configure the Operator Console service to use a Kubernetes Load Balancer, ingress, or similar control plane component that enables external access.

    Edit the service/console and set the spec.ports[0].nodePort and spec.type fields as follows:

    spec:
  ports:
  - name: http
    port: 9090
    protocol: TCP
    targetPort: 9090
    nodePort: 30090
type: NodePort

    You can attempt to connect to the MinIO Operator Console by specifying port 30090 on any of the worker nodes in the deployment.

  5. Retrieve the Console Access Token

    The MinIO Operator uses a JSON Web Token (JWT) saved as a Kubernetes Secret for controlling access to the Operator Console.

    Use the following command to retrieve the JWT for login. You must have permission within the Kubernetes cluster to read secrets:

    kubectl get secret/console-sa-secret -n minio-operator -o json | jq -r ".data.token" | base64 -d

    The output should resemble the following:

    eyJhbGciOiJSUzI1NiIsImtpZCI6IlRtV2x3Z1RILVREaThhQm9iemFfLW95NHFHT0ZZOHFBRjlZalBRcWZiSDgifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtaW5pby1vcGVyYXRvciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjb25zb2xlLXNhLXNlY3JldCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJjb25zb2xlLXNhIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiY2M1ZjEwYzktYzU1ZC00MjNiLTgxM2MtNmU5ZDY2ZGI5NDYyIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Om1pbmlvLW9wZXJhdG9yOmNvbnNvbGUtc2EifQ.F-Pt5nU9xaugjRksWAOTShBW_eNTf8UwXvLfGxEK6l3_41NYsLgvTg5m0hYLUiYr6v2HwkEu0XzqTJbPoeSrFds8BOjeiCoP2Lmw4tRPo9tSXhAq-_elWt83YpJl-zjUpna5nVSWJWXKgj1Iga-9gw-Q63UygEcyTJ9_AwCNU9T0HdPzqccS9XrEUdsXFQxR9RwZY4TGC8K7cD9sc_OmfEiuyilRgyC_gFRvtCQfFv1DP0GKyjMGo2ffu-2Tq2U7zK5epWdqmNSvbIa0ZRoPlPedZ6nYY935lNgTIIW1oykRYrgwZZiv4CzfTH2gPswjtPc5ICtDDRUjYEhdTq3gtw

    If the output includes a trailing % make sure to omit it from the result.

  6. Log into the MinIO Operator Console

    If you configured the svc/console service for access through ingress, a cluster load balancer, you can access the Console using the configured hostname and port.

    If you configured the service for access through NodePorts, specify the hostname of any worker node in the cluster with that port as HOSTNAME:NODEPORT to access the Console.

    Alternatively, you can use kubectl port forward to temporary forward ports for the Console:

    kubectl port-forward svc/console -n minio-operator 9090:9090

    You can then use http://localhost:9090 to access the MinIO Operator Console.

    Once you access the Console, use the Console JWT to log in.

You can now deploy and manage MinIO Tenants using the Operator Console <deploy-minio-distributed>.

You can also deploy a tenant using Helm Charts <deploy-tenant-helm>.

Install the MinIO Operator using Local Helm Charts

The following procedure installs the Operator using a local copy of the Helm Charts. This method may support easier pre-configuration of the Operator compared to the repo-based installation <minio-k8s-deploy-operator-helm-repo>

  1. Download the Helm charts

    On your local host, download the Operator Helm charts to a convenient directory:

    curl -O https://raw.githubusercontent.com/minio/operator/master/helm-releases/operator-|operator-version-stable|.tgz

    The chart contains a values.yaml file you can customize to suit your needs. For details on the options available in the MinIO Operator values.yaml, see minio-operator-chart-values.

    For more about customizations, see Helm Charts.

  2. Deploy Operator

    The following Helm command deploys the MinIO Operator using the downloaded chart:

    helm install \
--namespace minio-operator \
--create-namespace \
operator operator-|operator-version-stable|.tgz

  3. Configure Operator

    1. Create the YAML configuration files

      Use kubectl and yq to create the following files:

      • service.yaml:

        kubectl get service console -n minio-operator -o yaml > service.yaml
yq e -i '.spec.type="NodePort"' service.yaml
yq e -i '.spec.ports[0].nodePort = PORT_NUMBER' service.yaml

        Replace PORT_NUMBER with the port on which to serve the Operator GUI.

        The file contents resemble the following:

        Example service.yaml file

        apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: minio-operator
    meta.helm.sh/release-namespace: minio-operator
  creationTimestamp: "2023-05-11T14:57:42Z"
  labels:
    app.kubernetes.io/instance: minio-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: operator
    app.kubernetes.io/version: v5.0.4
    helm.sh/chart: operator-5.0.4
  name: console
  namespace: minio-operator
  resourceVersion: "907"
  uid: 9297fd97-806a-4715-8bd5-a1f6103149a8
spec:
  clusterIP: 10.96.157.135
  clusterIPs:
    - 10.96.157.135
  internalTrafficPolicy: Cluster
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  ports:
    - name: http
      port: 9090
      protocol: TCP
      targetPort: 9090
      nodePort: 30080
    - name: https
      port: 9443
      protocol: TCP
      targetPort: 9443
  selector:
    app.kubernetes.io/instance: minio-operator-console
    app.kubernetes.io/name: operator
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

      • operator.yaml:

        kubectl get deployment minio-operator -n minio-operator -o yaml > operator.yaml
yq -i -e '.spec.replicas |= 1' operator.yaml

        The file contents resemble the following:

        Example operator.yaml file

        apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
    meta.helm.sh/release-name: minio-operator
    meta.helm.sh/release-namespace: minio-operator
  creationTimestamp: "2023-05-11T14:57:43Z"
  generation: 1
  labels:
    app.kubernetes.io/instance: minio-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: operator
    app.kubernetes.io/version: v5.0.4
    helm.sh/chart: operator-5.0.4
  name: minio-operator
  namespace: minio-operator
  resourceVersion: "947"
  uid: f395171e-d17c-4645-9854-3dd92f23be59
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: minio-operator
      app.kubernetes.io/name: operator
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: minio-operator
        app.kubernetes.io/name: operator
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: name
                    operator: In
                    values:
                      - minio-operator
              topologyKey: kubernetes.io/hostname
      containers:
        - args:
            - controller
          image: quay.io/minio/operator:v5.0.4
          imagePullPolicy: IfNotPresent
          name: operator
          resources:
            requests:
              cpu: 200m
              ephemeral-storage: 500Mi
              memory: 256Mi
          securityContext:
            runAsGroup: 1000
            runAsNonRoot: true
            runAsUser: 1000
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1000
        runAsGroup: 1000
        runAsNonRoot: true
        runAsUser: 1000
      serviceAccount: minio-operator
      serviceAccountName: minio-operator
      terminationGracePeriodSeconds: 30
status:
  conditions:
    - lastTransitionTime: "2023-05-11T14:57:43Z"
      lastUpdateTime: "2023-05-11T14:57:43Z"
      message: Deployment does not have minimum availability.
      reason: MinimumReplicasUnavailable
      status: "False"
      type: Available
    - lastTransitionTime: "2023-05-11T14:57:43Z"
      lastUpdateTime: "2023-05-11T14:57:44Z"
      message: ReplicaSet "minio-operator-674cf5cf78" is progressing.
      reason: ReplicaSetUpdated
      status: "True"
      type: Progressing
  observedGeneration: 1
  replicas: 2
  unavailableReplicas: 2
  updatedReplicas: 2

      • console-secret.yaml:

        Create a console-secret.yaml file with the following contents:

        apiVersion: v1
kind: Secret
metadata:
  name: console-sa-secret
  namespace: minio-operator
  annotations:
    kubernetes.io/service-account.name: console-sa
type: kubernetes.io/service-account-token

    2. Apply the configuration to your deployment with kubectl apply:

      kubectl apply -f service.yaml
kubectl apply -f operator.yaml
kubectl apply -f console-secret.yaml

  4. To verify the installation, run the following command:

    kubectl get all --namespace minio-operator

    If you initialized the Operator with a custom namespace, replace minio-operator with that namespace.

    The output resembles the following:

    NAME                                  READY   STATUS    RESTARTS   AGE
pod/console-59b769c486-cv7zv          1/1     Running   0          81m
pod/minio-operator-7976b4df5b-rsskl   1/1     Running   0          81m

NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/console    ClusterIP   10.105.218.94    <none>        9090/TCP,9443/TCP   81m
service/operator   ClusterIP   10.110.113.146   <none>        4222/TCP,4233/TCP   81m

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/console          1/1     1            1           81m
deployment.apps/minio-operator   1/1     1            1           81m

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/console-59b769c486          1         1         1       81m
replicaset.apps/minio-operator-7976b4df5b   1         1         1       81m

  5. Connect to the Operator Console

    To connect to the Console, first retrieve the JSON Web Token (JWT) for your deployment and then forward the Console port.

    1. Retrieve the JWT

      The Operator Console uses a JWT to authenticate and log in. The following commands retrieve the token for your deployment:

      SA_TOKEN=$(kubectl -n minio-operator  get secret console-sa-secret -o jsonpath="{.data.token}" | base64 --decode)
echo $SA_TOKEN

    2. Forward the Operator Console port to allow access from another host.

      The following command temporarily forwards the Console to port 9090:

      kubectl --namespace minio-operator port-forward svc/console 9001:9001

      This command forwards the pod port 9090 to the matching port on the local machine while active in the shell. The kubectl port-forward command only functions while active in the shell session. Terminating the session closes the ports on the local machine.

    3. Access the Console by navigating to http://localhost:9001 in a browser and login with the JWT.

    Note

    Some Kubernetes deployments may experience issues with timeouts during port-forwarding operations with the Operator Console. Select the NodePorts section to view instructions for alternative access. You can alternatively configure your preferred Ingress to grant access to the Operator Console service. See https://github.com/kubernetes/kubectl/issues/1368 for more information.

NodePorts

Use the following command to identify the NodePorts <concepts/services-networking/service/#type-nodeport> configured for the Operator Console. If your local host does not have the jq utility installed, you can run the first command and locate the spec.ports section of the output.

kubectl get svc/console -n minio-operator -o json | jq -r '.spec.ports'

The output resembles the following:

[
   {
      "name": "http",
      "nodePort": 31055,
      "port": 9090,
      "protocol": "TCP",
      "targetPort": 9090
   },
   {
      "name": "https",
      "nodePort": 31388,
      "port": 9443,
      "protocol": "TCP",
      "targetPort": 9443
   }
]

Use the http or https port depending on whether you deployed the Operator with Console TLS enabled via kubectl minio init --console-tls.

Append the nodePort value to the externally-accessible IP address of a worker node in your Kubernetes cluster.

You can now deploy and manage MinIO Tenants using the Operator Console <deploy-minio-distributed>.

You can also deploy a tenant using Helm Charts <deploy-tenant-helm>.