From 265af648e2cd643f55f14b422ab68c9dda7f9cd6 Mon Sep 17 00:00:00 2001 From: Ravind Kumar Date: Tue, 11 Feb 2025 14:00:44 -0500 Subject: [PATCH] Removing references to openshift container platform, bumping CRD (#1415) # Summary We removed support for deploying via OpenShift operatorhub / marketplace a few months back. The burden of maintaining the certifications/process required was sufficient to pull that feature set out moving forward and migrate it to AIStor as part of a more complete enterprise offering. For now we need to remove references to the operator hub / marketplace. It is possible that the Openshift Kubernetes Engine may work w/ our stock kubernetes installation paths, but we do not have resources to test. So for the immediate term, we're just going to pull whats outdated. Also throwing in a fixup to the CRDs - I ran into a weird issue where I had to `pip3 install asciidoc` before the sync-operator-crd-docs job worked, am looking into that separately. Staged: http://192.241.195.202:9000/staging/openshift-fixup/openshift/index.html --- source/includes/k8s/deploy-operator.rst | 8 + source/includes/k8s/ext-tenant-crd.md | 77 ++++- source/includes/k8s/operator-values.yaml | 14 +- source/includes/k8s/tenant-values.yaml | 55 +++- .../openshift/deploy-minio-on-openshift.rst | 295 ------------------ source/index.rst | 7 +- .../deploy-minio-tenant.rst | 20 +- source/operations/installation.rst | 6 +- 8 files changed, 139 insertions(+), 343 deletions(-) delete mode 100644 source/includes/openshift/deploy-minio-on-openshift.rst diff --git a/source/includes/k8s/deploy-operator.rst b/source/includes/k8s/deploy-operator.rst index a5991c72..44e1365f 100644 --- a/source/includes/k8s/deploy-operator.rst +++ b/source/includes/k8s/deploy-operator.rst @@ -27,6 +27,14 @@ See the MinIO Operator :minio-git:`CRD Reference `. +.. cond:: openshift + + .. important:: + + Support for deploying the MinIO Operator via the RedHat Marketplace or OperatorHub was removed in 2024. + |subnet| customers can open an issue for further clarification and instructions on migrating to `AIStor `__. + + This documentation provides guidance through the general method of operator installation onto Kubernetes infrastructure. MinIO Operator Components ------------------------- diff --git a/source/includes/k8s/ext-tenant-crd.md b/source/includes/k8s/ext-tenant-crd.md index 97852e73..a6d8faee 100644 --- a/source/includes/k8s/ext-tenant-crd.md +++ b/source/includes/k8s/ext-tenant-crd.md @@ -981,6 +981,45 @@ should keep running this Pool without a Security Context

+## PoolsMetadata + +PoolsMetadata (`poolsMetadata`) defines custom labels and annotations +for the MinIO pool stateful sets / pods. + +- [TenantSpec](#tenantspec) + + ++++ + + + + + + + + + + + + + + + + +
FieldDescription

labels +object (keys:string, values:string)

Optional
+

+

If provided, append these labels to the MinIO statefulset / +pods

annotations +object (keys:string, values:string)

Optional
+

+

If provided, append these annotations to the MinIO statefulset / +pods

+ ## ServiceMetadata ServiceMetadata (`serviceMetadata`) defines custom labels and @@ -1033,6 +1072,22 @@ style="text-align: left;">

consoleServiceAnnotations

If provided, append these annotations to the Console service

+ +

kesServiceLabels +object (keys:string, values:string)

+

Optional
+

+

If provided, append these labels to the KES service

+ + +

kesServiceAnnotations +object (keys:string, values:string)

+

Optional
+

+

If provided, append these annotations to the KES service

+ @@ -1673,6 +1728,16 @@ href="#servicemetadata">ServiceMetadata

and/or Console service.

+

poolsMetadata +PoolsMetadata

+

Optional
+

+

Specify custom labels and annotations to append to all pool +statefulsets and pods.

+ +

users LocalObjectReference @@ -1704,7 +1769,7 @@ policy by default. You can change the assigned policy after the Tenant starts.

- +

buckets Bucket @@ -1714,7 +1779,7 @@ array

Create buckets when creating a new tenant. Skip if bucket with given name already exists

- +

logging Logging

@@ -1722,7 +1787,7 @@ href="#logging">Logging

Enable JSON, Anonymous logging for MinIO tenants.

- +

configuration - +

initContainers

Add custom initContainers to StatefulSet

- +

additionalVolumes

rules for the corresponding volumes and volume mounts. We will not test this rule, k8s will show the result.

- +

additionalVolumeMounts `__ to perform setup or configuration tasks before the main Tenant pods starts. + # + # Example of init container which waits for idenity provider to be reachable before starting MinIO Tenant: + # + # .. code-block:: yaml + # + # initContainers: + # - name: wait-for-idp + # image: busybox + # command: + # - sh + # - -c + # - | + # URL="https://idp-url" + # echo "Checking IdP reachability (${URL})" + # until $(wget -q -O "/dev/null" ${URL}) ; do + # echo "IdP (${URL}) not reachable. Waiting to be reachable..." + # sleep 5 + # done + # echo "IdP (${URL}) reachable. Starting MinIO..." + # + initContainers: [ ] + ### # The Kubernetes `Scheduler `__ to use for dispatching Tenant pods. # # Specify an empty dictionary ``{}`` to dispatch pods with the default scheduler. @@ -62,12 +86,24 @@ tenant: # # name: myminio-env-configuration # accessKey: minio - # secretKey: minio123 + # secretKey: minio123 # configSecret: name: myminio-env-configuration accessKey: minio secretKey: minio123 + #existingSecret: true + + ### + # Metadata that will be added to the statefulset and pods of all pools + poolsMetadata: + ### + # Specify `annotations `__ to associate to Tenant pods. + annotations: { } + ### + # Specify `labels `__ to associate to Tenant pods. + labels: { } + ### # If this variable is set to true, then enable the usage of an existing Kubernetes secret to set environment variables for the Tenant. # The existing Kubernetes secret name must be placed under .tenant.configuration.name e.g. existing-minio-env-configuration @@ -113,6 +149,9 @@ tenant: # Specify `storageAnnotations `__ to associate to PVCs. storageAnnotations: { } ### + # Specify `storageLabels `__ to associate to PVCs. + storageLabels: { } + ### # Specify `annotations `__ to associate to Tenant pods. annotations: { } ### @@ -253,8 +292,8 @@ tenant: # .. code-block:: yaml # # - name: my-minio-bucket - # objectLock: false # optional - # region: us-east-1 # optional + # objectLock: false # optional + # region: us-east-1 # optional buckets: [ ] ### # Array of Kubernetes secrets from which the Operator generates MinIO users during tenant provisioning. @@ -347,14 +386,14 @@ tenant: # # Image from tag (original behavior), for example: # # image: # # repository: quay.io/minio/kes - # # tag: 2024-08-16T14-39-28Z + # # tag: 2024-11-25T13-44-31Z # # Image from digest (added after original behavior), for example: # # image: # # repository: quay.io/minio/kes@sha256 # # digest: fb15af611149892f357a8a99d1bcd8bf5dae713bd64c15e6eb27fbdb88fc208b # image: # repository: quay.io/minio/kes - # tag: 2024-08-16T14-39-28Z + # tag: 2024-11-25T13-44-31Z # pullPolicy: IfNotPresent # env: [ ] # replicas: 2 diff --git a/source/includes/openshift/deploy-minio-on-openshift.rst b/source/includes/openshift/deploy-minio-on-openshift.rst deleted file mode 100644 index 13bb7885..00000000 --- a/source/includes/openshift/deploy-minio-on-openshift.rst +++ /dev/null @@ -1,295 +0,0 @@ - -.. _deploy-operator-openshift: - -========================================= -Deploy MinIO Operator on RedHat OpenShift -========================================= - -.. default-domain:: minio - -.. contents:: Table of Contents - :local: - :depth: 1 - -Overview --------- - -Red Hat® OpenShift® is an enterprise-ready Kubernetes container platform with full-stack automated operations to manage hybrid cloud, multi-cloud, and edge deployments. -OpenShift includes an enterprise-grade Linux operating system, container runtime, networking, monitoring, registry, and authentication and authorization solutions. - -You can deploy the MinIO Kubernetes Operator through the :openshift-docs:`Red Hat® OpenShift® Container Platform 4.8+ `. -You can deploy and manage MinIO Tenants through OpenShift after deploying the MinIO Operator. -This procedure includes instructions for the following deployment paths: - -- Purchase and Deploy MinIO through the `RedHat Marketplace `__. -- Deploy MinIO through the OpenShift `OperatorHub `__ - -After deploying the MinIO Operator into your OpenShift cluster, you can create and manage MinIO Tenants through the :openshift-docs:`OperatorHub ` user interface. - -This documentation assumes familiarity with all referenced Kubernetes and OpenShift concepts, utilities, and procedures. -While this documentation *may* provide guidance for configuring or deploying Kubernetes-related or OpenShift-related resources on a best-effort basis, it is not a replacement for the official :kube-docs:`Kubernetes Documentation <>` and :openshift-docs:`OpenShift Container Platform 4.8+ Documentation `. - -Prerequisites -------------- - -RedHat OpenShift 4.8+ -~~~~~~~~~~~~~~~~~~~~~ - -The MinIO Kubernetes Operator is available starting with `OpenShift 4.8+ `__. - -Red Hat Marketplace installation requires registration of the OpenShift cluster with the Marketplace for the necessary namespaces. -See `Register OpenShift cluster with Red Hat Marketplace `__ for complete instructions. - -For older versions of OpenShift, use the generic :ref:`deploy-operator-kubernetes` procedure. - -Administrator Access -~~~~~~~~~~~~~~~~~~~~ - -Installation of operators through the Red Hat Marketplace and the Operator Hub is restricted to OpenShift cluster administrators (``cluster-admin`` privileges). -This procedure requires logging into the Marketplace and/or OpenShift with an account that has those privileges. - -OpenShift ``oc`` CLI -~~~~~~~~~~~~~~~~~~~~ - -:openshift-docs:`Download and Install ` the OpenShift :abbr:`CLI (command-line interface)` ``oc`` for use in this procedure. - - -Procedure ---------- - -1) Access the MinIO Operator Installation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Select the tab that corresponds to your preferred installation method: - -.. tab-set:: - - .. tab-item:: Red Hat OperatorHub - - Log into the OpenShift Web Console as a user with ``cluster-admin`` privileges. - - From the :guilabel:`Administrator` panel, select :guilabel:`Operators`, then :guilabel:`OperatorHub`. - - From the :guilabel:`OperatorHub` page, type "MinIO" into the :guilabel:`Filter` text entry. Select the :guilabel:`MinIO Operator` tile from the search list. - - .. image:: /images/openshift/minio-openshift-select-minio.png - :align: center - :width: 90% - :class: no-scaled-link - :alt: From the OperatorHub, search for MinIO, then select the MinIO Tile. - - Select the :guilabel:`MinIO Operator` tile, then click :guilabel:`Install` to begin the installation. - - .. tab-item:: Red Hat Marketplace - - Open the `MinIO Red Hat Marketplace listing `__ in your browser. - Click :guilabel:`Login` to log in with your Red Hat Marketplace account. - - After logging in, click :guilabel:`Purchase` to purchase the MinIO Operator for your account. - - After completing the purchase, click :guilabel:`Workplace` from the top navigation and select :guilabel:`My Software`. - - .. image:: /images/openshift/minio-openshift-marketplace-my-software.png - :align: center - :width: 90% - :class: no-scaled-link - :alt: From the Red Hat Marketplace, select Workplace, then My Software - - Click :guilabel:`MinIO Hybrid Cloud Object Storage` and select :guilabel:`Install Operator` to start the Operator Installation procedure in OpenShift. - -2) Configure and Deploy the Operator -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :guilabel:`Install Operator` page provides a walkthrough for configuring the MinIO Operator installation. - -.. image:: /images/openshift/minio-openshift-operator-installation.png - :align: center - :width: 90% - :class: no-scaled-link - :alt: Complete the Operator Installation Walkthrough - -- For :guilabel:`Update channel`, select any of the available options. - -- For :guilabel:`Installation Mode`, select :guilabel:`All namespaces on the cluster` - -- For :guilabel:`Installed Namespace`, select :guilabel:`openshift-operators` - -- For :guilabel:`Approval Strategy`, select the approval strategy of your choice. - -See the :openshift-docs:`Operator Installation Documentation ` :guilabel:`Step 5` for complete descriptions of each displayed option. - -Click :guilabel:`Install` to start the installation procedure. -The web console displays a widget for tracking the installation progress. - -.. image:: /images/openshift/minio-openshift-operator-installation-progress.png - :align: center - :width: 70% - :class: no-scaled-link - :alt: Wait for Installation to Complete. - -Once installation completes, click :guilabel:`View Operator` to view the MinIO Operator page. - -3) Configure TLS Certificates -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you have installed the MinIO Operator from Red Hat OperatorHub, the installation process also configures the :openshift-docs:`OpenShift Service CA Operator `. -This Operator manages the TLS certificates required to access MinIO Tenants. -It automatically renews and rotates the certificates 13 months before expiration. -No additional action is required. - -For Operator installations deployed by other methods, configure the :openshift-docs:`Service CA certificates ` manually. -See the dropdowns below for details. - -.. dropdown:: OpenShift Service CA Certificate configuration - - To manually enable the ``service-ca`` Operator to manage TLS certificates: - - #. Use the following :openshift-docs:`oc ` command to edit the deployment: - - .. code-block:: shell - :class: copyable - - oc edit deployment minio-operator -n minio-operator - - If needed, replace ``minio-operator`` with the name and namespace of your deployment. - ``oc edit`` opens the deployment configuration file in an editor. - - #. In the ``spec`` section, add the highlighted MinIO Operator :ref:`environment variables `: - - .. code-block:: shell - :class: copyable - :emphasize-lines: 5-8 - - containers: - - args: - - controller - env: - - name: MINIO_CONSOLE_TLS_ENABLE - value: 'on' - - name: MINIO_OPERATOR_RUNTIME - value: OpenShift - - #. In the ``volumes`` section, add the following volumes and volume mounts: - - - ``sts-tls`` - - ``openshift-service-ca`` - - ``openshift-csr-signer-ca`` - - The added volume configuration resembles the following: - - .. code-block:: shell - :class: copyable - - volumes: - - name: sts-tls - projected: - sources: - - secret: - name: sts-tls - items: - - key: tls.crt - path: public.crt - - key: tls.key - path: private.key - optional: true - defaultMode: 420 - - name: openshift-service-ca - configMap: - name: openshift-service-ca.crt - items: - - key: service-ca.crt - path: service-ca.crt - defaultMode: 420 - optional: true - - name: openshift-csr-signer-ca - projected: - sources: - - secret: - name: openshift-csr-signer-ca - items: - - key: tls.crt - path: tls.crt - optional: true - defaultMode: 420 - volumeMounts: - - name: openshift-service-ca - mountPath: /tmp/service-ca - - name: openshift-csr-signer-ca - mountPath: /tmp/csr-signer-ca - - name: sts-tls - mountPath: /tmp/sts - -.. dropdown:: OpenShift Service CA Certificate for Helm deployments - - For Helm deployments on OpenShift, add the following :ref:`environment variables ` and volumes to the ``values.yaml`` in the Operator Helm chart before deploying. - - The added YAML configuration for the ``operator`` pod resembles the following: - - .. code-block:: - :class: copyable - - operator: - env: - - name: MINIO_OPERATOR_RUNTIME - value: "OpenShift" - - name: MINIO_CONSOLE_TLS_ENABLE - value: "on" - - volumes: - - name: sts-tls - projected: - sources: - - secret: - name: sts-tls - items: - - key: tls.crt - path: public.crt - - key: tls.key - path: private.key - optional: true - defaultMode: 420 - - name: openshift-service-ca - configMap: - name: openshift-service-ca.crt - items: - - key: service-ca.crt - path: service-ca.crt - defaultMode: 420 - optional: true - - name: openshift-csr-signer-ca - projected: - sources: - - secret: - name: openshift-csr-signer-ca - items: - - key: tls.crt - path: tls.crt - optional: true - defaultMode: 420 - volumeMounts: - - name: openshift-service-ca - mountPath: /tmp/service-ca - - name: openshift-csr-signer-ca - mountPath: /tmp/csr-signer-ca - - name: sts-tls - mountPath: /tmp/sts - - -4) Open the MinIO Operator Interface -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can find the MinIO Operator Interface from the :guilabel:`Operators` left-hand navigation header - -1. Go to :guilabel:`Operators`, then :guilabel:`Installed Operators`. - -2. For the :guilabel:`Project` dropdown, select :guilabel:`openshift-operators`. - -3. Select :guilabel:`MinIO Operators` from the list of installed operators. - The :guilabel:`Status` column must read :guilabel:`Success` to access the Operator interface. - - -5) Next Steps -~~~~~~~~~~~~~ - -After deploying the MinIO Operator, you can create a new MinIO Tenant. -To deploy a MinIO Tenant using OpenShift, see :ref:`deploy-minio-tenant-redhat-openshift`. diff --git a/source/index.rst b/source/index.rst index c962d141..bfbca86d 100644 --- a/source/index.rst +++ b/source/index.rst @@ -77,7 +77,12 @@ MinIO is built to deploy anywhere - public or private cloud, baremetal infrastru .. cond:: openshift - This site documents Operations, Administration, and Development of MinIO deployments on OpenShift 4.7+ through the :openshift-docs:`Red Hat® OpenShift® Container Platform 4.7+ ` for the latest stable version of the MinIO Operator: |operator-version-stable|. + This site documents Operations, Administration, and Development of MinIO deployments on Red Hat Kubernetes distributions for the latest stable version of the MinIO Operator: |operator-version-stable|. + + .. important:: + + Support for deploying the MinIO Operator via the RedHat Marketplace or OperatorHub was removed in 2024. + |subnet| customers can open an issue for further clarification and instructions on migrating to `AIStor `__. .. cond:: eks diff --git a/source/operations/install-deploy-manage/deploy-minio-tenant.rst b/source/operations/install-deploy-manage/deploy-minio-tenant.rst index a9348b2c..a59db27d 100644 --- a/source/operations/install-deploy-manage/deploy-minio-tenant.rst +++ b/source/operations/install-deploy-manage/deploy-minio-tenant.rst @@ -19,13 +19,7 @@ Deploy a MinIO Tenant :local: :depth: 1 -.. cond:: openshift - - This procedure documents deploying a MinIO Tenant through OpenShift 4.7+ using the OpenShift Web Console and the MinIO Kubernetes Operator. - -.. cond:: k8s and not openshift - - This procedure documents deploying a MinIO Tenant onto a stock Kubernetes cluster using either Kustomize or MinIO's Helm Charts. +This procedure documents deploying a MinIO Tenant onto a stock Kubernetes cluster using either Kustomize or MinIO's Helm Charts. .. screenshot temporarily removed @@ -68,18 +62,6 @@ See :ref:`deploy-operator-kubernetes` for complete documentation on deploying th MinIO **strongly recommends** upgrading Kubernetes clusters running with `End-Of-Life API versions `__. -.. cond:: openshift - - OpenShift 4.7+ and ``oc`` CLI Tool - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - This procedure assumes installation of the MinIO Operator using the OpenShift 4.7+ and the OpenShift OperatorHub. - - This procedure assumes your local machine has the OpenShift ``oc`` CLI tool installed and configured for access to the OpenShift Cluster. - :openshift-docs:`Download and Install ` the OpenShift :abbr:`CLI (command-line interface)` ``oc`` for use in this procedure. - - See :ref:`deploy-operator-openshift` for more complete instructions. - .. cond:: openshift Check Security Context Constraints diff --git a/source/operations/installation.rst b/source/operations/installation.rst index 1f768d3a..55c6cd7d 100644 --- a/source/operations/installation.rst +++ b/source/operations/installation.rst @@ -6,10 +6,6 @@ .. include:: /includes/container/installation.rst -.. cond:: openshift - - .. include:: /includes/openshift/deploy-minio-on-openshift.rst - .. cond:: eks .. include:: /includes/eks/deploy-minio-on-elastic-kubernetes-service.rst @@ -22,6 +18,6 @@ .. include:: /includes/aks/deploy-minio-on-azure-kubernetes-service.rst -.. cond:: k8s and not (openshift or eks or gke or aks) +.. cond:: k8s and not (eks or gke or aks) .. include:: /includes/k8s/deploy-operator.rst