Addresses #1273 # Summary This pass does three things: 1. Updates all tutorials related to Operator/Tenant deployment for Kustomize and Helm 2. Removes references to Operator Console + updates to reference Kustomize/Helm wherever possible 3. Slightly tidies up old or dangling references This pass does not do these things: - Link out heavily to Kubernetes docs (for later) - Clean up organization (singleplat build handles this) - Addresses OpenShift, Rancher, etc. --------- Co-authored-by: Andrea Longo <feorlen@users.noreply.github.com> Co-authored-by: Allan Roger Reid <allanrogerreid@gmail.com> Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com>
5.8 KiB
Expand a MinIO Tenant
minio
Table of Contents
This procedure documents expanding the available storage capacity of an existing MinIO tenant by deploying an additional pool of MinIO pods in the Kubernetes infrastructure.
Important
The MinIO Operator Console is deprecated and removed in Operator 6.0.0.
See minio-k8s-modify-minio-tenant
for instructions on
migrating Tenants installed via the Operator Console to
Kustomization.
Prerequisites
MinIO Kubernetes Operator
This procedure on this page requires a valid installation of the MinIO Kubernetes Operator and assumes the local host has a matching installation of the MinIO Kubernetes Operator. This procedure assumes the latest stable Operator, version |operator-version-stable|.
See deploy-operator-kubernetes
for complete documentation
on deploying the MinIO Operator.
Available Worker Nodes
MinIO deploys additional minio server <minio.server>
pods as part of the
new Tenant pool. The Kubernetes cluster must have sufficient
available worker nodes on which to schedule the new pods.
The MinIO Operator provides configurations for controlling pod affinity and anti-affinity to direct scheduling to specific workers.
Persistent Volumes
not eks
MinIO can use any Kubernetes Persistent Volume (PV) <concepts/storage/persistent-volumes>
that supports the ReadWriteOnce <concepts/storage/persistent-volumes/#access-modes>
access mode. MinIO's consistency guarantees require the exclusive
storage access that ReadWriteOnce
provides.
For Kubernetes clusters where nodes have Direct Attached Storage,
MinIO strongly recommends using the DirectPV CSI driver.
DirectPV provides a distributed persistent volume manager that can
discover, format, mount, schedule, and monitor drives across Kubernetes
nodes. DirectPV addresses the limitations of manually provisioning and
monitoring local persistent volumes <concepts/storage/volumes/#local>
.
eks
MinIO Tenants on EKS must use the EBS CSI Driver <kubernetes-sigs/aws-ebs-csi-driver>
to provision the necessary underlying persistent volumes. MinIO strongly
recommends using SSD-backed EBS volumes for best performance. For more
information on EBS resources, see EBS Volume
Types.
Procedure
The MinIO Operator supports expanding a MinIO Tenant by adding additional pools.
Kustomization
Review the Kustomization object which describes the Tenant object (
tenant.yaml
).The
spec.pools
array describes the current pool topology.Add a new entry to the
spec.pools
array.The new pool must reflect your intended combination of Worker nodes, volumes per server, storage class, and affinity/scheduler settings. See
minio-operator-crd
for more complete documentation on Pool-related configuration settings.Apply the updated Tenant configuration
Use the
kubectl apply
command to update the Tenant:kubectl apply -k ~/kustomization/TENANT-NAME
Modify the path to the Kustomization directory to match your local configuration.
Helm
Review the Helm
values.yaml
file.The
tenant.pools
array describes the current pool topology.Add a new entry to the
tenant.pools
array.The new pool must reflect your intended combination of Worker nodes, volumes per server, storage class, and affinity/scheduler settings. See
minio-tenant-chart-values
for more complete documentation on Pool-related configuration settings.Apply the updated Tenant configuration
Use the
helm upgrade
command to update the Tenant:helm upgrade TENANT-NAME minio-operator/tenant -f values.yaml -n TENANT-NAMESPACE
The command above assumes use of the MinIO Operator Chart repository. If you installed the Chart manually or by using a different repository name, specify that chart or name in the command.
Replace
TENANT-NAME
andTENANT-NAMESPACE
with the name and namespace of the Tenant respectively. You can usehelm list -n TENANT-NAMESPACE
to validate the Tenant name.
You can use the
kubectl get events -n TENANT-NAMESPACE --watch
to monitor
the progress of expansion. The MinIO Operator updates services to route
connections appropriately across the new nodes. If you use customized
services, routes, ingress, or similar Kubernetes network components, you
may need to update those components for the new pod hostname ranges.
Decommission a Tenant Server Pool
Decommissioning a server pool involves three steps:
- Run the
mc admin decommission start
command against the Tenant - Wait until decommissioning completes
- Modify the Tenant YAML to remove the decommissioned pool
When removing the Tenant pool, ensure the
spec.pools.[n].name
fields have values for all remaining
pools.
Important
You cannot reuse the same pool name or hostname sequence for a decommissioned pool.