Addressing hardware checklist critical feedback (#1080)

We're getting a lot of repeated requests during
inbounds on sizing VMs specifically.

Our baremetal and K8s checklists are slightly generic to this and imply
a baseline that may be uncommon within a virtualized context

This attempts to bring together a slightly wider band while emphasizing
that our high-watermark is intended for high performance, while the low
watermark is a bare minimum.


---------

Co-authored-by: Andrea Longo <feorlen@users.noreply.github.com>
Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com>

source/includes/common/common-checklist.rst

@@ -0,0 +1,82 @@
+.. start-linux-hardware-checklist
+
+.. list-table::
+   :header-rows: 1
+   :widths: 5 45 25 25
+   :width: 100%
+
+   * - 
+     - Description
+     - Minimum
+     - Recommended
+
+   * - :octicon:`circle`
+     - Dedicated Baremetal or Virtual Hosts ("hosts").
+
+     - 4 dedicated hosts
+     - 8+ dedicated hosts
+
+   * - :octicon:`circle`
+     - :ref:`Dedicated locally-attached drives for each host <minio-hardware-checklist-storage>`.
+
+     - 4 drives per MinIO Server
+     - 8+ drives per MinIO Server
+
+   * - :octicon:`circle`
+     - :ref:`High speed network infrastructure <minio-hardware-checklist-network>`.
+     - 25GbE
+     - 100GbE
+
+   * - :octicon:`circle`
+     - Server-grade CPUs with support for modern SIMD instructions (AVX-512), such as Intel® Xeon® Scalable or better.
+     - 8 CPU/socket or vCPU per host
+     - 16+ CPU/socket or vCPU per host
+
+   * - :octicon:`circle`
+     - :ref:`Available memory to meet or exceed per-server usage <minio-hardware-checklist-memory>` by a reasonable buffer.
+     - 32GB of available memory per host
+     - 128GB+ of available memory per host
+
+.. end-linux-hardware-checklist
+
+.. start-k8s-hardware-checklist
+
+.. list-table::
+   :header-rows: 1
+   :widths: 5 55 20 20
+   :width: 100%
+
+   * - 
+     - Description
+     - Minimum
+     - Recommended
+
+   * - :octicon:`circle`
+     - Kubernetes worker nodes to exclusively service the MinIO Tenant.
+
+     - 4 workers per Tenant
+     - 8+ workers per Tenant
+
+   * - :octicon:`circle`
+     - :ref:`Dedicated Persistent Volumes for the MinIO Tenant <minio-hardware-checklist-storage>`.
+
+     - 4 PV per MinIO Server pod
+     - 8+ PV per MinIO Server pod
+
+   * - :octicon:`circle`
+     - :ref:`High speed network infrastructure <minio-hardware-checklist-network>`.
+     - 25GbE
+     - 100GbE
+       
+
+   * - :octicon:`circle`
+     - Server-grade CPUs with support for modern SIMD instructions (AVX-512), such as Intel® Xeon® Scalable or better.
+     - 4 vCPU per MinIO Pod
+     - 8+ vCPU per MinIO Pod
+
+   * - :octicon:`circle`
+     - :ref:`Available memory to meet or exceed per-server usage <minio-hardware-checklist-memory>` by a reasonable buffer.
+     - 32GB of available memory per worker node
+     - 128GB+ of available memory per worker node
+
+.. end-k8s-hardware-checklist

source/operations/checklists/hardware.rst

@@ -37,51 +37,17 @@ The provided guidance is intended as a baseline and cannot replace |subnet| Perf
 
    See our `Reference Hardware <https://min.io/product/reference-hardware#hardware?ref-docs>`__ page for a curated selection of servers and storage components from our hardware partners.
 
-.. list-table::
+.. cond:: k8s
-   :widths: auto
-   :width: 100%
 
-   * - :octicon:`circle`
+   .. include:: /includes/common/common-checklist.rst
-     - Sufficient CPU cores to achieve performance goals for hashing (for example, for healing) and encryption
+      :start-after: start-k8s-hardware-checklist
-       
+      :end-before: end-k8s-hardware-checklist
-       MinIO recommends Single Socket Intel® Xeon® Scalable Gold CPUs (minimum 16 cores per socket).
 
-   * - :octicon:`circle`
+.. cond:: not k8s
-     - Sufficient RAM to achieve performance goals based on the number of drives and anticipated concurrent requests (see the :ref:`formula and reference table <minio-hardware-checklist-memory>`).
 
-       MinIO recommends a minimum of 128GB of memory per node for best performance.
+   .. include:: /includes/common/common-checklist.rst
-
+      :start-after: start-linux-hardware-checklist
-   * - :octicon:`circle`
+      :end-before: end-linux-hardware-checklist
-     - .. cond:: k8s
-
-          MinIO requires a *minimum* of 4 worker nodes per MinIO Tenant.
-
-          MinIO strongly recommends allocating worker nodes dedicated to servicing the MinIO Tenant.
-          Colocating multiple high-performance services on the same nodes can result in resource contention and reduced overall performance.
-
-       .. cond:: linux or container or macos or windows
-
-          MinIO recommends a *minimum* of 4 host servers per distributed deployment.
-
-          MinIO strongly recommends hardware dedicated to servicing the MinIO Tenant.
-          Colocating multiple high-performance services on the same servers can result in resource contention and reduced overall performance.
-
-   * - :octicon:`circle`
-     - .. cond:: k8s
-
-          MinIO recommends a minimum of 4 Persistent Volumes per MinIO Server pod.
-          For better performance and storage efficiency, use 8 or more PV per server.
-
-       .. cond:: linux or container or macos or windows
-
-          MinIO recommends a minimum of 4 locally attached drives per MinIO Server.
-          For better performance and storage efficiency, use 8 or more drives per server.
-
-       Use the same type of drive (NVMe, SSD, or HDD) with the same capacity across all nodes in the deployment.
-
-   * - :octicon:`circle`
-     - | 25GbE Network as a baseline 
-       | 100GbE Network for high performance
 
 .. important:: 
 
@@ -106,6 +72,8 @@ The provided guidance is intended as a baseline and cannot replace |subnet| Perf
 The minimum recommendations above reflect MinIO's experience with assisting enterprise customers in deploying on a variety of IT infrastructures while maintaining the desired SLA/SLO. 
 While MinIO may run on less than the minimum recommended topology, any potential cost savings come at the risk of decreased reliability, performance, or overall functionality.
 
+.. _minio-hardware-checklist-network:
+
 Networking
 ~~~~~~~~~~
 
@@ -119,16 +87,16 @@ This table assumes all network infrastructure components, such as routers, switc
    * - NIC Bandwidth (Gbps)
      - Estimated Aggregated Storage Throughput (GBps)
 
-   * - 10GbE
+   * - 10Gbps
      - 1.25GBps
 
-   * - 25GbE
+   * - 25Gbps
      - 3.125GBps
 
-   * - 50GbE
+   * - 50Gbps
      - 6.25GBps
 
-   * - 100GbE
+   * - 100Gbps
      - 12.5GBps
 
 Networking has the greatest impact on MinIO performance, where low per-host bandwidth artificially constrains the potential performance of the storage.
@@ -215,46 +183,62 @@ The following table provides general guidelines for allocating memory for use by
    * - More than 1 Pebibyte (Pi)
      - 128GiB
 
+.. _minio-hardware-checklist-storage:
+
 Storage
 ~~~~~~~
 
-MinIO recommends selecting the type of drive based on your performance objectives.
+.. cond:: k8s
-The following table highlights the general use case for each drive type based on cost and performance:
 
-NVMe/SSD - Hot Tier
+   MinIO recommends provisioning a storage class for each MinIO Tenant that meets the performance objectives for that tenant.
-HDD - Warm
 
-.. list-table::
+   Where possible, configure the Storage Class, CSI, or other provisioner underlying the PV to format volumes as XFS to ensure best performance.
-   :header-rows: 1
-   :widths: auto
-   :width: 100%
 
-   * - Type
+   Ensure a consistent underlying storage type (NVMe, SSD, HDD) for all PVs provisioned in a Tenant.
-     - Cost
+   
-     - Performance
+   Ensure the same presented capacity of each PV across all nodes in each Tenant :ref:`server pool <minio-intro-server-pool>`.
-     - Tier
+   MinIO limits the maximum usable size per PV to the smallest PV in the pool.
+   For example, if a pool has 15 10TB PVs and 1 1TB PV, MinIO limits the per-PV capacity to 1TB.
 
-   * - NVMe
+.. cond:: not k8s
-     - High
-     - High
-     - Hot
 
-   * - SSD
+   MinIO recommends selecting the type of drive based on your performance objectives.
-     - Balanced
+   The following table highlights the general use case for each drive type based on cost and performance:
-     - Balanced
-     - Hot/Warm
 
-   * - HDD
+   .. list-table::
-     - Low
+      :header-rows: 1
-     - Low
+      :widths: auto
-     - Cold/Archival
+      :width: 100%
 
-Use the same type of drive (NVME, SSD, HDD) with the same capacity across all nodes in a MinIO deployment.
+      * - Type
-MinIO does not distinguish drive types when using the underlying storage and does not benefit from mixed storage types.
+        - Cost
+        - Performance
+        - Tier
 
-Use the same capacity of drive across all nodes in the MinIO :ref:`server pool <minio-intro-server-pool>`. 
+      * - NVMe
-MinIO limits the maximum usable size per drive to the smallest size in the deployment.
+        - High
-For example, if a deployment has 15 10TB drives and 1 1TB drive, MinIO limits the per-drive capacity to 1TB.
+        - High
+        - Hot
+
+      * - SSD
+        - Balanced
+        - Balanced
+        - Hot/Warm
+
+      * - HDD
+        - Low
+        - Low
+        - Cold/Archival
+
+   Format drives as XFS and present them to MinIO as a :abbr:`JBOD (Just a Bunch of Disks)` array with no RAID or other pooling configurations.
+
+   Ensure a consistent drive type (NVMe, SSD, HDD) for the underyling storage. 
+   MinIO does not distinguish between storage types.
+   Mixing storage types provides no benefit to MinIO.
+
+   Use the same capacity of drive across all nodes in each MinIO :ref:`server pool <minio-intro-server-pool>`. 
+   MinIO limits the maximum usable size per drive to the smallest size in the deployment.
+   For example, if a deployment has 15 10TB drives and 1 1TB drive, MinIO limits the per-drive capacity to 1TB.
 
 Recommended Hardware Tests
 --------------------------

source/operations/install-deploy-manage/deploy-minio-tenant.rst

@@ -169,6 +169,8 @@ See :ref:`deploy-operator-kubernetes` for complete documentation on deploying th
 
    For more complete information on Azure Virtual Machine types and Storage resources, see :azure-docs:`Sizes for virtual machines in Azure <virtual-machines/sizes>` and :azure-docs:`Azure managed disk types <virtual-machines/disks-types>`
 
+.. _deploy-minio-tenant-pv:
+
 Persistent Volumes
 ~~~~~~~~~~~~~~~~~~
 
@@ -177,6 +179,7 @@ Persistent Volumes
    MinIO can use any Kubernetes :kube-docs:`Persistent Volume (PV) <concepts/storage/persistent-volumes>` that supports the :kube-docs:`ReadWriteOnce <concepts/storage/persistent-volumes/#access-modes>` access mode.
    MinIO's consistency guarantees require the exclusive storage access that ``ReadWriteOnce`` provides.
    Additionally, MinIO recommends setting a reclaim policy of ``Retain`` for the PVC :kube-docs:`StorageClass <concepts/storage/storage-classes>`.
+   Where possible, configure the Storage Class, CSI, or other provisioner underlying the PV to format volumes as XFS to ensure best performance.
 
    For Kubernetes clusters where nodes have Direct Attached Storage, MinIO strongly recommends using the `DirectPV CSI driver <https://min.io/directpv?ref=docs>`__. 
    DirectPV provides a distributed persistent volume manager that can discover, format, mount, schedule, and monitor drives across Kubernetes nodes.