mirror of
https://github.com/minio/docs.git
synced 2025-07-28 19:42:10 +03:00
Updates for minio RELEASE.2023-01-18T04-36-38Z (#711)
minio [RELEASE.2023-01-18T04-36-38Z](https://github.com/minio/minio/releases/tag/RELEASE.2023-01-18T04-36-38Z) affects the docs: - Decommissioning of multiple pools [PR #16416](https://github.com/minio/minio/pull/16416) - Automatic SUBNET license update [PR #16391](https://github.com/minio/minio/pull/16416) There is no doc issue to track this update.
This commit is contained in:
Daryl White
GitHub
@ -1,8 +1,8 @@
|
||||
.. _minio-decommissioning:
|
||||
|
||||
==========================
|
||||
Decommission a Server Pool
|
||||
==========================
|
||||
=========================
|
||||
Decommission Server Pools
|
||||
=========================
|
||||
|
||||
.. default-domain:: minio
|
||||
|
||||
@ -10,43 +10,36 @@ Decommission a Server Pool
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
Starting with :minio-release:`RELEASE.2022-01-25T19-56-04Z`, MinIO supports
|
||||
decommissioning and removing a :ref:`server pool <minio-intro-server-pool>`
|
||||
from a deployment. Decommissioning is designed for removing an older server pool
|
||||
whose hardware is no longer sufficient or performant compared to the pools in
|
||||
the deployment. MinIO automatically migrates data from the decommissioned pool
|
||||
to the remaining pools in the deployment based on the ratio of free space
|
||||
available in each pool.
|
||||
MinIO supports decommissioning and removing :ref:`server pools <minio-intro-server-pool>` from a deployment with two or more pools.
|
||||
To decommission, there must be at least one remaining pool with sufficient available space to receive the objects from the decommissioned pools.
|
||||
|
||||
During the decommissioning process, MinIO routes read operations (e.g. ``GET``,
|
||||
``LIST``, ``HEAD``) normally. MinIO routes write operations (e.g. ``PUT``,
|
||||
versioned ``DELETE``) to the remaining "active" pools in the deployment.
|
||||
Starting with ``RELEASE.2023-01-18T04-36-38Z``, MinIO supports queueing :ref:`multiple pools <minio-decommission-multiple-pools>` in a single decommission command.
|
||||
Each listed pool immediately enters a read-only status, but draining occurs one pool at a time.
|
||||
|
||||
Decommissioning is designed for removing an older server pool whose hardware is no longer sufficient or performant compared to the pools in the deployment.
|
||||
MinIO automatically migrates data from the decommissioned pools to the remaining pools in the deployment based on the ratio of free space available in each pool.
|
||||
|
||||
During the decommissioning process, MinIO routes read operations (e.g. ``GET``, ``LIST``, ``HEAD``) normally.
|
||||
MinIO routes write operations (e.g. ``PUT``, versioned ``DELETE``) to the remaining "active" pools in the deployment.
|
||||
Versioned objects maintain their ordering throughout the migration process.
|
||||
|
||||
The procedure on this page decommissions and removes a server pool from
|
||||
a :ref:`distributed <deploy-minio-distributed>` MinIO deployment with
|
||||
*at least* two server pools.
|
||||
The procedures on this page decommission and remove one or more server pools from a :ref:`distributed <deploy-minio-distributed>` MinIO deployment with *at least* two server pools.
|
||||
|
||||
.. admonition:: Decommissioning is Permanent
|
||||
:class: important
|
||||
|
||||
Once MinIO begins decommissioning a pool, it marks that pool as *permanently*
|
||||
inactive ("draining"). Cancelling or otherwise interrupting the
|
||||
decommissioning procedure does **not** restore the pool to an active
|
||||
state.
|
||||
Once MinIO begins decommissioning a pool, it marks that pool as *permanently* inactive ("draining").
|
||||
Cancelling or otherwise interrupting the decommissioning procedure does **not** restore the pool to an active state.
|
||||
Use extra caution when decommissioning multiple pools.
|
||||
|
||||
Decommissioning is a major administrative operation that requires care
|
||||
in planning and execution, and is not a trivial or 'daily' task.
|
||||
Decommissioning is a major administrative operation that requires care in planning and execution, and is not a trivial or 'daily' task.
|
||||
|
||||
`MinIO SUBNET <https://min.io/pricing?jmp=docs>`__ users can
|
||||
`log in <https://subnet.min.io/>`__ and create a new issue related to
|
||||
decommissioning. Coordination with MinIO Engineering via SUBNET can ensure
|
||||
successful decommissioning, including performance testing and health
|
||||
diagnostics.
|
||||
`MinIO SUBNET <https://min.io/pricing?jmp=docs>`__ users can `log in <https://subnet.min.io/>`__ and create a new issue related to decommissioning.
|
||||
Coordination with MinIO Engineering via SUBNET can ensure successful decommissioning, including performance testing and health diagnostics.
|
||||
|
||||
Community users can seek support on the `MinIO Community Slack <https://slack.min.io>`__.
|
||||
Community Support is best-effort only and has no SLAs around responsiveness.
|
||||
|
||||
Community users can seek support on the `MinIO Community Slack
|
||||
<https://slack.min.io>`__. Community Support is best-effort only and has
|
||||
no SLAs around responsiveness.
|
||||
|
||||
.. _minio-decommissioning-prereqs:
|
||||
|
||||
@ -354,3 +347,184 @@ to restart the MinIO service:
|
||||
|
||||
Once the deployment is online, use :mc:`mc admin info` to confirm the
|
||||
uptime of all remaining servers in the deployment.
|
||||
|
||||
.. _minio-decommission-multiple-pools:
|
||||
|
||||
Decommission Multiple Server Pools
|
||||
----------------------------------
|
||||
|
||||
.. versionchanged:: RELEASE.2023-01-18T04-36-38Z
|
||||
|
||||
You can start the decommission process for multiple server pools when issuing a decommission command.
|
||||
|
||||
After entering the command:
|
||||
|
||||
- MinIO immediately stops write access to all pools to be decommissioned.
|
||||
- Decommissioning happens one pool at a time.
|
||||
- Each pool completes the decommission draining process before MinIO begins draining the next pool.
|
||||
|
||||
To decommission multiple server pools from one command, add the full description of each server pool to decommission as a comma-separated list.
|
||||
|
||||
All other considerations about decommissioning apply when performing the process on multiple servers.
|
||||
|
||||
- Decommissioning is permanent.
|
||||
- Once you mark the pools as decommissioned, you **cannot** restore them.
|
||||
- Confirm you select the intended pools.
|
||||
|
||||
1) Review the MinIO Deployment Topology
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The :mc:`mc admin decommission` command returns a list of all pools in the MinIO deployment:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc admin decommission status myminio
|
||||
|
||||
The command returns output similar to the following:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
┌─────┬────────────────────────────────────────────────────────────────┬──────────────────────────────────┬────────┐
|
||||
│ ID │ Pools │ Capacity │ Status │
|
||||
│ 1st │ https://minio-{01...04}.example.com:9000/mnt/disk{1...4}/minio │ 10 TiB (used) / 10 TiB (total) │ Active │
|
||||
│ 2nd │ https://minio-{05...08}.example.com:9000/mnt/disk{1...4}/minio │ 95 TiB (used) / 100 TiB (total) │ Active │
|
||||
│ 3rd │ https://minio-{09...12}.example.com:9000/mnt/disk{1...4}/minio │ 40 TiB (used) / 500 TiB (total) │ Active │
|
||||
│ 4th │ https://minio-{13...16}.example.com:9000/mnt/disk{1...4}/minio │ 0 TiB (used) / 500 TiB (total) │ Active │
|
||||
└─────┴────────────────────────────────────────────────────────────────┴──────────────────────────────────┴────────┘
|
||||
|
||||
The example deployment above has three pools.
|
||||
Each pool has four servers with four drives each.
|
||||
|
||||
Identify the target pool for decommissioning and review the current capacity.
|
||||
The remaining pools in the deployment *must* have sufficient total capacity to migrate all object stored in the decommissioned pool.
|
||||
|
||||
In the example above, the deployment has 1110TiB total storage with 145TiB used.
|
||||
|
||||
- The first pool (``minio-{01...04}``) is the first decommissioning target, as it was provisioned when the MinIO deployment was created and is completely full.
|
||||
- The second pool (``minio-{05...08}``) is the second decommissioning target, as it was also provisioned when the MinIO deployment was created and is nearly full.
|
||||
- The fourth pool (``minio-{13...16}``) is a newly added pool with new hardware from a completed server expansion.
|
||||
|
||||
The third and fourth pools can absorb all objects stored on the first pool without significantly impacting total available storage.
|
||||
|
||||
.. important::
|
||||
|
||||
Complete any server expansion to add new storage resources _before_ beginning a decommission process.
|
||||
|
||||
2) Start the Decommissioning Process
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. admonition:: Decommissioning is Permanent
|
||||
:class: warning
|
||||
|
||||
Once MinIO begins decommissioning the pools, it marks those pools as *permanently* inactive ("draining").
|
||||
Cancelling or otherwise interrupting the decommissioning procedure does **not** restore the pools to an active state.
|
||||
|
||||
Review and validate that you are decommissioning the correct pools *before* running the following command.
|
||||
|
||||
Use the :mc-cmd:`mc admin decommission start` command to begin decommissioning the target pool.
|
||||
Specify the :ref:`alias <alias>` of the deployment and a comma-separated list of the full description of each pool to decommission, including all hosts, disks, and file paths.
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc admin decommission start myminio/ https://minio-{01...04}.example.net:9000/mnt/disk{1...4}/minio,https://minio-{05...08}.example.net:9000/mnt/disk{1...4}/minio
|
||||
|
||||
The example command begins decommissioning the two listed matching server pools on the ``myminio`` deployment.
|
||||
|
||||
During the decommissioning process, MinIO continues routing read operations (``GET``, ``LIST``, ``HEAD``) operations to the pools for those objects not yet migrated.
|
||||
MinIO routes all new write operations (``PUT``) to the remaining pools in the deployment not scheduled for decommissioning.
|
||||
|
||||
Draining of decommissioned pools happens one pool at a time, completing the decommission of each pool in sequence.
|
||||
Draining does _not_ happen concurrently for all decommissioning pools.
|
||||
|
||||
Load balancers, reverse proxy, or other network control components which manage connections to the deployment do not need to modify their configurations at this time.
|
||||
|
||||
3) Monitor the Decommissioning Process
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Use the :mc-cmd:`mc admin decommission status` command to monitor the decommissioning process.
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc admin decommission status myminio
|
||||
|
||||
The command returns output similar to the following:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
┌─────┬────────────────────────────────────────────────────────────────┬──────────────────────────────────┬──────────┐
|
||||
│ ID │ Pools │ Capacity │ Status │
|
||||
│ 1st │ https://minio-{01...04}.example.com:9000/mnt/disk{1...4}/minio │ 10 TiB (used) / 10 TiB (total) │ Draining │
|
||||
│ 2nd │ https://minio-{05...08}.example.com:9000/mnt/disk{1...4}/minio │ 95 TiB (used) / 100 TiB (total) │ Pending │
|
||||
│ 3rd │ https://minio-{09...12}.example.com:9000/mnt/disk{1...4}/minio │ 40 TiB (used) / 500 TiB (total) │ Active │
|
||||
│ 4th │ https://minio-{13...16}.example.com:9000/mnt/disk{1...4}/minio │ 0 TiB (used) / 500 TiB (total) │ Active │
|
||||
└─────┴────────────────────────────────────────────────────────────────┴──────────────────────────────────┴──────────┘
|
||||
|
||||
You can retrieve more detailed information by specifying the description of the server pool to the command:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc admin decommission status myminio https://minio-{01...04}.example.com:9000/mnt/disk{1...4}/minio
|
||||
|
||||
The command returns output similar to the following:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
Decommissioning rate at 100MiB/sec [1TiB/10TiB]
|
||||
Started: 30 minutes ago
|
||||
|
||||
:mc-cmd:`mc admin decommission status` marks the :guilabel:`Status` as :guilabel:`Complete` once decommissioning is completed.
|
||||
You can move on to the next step once MinIO completes decommissioning for all pools.
|
||||
|
||||
If :guilabel:`Status` reads as failed, you can re-run the :mc-cmd:`mc admin decommission start` command to resume the process.
|
||||
For persistent failures, use :mc:`mc admin console` or review the ``systemd`` logs (e.g. ``journalctl -u minio``) to identify more specific errors.
|
||||
|
||||
4) Remove the Decommissioned Pools from the Deployment Configuration
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Once decommissioning completes, you can safely remove the pools from the deployment configuration.
|
||||
Modify the startup command for each remaining MinIO server in the deployment and remove the decommissioned pool.
|
||||
|
||||
The ``.deb`` or ``.rpm`` packages install a `systemd <https://www.freedesktop.org/wiki/Software/systemd/>`__ service file to ``/etc/systemd/system/minio.service``.
|
||||
For binary installations, this procedure assumes the file was created manually as per the :ref:`deploy-minio-distributed` procedure.
|
||||
|
||||
The ``minio.service`` file uses an environment file located at ``/etc/default/minio`` for sourcing configuration settings, including the startup.
|
||||
Specifically, the ``MINIO_VOLUMES`` variable sets the startup command:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
cat /etc/default/minio | grep "MINIO_VOLUMES"
|
||||
|
||||
The command returns output similar to the following:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
MINIO_VOLUMES="https://minio-{1...4}.example.net:9000/mnt/disk{1...4}/minio https://minio-{5...8}.example.net:9000/mnt/disk{1...4}/minio https://minio-{9...12}.example.net:9000/mnt/disk{1...4}/minio"
|
||||
|
||||
Edit the environment file and remove the decommissioned pools from the ``MINIO_VOLUMES`` value.
|
||||
|
||||
5) Update Network Control Plane
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Update any load balancers, reverse proxies, or other network control planes to remove the decommissioned server pools from the connection configuration for the MinIO deployment.
|
||||
|
||||
Specific instructions for configuring network control plane components is out of scope for this procedure.
|
||||
|
||||
6) Restart the MinIO Deployment
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Issue the following commands on each node **simultaneously** in the deployment to restart the MinIO service:
|
||||
|
||||
.. include:: /includes/linux/common-installation.rst
|
||||
:start-after: start-install-minio-restart-service-desc
|
||||
:end-before: end-install-minio-restart-service-desc
|
||||
|
||||
.. include:: /includes/common-installation.rst
|
||||
:start-after: start-nondisruptive-upgrade-desc
|
||||
:end-before: end-nondisruptive-upgrade-desc
|
||||
|
||||
Once the deployment is online, use :mc:`mc admin info` to confirm the uptime of all remaining servers in the deployment.
|
||||
|
||||
Reference in New Issue
Block a user