mirror of
https://github.com/minio/docs.git
synced 2025-07-28 19:42:10 +03:00
DOCS-699: improve documentation on storage classes for remote targets (#700)
Closes #699
This commit is contained in:
@ -26,7 +26,7 @@ Object Transition ("Tiering")
|
|||||||
MinIO supports creating object transition lifecycle management rules, where MinIO can automatically move an object to a remote storage "tier".
|
MinIO supports creating object transition lifecycle management rules, where MinIO can automatically move an object to a remote storage "tier".
|
||||||
MinIO supports any of the following remote tier targets:
|
MinIO supports any of the following remote tier targets:
|
||||||
|
|
||||||
- :ref:`MinIO or other S3-compatible storage <minio-lifecycle-management-transition-to-s3>`
|
- :ref:`MinIO <minio-lifecycle-management-transition-to-minio>`
|
||||||
- :ref:`Amazon S3 <minio-lifecycle-management-transition-to-s3>`
|
- :ref:`Amazon S3 <minio-lifecycle-management-transition-to-s3>`
|
||||||
- :ref:`Google Cloud Storage <minio-lifecycle-management-transition-to-gcs>`
|
- :ref:`Google Cloud Storage <minio-lifecycle-management-transition-to-gcs>`
|
||||||
- :ref:`Microsoft Azure Blob Storage
|
- :ref:`Microsoft Azure Blob Storage
|
||||||
@ -126,6 +126,7 @@ Consider regularly checking cluster metrics, capacity, and resource usage to ens
|
|||||||
.. toctree::
|
.. toctree::
|
||||||
:hidden:
|
:hidden:
|
||||||
|
|
||||||
|
/administration/object-management/transition-objects-to-minio.rst
|
||||||
/administration/object-management/transition-objects-to-s3.rst
|
/administration/object-management/transition-objects-to-s3.rst
|
||||||
/administration/object-management/transition-objects-to-gcs.rst
|
/administration/object-management/transition-objects-to-gcs.rst
|
||||||
/administration/object-management/transition-objects-to-azure.rst
|
/administration/object-management/transition-objects-to-azure.rst
|
||||||
|
@ -68,13 +68,25 @@ Required Azure Permissions
|
|||||||
Object transition lifecycle management rules require additional permissions on
|
Object transition lifecycle management rules require additional permissions on
|
||||||
the remote storage tier. Specifically, MinIO requires the
|
the remote storage tier. Specifically, MinIO requires the
|
||||||
:abbr:`Azure (Microsoft Azure)` credentials provide read, write, list, and
|
:abbr:`Azure (Microsoft Azure)` credentials provide read, write, list, and
|
||||||
delete permissions for the remote bucket.
|
delete permissions for the remote storage account and container.
|
||||||
|
|
||||||
Refer to the `Azure RBAC
|
Refer to the `Azure RBAC
|
||||||
<https://docs.microsoft.com/en-us/azure/role-based-access-control/>`__
|
<https://docs.microsoft.com/en-us/azure/role-based-access-control/>`__
|
||||||
documentation for more complete guidance on configuring the required
|
documentation for more complete guidance on configuring the required
|
||||||
permissions.
|
permissions.
|
||||||
|
|
||||||
|
Remote Storage Account and Container Must Exist
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Create the remote :azure-docs:`Azure storage account <storage/common/storage-account-overview>` and container *prior* to configuring lifecycle management tiers or rules using that resource as the target.
|
||||||
|
When :azure-docs:`creating the Azure storage account <storage/common/storage-account-create>`, ensure the storage account corresponds to either Standard or Premium blob storage with the locally redundant storage (LRS) redundancy option.
|
||||||
|
The Azure Go SDK API used by MinIO does not support any other redundancy options.
|
||||||
|
|
||||||
|
If you set a Storage Account :azure-docs:`default access tier <storage/blobs/access-tiers-online-manage>`, MinIO uses that default *if* you do not specify a :mc-cmd:`storage class <mc ilm tier add --storage-class>` when defining the remote tier.
|
||||||
|
Ensure you document the settings of both your Azure storage account and MinIO tiering configuration to avoid any potential confusion, misconfiguration, or other unexpected outcomes.
|
||||||
|
|
||||||
|
For more information on Azure storage accounts, see :azure-docs:`Storage accounts <storage/common/storage-account-overview#types-of-storage-accounts>`.
|
||||||
|
|
||||||
Considerations
|
Considerations
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
@ -125,8 +137,9 @@ Use the :mc:`mc ilm tier add` command to add a new remote storage tier:
|
|||||||
mc ilm tier add azure TARGET TIER_NAME \
|
mc ilm tier add azure TARGET TIER_NAME \
|
||||||
--account-name ACCOUNT \
|
--account-name ACCOUNT \
|
||||||
--account-key KEY \
|
--account-key KEY \
|
||||||
--bucket BUCKET \
|
--bucket CONTAINER \
|
||||||
--prefix PREFIX
|
--prefix PREFIX \
|
||||||
|
--storage-class STORAGE_CLASS
|
||||||
|
|
||||||
|
|
||||||
The example above uses the following arguments:
|
The example above uses the following arguments:
|
||||||
@ -148,12 +161,25 @@ The example above uses the following arguments:
|
|||||||
remote storage tier. Specify the name in all-caps, e.g. ``AZURE_TIER``.
|
remote storage tier. Specify the name in all-caps, e.g. ``AZURE_TIER``.
|
||||||
This value is required in the next step.
|
This value is required in the next step.
|
||||||
|
|
||||||
* - :mc-cmd:`BUCKET <mc ilm tier add --bucket>`
|
* - :mc-cmd:`ACCOUNT <mc ilm tier add --account-name>`
|
||||||
- The name of the bucket on the :abbr:`Azure (Microsoft Azure)` storage
|
- The :azure-docs:`Storage Account <storage/common/storage-account-overview>` to use as the remote storage resource.
|
||||||
|
|
||||||
|
You cannot change this account name after creating the tier.
|
||||||
|
|
||||||
|
* - :mc-cmd:`KEY <mc ilm tier add --account-key>`
|
||||||
|
- The corresponding shared account key for the specified ``ACCOUNT``.
|
||||||
|
|
||||||
|
The account key must have an assigned Azure policy with the required :ref:`permissions
|
||||||
|
<minio-lifecycle-management-transition-to-azure-permissions-remote>`.
|
||||||
|
|
||||||
|
See :azure-docs:`Managing storage account access keys <storage/common/storage-account-keys-manage>` for more information.
|
||||||
|
|
||||||
|
* - :mc-cmd:`CONTAINER <mc ilm tier add --bucket>`
|
||||||
|
- The name of the container on the :abbr:`Azure (Microsoft Azure)` storage
|
||||||
backend to which MinIO transitions objects.
|
backend to which MinIO transitions objects.
|
||||||
|
|
||||||
* - :mc-cmd:`PREFIX <mc ilm tier add --prefix>`
|
* - :mc-cmd:`PREFIX <mc ilm tier add --prefix>`
|
||||||
- The optional bucket prefix within which MinIO transitions objects.
|
- The optional container prefix within which MinIO transitions objects.
|
||||||
|
|
||||||
MinIO stores all transitioned objects in the specified ``BUCKET`` under a
|
MinIO stores all transitioned objects in the specified ``BUCKET`` under a
|
||||||
unique per-deployment prefix value. Omit this argument to use only that
|
unique per-deployment prefix value. Omit this argument to use only that
|
||||||
@ -162,20 +188,21 @@ The example above uses the following arguments:
|
|||||||
MinIO recommends specifying this optional prefix for remote storage tiers
|
MinIO recommends specifying this optional prefix for remote storage tiers
|
||||||
which contain other data, including transitioned objects from other MinIO
|
which contain other data, including transitioned objects from other MinIO
|
||||||
deployments. This prefix should provide a clear reference back to the
|
deployments. This prefix should provide a clear reference back to the
|
||||||
source MinIO deployment to faciliate ease of operations related to
|
source MinIO deployment to facilitate ease of operations related to
|
||||||
diagnostics, maintenance, or disaster recovery.
|
diagnostics, maintenance, or disaster recovery.
|
||||||
|
|
||||||
* - :mc-cmd:`ACCOUNT <mc ilm tier add --account-name>`
|
* - :mc-cmd:`STORAGE_CLASS <mc ilm tier add --storage-class>`
|
||||||
- The account name MinIO uses to access the bucket. The account name
|
- The Azure access tier MinIO applies to objects transitioned to the Azure container.
|
||||||
*must* correspond to an :abbr:`Azure (Microsoft Azure)` user with the
|
|
||||||
required :ref:`permissions
|
|
||||||
<minio-lifecycle-management-transition-to-azure-permissions-remote>`.
|
|
||||||
|
|
||||||
You cannot change this account name after creating the tier.
|
MinIO tiering behavior depends on the remote storage returning objects immediately (milliseconds to seconds) upon request.
|
||||||
|
MinIO therefore *cannot* support remote storage which requires rehydration, wait periods, or manual intervention.
|
||||||
|
|
||||||
* - :mc-cmd:`KEY <mc ilm tier add --account-key>`
|
The following Azure access tiers meet MinIO's requirements as a remote tier:
|
||||||
- The corresponding key for the specified ``ACCOUNT``.
|
|
||||||
|
|
||||||
|
- ``Hot``
|
||||||
|
- ``Cool``
|
||||||
|
|
||||||
|
For more information, see :azure-docs:`Hot, cool, and archive access tiers for blob data <storage/blobs/access-tiers-overview.html>`.
|
||||||
|
|
||||||
3) Create and Apply the Transition Rule
|
3) Create and Apply the Transition Rule
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
@ -75,6 +75,14 @@ Refer to the `GCS IAM permissions
|
|||||||
documentation for more complete guidance on configuring the required
|
documentation for more complete guidance on configuring the required
|
||||||
permissions.
|
permissions.
|
||||||
|
|
||||||
|
Remote Bucket Must Exist
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Create the remote GCS bucket *prior* to configuring lifecycle management tiers or rules using that bucket as the target.
|
||||||
|
|
||||||
|
If you set a default GCS :gcs-docs:`storage class <storage-classes>`, MinIO uses that default *if* you do not specify a :mc-cmd:`storage class <mc ilm tier add --storage-class>` when defining the remote tier.
|
||||||
|
Ensure you document the settings of both your GCS bucket and MinIO tiering configuration to avoid any potential confusion, misconfiguration, or other unexpected outcomes.
|
||||||
|
|
||||||
Considerations
|
Considerations
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
@ -124,7 +132,8 @@ service as the remote storage tier:
|
|||||||
mc ilm tier add gcs TARGET TIER_NAME \
|
mc ilm tier add gcs TARGET TIER_NAME \
|
||||||
--bucket BUCKET \
|
--bucket BUCKET \
|
||||||
--prefix PREFIX \
|
--prefix PREFIX \
|
||||||
--credentials-file CREDENTIALS
|
--credentials-file CREDENTIALS \
|
||||||
|
--storage-class STORAGE_CLASS
|
||||||
|
|
||||||
The example above uses the following arguments:
|
The example above uses the following arguments:
|
||||||
|
|
||||||
@ -145,14 +154,6 @@ The example above uses the following arguments:
|
|||||||
remote storage tier. Specify the name in all-caps, e.g. ``GCS_TIER``.
|
remote storage tier. Specify the name in all-caps, e.g. ``GCS_TIER``.
|
||||||
This value is required in the next step.
|
This value is required in the next step.
|
||||||
|
|
||||||
* - :mc-cmd:`CREDENTIALS <mc ilm tier add --credentials-file>`
|
|
||||||
- The `credential file
|
|
||||||
<https://cloud.google.com/docs/authentication/getting-started>`__ for a
|
|
||||||
user on the remote GCS tier. The specified user credentials *must*
|
|
||||||
correspond to a GCS user with the required
|
|
||||||
:ref:`permissions
|
|
||||||
<minio-lifecycle-management-transition-to-gcs-permissions-remote>`.
|
|
||||||
|
|
||||||
* - :mc-cmd:`BUCKET <mc ilm tier add --bucket>`
|
* - :mc-cmd:`BUCKET <mc ilm tier add --bucket>`
|
||||||
- The name of the bucket on the :abbr:`GCS (Google Cloud Storage)` storage
|
- The name of the bucket on the :abbr:`GCS (Google Cloud Storage)` storage
|
||||||
backend to which MinIO transitions objects.
|
backend to which MinIO transitions objects.
|
||||||
@ -167,9 +168,31 @@ The example above uses the following arguments:
|
|||||||
MinIO recommends specifying this optional prefix for remote storage tiers
|
MinIO recommends specifying this optional prefix for remote storage tiers
|
||||||
which contain other data, including transitioned objects from other MinIO
|
which contain other data, including transitioned objects from other MinIO
|
||||||
deployments. This prefix should provide a clear reference back to the
|
deployments. This prefix should provide a clear reference back to the
|
||||||
source MinIO deployment to faciliate ease of operations related to
|
source MinIO deployment to facilitate ease of operations related to
|
||||||
diagnostics, maintenance, or disaster recovery.
|
diagnostics, maintenance, or disaster recovery.
|
||||||
|
|
||||||
|
* - :mc-cmd:`CREDENTIALS <mc ilm tier add --credentials-file>`
|
||||||
|
- The `credential file
|
||||||
|
<https://cloud.google.com/docs/authentication/getting-started>`__ for a
|
||||||
|
user on the remote GCS tier. The specified user credentials *must*
|
||||||
|
correspond to a GCS user with the required
|
||||||
|
:ref:`permissions
|
||||||
|
<minio-lifecycle-management-transition-to-gcs-permissions-remote>`.
|
||||||
|
|
||||||
|
* - :mc-cmd:`STORAGE_CLASS <mc ilm tier add --storage-class>`
|
||||||
|
- The :abbr:`GCS (Google Cloud Storage)` storage class MinIO applies to objects transitioned to the GCS bucket.
|
||||||
|
|
||||||
|
MinIO tiering behavior depends on the remote storage returning objects immediately (milliseconds to seconds) upon request.
|
||||||
|
MinIO therefore *cannot* support remote storage which requires rehydration, wait periods, or manual intervention.
|
||||||
|
|
||||||
|
The following GCS storage classes meet MinIO's requirements as a remote tier:
|
||||||
|
|
||||||
|
- ``STANDARD``
|
||||||
|
- ``NEARLINE``
|
||||||
|
- ``COLDLINE``
|
||||||
|
|
||||||
|
For more information, see :gcs-docs:`GCS storage class <storage-classes>`.
|
||||||
|
|
||||||
|
|
||||||
3) Create and Apply the Transition Rule
|
3) Create and Apply the Transition Rule
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
@ -0,0 +1,228 @@
|
|||||||
|
.. _minio-lifecycle-management-transition-to-minio:
|
||||||
|
|
||||||
|
=============================================
|
||||||
|
Transition Objects to Remote MinIO Deployment
|
||||||
|
=============================================
|
||||||
|
|
||||||
|
.. default-domain:: minio
|
||||||
|
|
||||||
|
.. contents:: Table of Contents
|
||||||
|
:local:
|
||||||
|
:depth: 2
|
||||||
|
|
||||||
|
The procedure on this page creates a new object lifecycle management rule that transitions objects from a bucket on a primary MinIO deployment to a bucket on a remote MinIO deployment.
|
||||||
|
This procedure supports cost-management strategies such as tiering objects from a "hot" MinIO deployment using NVMe storage to a "warm" MinIO deployment using SSD.
|
||||||
|
|
||||||
|
.. todo: diagram
|
||||||
|
|
||||||
|
Requirements
|
||||||
|
------------
|
||||||
|
|
||||||
|
Install and Configure ``mc``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This procedure uses :mc:`mc` for performing operations on the MinIO cluster.
|
||||||
|
Install :mc:`mc` on a machine with network access to both source and destination
|
||||||
|
clusters. See the ``mc`` :ref:`Installation Quickstart <mc-install>` for
|
||||||
|
instructions on downloading and installing ``mc``.
|
||||||
|
|
||||||
|
Use the :mc:`mc alias` command to create an alias for the source MinIO cluster.
|
||||||
|
Alias creation requires specifying an access key for a user on the source and
|
||||||
|
destination clusters. The specified users must have :ref:`permissions
|
||||||
|
<minio-lifecycle-management-transition-to-minio-permissions>` for configuring and
|
||||||
|
applying transition operations.
|
||||||
|
|
||||||
|
.. _minio-lifecycle-management-transition-to-minio-permissions:
|
||||||
|
|
||||||
|
Required Source MinIO Permissions
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
MinIO requires the following permissions scoped to the bucket or buckets
|
||||||
|
for which you are creating lifecycle management rules.
|
||||||
|
|
||||||
|
- :policy-action:`s3:PutLifecycleConfiguration`
|
||||||
|
- :policy-action:`s3:GetLifecycleConfiguration`
|
||||||
|
|
||||||
|
MinIO also requires the following administrative permissions on the cluster
|
||||||
|
in which you are creating remote tiers for object transition lifecycle
|
||||||
|
management rules:
|
||||||
|
|
||||||
|
- :policy-action:`admin:SetTier`
|
||||||
|
- :policy-action:`admin:ListTier`
|
||||||
|
|
||||||
|
For example, the following policy provides permission for configuring object
|
||||||
|
transition lifecycle management rules on any bucket in the cluster:
|
||||||
|
|
||||||
|
.. literalinclude:: /extra/examples/LifecycleManagementAdmin.json
|
||||||
|
:language: json
|
||||||
|
:class: copyable
|
||||||
|
|
||||||
|
.. _minio-lifecycle-management-transition-to-minio-permissions-remote:
|
||||||
|
|
||||||
|
Required Remote MinIO Permissions
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Object transition lifecycle management rules require additional permissions
|
||||||
|
on the remote storage tier. Specifically, MinIO requires the remote
|
||||||
|
tier credentials provide read, write, list, and delete permissions for the
|
||||||
|
remote bucket.
|
||||||
|
|
||||||
|
For example, the following policy on the remote MinIO deployment provides the necessary permission
|
||||||
|
for transitioning objects into and out of the remote tier:
|
||||||
|
|
||||||
|
.. literalinclude:: /extra/examples/LifecycleManagementUser.json
|
||||||
|
:language: json
|
||||||
|
:class: copyable
|
||||||
|
|
||||||
|
Modify the ``Resource`` for the bucket into which MinIO tiers objects.
|
||||||
|
|
||||||
|
Refer to the :ref:`minio-policy` documentation for more complete guidance on configuring the required permissions.
|
||||||
|
|
||||||
|
Remote Bucket Must Exist
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Create the remote bucket *prior* to configuring lifecycle management tiers or rules using that bucket as the target.
|
||||||
|
|
||||||
|
If the remote bucket contains existing data, use the :mc-cmd:`prefix <mc ilm tier add --prefix>` feature to isolate transitioned objects from any other objects on that bucket.
|
||||||
|
|
||||||
|
Considerations
|
||||||
|
--------------
|
||||||
|
|
||||||
|
Lifecycle Management Object Scanner
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
MinIO uses a scanner process to check objects against all configured
|
||||||
|
lifecycle management rules. Slow scanning due to high IO workloads or
|
||||||
|
limited system resources may delay application of lifecycle management
|
||||||
|
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
|
||||||
|
|
||||||
|
Exclusive Access to Remote Data
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
.. include:: /includes/common-minio-tiering.rst
|
||||||
|
:start-after: start-transition-bucket-access-desc
|
||||||
|
:end-before: end-transition-bucket-access-desc
|
||||||
|
|
||||||
|
Availability of Remote Data
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
.. include:: /includes/common-minio-tiering.rst
|
||||||
|
:start-after: start-transition-data-loss-desc
|
||||||
|
:end-before: end-transition-data-loss-desc
|
||||||
|
|
||||||
|
Procedure
|
||||||
|
---------
|
||||||
|
|
||||||
|
1) Configure User Accounts and Policies for Lifecycle Management
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
.. |permissions| replace:: :ref:`permissions <minio-lifecycle-management-transition-to-minio-permissions>`
|
||||||
|
|
||||||
|
.. include:: /includes/common-minio-tiering.rst
|
||||||
|
:start-after: start-create-transition-user-desc
|
||||||
|
:end-before: end-create-transition-user-desc
|
||||||
|
|
||||||
|
2) Configure the Remote Storage Tier
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Use the :mc:`mc ilm tier add` command to add the remote MinIO deployment as the
|
||||||
|
new remote storage tier:
|
||||||
|
|
||||||
|
.. code-block:: shell
|
||||||
|
:class: copyable
|
||||||
|
|
||||||
|
mc ilm tier add minio TARGET TIER_NAME \
|
||||||
|
--endpoint https://HOSTNAME \
|
||||||
|
--access-key ACCESS_KEY \
|
||||||
|
--secret-key SECRET_KEY \
|
||||||
|
--bucket BUCKET \
|
||||||
|
--prefix PREFIX \
|
||||||
|
--storage-class STORAGE_CLASS \
|
||||||
|
--region REGION
|
||||||
|
|
||||||
|
The example above uses the following arguments:
|
||||||
|
|
||||||
|
.. list-table::
|
||||||
|
:header-rows: 1
|
||||||
|
:widths: 30 70
|
||||||
|
:width: 100%
|
||||||
|
|
||||||
|
* - Argument
|
||||||
|
- Description
|
||||||
|
|
||||||
|
* - :mc-cmd:`ALIAS <mc ilm tier add TARGET>`
|
||||||
|
- The :mc:`alias <mc alias>` of the MinIO deployment on which to configure
|
||||||
|
the MinIO remote tier.
|
||||||
|
|
||||||
|
* - :mc-cmd:`TIER_NAME <mc ilm tier add TIER_NAME>`
|
||||||
|
- The name to associate with the new MinIO remote storage tier. Specify the
|
||||||
|
name in all-caps, e.g. ``MINIO_WARM_TIER``. This value is required in the next
|
||||||
|
step.
|
||||||
|
|
||||||
|
* - :mc-cmd:`HOSTNAME <mc ilm tier add --endpoint>`
|
||||||
|
- The URL endpoint for the MinIO storage backend.
|
||||||
|
|
||||||
|
* - :mc-cmd:`ACCESS_KEY <mc ilm tier add --access-key>`
|
||||||
|
- The access key MinIO uses to access the bucket. The
|
||||||
|
access key *must* correspond to an IAM user with the
|
||||||
|
required
|
||||||
|
:ref:`permissions
|
||||||
|
<minio-lifecycle-management-transition-to-minio-permissions-remote>`.
|
||||||
|
|
||||||
|
* - :mc-cmd:`SECRET_KEY <mc ilm tier add --secret-key>`
|
||||||
|
- The corresponding secret key for the specified ``ACCESS_KEY``.
|
||||||
|
|
||||||
|
* - :mc-cmd:`BUCKET <mc ilm tier add --bucket>`
|
||||||
|
- The name of the bucket on the remote MinIO deployment to which the ``SOURCE``
|
||||||
|
transitions objects.
|
||||||
|
|
||||||
|
* - :mc-cmd:`PREFIX <mc ilm tier add --prefix>`
|
||||||
|
- The optional bucket prefix within which MinIO transitions objects.
|
||||||
|
|
||||||
|
MinIO stores all transitioned objects in the specified ``BUCKET`` under a
|
||||||
|
unique per-deployment prefix value. Omit this argument to use only that
|
||||||
|
value for isolating and organizing data within the remote storage.
|
||||||
|
|
||||||
|
MinIO recommends specifying this optional prefix for remote storage tiers
|
||||||
|
which contain other data, including transitioned objects from other MinIO
|
||||||
|
deployments. This prefix should provide a clear reference back to the
|
||||||
|
source MinIO deployment to facilitate ease of operations related to
|
||||||
|
diagnostics, maintenance, or disaster recovery.
|
||||||
|
|
||||||
|
* - :mc-cmd:`STORAGE_CLASS <mc ilm tier add --storage-class>`
|
||||||
|
- The :ref:`Erasure Coding storage class <minio-ec-storage-class>` MinIO applies to objects transitions to the remote MinIO bucket.
|
||||||
|
Specify one of the following supported storage classes:
|
||||||
|
|
||||||
|
- ``STANDARD`` *Recommended*
|
||||||
|
- ``REDUCED``
|
||||||
|
|
||||||
|
* - :mc-cmd:`REGION <mc ilm tier add --region>`
|
||||||
|
- The MinIO region of the specified ``BUCKET``.
|
||||||
|
|
||||||
|
MinIO deployments typically do not require setting a region as part of setup.
|
||||||
|
Only include this option if you explicitly set the ``MINIO_SITE_REGION`` configuration setting for the deployment.
|
||||||
|
|
||||||
|
3) Create and Apply the Transition Rule
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
.. include:: /includes/common-minio-tiering.rst
|
||||||
|
:start-after: start-create-transition-rule-desc
|
||||||
|
:end-before: end-create-transition-rule-desc
|
||||||
|
|
||||||
|
|
||||||
|
4) Verify the Transition Rule
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Use the :mc:`mc ilm rule ls` command to review the configured transition
|
||||||
|
rules:
|
||||||
|
|
||||||
|
.. code-block:: shell
|
||||||
|
:class: copyable
|
||||||
|
|
||||||
|
mc ilm rule ls ALIAS/PATH --transition
|
||||||
|
|
||||||
|
- Replace :mc-cmd:`ALIAS <mc ilm rule ls ALIAS>` with the :mc:`alias <mc alias>`
|
||||||
|
of the MinIO deployment.
|
||||||
|
|
||||||
|
- Replace :mc-cmd:`PATH <mc ilm rule ls ALIAS>` with the name of the bucket for
|
||||||
|
which to retrieve the configured lifecycle management rules.
|
@ -84,6 +84,11 @@ Refer to the :aws-docs:`Amazon S3 Permissions
|
|||||||
documentation for more complete guidance on configuring the required
|
documentation for more complete guidance on configuring the required
|
||||||
permissions.
|
permissions.
|
||||||
|
|
||||||
|
Remote Bucket Must Exist
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Create the remote S3 bucket *prior* to configuring lifecycle management tiers or rules using that bucket as the target.
|
||||||
|
|
||||||
Considerations
|
Considerations
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
@ -189,11 +194,21 @@ The example above uses the following arguments:
|
|||||||
diagnostics, maintenance, or disaster recovery.
|
diagnostics, maintenance, or disaster recovery.
|
||||||
|
|
||||||
* - :mc-cmd:`STORAGE_CLASS <mc ilm tier add --storage-class>`
|
* - :mc-cmd:`STORAGE_CLASS <mc ilm tier add --storage-class>`
|
||||||
- The S3 storage class to which MinIO transitions objects. Specify
|
- The S3 storage class to which MinIO transitions objects.
|
||||||
one of the following supported storage classes:
|
|
||||||
|
MinIO tiering behavior depends on the remote storage returning objects immediately (milliseconds to seconds) upon request.
|
||||||
|
MinIO therefore *cannot* support remote storage which requires rehydration, wait periods, or manual intervention.
|
||||||
|
|
||||||
|
The following S3 storage classes meet MinIO's requirements as a remote tier:
|
||||||
|
|
||||||
- ``STANDARD``
|
- ``STANDARD``
|
||||||
- ``REDUCED``
|
- ``STANDARD_IA``
|
||||||
|
- ``STANDARD_ONEZONE``
|
||||||
|
|
||||||
|
Omit this value to use the default storage class for the bucket.
|
||||||
|
Specifying this value overrides the bucket storage class.
|
||||||
|
|
||||||
|
For more information, see :s3-docs:`Using Amazon S3 storage classes <storage-class-intro.html>`.
|
||||||
|
|
||||||
* - :mc-cmd:`REGION <mc ilm tier add --region>`
|
* - :mc-cmd:`REGION <mc ilm tier add --region>`
|
||||||
- The AWS S3 region of the specified ``BUCKET``. You can safely omit this
|
- The AWS S3 region of the specified ``BUCKET``. You can safely omit this
|
||||||
|
@ -72,6 +72,7 @@ extlinks = {
|
|||||||
'minio-docs' : ('https://min.io/docs/%s?ref=docs-internal',''),
|
'minio-docs' : ('https://min.io/docs/%s?ref=docs-internal',''),
|
||||||
'gke-docs' : ('https://cloud.google.com/kubernetes-engine/docs/%s',''),
|
'gke-docs' : ('https://cloud.google.com/kubernetes-engine/docs/%s',''),
|
||||||
'gcp-docs' : ('https://cloud.google.com/compute/docs/%s',''),
|
'gcp-docs' : ('https://cloud.google.com/compute/docs/%s',''),
|
||||||
|
'gcs-docs' : ('https://cloud.google.com/storage/docs/%s',''),
|
||||||
'aks-docs' : ('https://learn.microsoft.com/en-us/azure/aks/%s',''),
|
'aks-docs' : ('https://learn.microsoft.com/en-us/azure/aks/%s',''),
|
||||||
'azure-docs' : ('https://learn.microsoft.com/en-us/azure/%s',''),
|
'azure-docs' : ('https://learn.microsoft.com/en-us/azure/%s',''),
|
||||||
|
|
||||||
|
@ -111,9 +111,11 @@ This tutorial includes the necessary syntax for setting this prefix.
|
|||||||
|
|
||||||
.. start-transition-data-loss-desc
|
.. start-transition-data-loss-desc
|
||||||
|
|
||||||
MinIO creates metadata for each transitioned object that identifies its location
|
MinIO tiering behavior depends on the remote storage returning objects immediately (milliseconds to seconds) upon request.
|
||||||
on the remote storage. This metadata is required for accessing the object, such
|
MinIO therefore *cannot* support remote storage which requires rehydration, wait periods, or manual intervention.
|
||||||
that applications cannot access a transition object independent of MinIO.
|
|
||||||
|
MinIO creates metadata for each transitioned object that identifies its location on the remote storage.
|
||||||
|
Applications cannot trivially identify and access a transitioned object independent of MinIO.
|
||||||
Availability of the transitioned data therefore depends on the same core
|
Availability of the transitioned data therefore depends on the same core
|
||||||
protections that :ref:`erasure coding <minio-erasure-coding>` and distributed
|
protections that :ref:`erasure coding <minio-erasure-coding>` and distributed
|
||||||
deployment topologies provide for all objects on the MinIO deployment. Using
|
deployment topologies provide for all objects on the MinIO deployment. Using
|
||||||
|
@ -194,19 +194,20 @@ The command accepts the following arguments:
|
|||||||
.. mc-cmd:: --account-name
|
.. mc-cmd:: --account-name
|
||||||
:optional:
|
:optional:
|
||||||
|
|
||||||
The account name for a user on the remote Azure tier.
|
The :azure-docs:`Storage Account <storage/common/storage-account-overview>` to use as the remote storage resource.
|
||||||
The user must have permission to perform read/write/list/delete operations on the remote bucket or bucket prefix.
|
|
||||||
|
|
||||||
Required if :mc-cmd:`~mc ilm tier add TIER_TYPE` is ``azure``.
|
Required if :mc-cmd:`~mc ilm tier add TIER_TYPE` is ``azure``.
|
||||||
This option has no effect for any other value of ``TIER_TYPE``.
|
This option has no effect for any other value of ``TIER_TYPE``.
|
||||||
|
|
||||||
MinIO does *not* support changing the account name associated to an Azure remote tier.
|
MinIO does *not* support changing the storage account name associated to an Azure remote tier.
|
||||||
Azure storage backends are tied to the account, such that changing the account would change the storage backend and prevent access to any objects transitioned to the original account/backend.
|
Azure storage backends are tied to the storage account, such that changing this value would change the storage backend and prevent access to any objects transitioned to the original account/backend.
|
||||||
|
|
||||||
.. mc-cmd:: --account-key
|
.. mc-cmd:: --account-key
|
||||||
:optional:
|
:optional:
|
||||||
|
|
||||||
The account key for the :mc-cmd:`~mc ilm tier add --account-name` associated to the remote Azure tier.
|
The corresponding shared account key for the :mc-cmd:`~mc ilm tier add --account-name` associated to the remote Azure tier.
|
||||||
|
|
||||||
|
The account key must have an assigned Azure policy with the required :ref:`permissions <minio-lifecycle-management-transition-to-azure-permissions-remote>`.
|
||||||
|
|
||||||
Required if :mc-cmd:`~mc ilm tier add TIER_TYPE` is ``azure``.
|
Required if :mc-cmd:`~mc ilm tier add TIER_TYPE` is ``azure``.
|
||||||
This option has no effect for any other value of ``TIER_TYPE``.
|
This option has no effect for any other value of ``TIER_TYPE``.
|
||||||
@ -225,6 +226,8 @@ The command accepts the following arguments:
|
|||||||
|
|
||||||
The bucket on the remote tier to which MinIO transitions objects.
|
The bucket on the remote tier to which MinIO transitions objects.
|
||||||
|
|
||||||
|
For ``azure`` remote tiers, this value corresponds to the :azure-docs:`Container name <storage/blobs/storage-blobs-introduction#containers>`
|
||||||
|
|
||||||
.. mc-cmd:: --prefix
|
.. mc-cmd:: --prefix
|
||||||
:optional:
|
:optional:
|
||||||
|
|
||||||
@ -235,16 +238,47 @@ The command accepts the following arguments:
|
|||||||
.. mc-cmd:: --storage-class
|
.. mc-cmd:: --storage-class
|
||||||
:optional:
|
:optional:
|
||||||
|
|
||||||
The AWS storage class to use for objects transitioned by MinIO.
|
The storage class ("access tier" for Microsoft Azure) MinIO applies to objects transitioned to the remote bucket.
|
||||||
MinIO supports only the following storage classes:
|
|
||||||
|
|
||||||
- ``STANDARD``
|
The storage class to apply to objects transitioned by MinIO to the remote bucket.
|
||||||
- ``REDUCED_REDUNDANCY``
|
MinIO tiering behavior depends on the remote storage returning objects immediately (milliseconds to seconds) upon request.
|
||||||
|
MinIO therefore *cannot* support remote storage which requires rehydration, wait periods, or manual intervention.
|
||||||
|
|
||||||
|
Select the tab corresponding to the ``TIER_TYPE`` for a list of supported values for each tier:
|
||||||
|
|
||||||
Defaults to ``STANDARD`` if omitted.
|
.. tab-set::
|
||||||
|
|
||||||
This option only applies if :mc-cmd:`~mc ilm tier add TIER_TYPE` is ``s3`` or ``minio``.
|
.. tab-item:: minio
|
||||||
This option has no effect for any other value of ``TIER_TYPE``.
|
|
||||||
|
- ``STANDARD`` *Recommended*
|
||||||
|
- ``REDUCED``
|
||||||
|
|
||||||
|
For more information, see :ref:`Erasure Coding storage class <minio-ec-storage-class>`.
|
||||||
|
|
||||||
|
.. tab-item:: s3
|
||||||
|
|
||||||
|
- ``STANDARD``
|
||||||
|
- ``STANDARD_IA``
|
||||||
|
- ``ONEZONE_IA``
|
||||||
|
|
||||||
|
For more information, see :s3-docs:`Using Amazon S3 storage classes <storage-class-intro.html>`.
|
||||||
|
|
||||||
|
.. tab-item:: gcs
|
||||||
|
|
||||||
|
- ``STANDARD``
|
||||||
|
- ``NEARLINE``
|
||||||
|
- ``COLDLINE``
|
||||||
|
|
||||||
|
For more information, see :gcs-docs:`GCS storage class <storage-classes>`.
|
||||||
|
|
||||||
|
.. tab-item:: azure
|
||||||
|
|
||||||
|
- ``Hot``
|
||||||
|
- ``Cool``
|
||||||
|
|
||||||
|
For more information, see :azure-docs:`Hot, cool, and archive access tiers for blob data <storage/blobs/access-tiers-overview.html>`.
|
||||||
|
|
||||||
|
If omitted, objects use the default storage class defined for the remote bucket.
|
||||||
|
|
||||||
.. mc-cmd:: --region
|
.. mc-cmd:: --region
|
||||||
:optional:
|
:optional:
|
||||||
|
Reference in New Issue
Block a user