minio/docs
1
0
Fork 0
mirror of https://github.com/minio/docs.git synced 2026-01-04 02:44:36 +03:00
Code Activity

MinIO ILM Migration + Tiering

Browse Source 
Apply suggestions from code review + local changes

Co-authored-by: Poorna Krishnamoorthy <poornas@users.noreply.github.com>
Co-authored-by: Krishnan Parthasarathi <krisis@users.noreply.github.com>

Apply suggestions from code review

Co-authored-by: Krishnan Parthasarathi <krisis@users.noreply.github.com>
This commit is contained in:
ravindk89
2021-04-19 12:41:23 -04:00
committed by Ravind Kumar
parent 7736389e86
commit 5a5a8a6908
14 changed files with 1640 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
source/extra/examples/LifecycleManagementAdmin.json Normal file
View File

@@ -0,0 +1,24 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"admin:SetTier",
"admin:ListTier"
],
"Effect": "Allow",
"Sid": "EnableRemoteTierManagement"
},
{
"Action": [
"s3:PutBucketLifecycle",
"s3:GetBucketLifecycle"
],
"Resource": [
"arn:aws:s3:::*"
],
"Effect": "Allow",
"Sid": "EnableLifecycleManagementRules"
}
]
}

27
source/extra/examples/LifecycleManagementUser.json Normal file
View File

@@ -0,0 +1,27 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::MyDestinationBucket"
],
"Sid": ""
},
{
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::MyDestinationBucket/*"
],
"Sid": ""
}
]
}

2
source/extra/examples/ReplicationAdminPolicy.json
View File

@@ -27,4 +27,4 @@
"Sid": "EnableReplicationRuleConfiguration"
}
]
}
}

94
source/includes/common-minio-tiering.rst Normal file
View File

@@ -0,0 +1,94 @@
.. start-create-transition-rule-desc
Use the :mc-cmd:`mc ilm add` command to create a new transition rule
for the bucket. The following tabs contain examples for transitioning
objects on a calendar date *or* after a number of calendar days.
.. tabs::
.. tab:: Transition after Calendar Date
.. code-block:: shell
:class: copyable
mc ilm add ALIAS/BUCKET \
--storage-class TIERNAME \
--transition-date DATE \
--noncurrentversion-transition-date NONCURRENT_DAYS
.. tab:: Transition after Calendar Days
.. code-block:: shell
:class: copyable
mc ilm add ALIAS/BUCKET \
--storage-class TIERNAME \
--transition-days DAYS \
--noncurrentversion-transition-days NONCURRENT_DAYS
The examples above specify the following arguments:
.. list-table::
:header-rows: 1
:widths: 40 60
:width: 100%
* - Argument
- Description
* - :mc-cmd:`ALIAS <mc ilm add TARGET>`
- Specify the :mc:`alias <mc alias>` of the MinIO deployment for which
you are creating the lifecycle management rule.
* - :mc-cmd:`BUCKET <mc ilm add TARGET>`
- Specify the full path to the bucket for which you are
creating the lifecycle management rule.
* - :mc-cmd:`TIERNAME <mc ilm add storage-class>`
- The remote storage tier to which MinIO transitions objects.
Specify the remote storage tier name created in the previous step.
* - :mc-cmd:`DAYS <mc ilm add transition-days>`
- The number of calendar days after which MinIO marks an object as
eligible for transition.
* - :mc-cmd:`DATE <mc ilm add transition-date>`
- The ISO-8601-formatted calendar date after which MinIO marks an object
as eligible for transition.
* - :mc-cmd:`NONCURRENT_DAYS <mc ilm add noncurrentversion-transition-days>`
- The number of calendar days after which MinIO marks a noncurrent
object version as eligible for transition. Omit this value to
ignore noncurrent object versions.
This option has no effect on non-versioned buckets.
.. end-create-transition-rule-desc
.. start-create-transition-user-desc
This step creates users and policies on the MinIO deployment for supporting
lifecycle management operations. You can skip this step if the deployment
already has users with the necessary |permissions|.
The following example uses ``Alpha`` as a placeholder :mc:`alias <mc alias>` for
the MinIO deployment. Replace this value with the appropriate alias for the
MinIO deployment on which you are configuring lifecycle management rules.
Replace the password ``LongRandomSecretKey`` with a long, random, and secure
secret key as per your organizations best practices for password generation.
.. code-block:: shell
:class: copyable
wget -O - https://docs.min.io/minio/baremetal/examples/LifecycleManagementAdmin.json | \
mc admin policy add Alpha LifecycleAdminPolicy /dev/stdin
mc admin user add Alpha alphaLifecycleAdmin LongRandomSecretKey
mc admin policy set Alpha LifecycleAdminPolicy user=alphaLifecycleAdmin
This example assumes that the specified
aliases have the necessary permissions for creating policies and users
on the deployment. See :ref:`minio-users` and :ref:`minio-policy` for more
complete documentation on MinIO users and policies respectively.
.. end-create-transition-user-desc

1
source/index.rst
View File

@@ -150,6 +150,7 @@ such as versioning and replication.
/introduction/minio-overview
/concepts/feature-overview
/tutorials/minio-installation
/lifecycle-management/lifecycle-management-overview
/replication/replication-overview
/security/security-overview
/monitoring/monitoring-overview

139
source/lifecycle-management/create-lifecycle-management-expiration-rule.rst Normal file
View File

@@ -0,0 +1,139 @@
.. _minio-lifecycle-management-create-expiry-rule:
===========================
Automatic Object Expiration
===========================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 2
Each procedure on this page creates a new object lifecycle management rule that
expires objects on a MinIO bucket. This procedure supports use cases like
removing "old" objects after a certain time period or calendar date.
.. todo: diagram
Requirements
------------
Install and Configure ``mc``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This procedure uses :mc:`mc` for performing operations on the MinIO cluster.
Install :mc:`mc` on a machine with network access to both source and destination
clusters. See the ``mc`` :ref:`Installation Quickstart <mc-install>` for
instructions on downloading and installing ``mc``.
Use the :mc:`mc alias` command to create an alias for the source MinIO cluster
and the destination S3-compatible service. Alias creation requires specifying an
access key for a user on the source and destination clusters. The specified
users must have :ref:`permissions
<minio-lifecycle-management-create-expiry-rule-permissions>` for configuring
and applying expiry operations.
.. _minio-lifecycle-management-create-expiry-rule-permissions:
Required Permissions
~~~~~~~~~~~~~~~~~~~~
MinIO requires the following permissions scoped to the bucket or buckets
for which you are creating lifecycle management rules.
- :policy-action:`s3:PutBucketLifecycle`
- :policy-action:`s3:GetBucketLifecycle`
MinIO also requires the following administrative permissions on the cluster
in which you are creating remote tiers for object transition lifecycle
management rules:
- :policy-action:`admin:SetTier`
- :policy-action:`admin:ListTier`
For example, the following policy provides permission for configuring object
transition lifecycle management rules on any bucket in the cluster:.
.. literalinclude:: /extra/examples/LifecycleManagementAdmin.json
:language: json
:class: copyable
Expire Objects after Calendar Date
----------------------------------
Use :mc-cmd:`mc ilm add` with :mc-cmd-option:`~mc ilm add expiry-date` to
expire bucket contents after a specific date.
.. code-block:: shell
:class: copyable
mc ilm add ALIAS/PATH --expiry-date "DATE"
- Replace :mc-cmd:`ALIAS <mc ilm add TARGET>` with the
:mc:`alias <mc alias>` of the S3-compatible host.
- Replace :mc-cmd:`PATH <mc ilm add TARGET>` with the path to the bucket on the
S3-compatible host.
- Replace :mc-cmd:`DATE <mc ilm add expiry-date>` with the calendar date after
which to expire the object. For example, specify "2021-01-01" to expire
objects after January 1st, 2021.
Expire Objects after Number of Days
-----------------------------------
Use :mc-cmd:`mc ilm add` with :mc-cmd-option:`~mc ilm add expiry-days` to
expire bucket contents a number of days after object creation:
.. code-block:: shell
:class: copyable
mc ilm add ALIAS/PATH --expiry-days "DAYS"
- Replace :mc-cmd:`ALIAS <mc ilm add TARGET>` with the
:mc:`alias <mc alias>` of the S3-compatible host.
- Replace :mc-cmd:`PATH <mc ilm add TARGET>` with the path to the bucket on the
S3-compatible host.
- Replace :mc-cmd:`DATE <mc ilm add expiry-date>` with the number of days after
which to expire the object. For example, specify ``30d`` to expire the
object 30 days after creation.
Expire Versioned Objects
------------------------
Use :mc-cmd:`mc ilm add` to expiring noncurrent object versions and object
delete markers:
- To expire noncurrent object versions, specify the
:mc-cmd-option:`~mc ilm add noncurrentversion-expiration-days` option.
- To expire delete markers for objects with no remaining versions,
specify the :mc-cmd-option:`~mc ilm add expired-object-delete-marker`
option.
.. code-block:: shell
:class: copyable
mc ilm add ALIAS/PATH \
--noncurrentversion-expiration-days NONCURRENT_DAYS \
--expired-object-delete-marker EXPIRED_DAYS
- Replace :mc-cmd:`ALIAS <mc ilm add TARGET>` with the
:mc:`alias <mc alias>` of the S3-compatible host.
- Replace :mc-cmd:`PATH <mc ilm add TARGET>` with the path to the bucket on the
S3-compatible host.
- Replace :mc-cmd:`NONCURRENT_DAYS
<mc ilm add noncurrentversion-expiration-days>` with the number of days after
which to expire noncurrent object versions. For example, specify ``30d`` to
expire a version after it has been noncurrent for at least 30 days.
- Replace :mc-cmd:`EXPIRED_DAYS
<mc ilm add expired-object-delete-marker>` with the number of days after
which to expire the delete marker for an object with no remaining versions.
For example, specify ``30d`` to remove the delete marker after it has
been the only remaining "version" of that object for at least 30 days.

144
source/lifecycle-management/lifecycle-management-overview.rst Normal file
View File

@@ -0,0 +1,144 @@
.. _minio-lifecycle-management:
===========================
Object Lifecycle Management
===========================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 1
MinIO Object Lifecycle Management allows creating rules for time or date
based automatic transition or expiry of objects. For object transition,
MinIO automatically moves the object to a configured remote storage
tier. For object expiry, MinIO automatically deletes the object.
MinIO lifecycle management is built for behavior and syntax compatibility with
:s3-docs:`AWS S3 Lifecycle Management <object-lifecycle-mgmt.html>`. For
example, you can export S3 lifecycle management rules and import them into
MinIO or vice-versa. MinIO uses JSON to describe lifecycle management rules,
and conversion to or from XML may be required.
.. _minio-lifecycle-management-tiering:
Object Transition ("Tiering")
-----------------------------
MinIO supports creating object transition lifecycle management rules, where
MinIO can automatically move an object to a remote storage "tier". MinIO
supports any S3-compatible service as a remote tier *in addition to* the
following public cloud storage services:
- :ref:`Amazon S3 <minio-lifecycle-management-transition-to-s3>`
- :ref:`Google Cloud Storage <minio-lifecycle-management-transition-to-gcs>`
- :ref:`Microsoft Azure Blob Storage
<minio-lifecycle-management-transition-to-azure>`
MinIO object transition supports use cases like moving aged data from
MinIO clusters in private or public cloud infrastructure to low-cost
public cloud storage solutions.
MinIO manages retrieving tiered objects on-the-fly without any additional
application-side logic. MinIO retains the minimum object metadata required to
support retrieving objects transitioned to a remote tier. MinIO therefore
*requires* exclusive access to the data on the remote storage tier. Object
retrieval assumes no external mutation, migration, or deletion of stored
objects. Object transition is *not* a replacement for backup/recovery
strategies such as :ref:`minio-bucket-replication`.
Use the :mc-cmd:`mc admin tier` command to create a remote target for tiering
data to a supported Cloud Service Provider object storage You can then use the
:mc-cmd:`mc ilm add` command with one of the following commandline options to
create new transition rules on a bucket:
- :mc-cmd-option:`mc ilm add transition-date` to transition objects after
a specified calendar date.
- :mc-cmd-option:`mc ilm add transition-days` to transition object after
a specified number of calendar days.
Versioned Buckets
~~~~~~~~~~~~~~~~~
MinIO adopts :s3-docs:`S3 behavior
<intro-lifecycle-rules.html#intro-lifecycle-rules-actions>` for transition rules
on :ref:`versioned buckets <minio-bucket-versioning>`. Specifically, MinIO by
default applies the transition operation to the *current* object version.
To transition noncurrent object versions, specify the
:mc-cmd-option:`~mc ilm add noncurrentversion-transition-days` option
when creating the transition rule.
.. _minio-lifecycle-management-expiration:
Object Expiration
-----------------
MinIO lifecycle management supports expiring objects on a bucket. Object
"expiration" involves performing a ``DELETE`` operation on the object. For
example, you can create a lifecycle management rule to expire any object
older than 365 days.
.. todo: Diagram of MinIO Expiration
Use the :mc-cmd:`mc ilm add` command with one of the following commandline
options to create new expiration rules on a bucket:
- :mc-cmd-option:`mc ilm add expiry-date` to expire objects after
a specified calendar date.
- :mc-cmd-option:`mc ilm add expiry-days` to expire objects after a
specified number of calendar days.
Versioned Buckets
~~~~~~~~~~~~~~~~~
MinIO adopts :s3-docs:`S3 behavior
<intro-lifecycle-rules.html#intro-lifecycle-rules-actions>` for expiration rules
on :ref:`versioned buckets <minio-bucket-versioning>`. MinIO has two
specific default behaviors for versioned buckets:
- MinIO applies the expiration option to only the *current* object version by
creating a ``DeleteMarker`` as is normal with versioned delete.
To expire noncurrent object versions, specify the
:mc-cmd-option:`~mc ilm add noncurrentversion-expiration-days` option
when creating the expiration rule.
- MinIO does not expire ``DeleteMarkers`` *even if* no other versions of
that object exist.
To expire delete markers when there are no remaining versions for that
object, specify the :mc-cmd-option:`~mc ilm add expired-object-delete-marker`
option when creating the expiration rule.
.. _minio-lifecycle-management-scanner:
Lifecycle Management Object Scanner
-----------------------------------
MinIO uses a built-in scanner to actively check objects against all
configured lifecycle management rules. The scanner is a low-priority process
that yields to high IO workloads to prevent performance spikes triggered
by rule timing. The scanner may therefore not detect an object as eligible
for a configured transition or expiration lifecycle rule until *after*
the lifecycle rule period has passed.
Delayed application of lifecycle management rules is typically associated to
limited node resources and cluster size. Scanner speed tends to slow as
clusters grow as more time is required to visit all buckets and objects.
This can be exacerbated if the cluster hardware is undersized for regular
workloads, as the scanner will yield to high cluster load to avoid performance
loss. Consider regularly checking cluster metrics, capacity, and resource
usage to ensure the cluster hardware is scaling alongside cluster and workload
growth.
.. toctree::
:hidden:
/lifecycle-management/transition-objects-to-s3.rst
/lifecycle-management/transition-objects-to-gcs.rst
/lifecycle-management/transition-objects-to-azure.rst
/lifecycle-management/create-lifecycle-management-expiration-rule.rst

197
source/lifecycle-management/transition-objects-to-azure.rst Normal file
View File

@@ -0,0 +1,197 @@
.. _minio-lifecycle-management-transition-to-azure:
======================================
Transition Objects from MinIO to Azure
======================================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 2
The procedure on this page creates a new object lifecycle management rule that
transition objects from a MinIO bucket to a remote storage tier on the
:abbr:`Azure (Microsoft Azure)` storage backend. This procedure supports use
cases like moving aged data to low-cost public cloud storage solutions after a
certain time period or calendar date.
.. todo: diagram
Requirements
------------
Install and Configure ``mc``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This procedure uses :mc:`mc` for performing operations on the MinIO cluster.
Install :mc:`mc` on a machine with network access to both source and destination
clusters. See the ``mc`` :ref:`Installation Quickstart <mc-install>` for
instructions on downloading and installing ``mc``.
Use the :mc:`mc alias` command to create an alias for the source MinIO cluster.
Alias creation requires specifying an access key for a user on the source and
destination clusters. The specified users must have :ref:`permissions
<minio-lifecycle-management-transition-to-azure-permissions>` for configuring
and applying transition operations.
.. _minio-lifecycle-management-transition-to-azure-permissions:
Required MinIO Permissions
~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO requires the following permissions scoped to the bucket or buckets
for which you are creating lifecycle management rules.
- :policy-action:`s3:PutBucketLifecycle`
- :policy-action:`s3:GetBucketLifecycle`
MinIO also requires the following administrative permissions on the cluster
in which you are creating remote tiers for object transition lifecycle
management rules:
- :policy-action:`admin:SetTier`
- :policy-action:`admin:ListTier`
For example, the following policy provides permission for configuring object
transition lifecycle management rules on any bucket in the cluster:.
.. literalinclude:: /extra/examples/LifecycleManagementAdmin.json
:language: json
:class: copyable
.. _minio-lifecycle-management-transition-to-azure-permissions-remote:
Required Azure Permissions
~~~~~~~~~~~~~~~~~~~~~~~~~~
Object transition lifecycle management rules require additional permissions on
the remote storage tier. Specifically, MinIO requires the
:abbr:`Azure (Microsoft Azure)` credentials provide read, write, list, and
delete permissions for the remote bucket.
Refer to the `Azure RBAC
<https://docs.microsoft.com/en-us/azure/role-based-access-control/>`__
documentation for more complete guidance on configuring the required
permissions.
Considerations
--------------
Lifecycle Management Object Scanner
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
Exclusive Bucket Access
~~~~~~~~~~~~~~~~~~~~~~~
MinIO retains the minimum object metadata required to
support retrieving objects transitioned to a remote tier. MinIO therefore
*requires* exclusive access to the data on the remote storage tier. Object
retrieval assumes no external mutation, migration, or deletion of stored
objects.
MinIO also ignores any objects in the remote bucket or bucket prefix not
explicitly managed by MinIO.
Procedure
---------
1) Configure User Accounts and Policies for Lifecycle Management
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. |permissions| replace:: :ref:`permissions <minio-lifecycle-management-transition-to-azure-permissions>`
.. include:: /includes/common-minio-tiering.rst
:start-after: start-create-transition-user-desc
:end-before: end-create-transition-user-desc
2) Configure the Remote Storage Tier
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc admin tier add` command to add a new remote storage tier:
.. code-block:: shell
:class: copyable
mc admin tier add azure TARGET TIER_NAME \
--endpoint https://HOSTNAME
--bucket BUCKET \
--prefix PREFIX
--account-name ACCOUNT \
--account-key KEY \
--region REGION
The example above uses the following arguments:
.. list-table::
:header-rows: 1
:widths: 40 60
:width: 100%
* - Argument
- Description
* - :mc-cmd:`TARGET <mc admin tier add TARGET>`
- The :mc:`alias <mc alias>` of the MinIO deployment on which to configure
the remote tier.
* - :mc-cmd:`TIER_NAME <mc admin tier add TIER_NAME>`
- The name to associate with the new :abbr:`Azure (Microsoft Azure)` blob
remote storage tier. This value is required in the next step.
* - :mc-cmd:`HOSTNAME <mc admin tier add endpoint>`
- The URL endpoint for the :abbr:`Azure (Microsoft Azure)` storage
backend.
* - :mc-cmd:`BUCKET <mc admin tier add bucket>`
- The name of the bucket on the :abbr:`Azure (Microsoft Azure)` storage
backend to which MinIO transitions objects.
* - :mc-cmd:`PREFIX <mc admin tier add prefix>`
- The optional bucket prefix within which MinIO transitions objects.
Omit this argument to transition objects to the bucket root.
* - :mc-cmd:`ACCOUNT <mc admin tier add account-name>`
- The account name MinIO uses to access the bucket. The account name
*must* correspond to an :abbr:`Azure (Microsoft Azure)` user with the
required :ref:`permissions
<minio-lifecycle-management-transition-to-azure-permissions-remote>`.
* - :mc-cmd:`KEY <mc admin tier add account-key>`
- The corresponding key for the specified ``ACCOUNT``.
* - :mc-cmd:`REGION <mc admin tier add region>`
- The :abbr:`Azure (Microsoft Azure)` blob storage region of the specified
``BUCKET``. You can safely omit this option if the ``HOSTNAME`` includes
the region.
3) Create and Apply the Transition Rule
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common-minio-tiering.rst
:start-after: start-create-transition-rule-desc
:end-before: end-create-transition-rule-desc
3) Verify the Transition Rule
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc ilm list` command to review the configured transition
rules:
.. code-block:: shell
:class: copyable
mc ilm list ALIAS/PATH --transition
- Replace :mc-cmd:`ALIAS <mc ilm list TARGET>` with the :mc:`alias <mc alias>`
of the MinIO deployment.
- Replace :mc-cmd:`PATH <mc ilm list TARGET>` with the name of the bucket for
which to retrieve the configured lifecycle management rules.

195
source/lifecycle-management/transition-objects-to-gcs.rst Normal file
View File

@@ -0,0 +1,195 @@
.. _minio-lifecycle-management-transition-to-gcs:
====================================
Transition Objects from MinIO to GCS
====================================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 2
The procedure on this page creates a new object lifecycle management rule that
transition objects from a MinIO bucket to a remote storage tier on the Google
Cloud Storage backend. This procedure supports use cases like moving aged data
to low-cost public cloud storage solutions after a certain time period or
calendar date.
.. todo: diagram
Requirements
------------
Install and Configure ``mc``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This procedure uses :mc:`mc` for performing operations on the MinIO cluster.
Install :mc:`mc` on a machine with network access to both source and destination
clusters. See the ``mc`` :ref:`Installation Quickstart <mc-install>` for
instructions on downloading and installing ``mc``.
Use the :mc:`mc alias` command to create an alias for the source MinIO cluster.
Alias creation requires specifying an access key for a user on the source and
destination clusters. The specified users must have :ref:`permissions
<minio-lifecycle-management-transition-to-gcs-permissions>` for configuring and
applying transition operations.
.. _minio-lifecycle-management-transition-to-gcs-permissions:
Required MinIO Permissions
~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO requires the following permissions scoped to the bucket or buckets
for which you are creating lifecycle management rules.
- :policy-action:`s3:PutBucketLifecycle`
- :policy-action:`s3:GetBucketLifecycle`
MinIO also requires the following administrative permissions on the cluster
in which you are creating remote tiers for object transition lifecycle
management rules:
- :policy-action:`admin:SetTier`
- :policy-action:`admin:ListTier`
For example, the following policy provides permission for configuring object
transition lifecycle management rules on any bucket in the cluster:.
.. literalinclude:: /extra/examples/LifecycleManagementAdmin.json
:language: json
:class: copyable
.. _minio-lifecycle-management-transition-to-gcs-permissions-remote:
Required GCS Permissions
~~~~~~~~~~~~~~~~~~~~~~~~
Object transition lifecycle management rules require additional permissions on
the remote storage tier. Specifically, MinIO requires the
:abbr:`GCS (Google Cloud Storage)` credentials provide read, write, list, and
delete permissions for the remote bucket.
Refer to the `GCS IAM permissions
<https://cloud.google.com/storage/docs/access-control/iam-permissions>`__
documentation for more complete guidance on configuring the required
permissions.
Considerations
--------------
Lifecycle Management Object Scanner
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
Exclusive Bucket Access
~~~~~~~~~~~~~~~~~~~~~~~
MinIO retains the minimum object metadata required to
support retrieving objects transitioned to a remote tier. MinIO therefore
*requires* exclusive access to the data on the remote storage tier. Object
retrieval assumes no external mutation, migration, or deletion of stored
objects.
MinIO also ignores any objects in the remote bucket or bucket prefix not
explicitly managed by MinIO.
Procedure
---------
1) Configure User Accounts and Policies for Lifecycle Management
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. |permissions| replace:: :ref:`permissions <minio-lifecycle-management-transition-to-gcs-permissions>`
.. include:: /includes/common-minio-tiering.rst
:start-after: start-create-transition-user-desc
:end-before: end-create-transition-user-desc
2) Configure the Remote Storage Tier
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc admin tier add` command to add a new Google Cloud Storage
service as the remote storage tier:
.. code-block:: shell
:class: copyable
mc admin tier add gcs TARGET TIER_NAME \
--endpoint https://HOSTNAME \
--bucket BUCKET \
--prefix PREFIX
--credentials-file CREDENTIALS \
--region REGION
The example above uses the following arguments:
.. list-table::
:header-rows: 1
:widths: 40 60
:width: 100%
* - Argument
- Description
* - :mc-cmd:`TARGET <mc admin tier add TARGET>`
- The :mc:`alias <mc alias>` of the MinIO deployment on which to configure
the :abbr:`GCS (Google Cloud Storage)` remote tier.
* - :mc-cmd:`TIER_NAME <mc admin tier add TIER_NAME>`
- The name to associate with the new :abbr:`GCS (Google Cloud Storage)`
remote storage tier. This value is required in the next step.
* - :mc-cmd:`HOSTNAME <mc admin tier add endpoint>`
- The URL endpoint for the :abbr:`GCS (Google Cloud Storage)` storage
backend.
* - :mc-cmd:`BUCKET <mc admin tier add bucket>`
- The name of the bucket on the :abbr:`GCS (Google Cloud Storage)` storage
backend to which MinIO transitions objects.
* - :mc-cmd:`PREFIX <mc admin tier add prefix>`
- The optional bucket prefix within which MinIO transitions objects.
Omit this argument to transition objects to the bucket root.
* - :mc-cmd:`CREDENTIALS <mc admin tier add credentials-file>`
- The `credential file
<https://cloud.google.com/docs/authentication/getting-started>`__ for a
user on the remote GCS tier. The specified user credentials *must*
correspond to a GCS user with the required
:ref:`permissions
<minio-lifecycle-management-transition-to-gcs-permissions-remote>`.
* - :mc-cmd:`REGION <mc admin tier add region>`
- The :abbr:`GCS (Google Cloud Storage)` region of the specified
``BUCKET``. You can safely omit this
option if the ``HOSTNAME`` includes the region.
3) Create and Apply the Transition Rule
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common-minio-tiering.rst
:start-after: start-create-transition-rule-desc
:end-before: end-create-transition-rule-desc
3) Verify the Transition Rule
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc ilm list` command to review the configured transition
rules:
.. code-block:: shell
:class: copyable
mc ilm list ALIAS/PATH --transition
- Replace :mc-cmd:`ALIAS <mc ilm list TARGET>` with the :mc:`alias <mc alias>`
of the MinIO deployment.
- Replace :mc-cmd:`PATH <mc ilm list TARGET>` with the name of the bucket for
which to retrieve the configured lifecycle management rules.

213
source/lifecycle-management/transition-objects-to-s3.rst Normal file
View File

@@ -0,0 +1,213 @@
.. _minio-lifecycle-management-transition-to-s3:
===================================
Transition Objects from MinIO to S3
===================================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 2
The procedure on this page creates a new object lifecycle management rule that
transition objects from a MinIO bucket to a remote storage tier on the Amazon
Web Services S3 storage backend *or* an S3-compatible service. This procedure
supports use cases such as tiering objects to low-cost or archival storage after
a certain time period or calendar date.
.. todo: diagram
Requirements
------------
Install and Configure ``mc``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This procedure uses :mc:`mc` for performing operations on the MinIO cluster.
Install :mc:`mc` on a machine with network access to both source and destination
clusters. See the ``mc`` :ref:`Installation Quickstart <mc-install>` for
instructions on downloading and installing ``mc``.
Use the :mc:`mc alias` command to create an alias for the source MinIO cluster.
Alias creation requires specifying an access key for a user on the source and
destination clusters. The specified users must have :ref:`permissions
<minio-lifecycle-management-transition-to-s3-permissions>` for configuring and
applying transition operations.
.. _minio-lifecycle-management-transition-to-s3-permissions:
Required MinIO Permissions
~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO requires the following permissions scoped to the bucket or buckets
for which you are creating lifecycle management rules.
- :policy-action:`s3:PutBucketLifecycle`
- :policy-action:`s3:GetBucketLifecycle`
MinIO also requires the following administrative permissions on the cluster
in which you are creating remote tiers for object transition lifecycle
management rules:
- :policy-action:`admin:SetTier`
- :policy-action:`admin:ListTier`
For example, the following policy provides permission for configuring object
transition lifecycle management rules on any bucket in the cluster:.
.. literalinclude:: /extra/examples/LifecycleManagementAdmin.json
:language: json
:class: copyable
.. _minio-lifecycle-management-transition-to-s3-permissions-remote:
Required S3 Permissions
~~~~~~~~~~~~~~~~~~~~~~~
Object transition lifecycle management rules require additional permissions
on the remote storage tier. Specifically, MinIO requires the remote
tier credentials provide read, write, list, and delete permissions for the
remote bucket.
For example, the following policy provides the necessary permission
for transitioning objects into and out of the remote tier:
.. literalinclude:: /extra/examples/LifecycleManagementUser.json
:language: json
:class: copyable
Modify the ``Resource`` for the bucket into which MinIO tiers objects.
Refer to the :aws-docs:`Amazon S3 Permissions
<service-authorization/latest/reference/list_amazons3.html#amazons3-actions-as-permissions>`
documentation for more complete guidance on configuring the required
permissions.
Considerations
--------------
Lifecycle Management Object Scanner
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
Exclusive Bucket Access
~~~~~~~~~~~~~~~~~~~~~~~
MinIO retains the minimum object metadata required to
support retrieving objects transitioned to a remote tier. MinIO therefore
*requires* exclusive access to the data on the remote storage tier. Object
retrieval assumes no external mutation, migration, or deletion of stored
objects.
MinIO also ignores any objects in the remote bucket or bucket prefix not
explicitly managed by MinIO.
Procedure
---------
1) Configure User Accounts and Policies for Lifecycle Management
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. |permissions| replace:: :ref:`permissions <minio-lifecycle-management-transition-to-s3-permissions>`
.. include:: /includes/common-minio-tiering.rst
:start-after: start-create-transition-user-desc
:end-before: end-create-transition-user-desc
2) Configure the Remote Storage Tier
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc admin tier add` command to add an Amazon S3 service as the
new remote storage tier:
.. code-block:: shell
:class: copyable
mc admin tier add s3 TARGET TIER_NAME \
--endpoint https://HOSTNAME \
--bucket BUCKET \
--prefix PREFIX
--access-key ACCESS_KEY \
--secret-key SECRET_KEY \
--region REGION \
--storage-class STORAGE_CLASS
The example above uses the following arguments:
.. list-table::
:header-rows: 1
:widths: 40 60
:width: 100%
* - Argument
- Description
* - :mc-cmd:`TARGET <mc admin tier add TARGET>`
- The :mc:`alias <mc alias>` of the MinIO deployment on which to configure
the S3 remote tier.
* - :mc-cmd:`TIER_NAME <mc admin tier add TIER_NAME>`
- The name to associate with the new S3 remote storage tier. This value
is required in the next step.
* - :mc-cmd:`HOSTNAME <mc admin tier add endpoint>`
- The URL endpoint for the S3 storage backend.
* - :mc-cmd:`BUCKET <mc admin tier add bucket>`
- The name of the bucket on the S3 storage backend to which MinIO
transitions objects.
* - :mc-cmd:`PREFIX <mc admin tier add prefix>`
- The optional bucket prefix within which MinIO transitions objects.
Omit this argument to transition objects to the bucket root.
* - :mc-cmd:`ACCESS_KEY <mc admin tier add access-key>`
- The S3 access key MinIO uses to access the bucket. The
access key *must* correspond to an IAM user with the
required
:ref:`permissions
<minio-lifecycle-management-transition-to-s3-permissions-remote>`.
* - :mc-cmd:`SECRET_KEY <mc admin tier add secret-key>`
- The corresponding secret key for the specified ``ACCESS_KEY``.
* - :mc-cmd:`REGION <mc admin tier add region>`
- The AWS S3 region of the specified ``BUCKET``. You can safely omit this
option if the ``HOSTNAME`` includes the region.
* - :mc-cmd:`STORAGE_CLASS <mc admin tier add storage-class>`
- The S3 storage class to which MinIO transitions objects. Specify
one of the following supported storage classes:
- ``STANDARD``
- ``REDUCED``
3) Create and Apply the Transition Rule
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common-minio-tiering.rst
:start-after: start-create-transition-rule-desc
:end-before: end-create-transition-rule-desc
3) Verify the Transition Rule
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc ilm list` command to review the configured transition
rules:
.. code-block:: shell
:class: copyable
mc ilm list ALIAS/PATH --transition
- Replace :mc-cmd:`ALIAS <mc ilm list TARGET>` with the :mc:`alias <mc alias>`
of the MinIO deployment.
- Replace :mc-cmd:`PATH <mc ilm list TARGET>` with the name of the bucket for
which to retrieve the configured lifecycle management rules.

5
source/reference/minio-cli/minio-mc-admin.rst
View File

@@ -90,6 +90,11 @@ The following table lists :mc-cmd:`mc admin` commands:
:start-after: start-mc-admin-service-desc
:end-before: end-mc-admin-service-desc
* - :mc:`mc admin tier`
- .. include:: /reference/minio-cli/minio-mc-admin/mc-admin-tier.rst
:start-after: start-mc-admin-tier-desc
:end-before: end-mc-admin-tier-desc
* - :mc:`mc admin top`
- .. include:: /reference/minio-cli/minio-mc-admin/mc-admin-top.rst
:start-after: start-mc-admin-top-desc

386
source/reference/minio-cli/minio-mc-admin/mc-admin-tier.rst Normal file
View File

@@ -0,0 +1,386 @@
=================
``mc admin tier``
=================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 2
.. mc:: mc admin tier
Description
-----------
.. start-mc-admin-tier-desc
The :mc:`mc admin tier` command configures a remote supported S3-compatible
service for supporting MinIO
:ref:`Lifecycle Management: Object Transition ("Tiering")
<minio-lifecycle-management-expiration>`.
.. end-mc-admin-tier-desc
.. admonition:: Use ``mc admin`` on MinIO Deployments Only
:class: note
.. include:: /includes/facts-mc-admin.rst
:start-after: start-minio-only
:end-before: end-minio-only
Supported S3 Services
~~~~~~~~~~~~~~~~~~~~~
:mc:`mc admin tier` supports *only* the following S3-compatible services
as a remote target for object tiering:
- Amazon S3
- Google Cloud Storage
- Azure Blob Storage
Required Permissions
~~~~~~~~~~~~~~~~~~~~
MinIO requires the following permissions scoped to to the bucket or buckets
for which you are creating lifecycle management rules.
- :policy-action:`s3:PutBucketLifecycle`
- :policy-action:`s3:GetBucketLifecycle`
MinIO also requires the following administrative permissions on the cluster
in which you are creating remote tiers for object transition lifecycle
management rules:
- :policy-action:`admin:SetTier`
- :policy-action:`admin:ListTier`
For example, the following policy provides permission for configuring object
transition lifecycle management rules on any bucket in the cluster:.
.. literalinclude:: /extra/examples/LifecycleManagementAdmin.json
:language: json
:class: copyable
Transition Permissions
++++++++++++++++++++++
Object transition lifecycle management rules require additional permissions
on the remote storage tier. Specifically, MinIO requires the remote
tier credentials provide read, write, list, and delete permissions.
For example, if the remote storage tier implements AWS IAM policy-based
access control, the following policy provides the necessary permission
for transitioning objects into and out of the remote tier:
.. literalinclude:: /extra/examples/LifecycleManagementUser.json
:language: json
:class: copyable
Modify the ``Resource`` for the bucket into which MinIO tiers objects.
Defer to the documentation for the supported tiering targets for more complete
information on configuring users and permissions to support MinIO tiering:
- :aws-docs:`Amazon S3 Permissions <service-authorization/latest/reference/list_amazons3.html#amazons3-actions-as-permissions>`
- `Google Cloud Storage Access Control <https://cloud.google.com/storage/docs/access-control>`__
- `Authorizing access to data in Azure storage <https://docs.microsoft.com/en-us/azure/storage/common/storage-auth?toc=/azure/storage/blobs/toc.json>`__
Syntax
------
.. mc-cmd:: add
:fullpath:
Creates a new remote storage tier for transitioning objects using MinIO
lifecycle management rules.
.. important::
MinIO does not support removing remote storage tiers. Ensure the
storage backend supports the intended workload *prior* to
adding it as a remote tier target.
The command has the following syntax:
.. code-block:: shell
:class: copyable
mc admin tier add TIER_TYPE TARGET TIER_NAME [FLAGS]
The command accepts the following arguments:
.. mc-cmd:: TIER_TYPE
*Required*
The Cloud Service Provider storage backend ("Tier") to which MinIO
transitions objects. Specify *one* of the following supported values:
.. list-table::
:stub-columns: 1
:width: 100%
:widths: 30 70
* - ``s3``
- Use AWS S3 *or* a remote MinIO deployment as the storage
backend for the new Tier.
Requires specifying the following additional options:
- :mc-cmd-option:`~mc admin tier add access-key`
- :mc-cmd-option:`~mc admin tier add secret-key`
* - ``azure``
- Use :abbr:`Azure (Microsoft Azure)` Blob Storage as the storage
backend for the new Tier.
Requires specifying the following additional options:
- :mc-cmd-option:`~mc admin tier add account-name`
- :mc-cmd-option:`~mc admin tier add account-key`
* - ``gcs``
- Use :abbr:`GCP (Google Cloud Platform)` Cloud Storage as the
storage backend for the new Tier.
Requires specifying the following additional option:
- :mc-cmd-option:`~mc admin tier add credentials-file`
.. mc-cmd:: TARGET
*Required*
The :mc-cmd:`alias <mc alias>` of a configured MinIO deployment on which
the command creates the new remote tier.
.. mc-cmd:: TIER_NAME
*Required*
The name to associate with the new remote tier. The name *must*
be unique across all configured tiers on the MinIO cluster.
.. mc-cmd:: endpoint
:option:
*Required*
The URL endpoint for the cloud service provider. The URL endpoint
*must* resolve to the provider specified to
:mc-cmd:`~mc admin tier add TIER_TYPE`.
.. mc-cmd:: access-key
:option:
*Required*
The access key for a user on the remote S3 tier. The user must
have permission to perform read/write/list/delete operations on the
remote bucket or bucket prefix.
Required if :mc-cmd:`~mc admin tier add TIER_TYPE` is ``s3``.
This option has no effect for any other value of ``TIER_TYPE``.
.. mc-cmd:: secret-key
:option:
*Required*
The secret key for a user on the remote S3 tier.
Required if :mc-cmd:`~mc admin tier add TIER_TYPE` is ``s3``.
This option has no effect for any other value of ``TIER_TYPE``.
.. mc-cmd:: account-name
:option:
*Required*
The account name for a user on the remote Azure tier. The user
must have permission to perform read/write/list/delete operations on the
remote bucket or bucket prefix.
Required if :mc-cmd:`~mc admin tier add TIER_TYPE` is ``azure``.
This option has no effect for any other value of ``TIER_TYPE``.
.. mc-cmd:: account-key
:option:
*Required*
The account key for a user on the remote Azure tier.
Required if :mc-cmd:`~mc admin tier add TIER_TYPE` is ``azure``.
This option has no effect for any other value of ``TIER_TYPE``.
.. mc-cmd:: credentials-file
:option:
*Required*
The `credential file
<https://cloud.google.com/docs/authentication/getting-started>`__ for a
user on the remote GCS tier. The user must have permission to perform
read/write/list/delete operations on the remote bucket or bucket prefix.
Required if :mc-cmd:`~mc admin tier add TIER_TYPE` is ``gcs``.
This option has no effect for any other value of ``TIER_TYPE``.
.. mc-cmd:: bucket
:option:
*Required*
The bucket on the remote tier to which MinIO transitions objects.
.. mc-cmd:: prefix
:option:
*Optional*
The prefix path for the specified :mc-cmd:`~mc admin tier add bucket`
to which MinIO transitions objects.
Omit this field to transition objects into the bucket root.
.. mc-cmd:: storage-class
:option:
*Optional*
The AWS storage class to use for objects transitioned by
MinIO. MinIO supports only the following storage classes:
- ``STANDARD``
- ``REDUCED_REDUNDANCY``
Defaults to ``S3_STANDARD`` if omitted.
This option only applies if :mc-cmd:`~mc admin tier add TIER_TYPE` is
``s3``. This option has no effect for any other value of ``TIER_TYPE``.
.. mc-cmd:: region
:option:
*Optional*
The S3 backend region for the specified
:mc-cmd:`~mc admin tier add TIER_TYPE`, such as ``us-west-1``.
This option only applies if :mc-cmd:`~mc admin tier add TIER_TYPE` is
``s3``. This option has no effect for any other value of ``TIER_TYPE``.
.. mc-cmd:: edit
:fullpath:
Modify or remove a remote storage tier from a MinIO cluster. Remote
storage tiers support transitioning objects using MinIO lifecycle
management rules.
The command has the following syntax:
.. code-block:: shell
:class: copyable
mc admin tier edit TARGET TIER_NAME [FLAGS]
The command accepts the following arguments:
.. mc-cmd:: TARGET
*Required*
The :mc-cmd:`alias <mc alias>` of a configured MinIO deployment.
.. mc-cmd:: TIER_NAME
*Required*
The name of the remote tier the command modifies. The value
corresponds to the :mc-cmd:`mc admin tier add TIER_NAME` specified
when creating the remote tier.
.. mc-cmd:: access-key
:option:
*Optional*
The access key for a user on the remote S3 tier. The user must
have permission to perform read/write/list/delete operations on the
remote bucket or bucket prefix.
This option only applies to remote storage tiers with
:mc-cmd:`~mc admin tier add TIER_TYPE` is ``s3``.
This option has no effect for any other ``TIER_TYPE``.
.. mc-cmd:: secret-key
:option:
*Optional*
The secret key for a user on the remote S3 tier.
This option only applies to remote storage tiers with
:mc-cmd:`~mc admin tier add TIER_TYPE` is ``s3``.
This option has no effect for any other ``TIER_TYPE``.
.. mc-cmd:: account-name
:option:
*Required*
The account name for a user on the remote Azure tier. The user
must have permission to perform read/write/list/delete operations on the
remote bucket or bucket prefix.
This option only applies to remote storage tiers with
:mc-cmd:`~mc admin tier add TIER_TYPE` is ``azure``.
This option has no effect for any other ``TIER_TYPE``.
.. mc-cmd:: account-key
:option:
*Required*
The account key for a user on the remote Azure tier.
This option only applies to remote storage tiers with
:mc-cmd:`~mc admin tier add TIER_TYPE` is ``azure``.
This option has no effect for any other ``TIER_TYPE``.
.. mc-cmd:: credentials-file
:option:
*Required*
The credential file for a user on the remote GCS tier. The user must have
permission to perform read/write/list/delete operations on the remote bucket
or bucket prefix.
This option only applies to remote storage tiers with
:mc-cmd:`~mc admin tier add TIER_TYPE` is ``gcs``.
This option has no effect for any other ``TIER_TYPE``.
.. mc-cmd:: ls
:fullpath:
List all remote storage tiers on a MinIO cluster. Remote storage tiers
support transitioning objects using MinIO lifecycle management rules.
The command has the following syntax:
.. code-block:: shell
:class: copyable
mc admin tier ls TARGET [FLAGS]
The command accepts the following arguments:
.. mc-cmd:: TARGET
*Required*
The :mc-cmd:`alias <mc alias>` of a configured MinIO deployment.

258
source/reference/minio-cli/minio-mc/mc-ilm.rst
View File

@@ -22,6 +22,24 @@ information.
.. end-mc-ilm-desc
Lifecycle Management Object Scanner
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
Expiry vs Transition
~~~~~~~~~~~~~~~~~~~~
MinIO supports specifying both expiry and transition rules in the same
bucket or bucket prefix. MinIO can execute an expiration rule on an object
regardless of its transition status. Use
:mc-cmd:`mc ilm list` to review the currently configured object lifecycle
management rules for any potential interactions between expiry and transition
rules.
Examples
--------
@@ -34,14 +52,7 @@ expire bucket contents after a specific date.
.. code-block:: shell
:class: copyable
mc ilm add --id "RULE" --expiry-date "DATE" ALIAS/PATH
- Replace :mc-cmd:`RULE <mc ilm add id>` with the unique name of the lifecycle
management rule.
- Replace :mc-cmd:`DATE <mc ilm add expiry-date>` with the future date after
which to expire the object. For example, specify "2021-01-01" to expire
objects after January 1st, 2021.
mc ilm add ALIAS/PATH --expiry-date "DATE"
- Replace :mc-cmd:`ALIAS <mc ilm add TARGET>` with the
:mc:`alias <mc alias>` of the S3-compatible host.
@@ -49,6 +60,10 @@ expire bucket contents after a specific date.
- Replace :mc-cmd:`PATH <mc ilm add TARGET>` with the path to the bucket on the
S3-compatible host.
- Replace :mc-cmd:`DATE <mc ilm add expiry-date>` with the calendar date after
which to expire the object. For example, specify "2021-01-01" to expire
objects after January 1st, 2021.
Expire Bucket Contents After Number of Days
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -58,14 +73,7 @@ expire bucket contents a number of days after object creation:
.. code-block:: shell
:class: copyable
mc ilm add --id "RULE" --expiry-days "DAYS" ALIAS/PATH
- Replace :mc-cmd:`RULE <mc ilm add id>` with the unique name of the lifecycle
management rule.
- Replace :mc-cmd:`DATE <mc ilm add expiry-date>` with the number of days after
which to expire the object. For example, specify ``30d`` to expire the
object 30 days after creation.
mc ilm add ALIAS/PATH --expiry-days "DAYS"
- Replace :mc-cmd:`ALIAS <mc ilm add TARGET>` with the
:mc:`alias <mc alias>` of the S3-compatible host.
@@ -73,6 +81,10 @@ expire bucket contents a number of days after object creation:
- Replace :mc-cmd:`PATH <mc ilm add TARGET>` with the path to the bucket on the
S3-compatible host.
- Replace :mc-cmd:`DATE <mc ilm add expiry-date>` with the number of days after
which to expire the object. For example, specify ``30d`` to expire the
object 30 days after creation.
List Bucket Lifecycle Management Rules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -83,10 +95,10 @@ Use :mc-cmd:`mc ilm list` to list a bucket's lifecycle management rules:
mc ilm list ALIAS/PATH
- Replace :mc-cmd:`ALIAS <mc ilm add TARGET>` with the
- Replace :mc-cmd:`ALIAS <mc ilm list TARGET>` with the
:mc:`alias <mc alias>` of the S3-compatible host.
- Replace :mc-cmd:`PATH <mc ilm add TARGET>` with the path to the bucket on the
- Replace :mc-cmd:`PATH <mc ilm list TARGET>` with the path to the bucket on the
S3-compatible host.
Remove a Bucket Lifecycle Management Rule
@@ -99,13 +111,13 @@ Use :mc-cmd:`mc ilm remove` to remove a bucket lifecycle management rule:
mc ilm remove --id "RULE" ALIAS/PATH
- Replace :mc-cmd:`RULE <mc ilm add id>` with the unique name of the lifecycle
- Replace :mc-cmd:`RULE <mc ilm remove id>` with the unique name of the lifecycle
management rule.
- Replace :mc-cmd:`ALIAS <mc ilm add TARGET>` with the
- Replace :mc-cmd:`ALIAS <mc ilm remove TARGET>` with the
:mc:`alias <mc alias>` of the S3-compatible host.
- Replace :mc-cmd:`PATH <mc ilm add TARGET>` with the path to the bucket on the
- Replace :mc-cmd:`PATH <mc ilm remove TARGET>` with the path to the bucket on the
S3-compatible host.
@@ -121,7 +133,7 @@ Syntax
.. code-block:: shell
:class: copyable
mc ilm list [FLAGS] TARGET
mc ilm list TARGET [FLAGS]
The subcommand supports the following arguments:
@@ -161,36 +173,27 @@ Syntax
.. mc-cmd:: add
:fullpath:
Adds or modifies bucket lifecycle management rules. The command has
Adds a new bucket lifecycle management rules. The command has
the following syntax:
.. code-block:: shell
:class: copyable
mc ilm add [FLAGS] TARGET
mc ilm add TARGET [FLAGS]
.. mc-cmd:: TARGET
*Required*
The full path to the bucket from which to add or modify the lifecycle
The full path to the bucket from which to add the lifecycle
management rule. Specify the :mc-cmd:`alias <mc alias>` of a configured S3
service as the prefix to the ``TARGET`` path.
Specify all ``[FLAGS]`` *prior* to the ``TARGET``.
For example:
.. code-block:: shell
mc ilm list [FLAGS] play/mybucket
.. mc-cmd:: id
:option:
*Required* The unique name of the rule. Specify the
:mc-cmd-option:`mc ilm add id` of an existing rule to modify the
lifecycle configuration of that rule.
mc ilm add play/mybucket [FLAGS]
.. mc-cmd:: prefix
:option:
@@ -208,46 +211,190 @@ Syntax
One or more ampersand ``&``-delimited key-value pairs describing
the object tags to which to apply the lifecycle configuration rule.
This option is mutually exclusive with the following option:
- :mc-cmd-option:`~mc ilm add expired-object-delete-marker`
.. mc-cmd:: expiry-date
:option:
The ISO-8601-formatted date after which MinIO removes objects
covered by the rule. Specifying a date that is *prior* to the
current date marks all objects covered by the rule for removal.
The ISO-8601-formatted calendar date until which MinIO retains an object
after being created. MinIO marks the object for deletion once the
system host datetime passes that calendar date.
Specifying a calendar date that is *prior* to the current system host
datetime marks all objects covered by the rule for deletion.
For versioned buckets, the expiry rule applies only to the *current*
object version. Use the
:mc-cmd-option:`~mc ilm add noncurrentversion-expiration-days` option
to apply expiration behavior to noncurrent object versions.
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
.. mc-cmd:: expiry-days
:option:
The number of days from object creation after which MinIO removes
objects covered by the rule.
The number of days to retain an object after being created. MinIO
marks the object for deletion after the specified number of days pass.
For versioned buckets, the expiry rule applies only to the *current*
object version. Use the
:mc-cmd-option:`~mc ilm add noncurrentversion-expiration-days` option
to apply expiration behavior to noncurrent object versions.
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
.. mc-cmd:: noncurrentversion-expiration-days
:option:
The number of days to retain an object version after becoming
*non-current* (i.e. a different version of that object is now the `HEAD`).
MinIO marks noncurrent object versions for deletion after the
specified number of days pass.
This option has the same behavior as the
S3 ``NoncurrentVersionExpiration`` action.
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
.. mc-cmd:: expired-object-delete-marker
:option:
Specify ``true`` to direct MinIO to remove delete markers for
objects with no remaining object versions. Specifically, the delete
marker is the *only* remaining "version" of the given object.
Defaults to ``false``.
This option is mutually exclusive with the following option:
- :mc-cmd-option:`~mc ilm add tags`
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
.. mc-cmd:: transition-date
:option:
The ISO-8601-formatted date after which MinIO transitions objects
covered by the rule to the specified ``--storage-class``.
Specifying a date that is *prior* to the current date marks all
objects covered by the rule for transition.
The ISO-8601-formatted calendar date after which MinIO marks an object as
eligible for transition to the remote tier. MinIO transitions the object
to the configured remote storage tier specified to the
:mc-cmd-option:`~mc ilm add storage-class` once the system host datetime
passes that calendar date.
For versioned buckets, the transition rule applies only to the *current*
object version. Use the
:mc-cmd-option:`~mc ilm add noncurrentversion-transition-days` option
to apply transition behavior to noncurrent object versions.
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
.. mc-cmd:: transition-days
:option:
The number of days from object creation after which MinIO
transitions objects covered by the rule to the specified
``--storage-class``.
The number of calendar days from object creation after which MinIO marks
an object as eligible for transition. MinIO transitions the object to the
configured remote storage tier specified to the
:mc-cmd-option:`~mc ilm add storage-class`.
For versioned buckets, the transition rule applies only to the *current*
object version. Use the
:mc-cmd-option:`~mc ilm add noncurrentversion-transition-days` option
to apply transition behavior to noncurrent object versions.
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
.. mc-cmd:: noncurrentversion-transition-days
:option:
The number of days an object has been non-current (i.e. replaced by a
newer version of that same object) after which MinIO marks the object
version as eligible for transition. MinIO transitions the object to the
configured remote storage tier specified to the
:mc-cmd-option:`~mc ilm add storage-class` once the system host datetime
passes that calendar date.
This option has no effect on non-versioned buckets.
This option has the same behavior as the
S3 ``NoncurrentVersionTransition`` action.
MinIO uses a scanner process to check objects against all configured
lifecycle management rules. Slow scanning due to high IO workloads or
limited system resources may delay application of lifecycle management
rules. See :ref:`minio-lifecycle-management-scanner` for more information.
.. mc-cmd:: storage-class
:option:
The Amazon S3 storage class to transition objects covered by the
rule. See :s3-docs:`Transition objects using Amazon S3 Lifecycle
<lifecycle-transition-general-considerations.html>` for more
information on S3 storage classes.
The remote storage tier to which MinIO
:ref:`transition objects <minio-lifecycle-management-tiering>`.
Specify a remote storage tier created by :mc-cmd:`mc admin tier`.
If using :mc-cmd:`mc ilm add` against an Amazon S3 service, this argument
is the Amazon S3 storage class to transition objects covered by the rule.
See :s3-docs:`Transition objects using Amazon S3 Lifecycle
<lifecycle-transition-general-considerations.html>` for more information
on S3 storage classes.
.. mc-cmd:: disable
:option:
Disables the rule with matching :mc-cmd-option:`~mc ilm add id`.
Disables the rule.
.. mc-cmd:: edit
:fullpath:
Edits an existing lifecycle management rule in the bucket. The command
has the following syntax:
.. code-block:: shell
:class: copyable
mc ilm edit --id "RULE_ID" TARGET [FLAGS]
The command supports the following arguments:
.. mc-cmd:: TARGET
*Required*
The full path to the bucket from which to modify the
specified lifecycle management rule. Specify the :mc-cmd:`alias
<mc alias>` of a configured S3 service as the prefix to the
``TARGET`` path.
For example:
.. code-block:: shell
mc ilm edit --id "RULE_ID" play/mybucket [FLAGS]
.. mc-cmd:: id
:option:
*Required*
The unique ID of the rule. Use :mc-cmd:`mc ilm list` to list bucket rules
and retrieve the ``id`` for the rule you want to modify.
.. mc-cmd:: remove
:fullpath:
@@ -277,7 +424,10 @@ Syntax
.. mc-cmd:: id
*Required* The unique name of the rule.
*Required*
The unique ID of the rule. Use :mc-cmd:`mc ilm list` to list bucket rules
and retrieve the ``id`` for the rule you want to remove.
Mutually exclusive with :mc-cmd-option:`mc ilm remove all`

10
source/security/IAM/iam-policies.rst
View File

@@ -547,6 +547,16 @@ services:
Allows getting bucket targets
.. policy-action:: admin:SetTier
Allows creating and modifying remote storage tiers using the
:mc-cmd:`mc admin tier` command.
.. policy-action:: admin:ListTier
Allows listing configured remote storage tiers using the
:mc-cmd:`mc admin tier` command.
.. _minio-policy-conditions:
Supported S3 Policy Condition Keys