mirror of
https://github.com/minio/docs.git
synced 2025-07-30 07:03:26 +03:00
Updates for minio RELEASE.2022-08-22T23-53-06Z (#648)
Doc updates related to for minio RELEASE.2022-08-22T23-53-06Z - Adds note about idempotent delete markers, which diverges from standard S3 - Adds info about queueing and re-attempting failed bucket replication tasks - Cleans up `mc ilm add` examples throughout the docs that did not reflect recent flag name changes - Adds a missing metric to Metrics and Alerts Closes #537
This commit is contained in:
@ -256,6 +256,9 @@ workers operating on that queue. MinIO continuously works to replicate and
|
|||||||
remove objects from the queue while scanning for new unreplicated objects to
|
remove objects from the queue while scanning for new unreplicated objects to
|
||||||
add to the queue.
|
add to the queue.
|
||||||
|
|
||||||
|
MinIO queues failed replication operations and retries those operations until replication succeeds.
|
||||||
|
This helps keep replication up-to-date without relying on the MinIO scanner to notice unreplicated object versions.
|
||||||
|
|
||||||
.. versionchanged:: RELEASE.2022-08-11T04-37-28Z
|
.. versionchanged:: RELEASE.2022-08-11T04-37-28Z
|
||||||
|
|
||||||
Failed or pending replications requeue automatically when performing a list or any ``GET`` or ``HEAD`` API method.
|
Failed or pending replications requeue automatically when performing a list or any ``GET`` or ``HEAD`` API method.
|
||||||
|
@ -62,13 +62,13 @@ transition lifecycle management rules on any bucket in the cluster:.
|
|||||||
Expire Objects after Number of Days
|
Expire Objects after Number of Days
|
||||||
-----------------------------------
|
-----------------------------------
|
||||||
|
|
||||||
Use :mc:`mc ilm add` with :mc-cmd:`~mc ilm add --expiry-days` to
|
Use :mc:`mc ilm add` with :mc-cmd:`~mc ilm add --expire-days` to
|
||||||
expire bucket contents a number of days after object creation:
|
expire bucket contents a number of days after object creation:
|
||||||
|
|
||||||
.. code-block:: shell
|
.. code-block:: shell
|
||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc ilm add ALIAS/PATH --expiry-days "DAYS"
|
mc ilm add ALIAS/PATH --expire-days "DAYS"
|
||||||
|
|
||||||
- Replace :mc-cmd:`ALIAS <mc ilm add ALIAS>` with the
|
- Replace :mc-cmd:`ALIAS <mc ilm add ALIAS>` with the
|
||||||
:mc:`alias <mc alias>` of the S3-compatible host.
|
:mc:`alias <mc alias>` of the S3-compatible host.
|
||||||
@ -76,7 +76,7 @@ expire bucket contents a number of days after object creation:
|
|||||||
- Replace :mc-cmd:`PATH <mc ilm add ALIAS>` with the path to the bucket on the
|
- Replace :mc-cmd:`PATH <mc ilm add ALIAS>` with the path to the bucket on the
|
||||||
S3-compatible host.
|
S3-compatible host.
|
||||||
|
|
||||||
- Replace :mc-cmd:`DAYS <mc ilm add --expiry-days>` with the number of days after
|
- Replace :mc-cmd:`DAYS <mc ilm add --expire-days>` with the number of days after
|
||||||
which to expire the object. For example, specify ``30`` to expire the
|
which to expire the object. For example, specify ``30`` to expire the
|
||||||
object 30 days after creation.
|
object 30 days after creation.
|
||||||
|
|
||||||
@ -87,17 +87,17 @@ Use :mc:`mc ilm add` to expiring noncurrent object versions and object
|
|||||||
delete markers:
|
delete markers:
|
||||||
|
|
||||||
- To expire noncurrent object versions after a specific duration in days,
|
- To expire noncurrent object versions after a specific duration in days,
|
||||||
include :mc-cmd:`~mc ilm add --noncurrentversion-expiration-days`.
|
include :mc-cmd:`~mc ilm add --noncurrent-expire-days`.
|
||||||
|
|
||||||
- To expire delete markers for objects with no remaining versions,
|
- To expire delete markers for objects with no remaining versions,
|
||||||
include :mc-cmd:`~mc ilm add --expired-object-delete-marker`.
|
include :mc-cmd:`~mc ilm add --expire-delete-marker`.
|
||||||
|
|
||||||
.. code-block:: shell
|
.. code-block:: shell
|
||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc ilm add ALIAS/PATH \
|
mc ilm add ALIAS/PATH \
|
||||||
--noncurrentversion-expiration-days NONCURRENT_DAYS \
|
--noncurrent-expire-days NONCURRENT_DAYS \
|
||||||
--expired-object-delete-marker
|
--expire-delete-marker
|
||||||
|
|
||||||
- Replace :mc-cmd:`ALIAS <mc ilm add ALIAS>` with the
|
- Replace :mc-cmd:`ALIAS <mc ilm add ALIAS>` with the
|
||||||
:mc:`alias <mc alias>` of the S3-compatible host.
|
:mc:`alias <mc alias>` of the S3-compatible host.
|
||||||
@ -106,7 +106,7 @@ delete markers:
|
|||||||
S3-compatible host.
|
S3-compatible host.
|
||||||
|
|
||||||
- Replace :mc-cmd:`NONCURRENT_DAYS
|
- Replace :mc-cmd:`NONCURRENT_DAYS
|
||||||
<mc ilm add --noncurrentversion-expiration-days>` with the number of days after
|
<mc ilm add --noncurrent-expire-days>` with the number of days after
|
||||||
which to expire noncurrent object versions. For example, specify ``30d`` to
|
which to expire noncurrent object versions. For example, specify ``30d`` to
|
||||||
expire a version after it has been noncurrent for at least 30 days.
|
expire a version after it has been noncurrent for at least 30 days.
|
||||||
|
|
||||||
|
@ -69,8 +69,8 @@ on :ref:`versioned buckets <minio-bucket-versioning>`. Specifically, MinIO by
|
|||||||
default applies the transition operation to the *current* object version.
|
default applies the transition operation to the *current* object version.
|
||||||
|
|
||||||
To transition noncurrent object versions, specify the
|
To transition noncurrent object versions, specify the
|
||||||
:mc-cmd:`~mc ilm add --noncurrentversion-transition-days` and
|
:mc-cmd:`~mc ilm add --noncurrent-transition-days` and
|
||||||
:mc-cmd:`~mc ilm add --noncurrentversion-tier` options
|
:mc-cmd:`~mc ilm add --noncurrent-transition-tier` options
|
||||||
when creating the transition rule.
|
when creating the transition rule.
|
||||||
|
|
||||||
.. _minio-lifecycle-management-expiration:
|
.. _minio-lifecycle-management-expiration:
|
||||||
@ -85,7 +85,7 @@ older than 365 days.
|
|||||||
|
|
||||||
.. todo: Diagram of MinIO Expiration
|
.. todo: Diagram of MinIO Expiration
|
||||||
|
|
||||||
Use :mc-cmd:`mc ilm add --expiry-days` to expire objects after a specified
|
Use :mc-cmd:`mc ilm add --expire-days` to expire objects after a specified
|
||||||
number of calendar days.
|
number of calendar days.
|
||||||
|
|
||||||
For buckets with :ref:`replication <minio-bucket-replication>` configured, MinIO
|
For buckets with :ref:`replication <minio-bucket-replication>` configured, MinIO
|
||||||
@ -104,14 +104,14 @@ specific default behaviors for versioned buckets:
|
|||||||
creating a ``DeleteMarker`` as is normal with versioned delete.
|
creating a ``DeleteMarker`` as is normal with versioned delete.
|
||||||
|
|
||||||
To expire noncurrent object versions, specify the
|
To expire noncurrent object versions, specify the
|
||||||
:mc-cmd:`~mc ilm add --noncurrentversion-expiration-days` option
|
:mc-cmd:`~mc ilm add --noncurrent-expire-days` option
|
||||||
when creating the expiration rule.
|
when creating the expiration rule.
|
||||||
|
|
||||||
- MinIO does not expire ``DeleteMarkers`` *even if* no other versions of
|
- MinIO does not expire ``DeleteMarkers`` *even if* no other versions of
|
||||||
that object exist.
|
that object exist.
|
||||||
|
|
||||||
To expire delete markers when there are no remaining versions for that
|
To expire delete markers when there are no remaining versions for that
|
||||||
object, specify the :mc-cmd:`~mc ilm add --expired-object-delete-marker`
|
object, specify the :mc-cmd:`~mc ilm add --expire-delete-marker`
|
||||||
option when creating the expiration rule.
|
option when creating the expiration rule.
|
||||||
|
|
||||||
.. _minio-lifecycle-management-scanner:
|
.. _minio-lifecycle-management-scanner:
|
||||||
|
@ -143,6 +143,18 @@ return the ``DeleteMarker`` back as part of the response. Similarly, performing
|
|||||||
a ``LIST`` operation by default returns only objects which are *not* a
|
a ``LIST`` operation by default returns only objects which are *not* a
|
||||||
``DeleteMarker``.
|
``DeleteMarker``.
|
||||||
|
|
||||||
|
.. admonition:: MinIO Implements Idempotent Delete Markers
|
||||||
|
:class: note
|
||||||
|
|
||||||
|
.. versionchanged:: RELEASE.2022-08-22T23-53-06Z
|
||||||
|
|
||||||
|
Standard S3 implementations can create multiple sequential delete markers for the same object when processing simple ``DeleteObject`` requests with no version identifier.
|
||||||
|
See the S3 docs for details on :s3-docs:`managing delete markers <ManagingDelMarkers.html#RemDelMarker>``
|
||||||
|
|
||||||
|
MinIO diverges from standard S3 implementation by avoiding this potential duplication of delete markers.
|
||||||
|
When processing a ``Delete`` request with no version identifier, MinIO creates at most one Delete Marker for the specified object.
|
||||||
|
MinIO **does not** share S3's behavior in creating multiple sequential delete markers.
|
||||||
|
|
||||||
To permanently delete an object version, perform the ``DELETE`` operation and
|
To permanently delete an object version, perform the ``DELETE`` operation and
|
||||||
specify the version ID of the object to delete. Versioned delete operations
|
specify the version ID of the object to delete. Versioned delete operations
|
||||||
are **irreversible**.
|
are **irreversible**.
|
||||||
|
@ -8,10 +8,10 @@ specified number of calendar days:
|
|||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc ilm add ALIAS/BUCKET \
|
mc ilm add ALIAS/BUCKET \
|
||||||
--tier TIERNAME \
|
--transition-tier TIERNAME \
|
||||||
--transition-days DAYS \
|
--transition-days DAYS \
|
||||||
--noncurrentversion-transition-days NONCURRENT_DAYS
|
--noncurrent-transition-days NONCURRENT_DAYS
|
||||||
--noncurrentversion-tier TIERNAME
|
--noncurrent-transition-tier TIERNAME
|
||||||
|
|
||||||
The example above specifies the following arguments:
|
The example above specifies the following arguments:
|
||||||
|
|
||||||
@ -31,20 +31,20 @@ The example above specifies the following arguments:
|
|||||||
- Specify the full path to the bucket for which you are
|
- Specify the full path to the bucket for which you are
|
||||||
creating the lifecycle management rule.
|
creating the lifecycle management rule.
|
||||||
|
|
||||||
* - :mc-cmd:`TIERNAME <mc ilm add --tier>`
|
* - :mc-cmd:`TIERNAME <mc ilm add --transition-tier>`
|
||||||
- The remote storage tier to which MinIO transitions objects.
|
- The remote storage tier to which MinIO transitions objects.
|
||||||
Specify the remote storage tier name created in the previous step.
|
Specify the remote storage tier name created in the previous step.
|
||||||
|
|
||||||
If you want to transition noncurrent object versions to a distinct
|
If you want to transition noncurrent object versions to a distinct
|
||||||
remote tier, specify a different tier name for
|
remote tier, specify a different tier name for
|
||||||
:mc-cmd:`~mc ilm add --noncurrentversion-tier`.
|
:mc-cmd:`~mc ilm add --noncurrent-transition-tier`.
|
||||||
|
|
||||||
* - :mc-cmd:`DAYS <mc ilm add --transition-days>`
|
* - :mc-cmd:`DAYS <mc ilm add --transition-days>`
|
||||||
- The number of calendar days after which MinIO marks an object as
|
- The number of calendar days after which MinIO marks an object as
|
||||||
eligible for transition. Specify the number of days as an integer,
|
eligible for transition. Specify the number of days as an integer,
|
||||||
e.g. ``30`` for 30 days.
|
e.g. ``30`` for 30 days.
|
||||||
|
|
||||||
* - :mc-cmd:`NONCURRENT_DAYS <mc ilm add --noncurrentversion-transition-days>`
|
* - :mc-cmd:`NONCURRENT_DAYS <mc ilm add --noncurrent-transition-days>`
|
||||||
- The number of calendar days after which MinIO marks a noncurrent
|
- The number of calendar days after which MinIO marks a noncurrent
|
||||||
object version as eligible for transition. MinIO specifically measures
|
object version as eligible for transition. MinIO specifically measures
|
||||||
the time since an object *became* non-current instead of the object
|
the time since an object *became* non-current instead of the object
|
||||||
|
@ -97,6 +97,11 @@ Object and Bucket Metrics
|
|||||||
Total bucket size in bytes in a given bucket.
|
Total bucket size in bytes in a given bucket.
|
||||||
You can identify the bucket using the ``{ bucket="STRING" }`` label.
|
You can identify the bucket using the ``{ bucket="STRING" }`` label.
|
||||||
|
|
||||||
|
.. metric:: minio_bucket_quota_total_bytes
|
||||||
|
|
||||||
|
Total bucket quota size in bytes.
|
||||||
|
You can identify the bucket using the ``{ bucket="STRING" }`` label.
|
||||||
|
|
||||||
Replication Metrics
|
Replication Metrics
|
||||||
~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
@ -96,7 +96,7 @@ Parameters
|
|||||||
|
|
||||||
.. code-block:: none
|
.. code-block:: none
|
||||||
|
|
||||||
mc ilm add --prefix "meetingnotes/" myminio/mydata/ --expiry-days "90"
|
mc ilm add --prefix "meetingnotes/" myminio/mydata/ --expire-days "90"
|
||||||
|
|
||||||
The command creates a rule that expires objects in the ``mydata`` bucket of the ``myminio`` ALIAS after 90 days for any object with the ``meetingnotes/`` prefix.
|
The command creates a rule that expires objects in the ``mydata`` bucket of the ``myminio`` ALIAS after 90 days for any object with the ``meetingnotes/`` prefix.
|
||||||
|
|
||||||
@ -298,30 +298,30 @@ Examples
|
|||||||
Expire All Bucket Contents After Number of Days
|
Expire All Bucket Contents After Number of Days
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Use :mc:`mc ilm add` with :mc-cmd:`~mc ilm add --expiry-days` to mark bucket contents for expiration after a number of days pass from the object's creation:
|
Use :mc:`mc ilm add` with :mc-cmd:`~mc ilm add --expire-days` to mark bucket contents for expiration after a number of days pass from the object's creation:
|
||||||
|
|
||||||
.. code-block:: shell
|
.. code-block:: shell
|
||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc ilm add ALIAS/PATH --expiry-days "DAYS"
|
mc ilm add ALIAS/PATH --expire-days "DAYS"
|
||||||
|
|
||||||
- Replace :mc-cmd:`ALIAS <mc ilm add ALIAS>` with the :mc:`alias <mc alias>` of the S3-compatible host.
|
- Replace :mc-cmd:`ALIAS <mc ilm add ALIAS>` with the :mc:`alias <mc alias>` of the S3-compatible host.
|
||||||
|
|
||||||
- Replace :mc-cmd:`PATH <mc ilm add ALIAS>` with the path to the bucket on the S3-compatible host.
|
- Replace :mc-cmd:`PATH <mc ilm add ALIAS>` with the path to the bucket on the S3-compatible host.
|
||||||
|
|
||||||
- Replace :mc-cmd:`DATE <mc ilm add --expiry-days>` with the number of days after which to expire the object.
|
- Replace :mc-cmd:`DATE <mc ilm add --expire-days>` with the number of days after which to expire the object.
|
||||||
For example, specify ``30`` to expire the object 30 days after creation.
|
For example, specify ``30`` to expire the object 30 days after creation.
|
||||||
|
|
||||||
Transition Non-Current Object Versions at a Prefix to a Different Tier
|
Transition Non-Current Object Versions at a Prefix to a Different Tier
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Use the :mc:`mc ilm add` with :mc-cmd:`~mc ilm add --prefix` and :mc-cmd:`~mc ilm add --tier` to transition older non-current versions of an object to a different storage tier.
|
Use the :mc:`mc ilm add` with :mc-cmd:`~mc ilm add --prefix` and :mc-cmd:`~mc ilm add --transition-tier` to transition older non-current versions of an object to a different storage tier.
|
||||||
|
|
||||||
.. code-block:: shell
|
.. code-block:: shell
|
||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc ilm add --prefix "doc/" --transition-days "90" --tier "MINIOTIER-1" \
|
mc ilm add --prefix "doc/" --transition-days "90" --tier "MINIOTIER-1" \
|
||||||
--noncurrentversion-transition-days "45" --noncurrentversion-tier "MINIOTIER-2" \
|
--noncurrent-transition-days "45" --noncurrent-transition-tier "MINIOTIER-2" \
|
||||||
myminio/mybucket/
|
myminio/mybucket/
|
||||||
|
|
||||||
This command looks at the contents with the ``doc/`` prefix in the ``mybucket`` bucket on the ``myminio`` deployment.
|
This command looks at the contents with the ``doc/`` prefix in the ``mybucket`` bucket on the ``myminio`` deployment.
|
||||||
@ -333,12 +333,12 @@ This command looks at the contents with the ``doc/`` prefix in the ``mybucket``
|
|||||||
Expire All Objects at a Prefix, Retain Current Object Versions Longer Than Non-Current Object Versions
|
Expire All Objects at a Prefix, Retain Current Object Versions Longer Than Non-Current Object Versions
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Use the :mc:`mc ilm add` command with :mc-cmd:`~mc ilm add --prefix`, :mc-cmd:`~mc ilm add --expiry-days`, and :mc-cmd:`~mc ilm add --noncurrentversion-expiration-days` to expire current and non-current versions of an object at different times.
|
Use the :mc:`mc ilm add` command with :mc-cmd:`~mc ilm add --prefix`, :mc-cmd:`~mc ilm add --expire-days`, and :mc-cmd:`~mc ilm add --noncurrent-expire-days` to expire current and non-current versions of an object at different times.
|
||||||
|
|
||||||
.. code-block:: shell
|
.. code-block:: shell
|
||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc ilm add --prefix "doc/" --expiry-days "300" --noncurrentversion-expiration-days "100" myminio/mybucket/
|
mc ilm add --prefix "doc/" --expire-days "300" --noncurrent-expire-days "100" myminio/mybucket/
|
||||||
|
|
||||||
This command looks at the contents with the ``doc/`` prefix in the ``mybucket`` bucket on the ``myminio`` deployment.
|
This command looks at the contents with the ``doc/`` prefix in the ``mybucket`` bucket on the ``myminio`` deployment.
|
||||||
|
|
||||||
|
Reference in New Issue
Block a user