DOCS-370: Object Locking Feature Page
CR w/ Eco Addressing Eco suggestions Co-authored-by: Eco <41090896+eco-minio@users.noreply.github.com> Klaus fixups
@ -10,78 +10,310 @@ Bucket Versioning
|
||||
:local:
|
||||
:depth: 2
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
MinIO supports keeping multiple "versions" of an object in a single bucket.
|
||||
Write operations which would normally overwrite an existing object instead
|
||||
result in the creation of a new versioned object. MinIO versioning protects from
|
||||
unintended overwrites and deletions while providing support for "undoing" a
|
||||
write operation. Bucket versioning also supports retention and archive policies.
|
||||
write operation. Bucket versioning is a prerequisite for configuring
|
||||
:ref:`object locking and retention rules <minio-object-locking>`.
|
||||
|
||||
<Diagram>
|
||||
For versioned buckets, any write operation that mutates an object results in a
|
||||
new version of that object with a unique version ID. MinIO marks the "latest"
|
||||
version of the object that clients retrieve by default. Clients can then
|
||||
explicitly choose to list, retrieve, or remove a specific object version.
|
||||
|
||||
MinIO generates a unique immutable ID for each object. If a ``PUT`` request
|
||||
contains an object name which duplicates an existing object, MinIO does *not*
|
||||
overwrite the "older" object. Instead, MinIO retains all object versions while
|
||||
considering the most recently written "version" of the object as "latest".
|
||||
Applications retrieve the latest object version by default, but *may* retrieve
|
||||
any other version in the history of that object. To view all versions of an
|
||||
object or objects in a bucket, use the :mc-cmd-option:`mc ls versions` command.
|
||||
.. card-carousel:: 1
|
||||
|
||||
By default, deleting an object does *not* remove all existing versions of
|
||||
that object. Instead, MinIO places a "delete" marker for the object, such that
|
||||
applications requesting only the latest object versions do not see the object.
|
||||
Applications *may* retrieve earlier versions of that object. To completely
|
||||
delete an object and its entire version history, use the
|
||||
:mc-cmd-option:`mc rm versions` command.
|
||||
.. card:: Object with Single Version
|
||||
|
||||
Enable Bucket Versioning
|
||||
------------------------
|
||||
.. image:: /images/retention/minio-versioning-single-version.svg
|
||||
:alt: Object with single version
|
||||
:align: center
|
||||
|
||||
Enabling bucket versioning on a MinIO deployment requires that the deployment
|
||||
have *at least* four disks. Specifically, Bucket Versioning depends on
|
||||
:ref:`Erasure Coding <minio-erasure-coding>`. For MinIO deployments that
|
||||
meet the disk requirements, use the :mc-cmd:`mc version enable` command to
|
||||
enable versioning on a specific bucket.
|
||||
MinIO adds a unique version ID to each object as part of write operations.
|
||||
|
||||
The :mc-cmd:`mc version` command *may* work on other S3-compatible services
|
||||
depending on their implementation of and support for the AWS S3 API.
|
||||
.. card:: Object with Multiple Versions
|
||||
|
||||
Buckets with Existing Content
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
.. image:: /images/retention/minio-versioning-multiple-versions.svg
|
||||
:alt: Object with Multiple Versions
|
||||
:align: center
|
||||
|
||||
After enabling bucket versioning, MinIO begins generating version IDs for
|
||||
*new* objects created in the bucket. MinIO does *not* iterate through existing
|
||||
objects to generate version IDs. Existing unversioned objects in a versioned
|
||||
bucket have a ``null`` version ID.
|
||||
MinIO retains all versions of an object and marks the most recent
|
||||
version as the "latest".
|
||||
|
||||
Suspend Bucket Versioning
|
||||
-------------------------
|
||||
.. card:: Retrieving the Latest Object Version
|
||||
|
||||
You can suspend bucket versioning at any time using
|
||||
:mc-cmd:`mc version disable`. MinIO retains all existing object versions. During
|
||||
suspension, MinIO allows overwrites of any *unversioned* object. Applications
|
||||
can continue referencing any existing object version.
|
||||
.. image:: /images/retention/minio-versioning-retrieve-latest-version.svg
|
||||
:alt: Object with Multiple Versions
|
||||
:align: center
|
||||
|
||||
You can later re-enable object versioning on the bucket. MinIO resumes
|
||||
generating version IDs for *new* objects, and does not retroactively generate
|
||||
version IDs for existing unversioned objects. MinIO lists unversioned
|
||||
objects with a ``null`` version ID.
|
||||
.. card:: Retrieving a Specific Object Version
|
||||
|
||||
.. image:: /images/retention/minio-versioning-retrieve-single-version.svg
|
||||
:alt: Object with Multiple Versions
|
||||
:align: center
|
||||
|
||||
:ref:`Deleting <minio-bucket-versioning-delete>` an object results in a special
|
||||
``DeleteMarker`` tombstone that marks an object as deleted while retaining
|
||||
all previous versions of that object.
|
||||
|
||||
Versioning is Per-Namespace
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO uses the full namespace (the bucket and path to an object) for each object
|
||||
as part of determining object uniqueness. For example, all of the following
|
||||
namespaces are "unique" objects, where mutations of each object result in
|
||||
the creation of new object versions *at that namespace*:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
databucket/object.blob
|
||||
databucket/blobs/object.blob
|
||||
blobbucket/object.blob
|
||||
blobbucket/blobs/object.blob
|
||||
|
||||
While ``object.blob`` might be the same binary across all namespaces,
|
||||
MinIO only enforces versioning with a specific namespace and therefore
|
||||
considers each ``object.blob`` above as distinct and unique.
|
||||
|
||||
Versioning and Storage Capacity
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO does not perform incremental or differential-type versioning. For
|
||||
mutation-heavy workloads, this may result in substantial disk usage by
|
||||
older or aged object versions.
|
||||
|
||||
For example, consider a 1GB object containing log data. An application
|
||||
appends 100MB of data to the log and uploads to MinIO. MinIO would then contain
|
||||
both the 1GB and 1.1GB versions of the object. If the application repeated
|
||||
this process every day for 10 days, the bucket would eventually contain more
|
||||
than 14GB of data associated to a single object.
|
||||
|
||||
MinIO supports configuring configuring :ref:`object lifecycle management rules
|
||||
<minio-lifecycle-management>` to automatically expire or transition aged
|
||||
object versions and free up storage capacity. For example, you can configure
|
||||
a rule to automatically expire object versions 90 days after they become
|
||||
non-current (i.e. no longer the "latest" version of that object). See
|
||||
:ref:`MinIO Object Expiration <minio-lifecycle-management-expiration>` for
|
||||
more information.
|
||||
|
||||
You can alternatively perform manual removal of object versions using the
|
||||
following commands:
|
||||
|
||||
- :mc-cmd-option:`mc rm versions` - Removes all versions of an object.
|
||||
- :mc-cmd-option:`mc rm --versions --older-than <mc rm older-than>` -
|
||||
Removes all versions of an object older than the specified calendar date.
|
||||
|
||||
.. _minio-bucket-versioning-id:
|
||||
|
||||
Version ID Generation
|
||||
---------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO version ID's are DCE 1.1/RFC 4122-compliant Universally Unique Identifiers
|
||||
(UUID) version 4. Each UUID is a random 128-bit number intended to have a high
|
||||
likelihood of uniqueness over space and time, *and* that are computationally
|
||||
difficult to guess. UUID's are globally unique that can be generated without
|
||||
contacting a global registration authority.
|
||||
MinIO generates a unique and immutable identifier for each versioned object as
|
||||
part of write operations. Each object version ID consists of a 128-bit
|
||||
fixed-size :rfc:`UUIDv4 <4122#section-4.4>`. UUID generation is sufficiently
|
||||
random to ensure high likelihood of uniqueness for any environment, are
|
||||
computationally difficult to guess, and do not require centralized registration
|
||||
process and authority to guarantee uniqueness.
|
||||
|
||||
MinIO object version UUID's are *immutable* after creation.
|
||||
.. image:: /images/retention/minio-versioning-multiple-versions.svg
|
||||
:alt: Object with Multiple Versions
|
||||
:width: 600px
|
||||
:align: center
|
||||
|
||||
Versioning Dependent Features
|
||||
-----------------------------
|
||||
MinIO does not support client-managed version ID allocation. All version ID
|
||||
generation is handled by the MinIO server process.
|
||||
|
||||
The following MinIO features *require* bucket versioning for functionality:
|
||||
For objects created while versioning is disabled or suspended, MinIO
|
||||
uses a ``null`` version ID. You can access or remove these objects by specifying
|
||||
``null`` as the version ID as part of S3 operations.
|
||||
|
||||
.. _minio-bucket-versioning-delete:
|
||||
|
||||
Versioned Delete Operations
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Performing a ``DELETE`` operation on a versioned object creates a
|
||||
0-byte ``DeleteMarker`` as the latest version of that object. Clients performing
|
||||
``GET`` operations on that object do not return any results, as MinIO does not
|
||||
return the ``DeleteMarker`` back as part of the response. Similarly, performing
|
||||
a ``LIST`` operation by default returns only objects which are *not* a
|
||||
``DeleteMarker``.
|
||||
|
||||
To permanently delete an object version, perform the ``DELETE`` operation and
|
||||
specify the version ID of the object to delete. Versioned delete operations
|
||||
are **irreversible**.
|
||||
|
||||
.. card-carousel:: 1
|
||||
|
||||
.. card:: Deleting an Object
|
||||
|
||||
.. image:: /images/retention/minio-versioning-delete-object.svg
|
||||
:alt: Deleting an Object
|
||||
:align: center
|
||||
|
||||
Performing a ``DELETE`` operation on a versioned object produces a
|
||||
``DeleteMarker`` for that object.
|
||||
|
||||
.. card:: Reading a Deleted Object
|
||||
|
||||
.. image:: /images/retention/minio-versioning-retrieve-deleted-object.svg
|
||||
:alt: Object with Multiple Versions
|
||||
:align: center
|
||||
|
||||
Clients by default retrieve the "latest" object version. MinIO returns
|
||||
a ``404``-like response if the latest version is a ``DeleteMarker``.
|
||||
|
||||
.. card:: Retrieve Previous Version of Deleted Object
|
||||
|
||||
.. image:: /images/retention/minio-versioning-retrieve-version-before-delete.svg
|
||||
:alt: Retrieve Version of Deleted Object
|
||||
:align: center
|
||||
|
||||
Clients can retrieve any previous version of the object by specifying the
|
||||
version ID, even if the "Latest" version is a ``DeleteMarker``.
|
||||
|
||||
.. card:: Delete a Specific Object Version
|
||||
|
||||
.. image:: /images/retention/minio-versioning-delete-specific-version.svg
|
||||
:alt: Retrieve Version of Deleted Object
|
||||
:align: center
|
||||
|
||||
Clients can delete a specific object version by specifying the version ID
|
||||
as part of the ``DELETE`` operation. Deleting a specific version is
|
||||
**permanent** and does not result in the creation of a ``DeleteMarker``.
|
||||
|
||||
The following :mc:`mc` commands operate on ``DeleteMarkers`` or versioned
|
||||
objects:
|
||||
|
||||
- Use :mc-cmd-option:`mc ls versions` to view all versions of an object,
|
||||
including delete markers.
|
||||
|
||||
- Use :mc-cmd-option:`mc cp --version-id=UUID ... <mc cp version-id>` to
|
||||
retrieve the version of the "deleted" object with matching ``UUID``.
|
||||
|
||||
- Use :mc-cmd-option:`mc rm --version-id=UUID ... <mc rm version-id>` to delete
|
||||
the version of the object with matching ``UUID``.
|
||||
|
||||
- Use :mc-cmd-option:`mc rm versions` to delete *all* versions of an object.
|
||||
|
||||
Tutorials
|
||||
---------
|
||||
|
||||
Enable Bucket Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. note::
|
||||
|
||||
Versioning requires :ref:`erasure coding <minio-erasure-coding>` and is only
|
||||
available on :ref:`distributed MinIO deployments <deploy-minio-distributed>`.
|
||||
|
||||
You can enable versioning using the MinIO Console, the MinIO :mc:`mc` CLI, or
|
||||
using an S3-compatible SDK. Versioning is a bucket-scoped feature. You cannot
|
||||
enable versioning on only a prefix or subset of objects in a bucket.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MinIO Console
|
||||
|
||||
Select the :guilabel:`Buckets` section of the MinIO Console to access
|
||||
bucket creation and management functions. Select the bucket row from the
|
||||
list of buckets. You can use the :octicon:`search` :guilabel:`Search` bar
|
||||
to filter the list.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket Management
|
||||
:align: center
|
||||
|
||||
From the Bucket view, click the :guilabel:`Versioning`
|
||||
field to enable versioning on the bucket.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket-overview.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket minio-lifecycle-management
|
||||
:align: center
|
||||
|
||||
The MinIO Console also supports enabling versioning as part of bucket
|
||||
creation. See :ref:`minio-console-admin-buckets` for more information on
|
||||
bucket management using the MinIO Console.
|
||||
|
||||
.. tab-item:: MinIO CLI
|
||||
|
||||
Use the :mc-cmd:`mc version enable` command to enable versioning on an
|
||||
existing bucket:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc version enable ALIAS/BUCKET
|
||||
|
||||
- Replace ``ALIAS`` with the :mc:`alias <mc alias>` of a configured
|
||||
MinIO deployment.
|
||||
|
||||
- Replace ``BUCKET`` with the
|
||||
:mc:`target bucket <mc version enable TARGET>` on which to enable
|
||||
versioning.
|
||||
|
||||
Objects created prior to enabling versioning have a
|
||||
``null`` :ref:`version ID <minio-bucket-versioning-id>`.
|
||||
|
||||
Suspend Bucket Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can suspend bucket versioning at any time using the MinIO Console, the
|
||||
MinIO :mc:`mc` CLI, or using an S3-compatible SDK.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MinIO Console
|
||||
|
||||
Select the :guilabel:`Buckets` section of the MinIO Console to access
|
||||
bucket creation and management functions.
|
||||
|
||||
Select the bucket row from the list of buckets. You can use the
|
||||
:octicon:`search` :guilabel:`Search` bar to filter the list.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket Management
|
||||
:align: center
|
||||
|
||||
From the :guilabel:`Bucket` view, click the :guilabel:`Versioning` field
|
||||
to disable versioning on the bucket.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket-overview.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket minio-lifecycle-management
|
||||
:align: center
|
||||
|
||||
See :ref:`minio-console-admin-buckets` for more information on bucket
|
||||
management using the MinIO Console.
|
||||
|
||||
.. tab-item:: MinIO CLI
|
||||
|
||||
Use the :mc-cmd:`mc version disable` command to enable versioning on an
|
||||
existing bucket:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc version disable ALIAS/BUCKET
|
||||
|
||||
- Replace ``ALIAS`` with the :mc:`alias <mc alias>` of a configured
|
||||
MinIO deployment.
|
||||
|
||||
- Replace ``BUCKET`` with the
|
||||
:mc:`target bucket <mc version enable TARGET>` on which to disable
|
||||
versioning.
|
||||
|
||||
Objects created while versioning is suspended are assigned a
|
||||
``null`` :ref:`version ID <minio-bucket-versioning-id>`. Any mutations to an
|
||||
object while versioning is suspended results in overwriting that
|
||||
``null`` versioned object. MinIO does not remove or otherwise alter
|
||||
existing versioned objects as part of suspending versioning. Clients can
|
||||
continue interacting with any existing object versions in the bucket.
|
||||
|
||||
- Object Locking (:mc-cmd:`mc lock` and :mc-cmd-option:`mc mb with-lock`)
|
||||
- Object Legal Hold (:mc-cmd:`mc legalhold`)
|
||||
- Bucket Replication (:mc-cmd:`mc admin bucket remote` and :mc-cmd:`mc replicate`)
|
@ -44,5 +44,4 @@ The following table lists MinIO features and their corresponding documentation:
|
||||
:titlesonly:
|
||||
:hidden:
|
||||
|
||||
/concepts/bucket-versioning
|
||||
/concepts/erasure-coding
|
||||
|
@ -194,6 +194,8 @@ change or retrieve the credentials later. To rotate credentials for an
|
||||
application, create a new service account and delete the old one once the
|
||||
application updates to using the new credentials.
|
||||
|
||||
.. _minio-console-admin-buckets:
|
||||
|
||||
Admin: Buckets
|
||||
--------------
|
||||
|
||||
@ -216,7 +218,7 @@ You can select a bucket to view more specific details for that bucket:
|
||||
|
||||
.. image:: /images/minio-console/console-bucket-overview.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Create Bucket
|
||||
:alt: MinIO Console Bucket Management
|
||||
:align: center
|
||||
|
||||
- The :guilabel:`Summary` tab displays a summary of the bucket configuration.
|
||||
|
After Width: | Height: | Size: 28 KiB |
BIN
source/images/minio-console/console-bucket-locking.png
Normal file
After Width: | Height: | Size: 79 KiB |
Before Width: | Height: | Size: 44 KiB After Width: | Height: | Size: 85 KiB |
BIN
source/images/minio-console/console-object-browser-locking.png
Normal file
After Width: | Height: | Size: 55 KiB |
After Width: | Height: | Size: 68 KiB |
After Width: | Height: | Size: 187 KiB |
1
source/images/retention/minio-object-locking-delete.svg
Normal file
After Width: | Height: | Size: 148 KiB |
1
source/images/retention/minio-object-locking.svg
Normal file
After Width: | Height: | Size: 108 KiB |
1
source/images/retention/minio-versioning-default.svg
Normal file
After Width: | Height: | Size: 76 KiB |
After Width: | Height: | Size: 130 KiB |
After Width: | Height: | Size: 153 KiB |
After Width: | Height: | Size: 122 KiB |
After Width: | Height: | Size: 152 KiB |
After Width: | Height: | Size: 138 KiB |
After Width: | Height: | Size: 154 KiB |
After Width: | Height: | Size: 166 KiB |
After Width: | Height: | Size: 91 KiB |
@ -134,6 +134,7 @@ such as versioning and replication.
|
||||
|
||||
/introduction/minio-overview
|
||||
/concepts/feature-overview
|
||||
Object Retention </object-retention/minio-object-retention>
|
||||
/installation/deployment-and-management
|
||||
/lifecycle-management/lifecycle-management-overview
|
||||
/replication/replication-overview
|
||||
|
446
source/object-retention/minio-object-locking.rst
Normal file
@ -0,0 +1,446 @@
|
||||
.. _minio-object-locking:
|
||||
|
||||
====================
|
||||
MinIO Object Locking
|
||||
====================
|
||||
|
||||
.. default-domain:: minio
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 2
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
MinIO Object Locking ("Object Retention") enforces Write-Once Read-Many (WORM)
|
||||
immutability to protect :ref:`versioned objects <minio-bucket-versioning>` from
|
||||
deletion. MinIO supports both
|
||||
:ref:`duration based object retention <minio-object-locking-retention-modes>`
|
||||
and
|
||||
:ref:`indefinite Legal Hold retention <minio-object-locking-legalhold>`.
|
||||
|
||||
MinIO Object Locking provides key data retention compliance and meets
|
||||
SEC17a-4(f), FINRA 4511(C), and CFTC 1.31(c)-(d) requirements as per
|
||||
`Cohasset Associates <https://min.io/cohasset?ref-docs>`__.
|
||||
|
||||
.. card-carousel:: 1
|
||||
|
||||
.. card:: Bucket Without Locking
|
||||
|
||||
.. image:: /images/retention/minio-versioning-delete-object.svg
|
||||
:alt: Deleting an Object
|
||||
:align: center
|
||||
|
||||
MinIO versioning preserves the full history of object mutations.
|
||||
However, applications can explicitly delete specific object versions.
|
||||
|
||||
.. card:: Bucket With Locking
|
||||
|
||||
.. image:: /images/retention/minio-object-locking.svg
|
||||
:alt: 30 Day Locked Objects
|
||||
:align: center
|
||||
|
||||
Applying a default 30 Day WORM lock to objects in the bucket ensures
|
||||
a minimum period of retention and protection for all object versions.
|
||||
|
||||
.. card:: Delete Operations in Locked Bucket
|
||||
|
||||
.. image:: /images/retention/minio-object-locking-delete.svg
|
||||
:alt: Delete Operation in Locked Bucket
|
||||
:align: center
|
||||
|
||||
Delete operations follow normal behavior in
|
||||
:ref:`versioned buckets <minio-bucket-versioning-delete>`, where MinIO
|
||||
creates a ``DeleteMarker`` for the object. However, non-Delete Marker
|
||||
versions of the object remain under the retention rules and are protected
|
||||
from any specific deletion or overwrite attempts.
|
||||
|
||||
.. card:: Versioned Delete Operations in Locked Bucket
|
||||
|
||||
.. image:: /images/retention/minio-object-locking-delete-version.svg
|
||||
:alt: Versioned Delete Operation in a Locked Bucket
|
||||
:align: center
|
||||
|
||||
MinIO blocks any attempt to delete a specific object version held under
|
||||
WORM lock. The earliest possible time after which a client may delete
|
||||
the version is when the lock expires.
|
||||
|
||||
MinIO object locking is
|
||||
:s3-docs:`feature and API compatible with AWS S3 <object-lock.html>`.
|
||||
This page summarizes Object Locking / Retention concepts as implemented by
|
||||
MinIO. See the AWS S3 documentation on
|
||||
:s3-docs:`How S3 Object Lock works <object-lock.html>` for additional
|
||||
resources.
|
||||
|
||||
You can only enable object locking during bucket creation as per
|
||||
:s3-docs:`S3 behavior <object-lock-overview.html#object-lock-bucket-config>`.
|
||||
You cannot enable object locking on a bucket created without locking
|
||||
enabled. You can then configure object retention rules at any time.
|
||||
Object locking requires :ref:`versioning <minio-bucket-versioning>` and
|
||||
enables the feature implicitly.
|
||||
|
||||
.. _minio-bucket-locking-interactions-versioning:
|
||||
|
||||
Interaction with Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Objects held under WORM locked are immutable until the lock expires or is
|
||||
explicitly lifted. Locking is per-object version, where each version is
|
||||
independently immutable.
|
||||
|
||||
If an application performs an unversioned delete operation on a locked object,
|
||||
the operation produces a :ref:`delete marker <minio-bucket-versioning-delete>`.
|
||||
Attempts to explicitly delete any WORM-locked object fail with an error.
|
||||
Delete Markers are *not* eligible for protection under WORM locking.
|
||||
See the S3 documentation on
|
||||
:s3-docs:`Managing delete markers and object lifecycles
|
||||
<object-lock-managing.html#object-lock-managing-lifecycle>` for more
|
||||
information.
|
||||
|
||||
For example, consider the following bucket with
|
||||
:ref:`minio-object-locking-governance` locking enabled by default:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
$ mc ls --versions play/locking-guide
|
||||
|
||||
[DATETIME] 29B 62429eb1-9cb7-4dc5-b507-9cc23d0cc691 v3 PUT data.csv
|
||||
[DATETIME] 32B 78b3105a-02a1-4763-8054-e66add087710 v2 PUT data.csv
|
||||
[DATETIME] 23B c6b581ca-2883-41e2-9905-0a1867b535b8 v1 PUT data.csv
|
||||
|
||||
Attempting to perform a delete on a *specific version* of ``data.csv`` fails
|
||||
due to the object locking settings:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
$ mc rm --version-id 62429eb1-9cb7-4dc5-b507-9cc23d0cc691 play/data.csv
|
||||
|
||||
Removing `play/locking-guide/data.csv` (versionId=62429eb1-9cb7-4dc5-b507-9cc23d0cc691).
|
||||
mc: <ERROR> Failed to remove `play/locking-guide/data.csv`.
|
||||
Object, 'data.csv (Version ID=62429eb1-9cb7-4dc5-b507-9cc23d0cc691)' is
|
||||
WORM protected and cannot be overwritten
|
||||
|
||||
Attempting to perform an unversioned delete on ``data.csv`` succeeds and creates
|
||||
a new ``DeleteMarker`` for the object:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
$ mc rm play/locking-guide/data.csv
|
||||
|
||||
[DATETIME] 0B acce329f-ad32-46d9-8649-5fe8bf4ec6e0 v4 DEL data.csv
|
||||
[DATETIME] 29B 62429eb1-9cb7-4dc5-b507-9cc23d0cc691 v3 PUT data.csv
|
||||
[DATETIME] 32B 78b3105a-02a1-4763-8054-e66add087710 v2 PUT data.csv
|
||||
[DATETIME] 23B c6b581ca-2883-41e2-9905-0a1867b535b8 v1 PUT data.csv
|
||||
|
||||
Interaction with Lifecycle Management
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO :ref:`object expiration <minio-lifecycle-management-expiration>`
|
||||
respects any active object lock and retention settings for objects covered by
|
||||
the expiration rule.
|
||||
|
||||
- For expiration rules operating on only the *current* object version,
|
||||
MinIO creates a Delete Marker for the locked object.
|
||||
|
||||
- For expiration rules operating on *non-current object versions*,
|
||||
MinIO can only expire the non-current versions *after* the retention period
|
||||
has passed *or* has been explicitly lifted (e.g. Legal Holds).
|
||||
|
||||
For example, consider the following bucket with
|
||||
:ref:`minio-object-locking-governance` locking enabled by default for 45 days:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
$ mc ls --versions play/locking-guide
|
||||
|
||||
[7D] 29B 62429eb1-9cb7-4dc5-b507-9cc23d0cc691 v3 PUT data.csv
|
||||
[30D] 32B 78b3105a-02a1-4763-8054-e66add087710 v2 PUT data.csv
|
||||
[60D] 23B c6b581ca-2883-41e2-9905-0a1867b535b8 v1 PUT data.csv
|
||||
|
||||
Creating an expiration rule for *current* objects older than 7 days results in
|
||||
a Delete Marker for the object:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
$ mc ls --versions play/locking-guide
|
||||
|
||||
[0D] 0B acce329f-ad32-46d9-8649-5fe8bf4ec6e0 v4 DEL data.csv
|
||||
[7D] 29B 62429eb1-9cb7-4dc5-b507-9cc23d0cc691 v3 PUT data.csv
|
||||
[30D] 32B 78b3105a-02a1-4763-8054-e66add087710 v2 PUT data.csv
|
||||
[60D] 23B c6b581ca-2883-41e2-9905-0a1867b535b8 v1 PUT data.csv
|
||||
|
||||
However, an expiration rule for *non-current* objects older than 7 days would
|
||||
only take effect *after* the configured WORM lock expires. Since the bucket
|
||||
has a 45 day ``GOVERNANCE`` retention set, only the ``v1`` version of
|
||||
``data.csv`` is unlocked and therefore eligible for deletion.
|
||||
|
||||
Tutorials
|
||||
---------
|
||||
|
||||
Create Bucket with Object Locking Enabled
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You must enable object locking during bucket creation as per S3 behavior.
|
||||
You can create a bucket with object locking enabled using the MinIO Console,
|
||||
the MinIO :mc:`mc` CLI, or using an S3-compatible SDK.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MinIO Console
|
||||
:sync: console
|
||||
|
||||
Select the :guilabel:`Buckets` section of the MinIO Console to access
|
||||
bucket creation and management functions. Select the bucket row from the
|
||||
list of buckets. You can use the :octicon:`search` :guilabel:`Search` bar
|
||||
to filter the list.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket Management
|
||||
:align: center
|
||||
|
||||
Click the :guilabel:`Create Bucket` button to open the bucket creation
|
||||
model. Toggle the :guilabel:`Object Locking` selector to enable object
|
||||
locking on the bucket.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket-create-bucket.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket creation
|
||||
:align: center
|
||||
|
||||
.. tab-item:: MinIO CLI
|
||||
:sync: cli
|
||||
|
||||
Use the :mc-cmd:`mc mb` command with the :mc-cmd-option:`~mc mb with-lock`
|
||||
option to create a bucket with object locking enabled:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc mb --with-lock ALIAS/BUCKET
|
||||
|
||||
- Replace ``ALIAS`` with the :mc:`alias <mc alias>` of a configured
|
||||
MinIO deployment.
|
||||
|
||||
- Replace ``BUCKET`` with the
|
||||
:mc:`name <mc version enable TARGET>` of the bucket to create.
|
||||
|
||||
Configure Bucket-Default Object Retention
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can configure object locking rules ("object retention") using the
|
||||
MinIO Console, the MinIO :mc:`mc` CLI, or using an S3-compatible SDK.
|
||||
|
||||
MinIO supports setting both bucket-default *and* per-object retention rules.
|
||||
The following examples set bucket-default retention. For per-object retention
|
||||
settings, defer to the documentation for the ``PUT`` operation used by your
|
||||
preferred SDK.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MinIO Console
|
||||
:sync: console
|
||||
|
||||
Select the :guilabel:`Buckets` section of the MinIO Console to access
|
||||
bucket creation and management functions. Select the bucket row from the
|
||||
list of buckets. You can use the :octicon:`search` :guilabel:`Search` bar
|
||||
to filter the list.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket Management
|
||||
:align: center
|
||||
|
||||
From the :guilabel:`Bucket` view, look for the
|
||||
:guilabel:`Retention` section and click :guilabel:`Enabled`. This section
|
||||
is only visible if the bucket was created with object locking enabled.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket-overview.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket Management
|
||||
:align: center
|
||||
|
||||
From the :guilabel:`Set Retention Configuration` modal, set the
|
||||
desired bucket default retention settings.
|
||||
|
||||
.. image:: /images/minio-console/console-bucket-locking-compliance.png
|
||||
:width: 400px
|
||||
:alt: MinIO Console Bucket Default Retention
|
||||
:align: center
|
||||
|
||||
- For :guilabel:`Retention Mode`, select either
|
||||
:ref:`COMPLIANCE <minio-object-locking-compliance>` or
|
||||
:ref:`GOVERNANCE <minio-object-locking-governance>`.
|
||||
|
||||
- For :guilabel:`Duration`, select the retention duration units of
|
||||
:guilabel:`Days` or :guilabel:`Years`.
|
||||
|
||||
- For :guilabel:`Retention Validity`, set the duration of time for which
|
||||
MinIO holds objects under the specified retention mode for the bucket.
|
||||
|
||||
.. tab-item:: MinIO CLI
|
||||
:sync: cli
|
||||
|
||||
Use the :mc-cmd:`mc retention` command with the
|
||||
:mc-cmd-option:`--recursive <mc retention set recursive>` and
|
||||
:mc-cmd-option:`--default <mc retention set default>` options to set the
|
||||
default retention mode for a bucket:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc retention set --recursive --default MODE DURATION ALIAS/BUCKET
|
||||
|
||||
- Replace :mc-cmd:`MODE <mc retention set MODE>` with either either
|
||||
:ref:`COMPLIANCE <minio-object-locking-compliance>` or
|
||||
:ref:`GOVERNANCE <minio-object-locking-governance>`.
|
||||
|
||||
- Replace :mc-cmd:`DURATION <mc retention set VALIDITY>` with the
|
||||
duration for which the object lock remains in effect.
|
||||
|
||||
- Replace :mc-cmd:`ALIAS <mc retention set TARGET>` with the
|
||||
:mc:`alias <mc alias>` of a configured MinIO deployment.
|
||||
|
||||
- Replace :mc-cmd:`BUCKET <mc retention set TARGET>` with the
|
||||
name of the bucket on which to set the default retention rule.
|
||||
|
||||
Enable Legal Hold Retention
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can enable or disable indefinite Legal Hold retention for an object using
|
||||
the MinIO Console, the MinIO :mc:`mc` CLI, or using an S3-compatible SDK.
|
||||
You can place a legal hold on an object already held under a
|
||||
:ref:`COMPLIANCE <minio-object-locking-compliance>` or
|
||||
:ref:`GOVERNANCE <minio-object-locking-governance>` lock. The object remains
|
||||
WORM locked until the retention lock expires *and* the legal hold is lifted.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MinIO Console
|
||||
:sync: console
|
||||
|
||||
Select the :guilabel:`Object Browser` section of the MinIO Console. Select
|
||||
the bucket row from the list of buckets. You can use the :octicon:`search`
|
||||
:guilabel:`Search` bar to filter the list.
|
||||
|
||||
.. image:: /images/minio-console/console-object-browser-locking.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket Management
|
||||
:align: center
|
||||
|
||||
Browse to the object and select it to open the object details view.
|
||||
Click the :octicon:`pencil` icon on the :guilabel:`Legal Hold` row to
|
||||
toggle the Legal Hold status of the object.
|
||||
|
||||
.. image:: /images/minio-console/console-object-browser-object-details.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console Bucket Default Retention
|
||||
:align: center
|
||||
|
||||
.. tab-item:: MinIO CLI
|
||||
:sync: cli
|
||||
|
||||
Use the :mc-cmd:`mc legalhold` command to enable or disable the legal
|
||||
hold on an object.
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc legalhold set ALIAS/PATH
|
||||
|
||||
- Replace :mc-cmd:`ALIAS <mc legalhold set TARGET>` with the
|
||||
:mc:`alias <mc alias>` of a configured MinIO deployment.
|
||||
|
||||
- Replace :mc-cmd:`PATH <mc legalhold set TARGET>` with the
|
||||
path to the object for which to enable the legal hold.
|
||||
|
||||
.. _minio-object-locking-retention-modes:
|
||||
|
||||
Object Retention Modes
|
||||
----------------------
|
||||
|
||||
MinIO implements the following
|
||||
:s3-docs:`S3 Object Locking Modes <object-lock-overview.html>`:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 40 60
|
||||
:width: 100%
|
||||
|
||||
* - Mode
|
||||
- Summary
|
||||
|
||||
* - :ref:`minio-object-locking-governance`
|
||||
- Prevents any operation that would mutate or modify the object or its
|
||||
locking settings by non-privileged users.
|
||||
|
||||
Users with the :policy-action:`s3:BypassGovernanceRetention` permission
|
||||
on the bucket or object can modify the object or its locking settings.
|
||||
|
||||
MinIO lifts the lock automatically after the configured retention rule
|
||||
duration has passed.
|
||||
|
||||
* - :ref:`minio-object-locking-compliance`
|
||||
- Prevents any operation that would mutate or modify the object or its
|
||||
locking settings.
|
||||
|
||||
No MinIO user can modify the object or its settings, including the
|
||||
:ref:`MinIO root <minio-users-root>` user.
|
||||
|
||||
MinIO lifts the lock automatically after the configured retention rule
|
||||
duration has passed.
|
||||
|
||||
.. _minio-object-locking-governance:
|
||||
|
||||
GOVERNANCE Mode
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
An object under ``GOVERNANCE`` lock is protected from write operations by
|
||||
non-privileged users.
|
||||
|
||||
``GOVERNANCE`` locked objects enforce managed-immutability for locked objects,
|
||||
where users with the :policy-action:`s3:BypassGovernanceRetention` action can
|
||||
modify the locked object, change the retention duration, or lift the lock
|
||||
entirely. Bypassing ``GOVERNANCE`` retention also requires setting the
|
||||
``x-amz-bypass-governance-retention:true`` header as part of the request.
|
||||
|
||||
The MinIO ``GOVERNANCE`` lock is functionally identical to the
|
||||
:s3-docs:`S3 GOVERNANCE mode
|
||||
<object-lock-overview.html#object-lock-retention-modes>`.
|
||||
|
||||
.. _minio-object-locking-compliance:
|
||||
|
||||
COMPLIANCE Mode
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
An object under ``COMPLIANCE`` lock is protected from write operations by *all*
|
||||
users, including the :ref:`MinIO root <minio-users-root>` user.
|
||||
|
||||
``COMPLIANCE`` locked objects enforce complete immutability for locked objects.
|
||||
However, you cannot change or remove the lock before the configured retention
|
||||
duration has passed.
|
||||
|
||||
The MinIO ``COMPLIANCE`` lock is functionally identical to the
|
||||
:s3-docs:`S3 GOVERNANCE mode
|
||||
<object-lock-overview.html#object-lock-retention-modes>`.
|
||||
|
||||
.. _minio-object-locking-legalhold:
|
||||
|
||||
Legal Hold
|
||||
----------
|
||||
|
||||
An object under Legal Hold is protected from write operations by *all*
|
||||
users, including the :ref:`MinIO root <minio-users-root>` user.
|
||||
|
||||
Legal Holds are indefinite and enforce complete immutability for locked objects.
|
||||
Only privileged users with the :policy-action:`s3:PutObjectLegalHold` can set or
|
||||
lift the Legal Hold.
|
||||
|
||||
Legal holds are complementary to both :ref:`minio-object-locking-governance` and
|
||||
:ref:`minio-object-locking-compliance` retention settings. An object held under
|
||||
both legal hold *and* a ``GOVERNANCE/COMPLIANCE`` retention rule remains WORM
|
||||
locked until the legal hold is lifed *and* the rule expires.
|
||||
|
||||
For ``GOVERNANCE`` locked objects, the legal hold prevents mutating the object
|
||||
*even if* the user has the necessary privileges to bypass retention.
|
60
source/object-retention/minio-object-retention.rst
Normal file
@ -0,0 +1,60 @@
|
||||
.. _minio-object-retention:
|
||||
|
||||
======================
|
||||
MinIO Object Retention
|
||||
======================
|
||||
|
||||
.. default-domain:: minio
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
By default, each new write operation to a unique object name results in
|
||||
overwriting that object. You can configure MinIO to instead create
|
||||
:ref:`versions <minio-bucket-versioning>` of each object mutation, preserving
|
||||
the full history of that object. MinIO also supports
|
||||
:ref:`Write-Once Read-Many (WORM) locking <minio-object-locking>` versioned
|
||||
objects to ensure complete immutability for a specified duration *or* until the
|
||||
lock is explicitly lifted.
|
||||
|
||||
Both versioning and object locking features are available only with
|
||||
:ref:`distributed MinIO deployments <minio-installation-comparison>`.
|
||||
|
||||
.. card:: Bucket Versioning
|
||||
:link: minio-bucket-versioning
|
||||
:link-type: ref
|
||||
|
||||
MinIO keeps each mutation to an object as a full "version" of that object.
|
||||
|
||||
.. image:: /images/retention/minio-versioning-multiple-versions.svg
|
||||
:width: 600px
|
||||
:alt: Bucket with multiple versions of an object
|
||||
:align: center
|
||||
|
||||
Clients by default retrive the latest version of an object and can
|
||||
explicitly list and retrieve any other version in the object's history.
|
||||
|
||||
.. card:: WORM Object Locking
|
||||
:link: minio-object-locking
|
||||
:link-type: ref
|
||||
|
||||
MinIO enforces Write-Once Read Many (WORM) immutability on versioned objects.
|
||||
Clients cannot delete a WORM-locked object until the configured locking
|
||||
rules expire or are explicitly lifted.
|
||||
|
||||
.. image:: /images/retention/minio-object-locking.svg
|
||||
:width: 600px
|
||||
:alt: Bucket with multiple versions of an object
|
||||
:align: center
|
||||
|
||||
MinIO supports setting bucket-default or per-object WORM locking rules with
|
||||
either duration-based or indefinite lock expirations. MinIO object locking is
|
||||
feature-compatible with AWS S3 object locking.
|
||||
|
||||
.. toctree::
|
||||
:titlesonly:
|
||||
:hidden:
|
||||
|
||||
Object Versioning </concepts/bucket-versioning>
|
||||
Object Locking </object-retention/minio-object-locking>
|
@ -17,13 +17,16 @@ Description
|
||||
|
||||
.. start-mc-retention-desc
|
||||
|
||||
The :mc:`mc retention` command configures the Write-Once Read-Many (WORM) object
|
||||
lock settings for an object or object(s) in a bucket. You can also set the
|
||||
default object lock settings for a bucket, where all objects without explicit
|
||||
object lock settings inherit the bucket default.
|
||||
The :mc:`mc retention` command configures the :ref:`Write-Once Read-Many (WORM)
|
||||
locking <minio-object-locking>` settings for an object or object(s) in a bucket.
|
||||
You can also set the default object lock settings for a bucket, where all
|
||||
objects without explicit object lock settings inherit the bucket default.
|
||||
|
||||
.. end-mc-retention-desc
|
||||
|
||||
To lock an object under :ref:`legal hold <minio-object-locking-legalhold>`,
|
||||
use :mc:`mc legalhold`.
|
||||
|
||||
.. note::
|
||||
|
||||
Starting in version :mc-release:`RELEASE.2020-09-18T00-13-21Z`,
|
||||
|
@ -213,6 +213,19 @@ destination clusters to ensure the safety of objects during transmission.
|
||||
MinIO does *not* support replicating client-side encrypted objects
|
||||
(SSE-C).
|
||||
|
||||
Replication of Locked Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO supports replicating objects held under
|
||||
:ref:`WORM Locking <minio-object-locking>`. Both the source and destination
|
||||
buckets *must* have object locking enabled for MinIO to replicate the locked
|
||||
object.
|
||||
|
||||
You must enable object locking during bucket creation as per S3 behavior.
|
||||
You can then configure object retention rules at any time.
|
||||
Object locking requires :ref:`versioning <minio-bucket-versioning>` and
|
||||
enables the feature implicitly.
|
||||
|
||||
Procedure
|
||||
---------
|
||||
|
||||
|
@ -212,13 +212,20 @@ destination clusters to ensure the safety of objects during transmission.
|
||||
MinIO does *not* support replicating client-side encrypted objects
|
||||
(SSE-C).
|
||||
|
||||
Replication of Object Retention (WORM, Legal Hold)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Replication of Locked Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO supports replicating objects with retention settings, such as
|
||||
:abbr:`WORM (Write-Once Read-Many)` object locking or legal holds. Both the
|
||||
source and destination bucket *must* have object locking enabled for MinIO
|
||||
to replicate objects with their associated retention settings.
|
||||
MinIO supports replicating objects held under
|
||||
:ref:`WORM Locking <minio-object-locking>`. Both replication buckets *must* have
|
||||
object locking enabled for MinIO to replicate the locked object. For
|
||||
active-active configuration, MinIO recommends using the *same*
|
||||
retention rules on both buckets to ensure consistent behavior across
|
||||
sites.
|
||||
|
||||
You must enable object locking during bucket creation as per S3 behavior.
|
||||
You can then configure object retention rules at any time.
|
||||
Object locking requires :ref:`versioning <minio-bucket-versioning>` and
|
||||
enables the feature implicitly.
|
||||
|
||||
Procedure
|
||||
---------
|
||||
|