Aggregate information on deleting objects (#1218)

Creates a new object management page to aggregate information on
deleting objects.
Links to the page from other docs throughout.

Intentionally leaves most delete information where it is in existing
docs.
    
Closes #741

source/administration/batch-framework-job-expire.rst

@@ -86,7 +86,7 @@ Expire Batch Job Reference
        - ``object`` - Applies only to objects which do **not** have a ``DeleteMarker`` as the current version.
        - ``deleted`` - Applies only to objects which **do** Have a ``DeleteMarker`` as the current version.
 
-       See :ref:`minio-bucket-versioning-delete` for more complete documentation on ``DeleteMarker`` or delete operations in versioned buckets.
+       See :ref:`minio-object-delete` for more complete documentation on ``DeleteMarker`` or delete operations in versioned buckets.
 
    * - ``rules.[n].name``
      - *Optional*

source/administration/bucket-replication.rst

@@ -102,7 +102,7 @@ to avoid costs associated to the "lost" data.
 Replication of Delete Operations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-MinIO supports replicating delete operations, where MinIO synchronizes
+MinIO supports replicating :ref:`delete <minio-object-delete>` operations, where MinIO synchronizes
 deleting specific object versions *and* new 
 :s3-docs:`delete markers <delete-marker-replication.html>`. Delete operation
 replication uses the same :ref:`replication process <minio-replication-process>`

source/administration/bucket-replication/enable-server-side-multi-site-bucket-replication.rst

@@ -74,7 +74,7 @@ Click to expand any of the following:
 
    MinIO supports customizing the replication configuration to enable or disable the following replication behaviors:
 
-   - Replication of delete operations
+   - Replication of :ref:`delete operations <minio-object-delete>`
    - Replication of delete markers
    - Replication of existing objects
    - Replication of metadata-only changes
@@ -92,7 +92,7 @@ Click to expand any of the following:
 .. dropdown:: Replication of Delete Operations
    :icon: fold-down
 
-   MinIO supports replicating delete operations onto the target bucket. 
+   MinIO supports replicating :ref:`delete operations <minio-object-delete>` onto the target bucket. 
    Specifically, MinIO can replicate versioning :s3-docs:`Delete Markers <versioning-workflows.html>` and the deletion of specific versioned objects:
 
    - For delete operations on an object, MinIO replication also creates the delete marker on the target bucket.

source/administration/bucket-replication/enable-server-side-one-way-bucket-replication.rst

@@ -66,7 +66,7 @@ Click to expand any of the following:
 
    MinIO does *not* replicate delete operations resulting from the application of :ref:`lifecycle management expiration rules <minio-lifecycle-management-expiration>`.
 
-   See :ref:`minio-replication-behavior-delete` for more complete documentation.
+   See :ref:`minio-replication-behavior-delete` and :ref:`minio-object-delete` for more complete documentation.
 
 .. dropdown:: Multi-Site Replication
    :icon: fold-down

source/administration/bucket-replication/enable-server-side-two-way-bucket-replication.rst

@@ -66,7 +66,7 @@ Considerations
 
    MinIO supports customizing the replication configuration to enable or disable the following replication behaviors:
 
-   - Replication of delete operations
+   - Replication of :ref:`delete operations <minio-object-delete>`
    - Replication of delete markers
    - Replication of existing objects
    - Replication of metadata-only changes
@@ -97,7 +97,7 @@ Considerations
    MinIO does *not* replicate delete operations resulting from the application of :ref:`lifecycle management expiration rules <minio-lifecycle-management-expiration>`. 
    Configure matching expiration rules on both the source and destination bucket to ensure consistent application of object expiration.
 
-   See :ref:`minio-replication-behavior-delete` for more complete documentation.
+   See :ref:`minio-replication-behavior-delete` and :ref:`minio-object-delete` for more complete documentation.
 
 .. dropdown:: Multi-Site Replication
    :icon: fold-down

source/administration/console/managing-objects.rst

@@ -46,7 +46,7 @@ Example actions the user may be able to perform include:
 - Manage tags
 - Inspect
 - Display versions
-- Delete
+- :ref:`Delete <minio-object-delete>`
 
 .. versionadded:: Console v0.24.0
 

source/administration/identity-access-management/policy-based-access-control.rst

@@ -605,7 +605,7 @@ Bucket Replication
 
    MinIO Extension for controlling API operations related to :ref:`Server-Side Bucket Replication <minio-bucket-replication-serverside>`.
 
-   Required for synchronizing delete operations as part of MinIO server-side replication.
+   Required for synchronizing :ref:`delete operations <minio-object-delete>` as part of MinIO server-side replication.
    
    Supports the following additional :ref:`condition keys <minio-policy-conditions>`:
 

source/administration/object-management.rst

@@ -94,7 +94,7 @@ Object Organization and Planning
 --------------------------------
 
 Administrators typically control the creation and configuration of buckets.
-Client applications can then use :ref:`S3-compatible SDKs <minio-drivers>` to create, list, retrieve, and delete objects on the MinIO deployment.
+Client applications can then use :ref:`S3-compatible SDKs <minio-drivers>` to create, list, retrieve, and :ref:`delete <minio-object-delete>` objects on the MinIO deployment.
 Clients therefore drive the overall hierarchy of data within a given bucket or prefix, where Administrators can exercise control using :ref:`policies <minio-policy>` to grant or deny access to an action or resource.
 
 .. cond:: windows
@@ -129,7 +129,7 @@ Object Versioning
    :alt: Object with Multiple Versions
    :align: center
 
-The specific client behavior on write, list, get, or delete operations on a bucket depends on the versioning state of that bucket:
+The specific client behavior on write, list, get, or :ref:`delete <minio-object-delete>` operations on a bucket depends on the versioning state of that bucket:
 
 .. list-table::
    :stub-columns: 1
@@ -162,6 +162,8 @@ The specific client behavior on write, list, get, or delete operations on a buck
 
        Supports deleting any object version by version ID (hard delete).
        You cannot undo hard-delete operations.
+
+       Refer to :ref:`minio-object-delete` for more information.
      - Deletes the object
 
 See :ref:`minio-bucket-versioning` for more complete documentation.
@@ -200,7 +202,7 @@ Enabling bucket locking also enables :ref:`versioning <minio-bucket-versioning>`
 
 MinIO Object Locking provides key data retention compliance and meets SEC17a-4(f), FINRA 4511(C), and CFTC 1.31(c)-(d) requirements as per `Cohasset Associates <https://min.io/cohasset?ref=docs>`__.
 
-See :ref:`minio-object-locking` for more complete documentation.
+See :ref:`minio-object-locking` and :ref:`minio-object-delete` for more complete documentation.
 
 Object Lifecycle Management
 ---------------------------
@@ -224,5 +226,6 @@ See :ref:`minio-lifecycle-management` for more complete documentation.
 
    /administration/object-management/object-versioning
    /administration/object-management/object-retention
+   /administration/object-management/object-delete
    /administration/object-management/object-lifecycle-management
    /administration/object-management/data-compression

source/administration/object-management/object-delete.rst

@@ -0,0 +1,177 @@
+.. _minio-object-delete:
+
+===============
+Object Deletion
+===============
+
+.. default-domain:: minio
+
+.. contents:: Table of Contents
+   :local:
+   :depth: 2
+
+Overview
+--------
+
+This page summarizes how a ``DELETE`` operation affects objects depending on the configuration of the bucket that contains the object.
+
+Any combination of the following factors may impact how ``DELETE`` operations function:
+
+- :ref:`Bucket versioning <minio-bucket-versioning>`
+- :ref:`Object locking rules <minio-object-locking>`
+- :ref:`Object Lifecycle Management rules <minio-lifecycle-management>`
+- :ref:`Object tiering <minio-lifecycle-management-tiering>`
+- :ref:`Site <minio-site-replication-overview>` or :ref:`bucket <minio-replication-behavior-delete>` replication
+- :ref:`Scanner <minio-concepts-scanner>`
+
+Permissions
+-----------
+
+MinIO uses a :ref:`policy based access control <minio-policy>` system for access management.
+The user or service account must provide the correct policy action and conditions to allow a ``DELETE`` for the bucket and object.
+
+Unversioned Objects
+-------------------
+
+When performing a ``DELETE`` operation on an object in a bucket that does not have versioning enabled, the operation is straightforward.
+After verifying the user or service account has permission to perform the ``DELETE`` operation, MinIO permanently removes the object.
+
+The user or service account requesting the delete action the action must have the :policy-action:`s3:DeleteObject` action permission for the bucket and object.
+
+Versioned Objects
+-----------------
+
+``DELETE`` operations work differently when an object is versioned.
+
+The user or service account must have the :policy-action:`s3:DeleteObjectVersion` action permission for the bucket and object.
+
+Delete operations on the current version
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A ``DELETE`` operation on a versioned object that does not specify a version UUID results in the creation of a ``DeleteMarker`` placed as the ``head`` of the object.
+
+In this scenario, MinIO does not actually remove the object or any of its versions from the disk.
+All existing versions of the object remain available to access by specifying the version's UUID.
+When a ``DeleteMarker`` is the head for the object, MinIO does not serve the object for ``GET`` requests that do not specify a version ID.
+Instead, MinIO returns a ``404``-like response. 
+
+You can find the UUID of object versions with :mc-cmd:`mc ls --versions`.
+
+To remove the current version of the object from the drive, find the UUID of the version, and then use :mc-cmd:`mc rm --version-id=UUID ... <mc rm --version-id>` to delete the current version.
+In this scenario, the immediately preceding version of the object then becomes the current version of the object served for ``GET`` requests of the object with no UUID specified.
+
+.. warning::
+
+   Specifying a ``version-id`` in a DELETE operation is irreversible.
+   MinIO removes the specified version from the drive and **cannot** retrieve it.
+
+Delete operations on a prior version
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To delete prior versions of an object, specify the version's UUID.
+You can retrieve the version UUID with :mc-cmd:`mc ls --versions`. 
+When the ``DELETE`` request specifies a ``version-id`` and the user has the correct permissions to delete the object version`, MinIO permanently removes the specified version from the drive.
+
+.. warning::
+
+   Specifying a ``version-id`` in a DELETE operation is irreversible.
+   MinIO removes the specified version from the drive and **cannot** retrieve it.
+
+Delete all versions
+~~~~~~~~~~~~~~~~~~~
+
+Use :mc-cmd:`mc rm --versions` to delete *all* versions of an object.
+This is irreversible.
+
+Lifecycle Management Expiration
+-------------------------------
+
+You can define one or more :ref:`lifecycle management expiration rule(s) <minio-lifecycle-management-create-expiry-rule>` to expire objects after a certain version number count or a certain period of time.
+When more versions exist than the rule specifies, or when a version is older than specified, MinIO permanently removes the object version from the drive.
+
+These rules rely on the :ref:`scanner <minio-concepts-scanner>` to process the rule on the bucket.
+The scanner operates as a lower priority continuous process where ``READ`` and ``WRITE`` actions are preferred.
+Because of this, object versions that meet the requirements for expiration may not immediately be removed from MinIO.
+
+See the :ref:`scanner <minio-concepts-scanner>` page for more details on how the scanner works and configuration options.
+
+``DeleteMarkers`` are their own objects.
+Lifecycle rules can remove ``DeleteMarkers`` that are the only remaining versions of their objects.
+
+Retained Objects
+----------------
+
+MinIO protects objects subject to a :ref:`locking rule <minio-object-locking>` from being overwritten or deleted.
+These rules require that objects be retained until either the rule expires or is removed.
+
+``DELETE`` operations on locked objects without a specified version result in the creation of a `DeleteMarker` for the object.
+However, the object versions themselves are retained as required by the lock.
+
+``DELETE`` operations that specify an object version are subject to the retention rules.
+MinIO protects object versions subject to a lock from being overwritten or deleted until the lock expires or is removed.
+
+Replicated Objects
+------------------
+
+Replication duplicates objects from one location to another.
+MinIO supports replication at the bucket level or the cluster ("site") level.
+
+Delete operations may or may not replicate, depending on the type of replication and how the replication is configured.
+
+Site Replication
+~~~~~~~~~~~~~~~~
+
+For clusters with :ref:`multi-site replication <minio-site-replication-overview>` enabled, MinIO replicates all ``delete`` operations performed on any cluster to each of the other clusters in the peer group.
+
+Delete behavior on any single peer follows the same processes as any MinIO deployment.
+
+Bucket Replication
+~~~~~~~~~~~~~~~~~~
+
+With :ref:`bucket replication <minio-bucket-replication>`, MinIO supports replicating delete operations between a source bucket and a configured remote bucket.
+MinIO synchronizes deleting specific object versions *and* new  :s3-docs:`delete markers <delete-marker-replication.html>`. 
+Delete operation replication uses the same :ref:`replication process <minio-replication-process>` as all other replication operations. 
+
+MinIO requires *explicitly enabling* versioned deletes and delete marker replication. 
+Use the :mc-cmd:`mc replicate add --replicate` field to specify either ``delete`` and ``delete-marker`` or both to enable versioned deletes and delete marker replication, respectively. 
+To enable both, specify both strings using a comma separator: ``delete,delete-marker``.
+
+For delete marker replication, MinIO begins the replication process after a delete operation creates the delete marker. 
+MinIO uses the ``X-Minio-Replication-DeleteMarker-Status`` metadata field for tracking  delete marker replication status. 
+In :ref:`active-active <minio-bucket-replication-serverside-twoway>` replication configurations, MinIO may produce duplicate delete markers if both clusters concurrently create a delete marker for an object *or* if one or both clusters were down before the replication event synchronized.
+
+For replicating the deletion of a specific object version, MinIO marks the object version as ``PENDING`` until replication completes. 
+Once the remote target deletes that object version, MinIO deletes the object version on the source.
+While this process ensures near-synchronized version deletion, it may result in listing operations returning the object version after the initial delete operation. 
+MinIO uses the ``X-Minio-Replication-Delete-Status`` for tracking delete version replication status.
+
+MinIO only replicates explicit client-driven delete operations. 
+MinIO does *not* replicate objects deleted by :ref:`lifecycle management expiration rules <minio-lifecycle-management-expiration>`. 
+For :ref:`active-active <minio-bucket-replication-serverside-twoway>` configurations, set the same expiration rules on *all* of of the replication buckets to ensure consistent application of object expiration.
+
+.. admonition:: MinIO Trims Empty Object Prefixes on Source and Remote Bucket
+   :class: note, dropdown
+
+   If a delete operation removes the last object in a bucket prefix, MinIO recursively removes each empty part of the prefix up to the bucket root.
+   MinIO only applies the recursive removal to prefixes created *implicitly* as part of object write operations.
+   MinIO does not recursively remove prefixes created using an explicit directory creation command, such as :mc:`mc mb`.
+
+   If a replication rule enables replication delete operations, the replication process *also* applies the implicit prefix trimming behavior on the destination MinIO cluster.
+
+   For example, consider a bucket ``photos`` with the following object prefixes:
+   
+   - ``photos/2021/january/myphoto.jpg`` // ``2021/january/`` created implicitly based on the object name
+   - ``photos/2021/february/myotherphoto.jpg``  // ``2021/february/`` created implicitly based on the object name
+   - ``photos/NYE21/NewYears.jpg``  // ``NYE21/`` explicitly created in the bucket
+
+   ``photos/NYE21`` is the *only* prefix explicitly created using :mc:`mc mb`.
+   All other prefixes were *implicitly* created as part of writing the object located at that prefix. 
+   
+   - A command removes ``myphoto.jpg``. 
+     MinIO automatically trims the empty ``/january/`` prefix. 
+   
+   - A command then removes the ``myotherphoto.jpg``. 
+     MinIO automatically trims the ``/february/`` prefix *and* the now-empty ``/2021`` prefix. 
+   
+   - A command removes the ``NewYears.jpg`` object. 
+     MinIO leaves the ``/NYE21/`` prefix remains in place since it was *explicitly* created.

source/administration/object-management/object-retention.rst

@@ -59,7 +59,7 @@ SEC17a-4(f), FINRA 4511(C), and CFTC 1.31(c)-(d) requirements as per
          :align: center
          :width: 600px
 
-      Delete operations follow normal behavior in 
+      :ref:`Delete operations <minio-object-delete>` follow normal behavior in 
       :ref:`versioned buckets <minio-bucket-versioning-delete>`, where MinIO
       creates a ``DeleteMarker`` for the object. However, non-Delete Marker 
       versions of the object remain under the retention rules and are protected 
@@ -72,7 +72,7 @@ SEC17a-4(f), FINRA 4511(C), and CFTC 1.31(c)-(d) requirements as per
          :align: center
          :width: 600px
 
-      MinIO blocks any attempt to delete a specific object version held under
+      MinIO blocks any attempt to :ref:`delete <minio-object-delete>` a specific object version held under
       WORM lock. The earliest possible time after which a client may delete
       the version is when the lock expires.
 

source/reference/deprecated/mc-ilm-add.rst

@@ -130,7 +130,7 @@ Parameters
 
    - :mc-cmd:`~mc ilm add --expire-delete-marker`
 
-   For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration`.
+   For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration` and :ref:`minio-object-delete`.
 
 .. mc-cmd:: --expire-delete-marker
    :optional:
@@ -147,8 +147,7 @@ Parameters
    Slow scanning due to high IO workloads or limited system resources may delay application of lifecycle management rules. 
    See :ref:`minio-lifecycle-management-scanner` for more information.
 
-   For more complete documentation on object expiration, see
+   For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration` and :ref:`minio-object-delete`.
-   :ref:`minio-lifecycle-management-expiration`.
 
 .. mc-cmd:: --transition-days
    :optional:

source/reference/deprecated/mc-ilm-edit.rst

@@ -146,6 +146,8 @@ Parameters
 
    - :mc-cmd:`~mc ilm edit --expired-object-delete-marker`
 
+   See :ref:`minio-object-delete` for more information.
+
 .. mc-cmd:: --expired-object-delete-marker
    :optional:
 
@@ -161,7 +163,7 @@ Parameters
    MinIO uses a :ref:`scanner process <minio-concepts-scanner>` to check objects against all configured
    lifecycle management rules. Slow scanning due to high IO workloads or
    limited system resources may delay application of lifecycle management
-   rules. See :ref:`minio-lifecycle-management-scanner` for more information.
+   rules. See :ref:`minio-lifecycle-management-scanner` and :ref:`minio-object-delete` for more information.
 
 .. mc-cmd:: --noncurrentversion-expiration-days
    :optional:

source/reference/minio-mc/mc-ilm-rule-add.rst

@@ -144,7 +144,7 @@ Parameters
 
    - :mc-cmd:`~mc ilm rule add --expire-delete-marker`
 
-   For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration`.
+   For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration` and :ref:`minio-object-delete`.
 
 .. mc-cmd:: --expire-delete-marker
    :optional:
@@ -161,8 +161,7 @@ Parameters
    Slow scanning due to high IO workloads or limited system resources may delay application of lifecycle management rules. 
    See :ref:`minio-lifecycle-management-scanner` for more information.
 
-   For more complete documentation on object expiration, see
+   For more complete documentation on object expiration, see :ref:`minio-lifecycle-management-expiration` and :ref:`minio-object-delete`.
-   :ref:`minio-lifecycle-management-expiration`.
 
 .. mc-cmd:: --transition-days
    :optional:

source/reference/minio-mc/mc-ilm-rule-edit.rst

@@ -162,7 +162,7 @@ Parameters
    MinIO uses a :ref:`scanner process <minio-concepts-scanner>` to check objects against all configured
    lifecycle management rules. Slow scanning due to high IO workloads or
    limited system resources may delay application of lifecycle management
-   rules. See :ref:`minio-lifecycle-management-scanner` for more information.
+   rules. See :ref:`minio-lifecycle-management-scanner` and :ref:`minio-object-delete` for more information.
 
 .. mc-cmd:: --noncurrent-expire-days
    :optional:

source/reference/minio-mc/mc-replicate-add.rst

@@ -252,7 +252,7 @@ Parameters
 
    Specify a comma-separated list of the following values to enable extended replication features. 
 
-   - ``delete`` - Directs MinIO to replicate DELETE operations to the destination bucket.
+   - ``delete`` - Directs MinIO to replicate :ref:`DELETE operations <minio-object-delete>` to the destination bucket.
 
    - ``delete-marker`` - Directs MinIO to replicate delete markers to the destination bucket. 
 

source/reference/minio-mc/mc-replicate-update.rst

@@ -232,7 +232,7 @@ Parameters
 
    Specify a comma-separated list of the following values to enable extended replication features:
 
-   - ``delete`` - Directs MinIO to replicate DELETE operations to the destination bucket.
+   - ``delete`` - Directs MinIO to replicate :ref:`DELETE operations <minio-object-delete>` to the destination bucket.
 
    - ``delete-marker`` - Directs MinIO to replicate delete markers to the destination bucket. 
 

source/reference/minio-mc/mc-rm.rst

@@ -21,7 +21,7 @@ Syntax
 
 .. start-mc-rm-desc
 
-The :mc:`mc rm` command removes objects from a bucket on a MinIO deployment. 
+The :mc:`mc rm` command :ref:`removes objects <minio-object-delete>` from a bucket on a MinIO deployment. 
 To completely remove a bucket, use :mc:`mc rb` instead.
 
 .. end-mc-rm-desc
@@ -29,6 +29,8 @@ To completely remove a bucket, use :mc:`mc rb` instead.
 You can also use :mc:`mc rm` against the local filesystem to produce similar
 results to the ``rm`` commandline tool.
 
+For more information on how MinIO performs ``DELETE`` actions on objects, see :ref:`minio-object-delete`.
+
 .. important::
 
    :mc:`mc rm` supports removing multiple objects *or* files in a single

source/url-excludes.yaml

@@ -3,6 +3,7 @@ excludes:
 - 'operations/install-deploy-manage/deploy-minio-tenant.rst'
 - 'operations/install-deploy-manage/modify-minio-tenant.rst'
 - 'operations/install-deploy-manage/deploy-operator-helm.rst'
+- 'operations/install-deploy-manage/deploy-operator-kustomize.rst'
 - 'operations/install-deploy-manage/expand-minio-tenant.rst'
 - 'operations/install-deploy-manage/upgrade-minio-tenant.rst'
 - 'operations/install-deploy-manage/upgrade-minio-operator.rst'
@@ -37,6 +38,7 @@ excludes:
 - 'operations/install-deploy-manage/decommission-server-pool.rst'
 - 'operations/install-deploy-manage/deploy-minio-tenant.rst'
 - 'operations/install-deploy-manage/deploy-operator-helm.rst'
+- 'operations/install-deploy-manage/deploy-operator-kustomize.rst'
 - 'operations/install-deploy-manage/modify-minio-tenant.rst'
 - 'operations/install-deploy-manage/expand-minio-tenant.rst'
 - 'operations/install-deploy-manage/upgrade-minio-tenant.rst'
@@ -68,6 +70,7 @@ excludes:
 - 'operations/install-deploy-manage/deploy-minio-single-node-multi-drive.rst'
 - 'operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.rst'
 - 'operations/install-deploy-manage/deploy-operator-helm.rst'
+- 'operations/install-deploy-manage/deploy-operator-kustomize.rst'
 - 'operations/install-deploy-manage/multi-site-replication.rst'
 - 'operations/install-deploy-manage/deploy-minio-tenant.rst'
 - 'operations/install-deploy-manage/modify-minio-tenant.rst'
@@ -95,6 +98,7 @@ tag: container
 excludes:
 - 'operations/install-deploy-manage/deploy-minio-tenant.rst'
 - 'operations/install-deploy-manage/deploy-operator-helm.rst'
+- 'operations/install-deploy-manage/deploy-operator-kustomize.rst'
 - 'operations/install-deploy-manage/modify-minio-tenant.rst'
 - 'operations/install-deploy-manage/expand-minio-tenant.rst'
 - 'operations/install-deploy-manage/upgrade-minio-tenant.rst'