diff --git a/source/includes/common-minio-tiering.rst b/source/includes/common-minio-tiering.rst index cef1069e..ec11f96f 100644 --- a/source/includes/common-minio-tiering.rst +++ b/source/includes/common-minio-tiering.rst @@ -81,4 +81,49 @@ aliases have the necessary permissions for creating policies and users on the deployment. See :ref:`minio-users` and :ref:`MinIO Policy Based Access Control ` for more complete documentation on MinIO users and policies respectively. -.. end-create-transition-user-desc \ No newline at end of file +.. end-create-transition-user-desc + +.. start-transition-bucket-access-desc + +MinIO *requires* exclusive access to the transitioned data on the remote storage +tier. MinIO ignores any objects in the remote bucket or bucket prefix not +explicitly managed by the MinIO deployment. Automatic transition and transparent +object retrieval depend on the following assumptions: + +- No external mutation, migration, or deletion of objects on the remote storage. +- No lifecycle management rules (e.g. transition or expiration) on the remote + storage bucket. + +MinIO stores all transitioned objects in the remote storage bucket or resource +under a unique per-deployment prefix value. This value is not intended to +support identifying the source deployment from the backend. MinIO supports an +additional optional human-readable prefix when configuring the remote target, +which may facilitate operations related to diagnostics, maintenance, or disaster +recovery. + +MinIO recommends specifying this optional prefix for remote storage tiers which +contain other data, including transitioned objects from other MinIO deployments. +This tutorial includes the necessary syntax for setting this prefix. + +.. end-transition-bucket-access-desc + +.. start-transition-data-loss-desc + +MinIO creates metadata for each transitioned object that identifies its location +on the remote storage. This metadata is required for accessing the object, such +that applications cannot access a transition object independent of MinIO. +Availability of the transitioned data therefore depends on the same core +protections that :ref:`erasure coding ` and distributed +deployment topologies provide for all objects on the MinIO deployment. Using +object transition does not provide any additional business continuity or +disaster recovery benefits. + +Workloads that require :abbr:`BC/DR (Business Continuity/Disaster Recovery)` +protections should implement MinIO :ref:`Server-Side replication +`. Replication ensures objects remains +preserved on the remote replication site, such that you can resynchronize from +the remote in the event of partial or total data loss. See +:ref:`minio-replication-behavior-resync` for more complete documentation on +using replication to recover after partial or total data loss. + +.. end-transition-data-loss-desc \ No newline at end of file diff --git a/source/lifecycle-management/lifecycle-management-overview.rst b/source/lifecycle-management/lifecycle-management-overview.rst index ba76c77e..dfd4a49f 100644 --- a/source/lifecycle-management/lifecycle-management-overview.rst +++ b/source/lifecycle-management/lifecycle-management-overview.rst @@ -36,23 +36,30 @@ following public cloud storage services: - :ref:`Microsoft Azure Blob Storage ` -MinIO object transition supports use cases like moving aged data from -MinIO clusters in private or public cloud infrastructure to low-cost -public cloud storage solutions. - -MinIO manages retrieving tiered objects on-the-fly without any additional -application-side logic. MinIO retains the minimum object metadata required to -support retrieving objects transitioned to a remote tier. MinIO therefore -*requires* exclusive access to the data on the remote storage tier. Object -retrieval assumes no external mutation, migration, or deletion of stored -objects. Object transition is *not* a replacement for backup/recovery -strategies such as :ref:`minio-bucket-replication`. +MinIO object transition supports use cases like moving aged data from MinIO +clusters in private or public cloud infrastructure to low-cost private or public cloud +storage solutions. MinIO manages retrieving tiered objects on-the-fly without +any additional application-side logic. Use the :mc-cmd:`mc admin tier` command to create a remote target for tiering data to a supported Cloud Service Provider object storage. You can then use the :mc-cmd-option:`mc ilm add transition-days` command to transition objects to the remote tier after a specified number of calendar days. +Exclusive Access to Remote Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-tiering.rst + :start-after: start-transition-bucket-access-desc + :end-before: end-transition-bucket-access-desc + +Availability of Remote Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-tiering.rst + :start-after: start-transition-data-loss-desc + :end-before: end-transition-data-loss-desc + Versioned Buckets ~~~~~~~~~~~~~~~~~ diff --git a/source/lifecycle-management/transition-objects-to-azure.rst b/source/lifecycle-management/transition-objects-to-azure.rst index 0570b7a1..3a86d97b 100644 --- a/source/lifecycle-management/transition-objects-to-azure.rst +++ b/source/lifecycle-management/transition-objects-to-azure.rst @@ -78,25 +78,12 @@ permissions. Considerations -------------- -Lifecycle Management Object Scanner -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Exclusive Access to Remote Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -MinIO uses a scanner process to check objects against all configured -lifecycle management rules. Slow scanning due to high IO workloads or -limited system resources may delay application of lifecycle management -rules. See :ref:`minio-lifecycle-management-scanner` for more information. - -Exclusive Bucket Access -~~~~~~~~~~~~~~~~~~~~~~~ - -MinIO retains the minimum object metadata required to -support retrieving objects transitioned to a remote tier. MinIO therefore -*requires* exclusive access to the data on the remote storage tier. Object -retrieval assumes no external mutation, migration, or deletion of stored -objects. - -MinIO also ignores any objects in the remote bucket or bucket prefix not -explicitly managed by MinIO. +.. include:: /includes/common-minio-tiering.rst + :start-after: start-transition-bucket-access-desc + :end-before: end-transition-bucket-access-desc .. important:: @@ -108,6 +95,13 @@ explicitly managed by MinIO. Please contact `MinIO Support `__ if you need situation-specific guidance around configuring Azure remote tiers. +Availability of Remote Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-tiering.rst + :start-after: start-transition-data-loss-desc + :end-before: end-transition-data-loss-desc + Procedure --------- @@ -166,7 +160,16 @@ The example above uses the following arguments: * - :mc-cmd:`PREFIX ` - The optional bucket prefix within which MinIO transitions objects. - Omit this argument to transition objects to the bucket root. + + MinIO stores all transitioned objects in the specified ``BUCKET`` under a + unique per-deployment prefix value. Omit this argument to use only that + value for isolating and organizing data within the remote storage. + + MinIO recommends specifying this optional prefix for remote storage tiers + which contain other data, including transitioned objects from other MinIO + deployments. This prefix should provide a clear reference back to the + source MinIO deployment to faciliate ease of operations related to + diagnostics, maintenance, or disaster recovery. * - :mc-cmd:`ACCOUNT ` - The account name MinIO uses to access the bucket. The account name @@ -192,7 +195,7 @@ The example above uses the following arguments: :end-before: end-create-transition-rule-desc -3) Verify the Transition Rule +4) Verify the Transition Rule ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Use the :mc-cmd:`mc ilm ls` command to review the configured transition diff --git a/source/lifecycle-management/transition-objects-to-gcs.rst b/source/lifecycle-management/transition-objects-to-gcs.rst index 6ce20339..8ea6236a 100644 --- a/source/lifecycle-management/transition-objects-to-gcs.rst +++ b/source/lifecycle-management/transition-objects-to-gcs.rst @@ -86,17 +86,19 @@ lifecycle management rules. Slow scanning due to high IO workloads or limited system resources may delay application of lifecycle management rules. See :ref:`minio-lifecycle-management-scanner` for more information. -Exclusive Bucket Access -~~~~~~~~~~~~~~~~~~~~~~~ +Exclusive Access to Remote Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -MinIO retains the minimum object metadata required to -support retrieving objects transitioned to a remote tier. MinIO therefore -*requires* exclusive access to the data on the remote storage tier. Object -retrieval assumes no external mutation, migration, or deletion of stored -objects. +.. include:: /includes/common-minio-tiering.rst + :start-after: start-transition-bucket-access-desc + :end-before: end-transition-bucket-access-desc -MinIO also ignores any objects in the remote bucket or bucket prefix not -explicitly managed by MinIO. +Availability of Remote Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-tiering.rst + :start-after: start-transition-data-loss-desc + :end-before: end-transition-data-loss-desc Procedure --------- @@ -122,7 +124,7 @@ service as the remote storage tier: mc admin tier add gcs TARGET TIER_NAME \ --endpoint https://HOSTNAME \ --bucket BUCKET \ - --prefix PREFIX + --prefix PREFIX \ --credentials-file CREDENTIALS \ --region REGION @@ -155,7 +157,16 @@ The example above uses the following arguments: * - :mc-cmd:`PREFIX ` - The optional bucket prefix within which MinIO transitions objects. - Omit this argument to transition objects to the bucket root. + + MinIO stores all transitioned objects in the specified ``BUCKET`` under a + unique per-deployment prefix value. Omit this argument to use only that + value for isolating and organizing data within the remote storage. + + MinIO recommends specifying this optional prefix for remote storage tiers + which contain other data, including transitioned objects from other MinIO + deployments. This prefix should provide a clear reference back to the + source MinIO deployment to faciliate ease of operations related to + diagnostics, maintenance, or disaster recovery. * - :mc-cmd:`CREDENTIALS ` - The `credential file @@ -178,7 +189,7 @@ The example above uses the following arguments: :end-before: end-create-transition-rule-desc -3) Verify the Transition Rule +4) Verify the Transition Rule ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Use the :mc-cmd:`mc ilm ls` command to review the configured transition diff --git a/source/lifecycle-management/transition-objects-to-s3.rst b/source/lifecycle-management/transition-objects-to-s3.rst index 62ad8a25..bfce401c 100644 --- a/source/lifecycle-management/transition-objects-to-s3.rst +++ b/source/lifecycle-management/transition-objects-to-s3.rst @@ -95,17 +95,19 @@ lifecycle management rules. Slow scanning due to high IO workloads or limited system resources may delay application of lifecycle management rules. See :ref:`minio-lifecycle-management-scanner` for more information. -Exclusive Bucket Access -~~~~~~~~~~~~~~~~~~~~~~~ +Exclusive Access to Remote Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -MinIO retains the minimum object metadata required to -support retrieving objects transitioned to a remote tier. MinIO therefore -*requires* exclusive access to the data on the remote storage tier. Object -retrieval assumes no external mutation, migration, or deletion of stored -objects. +.. include:: /includes/common-minio-tiering.rst + :start-after: start-transition-bucket-access-desc + :end-before: end-transition-bucket-access-desc -MinIO also ignores any objects in the remote bucket or bucket prefix not -explicitly managed by MinIO. +Availability of Remote Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-tiering.rst + :start-after: start-transition-data-loss-desc + :end-before: end-transition-data-loss-desc Procedure --------- @@ -165,7 +167,16 @@ The example above uses the following arguments: * - :mc-cmd:`PREFIX ` - The optional bucket prefix within which MinIO transitions objects. - Omit this argument to transition objects to the bucket root. + + MinIO stores all transitioned objects in the specified ``BUCKET`` under a + unique per-deployment prefix value. Omit this argument to use only that + value for isolating and organizing data within the remote storage. + + MinIO recommends specifying this optional prefix for remote storage tiers + which contain other data, including transitioned objects from other MinIO + deployments. This prefix should provide a clear reference back to the + source MinIO deployment to faciliate ease of operations related to + diagnostics, maintenance, or disaster recovery. * - :mc-cmd:`ACCESS_KEY ` - The S3 access key MinIO uses to access the bucket. The @@ -196,7 +207,7 @@ The example above uses the following arguments: :end-before: end-create-transition-rule-desc -3) Verify the Transition Rule +4) Verify the Transition Rule ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Use the :mc-cmd:`mc ilm ls` command to review the configured transition diff --git a/source/replication/enable-server-side-multi-site-bucket-replication.rst b/source/replication/enable-server-side-multi-site-bucket-replication.rst index 9221f617..d8978ab6 100644 --- a/source/replication/enable-server-side-multi-site-bucket-replication.rst +++ b/source/replication/enable-server-side-multi-site-bucket-replication.rst @@ -232,6 +232,12 @@ MinIO requires explicitly enabling replication of delete operations using the :mc-cmd-option:`mc replicate edit replicate`. This procedure includes the required flags for enabling replication of delete operations and delete markers. +MinIO does *not* replicate delete operations resulting from the application of +:ref:`lifecycle management expiration rules +`. Configure matching expiration rules +for the bucket on all replication sites to ensure consistent application of +object expiration. + Replication of Encrypted Objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -412,7 +418,7 @@ the remote as a target: the following replication features: - :ref:`Replication of Deletes ` - - :ref:`Replication of existing Objects ` + - :ref:`Replication of Existing Objects ` See :mc-cmd-option:`mc replicate add replicate` for more complete documentation. Omit these fields to disable replication of delete operations diff --git a/source/replication/enable-server-side-one-way-bucket-replication.rst b/source/replication/enable-server-side-one-way-bucket-replication.rst index a18d8a61..c7652f8e 100644 --- a/source/replication/enable-server-side-one-way-bucket-replication.rst +++ b/source/replication/enable-server-side-one-way-bucket-replication.rst @@ -181,7 +181,7 @@ required flags for enabling replication of existing objects. Replication of Delete Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -MinIO supports replicating delete operations onto the target bucket. +MinIO supports replicating S3 ``DELETE`` operations onto the target bucket. Specifically, MinIO can replicate versioning :s3-docs:`Delete Markers ` and the deletion of specific versioned objects: @@ -197,6 +197,12 @@ MinIO requires explicitly enabling replication of delete operations using the :mc-cmd-option:`mc replicate edit replicate`. This procedure includes the required flags for enabling replication of delete operations and delete markers. +MinIO does *not* replicate delete operations resulting from the +application of :ref:`lifecycle management expiration rules +`. + +See :ref:`minio-replication-behavior-delete` for more complete documentation. + Replication of Encrypted Objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/replication/enable-server-side-two-way-bucket-replication.rst b/source/replication/enable-server-side-two-way-bucket-replication.rst index ebdffb2f..3b336f0c 100644 --- a/source/replication/enable-server-side-two-way-bucket-replication.rst +++ b/source/replication/enable-server-side-two-way-bucket-replication.rst @@ -219,6 +219,14 @@ MinIO requires explicitly enabling replication of delete operations using the :mc-cmd-option:`mc replicate edit replicate`. This procedure includes the required flags for enabling replication of delete operations and delete markers. +MinIO does *not* replicate delete operations resulting from the +application of :ref:`lifecycle management expiration rules +`. Configure matching expiration rules +on both the source and destination bucket to ensure consistent application +of object expiration. + +See :ref:`minio-replication-behavior-delete` for more complete documentation. + Replication of Encrypted Objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/replication/replication-overview.rst b/source/replication/replication-overview.rst index 1c132a5e..daf80189 100644 --- a/source/replication/replication-overview.rst +++ b/source/replication/replication-overview.rst @@ -63,10 +63,198 @@ replication while adding the following MinIO-only features: - Multi-Site replication of objects between three or more MinIO deployments (New in :minio-release:`RELEASE.2021-09-23T04-46-24Z`). +.. _minio-replication-behavior-resync: + +Resynchronization (Disaster Recovery) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Resynchronization primarily supports recovery after partial or total loss of the +data on a MinIO deployment using a healthy deployment in the replica +configuration. Use the :mc-cmd:`mc replicate resync` command completely +resynchronize the remote target (:mc-cmd:`mc admin bucket remote`) using the +specified source bucket. + +The resynchronization process checks all objects in the source bucket against +all configured replication rules that include :ref:`existing object replication +`. For each object which matches a +rule, the resynchronization process places the object into the replication +:ref:`queue ` regardless of the object's current +:ref:`replication status `. + +MinIO skips synchronizing those objects whose remote copy exactly match the +source, including object metadata. MinIO otherwise does not prioritize or modify +the queue with regards to the existing contents of the target. + +:mc-cmd:`mc replicate resync` operates at the bucket level and does +*not* support prefix-level granularity. Initiating resynchronization on a large +bucket may result in a significant increase in replication-related load +and traffic. Use this command with caution and only when necessary. + +For buckets with :ref:`object transition (Tiering) +` configured, replication resynchronization +restores objects in a non-transitioned state with no associated transition +metadata. Any data previously transitioned to the remote storage is therefore +permanently disconnected from the remote MinIO deployment. For tiering +configurations which specify an explicit human-readable prefix as part of the +remote configuration, you can safely purge the transitioned data in that prefix +to avoid costs associated to the "lost" data. + +.. _minio-replication-behavior-delete: + +Replication of Delete Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO supports replicating delete operations, where MinIO synchronizes +deleting specific object versions *and* new :s3-docs:`delete markers +`. Delete operation replication uses +the same :ref:`replication process ` as all other +replication operations. + +For delete marker replication, MinIO begins the replication process after +a delete operation creates the delete marker. MinIO uses the +``X-Minio-Replication-DeleteMarker-Status`` metadata field for tracking +delete marker replication status. In +:ref:`active-active ` +replication configurations, MinIO may produce duplicate delete markers if +both clusters concurrently create a delete marker for an object *or* +if one or both clusters were down before the replication event synchronized. + +For replicating the deletion of a specific object version, MinIO marks the +object version as ``PENDING`` until replication completes. Once the remote +target deletes that object version, MinIO deletes the object on the source. +While this process ensures near-synchronized version deletion, it may result +in listing operations returning the object version after the initial +delete operation. MinIO uses the ``X-Minio-Replication-Delete-Status`` for +tracking delete version replication status. + +MinIO requires explicitly enabling versioned deletes and delete marker +replication . Use the :mc-cmd-option:`mc replicate add replicate` field to +specify both or either ``delete`` and ``delete-marker`` to enable versioned +deletes and delete marker replication respectively. To enable both, specify both +strings using a comma separator ``delete,delete-marker``. + +MinIO only replicates explicit client-driven delete operations. MinIO does *not* +replicate objects deleted from the application of +:ref:`lifecycle management expiration rules +`. For :ref:`active-active +` configurations, set the same +expiration rules on *all* of of the replication buckets to ensure consistent +application of object expiration. + +.. admonition:: MinIO Trims Empty Object Prefixes on Source and Remote Bucket + :class: note, dropdown + + If a delete operation removes the last object in a bucket prefix, MinIO + recursively removes each empty part of the prefix up to the bucket root. + MinIO only applies the recursive removal to prefixes created *implicitly* as + part of object write operations - that is, the prefix was not created using + an explicit directory creation command such as :mc:`mc mb`. + + If a replication rule enables replication delete operations, the replication + process *also* applies the implicit prefix trimming behavior on the + destination MinIO cluster. + + For example, consider a bucket ``photos`` with the following object prefixes: + + - ``photos/2021/january/myphoto.jpg`` + - ``photos/2021/february/myotherphoto.jpg`` + - ``photos/NYE21/NewYears.jpg`` + + ``photos/NYE21`` is the *only* prefix explicitly created using :mc:`mc mb`. + All other prefixes were *implicitly* created as part of writing the object + located at that prefix. + + - A command removes ``myphoto.jpg``. MinIO automatically trims the empty + ``/janaury`` prefix. + + - A command then removes the ``myotherphoto.jpg``. MinIO automatically + trims the ``/february`` prefix *and* the now-empty ``/2021`` prefix. + + - A command removes the ``NewYears.jpg`` object. MinIO leaves the + ``/NYE21`` prefix remains in place since it was *explicitly* created. + +.. _minio-replication-behavior-existing-objects: + +Replication of Existing Objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO by default does not enable existing object replication. Objects +created before replication was configured *or* while replication is +disabled are not synchronized to the target deployment. +Starting with :mc:`mc` :minio-git:`RELEASE.2021-06-13T17-48-22Z +` and :mc:`minio` +:minio-git:`RELEASE.2021-06-07T21-40-51Z +`, MinIO supports enabling +replication of existing objects in a bucket. + +Enabling existing object replication marks all objects or object prefixes that +satisfy the replication rules as eligible for synchronization to the source +cluster, *even if* those objects were created prior to configuring or enabling +replication. You can enable existing object replication while configuring +or modifying a replication rule: + +- For new replication rules, include ``"existing-objects"`` to the list of + replication features specified to :mc-cmd-option:`mc replicate add replicate`. + +- For existing replication rules, add ``"existing-objects"`` to the list of + existing replication features using + :mc-cmd-option:`mc replicate edit replicate`. You must specify *all* desired + replication features when editing the replication rule. + +Enabling existing object replication does not increase the priority of objects +pending replication. MinIO uses the same core +:ref:`replication scanner and queue system ` for +detecting and synchronizing objects regardless of the enabled replication +feature. The time required to fully synchronize a bucket depends on a number of +factors, including but not limited to the current cluster replication load, +overall cluster load, and the size of the namespace (all objects in the bucket). + +MinIO does not synchronize existing unversioned objects. Specifically, the +bucket *must* have :ref:`versioning ` enabled when the +object was created. You can use the :mc-cmd:`mc cp` command to create a +"versioned" copy of that object. Once that object replicates successfully, +you can delete the unversioned object (versionid = ``null``). + +MinIO existing object replication +implements functionality similar to +`AWS: Replicating existing objects between S3 buckets +`__ +without the overhead of contacting technical support. + +Synchronous vs Asynchronous Replication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO supports specifying either asynchronous (default) or synchronous +replication for a given remote target. + +With the default asynchronous replication, MinIO completes the originating +``PUT`` operation *before* placing the object into a :ref:`replication queue +`. The originating client may therefore see a +successful ``PUT`` operation *before* the object is replicated. While +this may result in stale or missing objects on the remote, it mitigates +the risk of slow write operations due to replication load. + +With synchronous replication, MinIO attempts to replicate the object *prior* to +completing the originating ``PUT`` operation. MinIO returns a successful ``PUT`` +operation whether or not the replication attempts succeeds. While this may +result in more reliable synchronization between the source and remote target, +it may also increase the time of each write operation due to replication load. + +You must explicitly enable synchronous replication when configuring the remote +target target using the :mc-cmd:`mc admin bucket remote add` command with the +:mc-cmd-option:`~mc admin bucket remote add sync` flag. + +Replication Internals +~~~~~~~~~~~~~~~~~~~~~ + +This section documents internal replication behavior and is not critical to +using or implementing replication. This documentation is provided strictly +for learning and educational purposes. + .. _minio-replication-process: Replication Process -~~~~~~~~~~~~~~~~~~~ ++++++++++++++++++++ MinIO uses a replication queuing system with multiple concurrent replication workers operating on that queue. MinIO continuously works to replicate and @@ -117,176 +305,6 @@ The replication process generally has one of the following flows: - ``PENDING -> COMPLETED`` - ``PENDING -> FAILED -> COMPLETED`` -Synchronous vs Asynchronous Replication -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -MinIO supports specifying either asynchronous (default) or synchronous -replication for a given remote target. - -With the default asynchronous replication, MinIO completes the originating -``PUT`` operation *before* placing the object into a :ref:`replication queue -`. The originating client may therefore see a -successful ``PUT`` operation *before* the object is replicated. While -this may result in stale or missing objects on the remote, it mitigates -the risk of slow write operations due to replication load. - -With synchronous replication, MinIO attempts to replicate the object *prior* to -completing the originating ``PUT`` operation. MinIO returns a successful ``PUT`` -operation whether or not the replication attempts succeeds. While this may -result in more reliable synchronization between the source and remote target, -it may also increase the time of each write operation due to replication load. - -You must explicitly enable synchronous replication when configuring the remote -target target using the :mc-cmd:`mc admin bucket remote add` command with the -:mc-cmd-option:`~mc admin bucket remote add sync` flag. - -.. _minio-replication-behavior-resync: - -Resynchronization -~~~~~~~~~~~~~~~~~ - -The :mc-cmd:`mc replicate resync` command can reset replication synchronization -for a remote target. The resynchronization process attempts to queue all -objects on the source bucket regardless of their -:ref:`replication status `. Resynchronization -primarily supports recovery after partial or total loss of a remote -replication target. - -Initiating resynchronization on a bucket uses the same core -:ref:`replication scanner and queue system ` -for detecting and synchronizing objects. MinIO does not prioritize objects -in the target bucket, nor does it empty or otherwise modify the queue to -favor objects in the target bucket. MinIO skips synchronizing those -objects whose remote copy exactly match the source, including object -metadata. - -:mc-cmd:`mc replicate resync` operates at the bucket level and does -*not* support prefix-level granularity. Initiating resynchronization on a large -bucket may result in a significant increase in replication-related load -and traffic. Use this command with caution and only when necessary. - -Replication Behavior -~~~~~~~~~~~~~~~~~~~~ - -.. _minio-replication-behavior-delete: - -Replication of Delete Operations -++++++++++++++++++++++++++++++++ - -MinIO supports replicating delete operations, where MinIO synchronizes -deleting specific object versions *and* new :s3-docs:`delete markers -`. Delete operation replication uses -the same :ref:`replication process ` as all other -replication operations. - -MinIO only replicates explicit client-driven delete operations. MinIO does *not* -replicate objects deleted due to :ref:`lifecycle management expiration rules -`. - -For delete marker replication, MinIO begins the replication process after -a delete operation creates the delete marker. MinIO uses the -``X-Minio-Replication-DeleteMarker-Status`` metadata field for tracking -delete marker replication status. In -:ref:`active-active ` -replication configurations, MinIO may produce duplicate delete markers if -both clusters concurrently create a delete marker for an object *or* -if one or both clusters were down before the replication event synchronized. - -For replicating the deletion of a specific object version, MinIO marks the -object version as ``PENDING`` until replication completes. Once the remote -target deletes that object version, MinIO deletes the object on the source. -While this process ensures near-synchronized version deletion, it may result -in listing operations returning the object version after the initial -delete operation. MinIO uses the ``X-Minio-Replication-Delete-Status`` for -tracking delete version replication status. - -MinIO requires explicitly enabling versioned deletes and delete marker -replication . Use the :mc-cmd-option:`mc replicate add replicate` field to -specify both or either ``delete`` and ``delete-marker`` to enable versioned -deletes and delete marker replication respectively. To enable both, specify both -strings using a comma separator ``delete,delete-marker``. - -.. admonition:: MinIO Trims Empty Object Prefixes on Source and Remote Bucket - :class: note, dropdown - - If a delete operation removes the last object in a bucket prefix, MinIO - recursively removes each empty part of the prefix up to the bucket root. - MinIO only applies the recursive removal to prefixes created *implicitly* as - part of object write operations - that is, the prefix was not created using - an explicit directory creation command such as :mc:`mc mb`. - - If a replication rule enables replication delete operations, the replication - process *also* applies the implicit prefix trimming behavior on the - destination MinIO cluster. - - For example, consider a bucket ``photos`` with the following object prefixes: - - - ``photos/2021/january/myphoto.jpg`` - - ``photos/2021/february/myotherphoto.jpg`` - - ``photos/NYE21/NewYears.jpg`` - - ``photos/NYE21`` is the *only* prefix explicitly created using :mc:`mc mb`. - All other prefixes were *implicitly* created as part of writing the object - located at that prefix. - - - A command removes ``myphoto.jpg``. MinIO automatically trims the empty - ``/janaury`` prefix. - - - A command then removes the ``myotherphoto.jpg``. MinIO automatically - trims the ``/february`` prefix *and* the now-empty ``/2021`` prefix. - - - A command removes the ``NewYears.jpg`` object. MinIO leaves the - ``/NYE21`` prefix remains in place since it was *explicitly* created. - -.. _minio-replication-behavior-existing-objects: - -Replication of Existing Objects -+++++++++++++++++++++++++++++++ - -MinIO by default does not enable existing object replication. Objects -created before replication was configured *or* while replication is -disabled are not synchronized to the target deployment. -Starting with :mc:`mc` :minio-git:`RELEASE.2021-06-13T17-48-22Z -` and :mc:`minio` -:minio-git:`RELEASE.2021-06-07T21-40-51Z -`, MinIO supports enabling -replication of existing objects in a bucket. - -Enabling existing object replication marks all objects or object prefixes that -satisfy the replication rules as eligible for synchronization to the source -cluster, *even if* those objects were created prior to configuring or enabling -replication. You can enable existing object replication while configuring -or modifying a replication rule: - -- For new replication rules, include ``"existing-objects"`` to the list of - replication features specified to :mc-cmd-option:`mc replicate add replicate`. - -- For existing replication rules, add ``"existing-objects"`` to the list of - existing replication features using - :mc-cmd-option:`mc replicate edit replicate`. You must specify *all* desired - replication features when editing the replication rule. - -Enabling existing object replication does not increase the priority of objects -pending replication. MinIO uses the same core -:ref:`replication scanner and queue system ` for -detecting and synchronizing objects regardless of the enabled replication -feature. The time required to fully synchronize a bucket depends on a number of -factors, including but not limited to the current cluster replication load, -overall cluster load, and the size of the namespace (all objects in the bucket). - -MinIO does not synchronize existing unversioned objects. Specifically, the -bucket *must* have :ref:`versioning ` enabled when the -object was created. You can use the :mc-cmd:`mc cp` command to create a -"versioned" copy of that object. Once that object replicates successfully, -you can delete the unversioned object (versionid = ``null``). - -MinIO existing object replication -implements functionality similar to -`AWS: Replicating existing objects between S3 buckets -`__ -without the overhead of contacting technical support. - - .. toctree:: :hidden: :titlesonly: