minio/docs
1
0
Fork 0
mirror of https://github.com/minio/docs.git synced 2025-07-24 10:22:32 +03:00
Code Activity

DOCS-330: Continuing Clean Up Work

Browse Source 
CR updates
This commit is contained in:
ravindk89
2021-04-12 16:55:22 -04:00
committed by Ravind Kumar
parent d5d39def55
commit ae77ac4740
6 changed files with 324 additions and 148 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
source/concepts/feature-overview.rst
View File

@ -34,6 +34,12 @@ The following table lists MinIO features and their corresponding documentation:
coding provides object-level healing with less overhead than adjacent
technologies such as RAID or replication.
* - :ref:`Server-Side Bucket Replication <minio-bucket-replication-serverside>`
- MinIO Server-Side Bucket Replication is an automatic bucket-level
configuration that synchronizes objects between MinIO clusters.
MinIO supports one-way "active-passive" and two-way "active-active"
configurations.
.. toctree::
:titlesonly:
:hidden:

2
source/images/active-active-replication.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 22 KiB

2
source/images/active-passive-replication.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

309
source/replication/enable-server-side-one-way-bucket-replication.rst
View File

@ -11,32 +11,34 @@ Enable One-Way Server-Side Bucket Replication
:depth: 2
MinIO Server-Side Bucket Replication supports automatically synchronizing
objects between a source and destination bucket. The destination bucket
can be on the same MinIO cluster as the source *or* an independent remote
MinIO cluster.
The procedure on this page creates a new bucket replication rule for
one-way synchronization of objects from a source MinIO cluster to a destination
MinIO cluster.
one-way synchronization of objects between MinIO buckets.
- Use the :mc-cmd:`mc replicate edit` command to modify an existing
replication rule.
.. image:: /images/active-active-replication.svg
:width: 600px
:alt: Active-Active Replication synchronizes data between two remote clusters.
:align: center
- Use the :mc-cmd-option:`mc replicate edit` command with the
:mc-cmd-option:`--state "disable" <mc replicate edit state>` flag to
disable an existing replication rule.
MinIO server-side replication supports at most *two* MinIO clusters. Both
clusters *must* run MinIO.
- Use the :mc-cmd:`mc replicate rm` command to remove an existing replication
rule.
- To configure replication between arbitrary S3-compatible services, use
:mc-cmd:`mc mirror`.
- To configure two-way "active-active" replication between MinIO clusters,
see :ref:`minio-bucket-replication-serverside-twoway`.
.. todo: Diagram
.. seealso::
.. todo
This procedure specifically enables only one-way replication between the
source and destination buckets. For a procedure on two-way "active-active"
replication, see <tutorial>.
- Use the :mc-cmd:`mc replicate edit` command to modify an existing
replication rule.
- Use the :mc-cmd-option:`mc replicate edit` command with the
:mc-cmd-option:`--state "disable" <mc replicate edit state>` flag to
disable an existing replication rule.
- Use the :mc-cmd:`mc replicate rm` command to remove an existing replication
rule.
.. _minio-bucket-replication-serverside-oneway-requirements:
@ -72,13 +74,26 @@ Use the :mc-cmd:`mc version enable` command to enable versioning on
- Replace :mc-cmd:`PATH <mc version enable TARGET>` with the bucket on which
to enable versioning.
``mc`` Command Line Interface
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Install and Configure ``mc`` with Access to Both Clusters.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This procedure uses :mc:`mc` for performing operations on both the source
and destination MinIO cluster. See the ``mc``
:ref:`Installation Quickstart <mc-install>` for instructions on downloading
and installing ``mc``.
This procedure uses :mc:`mc` for performing operations on both the source and
destination MinIO cluster. Install :mc:`mc` on a machine with network access to
both source and destination clusters. See the ``mc``
:ref:`Installation Quickstart <mc-install>` for instructions on downloading and
installing ``mc``.
Use the :mc:`mc alias` command to create an alias for both MinIO clusters.
Alias creation requires specifying an access key for a user on the cluster.
This user **must** have permission to create and manage users and policies
on the cluster. Specifically, ensure the user has *at minimum*:
- :policy-action:`admin:CreateUser`
- :policy-action:`admin:ListUsers`
- :policy-action:`admin:GetUser`
- :policy-action:`admin:CreatePolicy`
- :policy-action:`admin:GetPolicy`
- :policy-action:`admin:AttachUserOrGroupPolicy`
.. _minio-bucket-replication-serverside-oneway-permissions:
@ -90,10 +105,10 @@ source and destination clusters:
.. tabs::
.. tab:: Source Policy
.. tab:: Replication Admin
The source cluster *must* have a user with *at minimum* following attached
*or* inherited policy:
The following policy provides permissions for configuring and enabling
replication on a cluster.
.. code-block:: shell
:class: copyable
@ -107,7 +122,7 @@ source and destination clusters:
"admin:GetBucketTarget"
],
"Effect": "Allow",
"Sid": ""
"Sid": "EnableRemoteBucketConfiguration"
},
{
"Effect": "Allow",
@ -116,25 +131,37 @@ source and destination clusters:
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:GetBucketLocation",
"s3:GetBucketVersioning"
"s3:GetBucketVersioning",
"s3:GetObjectRetention",
"s3:GetObjectLegalHold",
"s3:PutReplicationConfiguration"
],
"Resource": [
"arn:aws:s3:::SOURCEBUCKETNAME"
]
"arn:aws:s3:::*"
],
"Sid": "EnableReplicationRuleConfiguration"
}
]
}
Replace ``SOURCEBUCKETNAME`` with the name of the source bucket from which
MinIO replicates objects.
- The ``"EnableRemoteBucketConfiguration"`` statement grants permission
for creating a remote target for supporting replication.
Use the :mc-cmd:`mc admin policy set` command to associate the policy to
a user on the source MinIO cluster.
- The ``"EnableReplicationRuleConfiguration"`` statement grants permission
for creating replication rules on a bucket. The ``"arn:aws:s3:::*``
resource applies the replication permissions to *any* bucket on the
source cluster. You can restrict the user policy to specific buckets
as-needed.
.. tab:: Destination Policy
Use the :mc-cmd:`mc admin policy add` to add this policy to the
source cluster. Use :mc-cmd:`mc admin user add` to create a user
on the source cluster and :mc-cmd:`mc admin policy set` to associate
the policy to that new user.
The destination cluster *must* have a user with *at minimum* the
following attached *or* inherited policy:
.. tab:: Replication Remote User
The following policy provides permissions for enabling synchronization of
replicated data *into* the cluster.
.. code-block:: shell
:class: copyable
@ -150,11 +177,13 @@ source and destination clusters:
"s3:ListBucketMultipartUploads",
"s3:GetBucketLocation",
"s3:GetBucketVersioning",
"s3:GetBucketObjectLockConfiguration"
"s3:GetBucketObjectLockConfiguration",
"s3:GetEncryptionConfiguration"
],
"Resource": [
"arn:aws:s3:::DESTINATIONBUCKETNAME"
]
"arn:aws:s3:::*"
],
"Sid": "EnableReplicationOnBucket"
},
{
"Effect": "Allow",
@ -166,23 +195,39 @@ source and destination clusters:
"s3:GetObjectVersion",
"s3:GetObjectVersionTagging",
"s3:PutObject",
"s3:PutObjectRetention",
"s3:PutBucketObjectLockConfiguration",
"s3:PutObjectLegalHold",
"s3:DeleteObject",
"s3:ReplicateObject",
"s3:ReplicateDelete"
],
"Resource": [
"arn:aws:s3:::DESTINATIONBUCKETNAME/*"
]
"arn:aws:s3:::*"
],
"Sid": "EnableReplicatingDataIntoBucket"
}
]
}
Replace ``DESTINATIONBUCKETNAME`` with the name of the target bucket to
which MinIO replicates objects.
- The ``"EnableReplicationOnBucket"`` statement grants permission for
a remote target to retrieve bucket-level configuration for supporting
replication operations on *all* buckets in the MinIO cluster. To
restrict the policy to specific buckets, specify those buckets as an
element in the ``Resource`` array similar to
``"arn:aws:s3:::bucketName"``.
Use the :mc-cmd:`mc admin policy set` command to associate the policy
to a user on the target MinIO cluster.
- The ``"EnableReplicatingDataIntoBucket"`` statement grants permission
for a remote target to synchronize data into *any* bucket in the MinIO
cluster. To restrict the policy to specific buckets, specify those
buckets as an element in the ``Resource`` array similar to
``"arn:aws:s3:::bucketName/*"``.
Use the :mc-cmd:`mc admin policy add` to add this policy to the
destination cluster. Use :mc-cmd:`mc admin user add` to create a user
on the destination cluster and :mc-cmd:`mc admin policy set` to associate
the policy to that new user.
MinIO strongly recommends creating users specifically for supporting
bucket replication operations. See
:mc:`mc admin user` and :mc:`mc admin policy` for more complete
@ -199,6 +244,14 @@ does *not* apply replication rules to existing objects in the bucket. Use
:mc:`mc cp` or :mc:`mc mirror` to migrate existing objects to the destination
bucket.
For buckets with active write operations during the procedure, any objects
written *before* configuring bucket replication remain unreplicated.
Consider scheduling a maintenance period during which applications stop
all write operations to the bucket or buckets for which you are configuring
bucket replication. Restart write operations at the completion of the
procedure to ensure consistent object replication.
Replication of Delete Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -237,51 +290,66 @@ MinIO does *not* support replicating client-side encrypted objects
Procedure
---------
1) Configure ``mc`` Access to the Source and Destination Cluster
1) Configure User Accounts and Policies for Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This step creates users and policies on both MinIO clusters for
supporting replication operations. You can skip this step if both
clusters already have users with the necessary
:ref:`permissions <minio-bucket-replication-serverside-oneway-permissions>`.
The following examples use ``Alpha`` and ``Baker`` as placeholder :mc:`aliases
<mc alias>` for each MinIO cluster. You should replace these values with the
appropriate aliases for the MinIO clusters on which you are configuring bucket
replication. These examples assume that the specified aliases have
the necessary permissions for creating policies and users on both clusters. See
:ref:`minio-users` and :ref:`minio-policy` for more complete documentation on
MinIO users and policies respectively.
A\) Create Replication Administrator
The following code creates a user and policy for supporting configuring
replication on the ``Alpha`` cluster. Replace the
password ``LongRandomSecretKey`` with a long, random, and secure secret key
as per your organizations best practices for password generation.
.. code-block:: shell
:class: copyable
wget -O - https://docs.min.io/minio/baremetal/examples/ReplicationAdminPolicy.json | \
mc admin policy add Alpha ReplicationAdminPolicy /dev/stdin
mc admin user add Alpha alphaReplicationAdmin LongRandomSecretKey
mc admin policy set Alpha ReplicationAdminPolicy user=alphaReplicationAdmin
B\) Create Remote Replication User
The following code creates a user and policy for supporting synchronizing
data into the ``Baker`` cluster. Replace the password
``LongRandomSecretKey`` with a long, random, and secure secret key as per
your organizations best practices for password generation.
.. code-block:: shell
:class: copyable
wget -O - https://docs.min.io/minio/baremetal/examples/ReplicationRemoteUserPolicy.json | \
mc admin policy add Baker ReplicationRemoteUserPolicy /dev/stdin
mc admin user add Baker bakerReplicationRemoteUser LongRandomSecretKey
mc admin policy set Baker ReplicationRemoteUserPolicy user=bakerReplicationRemoteUser
2) Configure ``mc`` Access to the Source and Destination Cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc alias set` command to add an alias for both source
and destination MinIO clusters.
.. tabs::
Use the :mc-cmd:`mc alias set` command to add a replication-specific alias for
both remote clusters:
.. tab: Source Cluster
.. code-block:: shell
:class: copyable
.. code-block:: shell
:class: copyable
mc admin set AlphaReplication HOSTNAME AlphaReplicationAdmin LongRandomSecretKey
mc admin set BakerReplication HOSTNAME BakerReplicationUser LongRandomSecretKey
mc alias set SourceCluster HOSTNAME ACCESSKEY SECRETKEY
- Replace :mc-cmd:`~mc alias set HOSTNAME` with the hostname or IP address
of a node in the source MinIO cluster. For distributed MinIO clusters
using a load balancer, specify the hostname or IP address of that load
balancer.
- Replace :mc-cmd:`~mc alias set ACCESSKEY` and
:mc-cmd:`~mc alias set SECRETKEY` with the access and secret key for a
user with the :ref:`required permissions
<minio-bucket-replication-serverside-oneway-permissions>` on the source
MinIO cluster.
.. tab:: DestinationCluster
.. code-block:: shell
:class: copyable
mc alias set DestinationCluster HOSTNAME ACCESSKEY SECRETKEY
- Replace :mc-cmd:`~mc alias set HOSTNAME` with the hostname or IP address
of a node in the destination MinIO cluster. For distributed MinIO
clusters using a load balancer, specify the hostname or IP address of
that load balancer.
- Replace :mc-cmd:`~mc alias set ACCESSKEY` and
:mc-cmd:`~mc alias set SECRETKEY` with the access and secret key for a
user with the :ref:`required permissions
<minio-bucket-replication-serverside-oneway-permissions>` on the
destination MinIO cluster.
2) Create a Replication Target for the Destination Cluster
3) Create a Replication Target for the Destination Cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc admin bucket remote` command to create a replication target
@ -292,35 +360,24 @@ bucket.
.. code-block:: shell
:class: copyable
mc admin bucket remote add SourceCluster/SOURCEBUCKET \
https://ACCESSKEY:SECRETKEY@HOSTNAME/DESTINATIONBUCKET
mc admin bucket remote add AlphaReplication/SOURCEBUCKET \
https://bakerReplicationRemoteUser:LongRandomSecretKey@HOSTNAME/DESTINATIONBUCKET
--service "replication"
[--sync]
- Replace ``SOURCEBUCKET`` with the name of the bucket on the source cluster.
The name of the bucket *must* match the bucket name specified to the source
user's replication :ref:`policy
<minio-bucket-replication-serverside-oneway-permissions>`.
- Replace ``SOURCEBUCKET`` with the name of the source bucket on the
``Alpha`` cluster.
- Replace ``ACCESSKEY`` and ``SECRETKEY`` with the access and secret key for
a user with the :ref:`required permissions
<minio-bucket-replication-serverside-oneway-permissions>` on the destination
MinIO cluster.
- Replace ``HOSTNAME`` with the URL of the ``Baker`` cluster.
- Replace the ``HOSTNAME`` with the hostname or IP address of
a node in the MinIO cluster. For distributed MinIO clusters using a
load balancer, specify the hostname or IP address of that load balancer.
- Replace ``DESTINATIONBUCKET`` with the name of the target bucket on the
``Baker`` cluster.
- Replace the ``DESTINATIONBUCKET`` with the name of the bucket on the
destination cluster. The name of the bucket *must* match the bucket name
specified to the destination user's replication :ref:`policy
<minio-bucket-replication-serverside-oneway-permissions>`.
- MinIO defaults to using asynchronous object replication, where MinIO
replicates objects *after* returning the PUT object response. Specify the
:mc-cmd-option:`~mc admin bucket remote add sync` option to enable
synchronous replication, where MinIO attempts to replicate the object
*prior* to returning the PUT object response.
- Specify the :mc-cmd-option:`~mc admin bucket remote add sync` option to
enable synchronous replication. Omit the option to use the default of
asynchronous replication. See the reference documentation for
:mc-cmd-option:`~mc admin bucket remote add sync` for more information
on synchronous vs asynchronous replication.
The command returns an ARN similar to the following:
@ -330,7 +387,7 @@ The command returns an ARN similar to the following:
Copy the ARN string for use in the next step.
3) Create a New Bucket Replication Rule
4) Create a New Bucket Replication Rule
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc-cmd:`mc replicate add` command to add the new server-side
@ -339,29 +396,35 @@ replication rule to the source MinIO cluster.
.. code-block:: shell
:class: copyable
mc replicate add SourceCluster/SOURCEBUCKET \
mc replicate add AlphaReplication/SOURCEBUCKET \
--remote-bucket DESTINATIONBUCKET \
--arn 'arn:minio:replication::<UUID>:DESTINATIONBUCKET' \
--replicate "delete,delete-marker"
- Replace ``SOURCEBUCKET`` with the name of the bucket on the source cluster.
- Replace ``SOURCEBUCKET`` with the name of the bucket from which Alpha
replicates data. The name *must* match the bucket specified when
creating the remote target in the previous step.
- Replace the ``DESTINATIONBUCKET`` with the name of the bucket on the
destination cluster.
- Replace the ``DESTINATIONBUCKET`` with the name of the ``Baker`` bucket to
which Alpha replicates data. The name *must* match the bucket specified
when creating the remote target in the previous step.
- Replace the ``--arn`` value with the ARN returned in the previous step.
- Replace ``PRIORITY`` with the integer priority of this replication rule.
- Replace the ``--arn`` value with the ARN returned in the previous step.
Ensure you specify the ARN created on the ``Alpha`` cluster. You can use
:mc-cmd:`mc admin bucket remote ls` to list all remote ARNs configured
on the cluster.
- The ``--replicate "delete,delete-marker"`` flag enables replicating delete
markers and deletion of object versions. See
:mc-cmd-option:`mc replicate add replicate` for more complete
documentation. Omit these fields to disable replication of delete
operations.
- The ``--replicate "delete,delete-marker"`` flag enables replicating delete
markers and deletion of object versions. See
:mc-cmd-option:`mc replicate add replicate` for more complete documentation.
Omit this field to disable replication of delete operations.
Specify any other supported optional arguments for
:mc-cmd:`mc replicate add`.
4) Validate the Replication Configuration
5) Validate the Replication Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use :mc-cmd:`mc cp` to copy a new object to the source bucket.
@ -369,14 +432,14 @@ Use :mc-cmd:`mc cp` to copy a new object to the source bucket.
.. code-block:: shell
:class: copyable
mc cp ~/foo.txt SourceCluster/SOURCEBUCKET
mc cp ~/foo.txt Alpha/SOURCEBUCKET
Use :mc-cmd:`mc ls` to verify the object exists on the destination bucket:
.. code-block:: shell
:class: copyable
mc ls DestinationCluster/DESTINATIONBUCKET
mc ls Baker/DESTINATIONBUCKET
If the remote target was configured *without* the
:mc-cmd-option:`~mc admin bucket remote add sync` option, the destination

44
source/replication/enable-server-side-two-way-bucket-replication.rst
View File

@ -14,11 +14,13 @@ Enable Two-Way Server-Side Bucket Replication
The procedure on this page creates a new bucket replication rule for two-way
"active-active" synchronization of objects between MinIO buckets.
.. todo: Diagram
.. image:: /images/active-active-replication.svg
:width: 600px
:alt: Active-Active Replication synchronizes data between two remote clusters.
:align: center
MinIO server-side replication supports active-active replication between
at most *two* MinIO clusters. MinIO does not support arbitrary S3-compatible
services.
MinIO server-side replication supports at most *two* MinIO clusters. Both
clusters *must* run MinIO.
- To configure replication between arbitrary S3-compatible services, use
:mc-cmd:`mc mirror`.
@ -26,6 +28,13 @@ services.
- To configure one-way "active-passive" replication between MinIO clusters,
see :ref:`minio-bucket-replication-serverside-oneway`.
MinIO Active-Active replication is designed for synchronizing objects between
two MinIO clusters. MinIO does not support Active-Active replication between
more than two clusters (multi-site). Enterprises looking to implement multi-site
replication should consider leveraging `MinIO SUBNET
<https://min.io/pricing?ref=docs>`__ support to access the expertise, planning,
and engineering resources required for addressing that use case.
.. seealso::
- Use the :mc-cmd:`mc replicate edit` command to modify an existing
@ -38,13 +47,6 @@ services.
- Use the :mc-cmd:`mc replicate rm` command to remove an existing replication
rule.
.. todo: Diagram
.. todo
This procedure specifically enables only two-way replication between the
source and destination buckets. For a procedure on two-way "active-active"
replication, see <tutorial>.
.. _minio-bucket-replication-serverside-twoway-requirements:
Requirements
@ -193,6 +195,7 @@ destination clusters to configure and enable replication rules.
"s3:GetObjectVersionTagging",
"s3:PutObject",
"s3:PutObjectRetention",
"s3:PutBucketObjectLockConfiguration",
"s3:PutObjectLegalHold",
"s3:DeleteObject",
"s3:ReplicateObject",
@ -291,8 +294,6 @@ MinIO supports replicating objects with retention settings, such as
source and destination bucket *must* have object locking enabled for MinIO
to replicate objects with their associated retention settings.
Procedure
---------
@ -304,14 +305,13 @@ supporting replication operations. You can skip this step if both
clusters already have users with the necessary
:ref:`permissions <minio-bucket-replication-serverside-twoway-permissions>`.
The following examples that the :mc:`alias <mc alias>` for each cluster
provides the necessary permissions for creating policies and users on both
clusters. See :ref:`minio-users` and :ref:`minio-policy` for more complete
documentation on MinIO users and policies respectively.
Replace ``Alpha`` and ``Baker`` in the following examples with the appropriate
:mc:`aliases <mc alias>` for the MinIO clusters on which you are configuring
bucket replication.
The following examples use ``Alpha`` and ``Baker`` as placeholder :mc:`aliases
<mc alias>` for each MinIO cluster. You should replace these values with the
appropriate aliases for the MinIO clusters on which you are configuring bucket
replication. These examples assume that the specified aliases have
the necessary permissions for creating policies and users on both clusters. See
:ref:`minio-users` and :ref:`minio-policy` for more complete documentation on
MinIO users and policies respectively.
A\) Create Replication Administrators
The following code creates policies and users for supporting configuring
@ -332,7 +332,7 @@ A\) Create Replication Administrators
mc admin user add Baker bakerReplicationAdmin LongRandomSecretKey
mc admin policy set baker ReplicationAdminPolicy user=bakerReplicationAdmin
B\) Create Remote Replication Administrators
B\) Create Remote Replication User
The following code creates policies and users for supporting synchronizing data
to the ``Alpha`` and ``Baker`` clusters. Replace the password
``LongRandomSecretKey`` with a long, random, and secure secret key as per your

109
source/replication/replication-overview.rst
View File

@ -8,7 +8,7 @@ Bucket Replication
.. contents:: Table of Contents
:local:
:depth: 1
:depth: 2
MinIO supports server-side and client-side replication of objects between source
and destination buckets. MinIO offers both active-passive (one-way) and
@ -54,8 +54,6 @@ replication while adding the following MinIO-only features:
- Source and destination bucket names can match, supporting site-to-site
use cases such as Splunk or Veeam BC/DR.
- Replicate object locking and retention settings from source to destination.
- Simplified implementation than S3 bucket replication configuration, removing
the need to configure settings like AccessControlTranslation, Metrics, and
SourceSelectionCriteria.
@ -63,6 +61,111 @@ replication while adding the following MinIO-only features:
- Active-Active (Two-Way) replication of objects between source and destination
buckets.
.. _minio-replication-process:
Replication Process
~~~~~~~~~~~~~~~~~~~
MinIO uses a replication queuing system with multiple concurrent replication
workers operating on that queue. MinIO continuously works to replicate and
remove objects from the queue while scanning for new unreplicated objects to
add to the queue.
MinIO sets the ``X-Amz-Replication-Status`` metadata field according to the
replication state of the object:
.. list-table::
:header-rows: 1
:width: 100%
:widths: 30 70
* - Replication State
- Description
* - ``PENDING``
- The object has not yet been replicated. MinIO applies this state
if the object meets one of the configured replication rules on the
bucket. MinIO continuously scans for ``PENDING`` objects not yet in the
replication queue and adds them to the queue as space is available.
* - ``COMPLETED``
- The object has successfully replicated to the remote cluster.
* - ``FAILED``
- The object failed to replicate to the remote cluster.
MinIO continuously scans for ``FAILED`` objects not yet in the
replication queue and adds them to the queue as space is available.
* - ``REPLICA``
- The object is itself a replica from a remote source.
MinIO ignores ``REPLICA`` objects, including any metadata-only changes
to that object. MinIO does not support
AWS `replica modification sync
<https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-s3-replication-adds-support-two-way-replication/>`__
at this time.
The replication process generally has one of the following flows:
- ``PENDING -> COMPLETED``
- ``PENDING -> FAILED -> COMPLETED``
Synchronous vs Asynchronous Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO supports specifying either asynchronous (default) or synchronous
replication for a given remote target.
With the default asynchronous replication, MinIO completes the originating
``PUT`` operation *before* placing the object into a :ref:`replication queue
<minio-replication-process>`. The originating client may therefore see a
successful ``PUT`` operation *before* the object is replicated. While
this may result in stale or missing objects on the remote, it mitigates
the risk of slow write operations due to replication load.
With synchronous replication, MinIO attempts to replicate the object *prior* to
completing the originating ``PUT`` operation. MinIO returns a successful ``PUT``
operation whether or not the replication attempts succeeds. While this may
result in more reliable synchronization between the source and remote target,
it may also increase the time of each write operation due to replication load.
You must explicitly enable synchronous replication when configuring the remote
target target using the :mc-cmd:`mc admin bucket remote add` command with the
:mc-cmd-option:`~mc admin bucket remote add sync` flag.
Replication of Delete Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO supports replicating delete operations, where MinIO synchronizes
deleting specific object versions *and* new :s3-docs:`delete markers
<delete-marker-replication.html>`. Delete operation replication uses
the same :ref:`replication process <minio-replication-process>` as all other
replication operations.
For delete marker replication, MinIO begins the replication process after
a delete operation creates the delete marker. MinIO uses the
``X-Minio-Replication-DeleteMarker-Status`` metadata field for tracking
delete marker replication status. In
:ref:`active-active <minio-bucket-replication-serverside-twoway>`
replication configurations, MinIO may produce duplicate delete markers if
both clusters concurrently create a delete marker for an object *or*
if one or both clusters were down before the replication event synchronized.
For replicating the deletion of a specific object version, MinIO marks the
object version as ``PENDING`` until replication completes. Once the remote
target deletes that object version, MinIO deletes the object on the source.
While this process ensures near-synchronized version deletion, it may result
in listing operations returning the object version after the initial
delete operation. MinIO uses the ``X-Minio-Replication-Delete-Status`` for
tracking delete version replication status.
MinIO requires explicitly enabling versioned deletes and delete marker
replication . Use the :mc-cmd-option:`mc replicate add replicate` field to
specify both or either ``delete`` and ``delete-marker`` to enable versioned
deletes and delete marker replication respectively. To enable both, specify both
strings using a comma separator ``delete,delete-marker``.
.. toctree::
:hidden:
:titlesonly: