.. _minio-mc-replicate-add: ==================== ``mc replicate add`` ==================== .. default-domain:: minio .. contents:: Table of Contents :local: :depth: 2 .. mc:: mc replicate .. mc:: mc replicate add Syntax ------ .. start-mc-replicate-add-desc The :mc:`mc replicate add` command creates a new :ref:`server-side replication ` rule for a bucket on a MinIO deployment. .. end-mc-replicate-add-desc The MinIO deployment automatically begins synchronizing new objects to the remote MinIO deployment after creating the rule. You can optionally configure synchronization of existing objects, delete operations, and fully-deleted objects. This command *requires* first configuring the remote bucket target using the :mc-cmd:`mc admin bucket remote add` command. You must specify the resulting remote ARN as part of running :mc-cmd:`mc replicate add`. .. tab-set:: .. tab-item:: EXAMPLE The following command adds a new replication rule for the ``mydata`` bucket on the ``myminio`` MinIO deployment: .. code-block:: shell :class: copyable mc replicate add \ --remote-bucket "arn:minio:replication:aefc8b3a-1f6c-4d7a-86dc-1b0bdffa9100:mydata" \ --replicate "delete,delete-marker,existing-objects" \ myminio/mydata The replication rule synchronizes versioned delete operations, delete markers, and existing objects to the remote MinIO deployment. .. tab-item:: SYNTAX The command has the following syntax: .. code-block:: shell :class: copyable mc [GLOBALFLAGS] replicate add \ --remote-bucket "string" \ [--disable] \ [--id "string"] \ [--replicate "string"] \ [--storage-class "string"] \ [--tags "string"] \ [--priority int] \ ALIAS .. include:: /includes/common-minio-mc.rst :start-after: start-minio-syntax :end-before: end-minio-syntax Parameters ~~~~~~~~~~ .. mc-cmd:: ALIAS *Required* the :ref:`alias ` of the MinIO deployment and full path to the bucket or bucket prefix on which to create the replication rule. For example: .. code-block:: none mc replicate add --remote-bucket "arn:minio:replica::UUID" play/mybucket .. mc-cmd:: --remote-bucket *Required* Specify the ARN for the destination deployment and bucket. You can retrieve the ARN using :mc-cmd:`mc admin bucket remote`: - Use the :mc-cmd:`mc admin bucket remote ls` to retrieve a list of ARNs for the bucket on the destination deployment. - Use the :mc-cmd:`mc admin bucket remote add` to create a replication ARN for the bucket on the destination deployment. .. mc-cmd:: --disable *Optional* Creates the replication rule in the "disabled" state. MinIO does not begin replicating objects using the rule until it is enabled using :mc-cmd:`mc replicate edit`. Objects created while replication is disabled are not immediately eligible for replication after enabling the rule. You must explicitly enable replication of existing objects by including ``"existing-objects"`` to the list of replication features specified to :mc-cmd:`mc replicate edit --replicate`. See :ref:`minio-replication-behavior-existing-objects` for more information. .. mc-cmd:: --id *Optional* Specify a unique ID for the replication rule. MinIO automatically generates an ID if one is not specified. .. mc-cmd:: --replicate *Optional* Specify a comma-separated list of the following values to enable extended replication features. - ``delete`` - Directs MinIO to replicate DELETE operations to the destination bucket. - ``delete-marker`` - Directs MinIO to replicate delete markers to the destination bucket. - ``existing-objects`` - Directs MinIO to replicate objects created before replication was enabled *or* while replication was suspended. .. mc-cmd:: --storage-class *Optional* Specify the MinIO :ref:`storage class ` to apply to replicated objects. .. mc-cmd:: --tags *Optional* Specify one or more ampersand ``&`` separated key-value pair tags which MinIO uses for filtering objects to replicate. For example: .. code-block:: shell mc replicate add --tags "TAG1=VALUE&TAG2=VALUE&TAG3=VALUE" ALIAS MinIO applies the replication rule to any object whose tag set contains the specified replication tags. .. mc-cmd:: --priority *Optional* Specify the integer priority of the replication rule. The value *must* be unique among all other rules on the source bucket. Higher values imply a *higher* priority than all other rules. The default value is ``0``. Global Flags ~~~~~~~~~~~~ .. include:: /includes/common-minio-mc.rst :start-after: start-minio-mc-globals :end-before: end-minio-mc-globals Examples -------- Configure Bucket Replication ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following :mc-cmd:`mc replicate add` command creates a replication configuration that synchronizes all new objects, existing objects, delete operations, and delete markers to the remote target: .. code-block:: shell :class: copyable mc replicate add myminio/mybucket \ --remote-bucket "arn:minio:replica::UUID" \ --replicate "delete,delete-marker,existing-objects" - Replace ``myminio/mybucket`` with the :mc-cmd:`~mc replicate add ALIAS` and full bucket path for which to create the replication configuration. - Replace the :mc-cmd:`~mc replicate add --remote-bucket` value with the ARN of the remote target. Use :mc-cmd:`mc admin bucket remote ls` to list all configured remote replication targets. - The :mc-cmd:`~mc replicate add --replicate` flag directs MinIO to replicate all delete operations, delete markers, and existing objects to the remote. See :ref:`minio-replication-behavior-delete` and :ref:`minio-replication-behavior-existing-objects` for more information on replication behavior. Configure Bucket Replication for Historical Data Record ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following :mc-cmd:`mc replicate add` command creates a new bucket replication configuration that synchronizes all new and existing objects to the remote target: .. code-block:: shell :class: copyable mc replicate add myminio/mybucket \ --remote-bucket "arn:minio:replica::UUID" \ --replicate "existing-objects" - Replace ``myminio/mybucket`` with the :mc-cmd:`~mc replicate add ALIAS` and full bucket path for which to create the replication configuration. - Replace the :mc-cmd:`~mc replicate add --remote-bucket` value with the ARN of the remote target. Use :mc-cmd:`mc admin bucket remote ls` to list all configured remote replication targets. - The :mc-cmd:`~mc replicate add --replicate` flag directs MinIO to replicate all existing objects to the remote. See :ref:`minio-replication-behavior-existing-objects` for more information on replication behavior. The resulting remote copy represents a historical record of objects on the remote, where delete operations on the source have no effect on the remote copy. Behavior -------- Server-Side Replication Requires MinIO Source and Destination ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MinIO server-side replication only works between MinIO deployments. Both the source and destination deployments *must* run MinIO. To configure replication between arbitrary S3-compatible services, use :mc-cmd:`mc mirror`. Enable Versioning on Source and Destination Buckets ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MinIO relies on the immutability protections provided by versioning to synchronize objects between the source and replication target. Use the :mc-cmd:`mc version suspend` command to enable versioning on *both* the source and destination bucket before starting this procedure: .. code-block:: shell :class: copyable mc version ALIAS/PATH - Replace :mc-cmd:`ALIAS ` with the :mc:`alias ` of the MinIO deployment. - Replace :mc-cmd:`PATH ` with the bucket on which to enable versioning. Required Permissions ~~~~~~~~~~~~~~~~~~~~ MinIO strongly recommends creating users specifically for supporting bucket replication operations. See :mc:`mc admin user` and :mc:`mc admin policy` for more complete documentation on adding users and policies to a MinIO deployment. .. tab-set:: .. tab-item:: Replication Admin The following policy provides permissions for configuring and enabling replication on a deployment. .. literalinclude:: /extra/examples/ReplicationAdminPolicy.json :class: copyable :language: json - The ``"EnableRemoteBucketConfiguration"`` statement grants permission for creating a remote target for supporting replication. - The ``"EnableReplicationRuleConfiguration"`` statement grants permission for creating replication rules on a bucket. The ``"arn:aws:s3:::*`` resource applies the replication permissions to *any* bucket on the source deployment. You can restrict the user policy to specific buckets as-needed. Use the :mc-cmd:`mc admin policy add` to add this policy to each deployment acting as a replication source. Use :mc-cmd:`mc admin user add` to create a user on the deployment and :mc-cmd:`mc admin policy set` to associate the policy to that new user. .. tab-item:: Replication Remote User The following policy provides permissions for enabling synchronization of replicated data *into* the deployment. .. literalinclude:: /extra/examples/ReplicationRemoteUserPolicy.json :class: copyable :language: json - The ``"EnableReplicationOnBucket"`` statement grants permission for a remote target to retrieve bucket-level configuration for supporting replication operations on *all* buckets in the MinIO deployment. To restrict the policy to specific buckets, specify those buckets as an element in the ``Resource`` array similar to ``"arn:aws:s3:::bucketName"``. - The ``"EnableReplicatingDataIntoBucket"`` statement grants permission for a remote target to synchronize data into *any* bucket in the MinIO deployment. To restrict the policy to specific buckets, specify those buckets as an element in the ``Resource`` array similar to ``"arn:aws:s3:::bucketName/*"``. Use the :mc-cmd:`mc admin policy add` to add this policy to each deployment acting as a replication target. Use :mc-cmd:`mc admin user add` to create a user on the deployment and :mc-cmd:`mc admin policy set` to associate the policy to that new user. Replication of Existing Objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting with :mc:`mc` :minio-git:`RELEASE.2021-06-13T17-48-22Z ` and :mc:`minio` :minio-git:`RELEASE.2021-06-07T21-40-51Z `, MinIO supports automatically replicating existing objects in a bucket. MinIO existing object replication implements functionality similar to `AWS Replicating existing objects between S3 buckets `__ without the overhead of contacting technical support. - To enable replication of existing objects when creating a new replication rule, include ``"existing-objects"`` to the list of replication features specified to :mc-cmd:`mc replicate add --replicate`. - To enable replication of existing objects for an existing replication rule, add ``"existing-objects"`` to the list of existing replication features using :mc-cmd:`mc replicate add --replicate`. You must specify *all* desired replication features when editing the replication rule. See :ref:`minio-replication-behavior-existing-objects` for more complete documentation on this behavior. Synchronization of Metadata Changes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MinIO supports :ref:`two-way active-active ` replication configurations, where MinIO synchronizes new and modified objects between a bucket on two MinIO deployments. Starting with :mc:`mc` :minio-git:`RELEASE.2021-05-18T03-39-44Z `, MinIO by default synchronizes metadata-only changes to a replicated object back to the "source" deployment. Prior to the this update, MinIO did not support synchronizing metadata-only changes to a replicated object. With metadata synchronization enabled, MinIO resets the object :ref:`replication status ` to indicate replication eligibility. Specifically, when an application performs a metadata-only update to an object with the ``REPLICA`` status, MinIO marks the object as ``PENDING`` and eligible for replication. To disable metadata synchronization, use the :mc-cmd:`mc replicate edit --replicate` command and omit ``replica-metadata-sync`` from the replication feature list. Replication of Delete Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MinIO supports replicating delete operations onto the target bucket. Specifically, MinIO can replicate both :s3-docs:`Delete Markers ` *and* the deletion of specific versioned objects: - For delete operations on an object, MinIO replication also creates the delete marker on the target bucket. - For delete operations on versions of an object, MinIO replication also deletes those versions on the target bucket. MinIO does *not* replicate objects deleted due to :ref:`lifecycle management expiration rules `. MinIO only replicates explicit client-driven delete operations. MinIO requires explicitly enabling replication of delete operations using the :mc-cmd:`mc replicate add --replicate` flag. This procedure includes the required flags for enabling replication of delete operations and delete markers. See :ref:`minio-replication-behavior-delete` for more complete documentation on this behavior. Replication of Encrypted Objects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MinIO supports replicating objects encrypted with automatic Server-Side Encryption (SSE-S3). Both the source and destination buckets *must* have automatic SSE-S3 enabled for MinIO to replicate an encrypted object. As part of the replication process, MinIO *decrypts* the object on the source bucket and transmits the unencrypted object. The destination MinIO deployment then re-encrypts the object using the destination bucket SSE-S3 configuration. MinIO *strongly recommends* :ref:`enabling TLS ` on both source and destination deployments to ensure the safety of objects during transmission. MinIO does *not* support replicating client-side encrypted objects (SSE-C). S3 Compatibility ~~~~~~~~~~~~~~~~ .. include:: /includes/common-minio-mc.rst :start-after: start-minio-mc-s3-compatibility :end-before: end-minio-mc-s3-compatibility