mirror of
https://github.com/minio/docs.git
synced 2025-09-02 00:02:14 +03:00
2f8c5a94f6
Changes to the docs related to mc [RELEASE.2022-11-07T23-47-39Z](https://github.com/minio/mc/releases/tag/RELEASE.2022-11-07T23-47-39Z) - Replaces `mc replicate edit` with `mc replicate update` - Adds `mc admin idp oidc | ldap` commands - Update to `mc support inspect` format - Adds `mc admin rebalance` command - Updates mc replicate edit references throughout docs to point to mc replicate update. - Adds a rebalance pools section to the expand deployment doc. - Clarifies precedence for environment variables vs config settings. Closes #636 Closes #632
424 lines
15 KiB
ReStructuredText
424 lines
15 KiB
ReStructuredText
.. _minio-mc-replicate-add:
|
|
|
|
====================
|
|
``mc replicate add``
|
|
====================
|
|
|
|
.. default-domain:: minio
|
|
|
|
.. contents:: Table of Contents
|
|
:local:
|
|
:depth: 2
|
|
|
|
.. mc:: mc replicate
|
|
.. mc:: mc replicate add
|
|
|
|
Syntax
|
|
------
|
|
|
|
.. start-mc-replicate-add-desc
|
|
|
|
The :mc:`mc replicate add` command creates a new :ref:`server-side replication
|
|
<minio-bucket-replication-serverside>` rule for a bucket on a MinIO deployment.
|
|
|
|
.. end-mc-replicate-add-desc
|
|
|
|
.. note::
|
|
|
|
Where :mc:`mc mirror` only synchronizes the current version of an object, ``mc replicate`` synchronizes all versions, version information, and metadata for the objects.
|
|
|
|
The MinIO deployment automatically begins synchronizing new objects to the remote MinIO deployment after creating the rule.
|
|
You can optionally configure synchronization of existing objects, delete operations, and fully-deleted objects.
|
|
|
|
This command *requires* first configuring the remote bucket target using the
|
|
:mc-cmd:`mc admin bucket remote add` command. You must specify the resulting
|
|
remote ARN as part of running :mc:`mc replicate add`.
|
|
|
|
.. tab-set::
|
|
|
|
.. tab-item:: EXAMPLE
|
|
|
|
The following command adds a new replication rule for the ``mydata``
|
|
bucket on the ``myminio`` MinIO deployment:
|
|
|
|
.. code-block:: shell
|
|
:class: copyable
|
|
|
|
mc replicate add \
|
|
--remote-bucket "arn:minio:replication:aefc8b3a-1f6c-4d7a-86dc-1b0bdffa9100:mydata" \
|
|
--replicate "delete,delete-marker,existing-objects" \
|
|
myminio/mydata
|
|
|
|
The replication rule synchronizes versioned delete operations, delete
|
|
markers, and existing objects to the remote MinIO deployment.
|
|
|
|
.. tab-item:: SYNTAX
|
|
|
|
The command has the following syntax:
|
|
|
|
.. code-block:: shell
|
|
:class: copyable
|
|
|
|
mc [GLOBALFLAGS] replicate add \
|
|
--remote-bucket "string" \
|
|
[--disable] \
|
|
[--id "string"] \
|
|
[--replicate "string"] \
|
|
[--storage-class "string"] \
|
|
[--tags "string"] \
|
|
[--priority int] \
|
|
ALIAS
|
|
|
|
.. include:: /includes/common-minio-mc.rst
|
|
:start-after: start-minio-syntax
|
|
:end-before: end-minio-syntax
|
|
|
|
Parameters
|
|
~~~~~~~~~~
|
|
|
|
.. mc-cmd:: ALIAS
|
|
|
|
*Required* the :ref:`alias <alias>` of the MinIO deployment and full path to
|
|
the bucket or bucket prefix on which to create the replication rule. For
|
|
example:
|
|
|
|
.. code-block:: none
|
|
|
|
mc replicate add --remote-bucket "arn:minio:replica::UUID" play/mybucket
|
|
|
|
|
|
.. mc-cmd:: --remote-bucket
|
|
|
|
|
|
*Required* Specify the ARN for the destination deployment and bucket. You
|
|
can retrieve the ARN using :mc-cmd:`mc admin bucket remote`:
|
|
|
|
- Use the :mc-cmd:`mc admin bucket remote ls` to retrieve a list of
|
|
ARNs for the bucket on the destination deployment.
|
|
|
|
- Use the :mc-cmd:`mc admin bucket remote add` to create a replication ARN
|
|
for the bucket on the destination deployment.
|
|
|
|
.. mc-cmd:: --disable
|
|
|
|
|
|
*Optional* Creates the replication rule in the "disabled" state. MinIO does
|
|
not begin replicating objects using the rule until it is enabled using
|
|
:mc:`mc replicate update`.
|
|
|
|
Objects created while replication is disabled are not
|
|
immediately eligible for replication after enabling the rule.
|
|
You must explicitly enable replication of existing
|
|
objects by including ``"existing-objects"`` to the list of
|
|
replication features specified to
|
|
:mc-cmd:`mc replicate update --replicate`. See
|
|
:ref:`minio-replication-behavior-existing-objects` for more
|
|
information.
|
|
|
|
.. mc-cmd:: --id
|
|
|
|
|
|
*Optional* Specify a unique ID for the replication rule. MinIO automatically
|
|
generates an ID if one is not specified.
|
|
|
|
.. mc-cmd:: --replicate
|
|
|
|
|
|
*Optional* Specify a comma-separated list of the following values to enable
|
|
extended replication features.
|
|
|
|
- ``delete`` - Directs MinIO to replicate DELETE operations to the
|
|
destination bucket.
|
|
|
|
- ``delete-marker`` - Directs MinIO to replicate delete markers to the
|
|
destination bucket.
|
|
|
|
- ``existing-objects`` - Directs MinIO to replicate objects created
|
|
before replication was enabled *or* while replication was suspended.
|
|
|
|
.. mc-cmd:: --storage-class
|
|
|
|
|
|
*Optional*
|
|
|
|
Specify the MinIO :ref:`storage class <minio-ec-storage-class>` to
|
|
apply to replicated objects.
|
|
|
|
.. mc-cmd:: --tags
|
|
|
|
|
|
*Optional* Specify one or more ampersand ``&`` separated key-value pair tags
|
|
which MinIO uses for filtering objects to replicate. For example:
|
|
|
|
.. code-block:: shell
|
|
|
|
mc replicate add --tags "TAG1=VALUE&TAG2=VALUE&TAG3=VALUE" ALIAS
|
|
|
|
MinIO applies the replication rule to any object whose tag set
|
|
contains the specified replication tags.
|
|
|
|
.. mc-cmd:: --priority
|
|
|
|
|
|
*Optional* Specify the integer priority of the replication rule. The value
|
|
*must* be unique among all other rules on the source bucket. Higher values
|
|
imply a *higher* priority than all other rules.
|
|
|
|
The default value is ``0``.
|
|
|
|
Global Flags
|
|
~~~~~~~~~~~~
|
|
|
|
.. include:: /includes/common-minio-mc.rst
|
|
:start-after: start-minio-mc-globals
|
|
:end-before: end-minio-mc-globals
|
|
|
|
Examples
|
|
--------
|
|
|
|
Configure Bucket Replication
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The following :mc:`mc replicate add` command creates a replication
|
|
configuration that synchronizes all new objects, existing objects, delete
|
|
operations, and delete markers to the remote target:
|
|
|
|
.. code-block:: shell
|
|
:class: copyable
|
|
|
|
mc replicate add myminio/mybucket \
|
|
--remote-bucket "arn:minio:replica::UUID" \
|
|
--replicate "delete,delete-marker,existing-objects"
|
|
|
|
- Replace ``myminio/mybucket`` with the :mc-cmd:`~mc replicate add ALIAS` and
|
|
full bucket path for which to create the replication configuration.
|
|
|
|
- Replace the :mc-cmd:`~mc replicate add --remote-bucket` value with the
|
|
ARN of the remote target. Use :mc-cmd:`mc admin bucket remote ls` to list
|
|
all configured remote replication targets.
|
|
|
|
- The :mc-cmd:`~mc replicate add --replicate` flag directs MinIO to
|
|
replicate all delete operations, delete markers, and existing objects to the
|
|
remote. See :ref:`minio-replication-behavior-delete` and
|
|
:ref:`minio-replication-behavior-existing-objects` for more information on
|
|
replication behavior.
|
|
|
|
Configure Bucket Replication for Historical Data Record
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The following :mc:`mc replicate add` command creates a new bucket
|
|
replication configuration that synchronizes all new and existing objects
|
|
to the remote target:
|
|
|
|
.. code-block:: shell
|
|
:class: copyable
|
|
|
|
mc replicate add myminio/mybucket \
|
|
--remote-bucket "arn:minio:replica::UUID" \
|
|
--replicate "existing-objects"
|
|
|
|
- Replace ``myminio/mybucket`` with the :mc-cmd:`~mc replicate add ALIAS` and
|
|
full bucket path for which to create the replication configuration.
|
|
|
|
- Replace the :mc-cmd:`~mc replicate add --remote-bucket` value with the
|
|
ARN of the remote target. Use :mc-cmd:`mc admin bucket remote ls` to list
|
|
all configured remote replication targets.
|
|
|
|
- The :mc-cmd:`~mc replicate add --replicate` flag directs MinIO to
|
|
replicate all existing objects to the remote. See
|
|
:ref:`minio-replication-behavior-existing-objects` for more information on
|
|
replication behavior.
|
|
|
|
The resulting remote copy represents a historical record of objects on the
|
|
remote, where delete operations on the source have no effect on the remote
|
|
copy.
|
|
|
|
Behavior
|
|
--------
|
|
|
|
Server-Side Replication Requires MinIO Source and Destination
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
MinIO server-side replication only works between MinIO deployments. Both the
|
|
source and destination deployments *must* run MinIO.
|
|
|
|
To configure replication between arbitrary S3-compatible services,
|
|
use :mc:`mc mirror`.
|
|
|
|
Enable Versioning on Source and Destination Buckets
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
MinIO relies on the immutability protections provided by versioning to
|
|
synchronize objects between the source and replication target.
|
|
|
|
Use the :mc:`mc version suspend` command to enable versioning on
|
|
*both* the source and destination bucket before starting this procedure:
|
|
|
|
.. code-block:: shell
|
|
:class: copyable
|
|
|
|
mc version ALIAS/PATH
|
|
|
|
- Replace :mc-cmd:`ALIAS <mc version ALIAS>` with the
|
|
:mc:`alias <mc alias>` of the MinIO deployment.
|
|
|
|
- Replace :mc-cmd:`PATH <mc version ALIAS>` with the bucket on which
|
|
to enable versioning.
|
|
|
|
Required Permissions
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
MinIO strongly recommends creating users specifically for supporting
|
|
bucket replication operations. See
|
|
:mc:`mc admin user` and :mc:`mc admin policy` for more complete
|
|
documentation on adding users and policies to a MinIO deployment.
|
|
|
|
.. tab-set::
|
|
|
|
.. tab-item:: Replication Admin
|
|
|
|
The following policy provides permissions for configuring and enabling
|
|
replication on a deployment.
|
|
|
|
.. literalinclude:: /extra/examples/ReplicationAdminPolicy.json
|
|
:class: copyable
|
|
:language: json
|
|
|
|
- The ``"EnableRemoteBucketConfiguration"`` statement grants permission
|
|
for creating a remote target for supporting replication.
|
|
|
|
- The ``"EnableReplicationRuleConfiguration"`` statement grants permission
|
|
for creating replication rules on a bucket. The ``"arn:aws:s3:::*``
|
|
resource applies the replication permissions to *any* bucket on the
|
|
source deployment. You can restrict the user policy to specific buckets
|
|
as-needed.
|
|
|
|
Use the :mc-cmd:`mc admin policy add` to add this policy to each
|
|
deployment acting as a replication source. Use :mc-cmd:`mc admin user add`
|
|
to create a user on the deployment and :mc-cmd:`mc admin policy set`
|
|
to associate the policy to that new user.
|
|
|
|
.. tab-item:: Replication Remote User
|
|
|
|
The following policy provides permissions for enabling synchronization of
|
|
replicated data *into* the deployment.
|
|
|
|
.. literalinclude:: /extra/examples/ReplicationRemoteUserPolicy.json
|
|
:class: copyable
|
|
:language: json
|
|
|
|
- The ``"EnableReplicationOnBucket"`` statement grants permission for
|
|
a remote target to retrieve bucket-level configuration for supporting
|
|
replication operations on *all* buckets in the MinIO deployment. To
|
|
restrict the policy to specific buckets, specify those buckets as an
|
|
element in the ``Resource`` array similar to
|
|
``"arn:aws:s3:::bucketName"``.
|
|
|
|
- The ``"EnableReplicatingDataIntoBucket"`` statement grants permission
|
|
for a remote target to synchronize data into *any* bucket in the MinIO
|
|
deployment. To restrict the policy to specific buckets, specify those
|
|
buckets as an element in the ``Resource`` array similar to
|
|
``"arn:aws:s3:::bucketName/*"``.
|
|
|
|
Use the :mc-cmd:`mc admin policy add` to add this policy to each
|
|
deployment acting as a replication target. Use :mc-cmd:`mc admin user add`
|
|
to create a user on the deployment and :mc-cmd:`mc admin policy set`
|
|
to associate the policy to that new user.
|
|
|
|
Replication of Existing Objects
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Starting with :mc:`mc`
|
|
:minio-git:`RELEASE.2021-06-13T17-48-22Z <mc/releases/tag/RELEASE.2021-06-13T17-48-22Z>`
|
|
and :mc:`minio`
|
|
:minio-git:`RELEASE.2021-06-07T21-40-51Z <minio/releases/tag/RELEASE.2021-06-07T21-40-51Z>`,
|
|
|
|
MinIO supports automatically replicating existing objects in a bucket. MinIO
|
|
existing object replication implements functionality similar to
|
|
`AWS Replicating existing objects between S3 buckets
|
|
<https://aws.amazon.com/blogs/storage/replicating-existing-objects-between-s3-buckets/>`__
|
|
without the overhead of contacting technical support.
|
|
|
|
- To enable replication of existing objects when creating a new replication
|
|
rule, include ``"existing-objects"`` to the list of replication features
|
|
specified to :mc-cmd:`mc replicate add --replicate`.
|
|
|
|
- To enable replication of existing objects for an existing replication rule,
|
|
add ``"existing-objects"`` to the list of existing replication features using
|
|
:mc-cmd:`mc replicate add --replicate`. You must specify *all*
|
|
desired replication features when editing the replication rule.
|
|
|
|
See :ref:`minio-replication-behavior-existing-objects` for more complete
|
|
documentation on this behavior.
|
|
|
|
Synchronization of Metadata Changes
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
MinIO supports :ref:`two-way active-active
|
|
<minio-bucket-replication-serverside-twoway>` replication configurations, where
|
|
MinIO synchronizes new and modified objects between a bucket on two MinIO
|
|
deployments. Starting with :mc:`mc`
|
|
|
|
:minio-git:`RELEASE.2021-05-18T03-39-44Z <mc/releases/tag/RELEASE.2021-05-18T03-39-44Z>`,
|
|
MinIO by default synchronizes metadata-only changes to a replicated object back
|
|
to the "source" deployment. Prior to the this update, MinIO did not support
|
|
synchronizing metadata-only changes to a replicated object.
|
|
|
|
With metadata synchronization enabled, MinIO resets the object
|
|
:ref:`replication status <minio-replication-process>` to indicate
|
|
replication eligibility. Specifically, when an application performs a
|
|
metadata-only update to an object with the ``REPLICA`` status, MinIO marks the
|
|
object as ``PENDING`` and eligible for replication.
|
|
|
|
To disable metadata synchronization, use the
|
|
:mc-cmd:`mc replicate update --replicate` command and omit
|
|
``replica-metadata-sync`` from the replication feature list.
|
|
|
|
Replication of Delete Operations
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
MinIO supports replicating delete operations onto the target bucket.
|
|
Specifically, MinIO can replicate both
|
|
:s3-docs:`Delete Markers <versioning-workflows.html>` *and* the deletion
|
|
of specific versioned objects:
|
|
|
|
- For delete operations on an object, MinIO replication also creates the delete
|
|
marker on the target bucket.
|
|
|
|
- For delete operations on versions of an object,
|
|
MinIO replication also deletes those versions on the target bucket.
|
|
|
|
MinIO does *not* replicate objects deleted due to
|
|
:ref:`lifecycle management expiration rules
|
|
<minio-lifecycle-management-expiration>`. MinIO only replicates explicit
|
|
client-driven delete operations.
|
|
|
|
MinIO requires explicitly enabling replication of delete operations using the
|
|
:mc-cmd:`mc replicate add --replicate` flag. This procedure includes the
|
|
required flags for enabling replication of delete operations and delete markers.
|
|
See :ref:`minio-replication-behavior-delete` for more complete documentation
|
|
on this behavior.
|
|
|
|
Replication of Encrypted Objects
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
MinIO supports replicating objects encrypted with automatic
|
|
Server-Side Encryption (SSE-S3). Both the source and destination buckets
|
|
*must* have automatic SSE-S3 enabled for MinIO to replicate an encrypted object.
|
|
|
|
As part of the replication process, MinIO *decrypts* the object on the source
|
|
bucket and transmits the unencrypted object. The destination MinIO deployment then
|
|
re-encrypts the object using the destination bucket SSE-S3 configuration. MinIO
|
|
*strongly recommends* :ref:`enabling TLS <minio-TLS>` on both source and
|
|
destination deployments to ensure the safety of objects during transmission.
|
|
|
|
MinIO does *not* support replicating client-side encrypted objects
|
|
(SSE-C).
|
|
|
|
S3 Compatibility
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
.. include:: /includes/common-minio-mc.rst
|
|
:start-after: start-minio-mc-s3-compatibility
|
|
:end-before: end-minio-mc-s3-compatibility
|