mirror of
https://github.com/minio/docs.git
synced 2025-07-27 08:41:57 +03:00
DOCS-412: Add replication resynchronization procedure
Includes refinements to other replication procedures and single sourcing
This commit is contained in:
ravindk89
Ravind Kumar
@ -8,7 +8,7 @@ Enable Multi-Site Server-Side Bucket Replication
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 2
|
||||
:depth: 1
|
||||
|
||||
|
||||
The procedure on this page configures automatic server-side bucket
|
||||
@ -61,38 +61,6 @@ MinIO multi-site replication requires MinIO server
|
||||
Requirements
|
||||
------------
|
||||
|
||||
Replication Requires MinIO Remote Targets
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO server-side replication only works between MinIO deployments. All
|
||||
deployments participating in the multi-site replication configuration
|
||||
*must* run MinIO. MinIO strongly recommends using the *same* MinIO server
|
||||
version across all sites.
|
||||
|
||||
To configure replication between arbitrary S3-compatible services,
|
||||
use :mc-cmd:`mc mirror`.
|
||||
|
||||
Replication Requires Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO relies on the immutability protections provided by versioning to
|
||||
synchronize objects as part of replication.
|
||||
|
||||
Use the :mc-cmd:`mc version suspend` command to enable versioning for the bucket
|
||||
across *all* MinIO deployments participating in the multi-site replication
|
||||
configuration.
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc version ALIAS/PATH
|
||||
|
||||
- Replace :mc-cmd:`ALIAS <mc version ALIAS>` with the
|
||||
:mc:`alias <mc alias>` of the MinIO deployment.
|
||||
|
||||
- Replace :mc-cmd:`PATH <mc version ALIAS>` with the bucket on which
|
||||
to enable versioning.
|
||||
|
||||
Install and Configure ``mc`` with Access to Both Clusters.
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
@ -142,11 +110,6 @@ destination deployments to configure and enable replication rules.
|
||||
source cluster. You can restrict the user policy to specific buckets
|
||||
as-needed.
|
||||
|
||||
Use the :mc-cmd:`mc admin policy add` to add this policy to *both*
|
||||
deployments. You can then create a user on both deployments using
|
||||
:mc-cmd:`mc admin user add` and associate the policy to those users
|
||||
with :mc-cmd:`mc admin policy set`.
|
||||
|
||||
.. tab-item:: Replication Remote User
|
||||
|
||||
The following policy provides permissions for enabling synchronization of
|
||||
@ -170,15 +133,41 @@ destination deployments to configure and enable replication rules.
|
||||
buckets as an element in the ``Resource`` array similar to
|
||||
``"arn:aws:s3:::bucketName/*"``.
|
||||
|
||||
Use the :mc-cmd:`mc admin policy add` to add this policy to *both*
|
||||
deployments. You can then create a user on both deployments using
|
||||
:mc-cmd:`mc admin user add` and associate the policy to those users
|
||||
with :mc-cmd:`mc admin policy set`.
|
||||
Use the reference policies above to create the necessary policies using :mc-cmd:`mc admin policy add`. After creating the necessary policy, MinIO strongly recommends creating a dedicated user or service account for supporting bucket replication operations.
|
||||
|
||||
MinIO strongly recommends creating users specifically for supporting
|
||||
bucket replication operations. See
|
||||
:mc:`mc admin user` and :mc:`mc admin policy` for more complete
|
||||
documentation on adding users and policies to a MinIO cluster.
|
||||
- MinIO deployments configured for :ref:`Active Directory/LDAP <minio-external-identity-management-ad-ldap>` or :ref:`OpenID Connect <minio-external-identity-management-openid>` user management can create a dedicated :ref:`service account <minio-idp-service-account>` for bucket replication.
|
||||
|
||||
- MinIO deployments configured for :ref:`MinIO <minio-users>` user management can create either a dedicated user *or* service account for bucket replication.
|
||||
|
||||
See :mc:`mc admin user`, :mc:`mc admin user svcacct`, and :mc:`mc admin policy` for more complete documentation on adding users, service accounts, and policies to a MinIO cluster.
|
||||
|
||||
Replication Requires Matching Object Encryption Settings
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-encrypted-objects
|
||||
:end-before: end-replication-encrypted-objects
|
||||
|
||||
Replication Requires MinIO Deployments
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-minio-only
|
||||
:end-before: end-replication-minio-only
|
||||
|
||||
Replication Requires Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-versioning
|
||||
:end-before: end-replication-requires-versioning
|
||||
|
||||
Replication Requires Matching Object Locking State
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-object-locking
|
||||
:end-before: end-replication-requires-object-locking
|
||||
|
||||
Considerations
|
||||
--------------
|
||||
@ -201,10 +190,7 @@ behaviors to ensure consistent and predictable synchronization of objects.
|
||||
Replication of Existing Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Starting with :mc:`mc` :minio-git:`RELEASE.2021-06-13T17-48-22Z
|
||||
<mc/releases/tag/RELEASE.2021-06-13T17-48-22Z>` and :mc:`minio`
|
||||
:minio-git:`RELEASE.2021-06-07T21-40-51Z
|
||||
<minio/releases/tag/RELEASE.2021-06-07T21-40-51Z>`, MinIO supports automatically
|
||||
MinIO supports automatically
|
||||
replicating existing objects in a bucket.
|
||||
|
||||
MinIO requires explicitly enabling replication of existing objects using the
|
||||
@ -238,37 +224,6 @@ MinIO does *not* replicate delete operations resulting from the application of
|
||||
for the bucket on all replication sites to ensure consistent application of
|
||||
object expiration.
|
||||
|
||||
Replication of Encrypted Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO supports replicating objects encrypted with automatic
|
||||
Server-Side Encryption (SSE-S3). Both the source and destination buckets
|
||||
*must* have automatic SSE-S3 enabled for MinIO to replicate an encrypted object.
|
||||
|
||||
As part of the replication process, MinIO *decrypts* the object on the source
|
||||
bucket and transmits the unencrypted object. The destination MinIO cluster then
|
||||
re-encrypts the object using the destination bucket SSE-S3 configuration. MinIO
|
||||
*strongly recommends* :ref:`enabling TLS <minio-TLS>` on both source and
|
||||
destination deployments to ensure the safety of objects during transmission.
|
||||
|
||||
MinIO does *not* support replicating client-side encrypted objects
|
||||
(SSE-C).
|
||||
|
||||
Replication of Locked Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO supports replicating objects held under
|
||||
:ref:`WORM Locking <minio-object-locking>`. Both replication buckets *must* have
|
||||
object locking enabled for MinIO to replicate the locked object. For
|
||||
active-active configuration, MinIO recommends using the *same*
|
||||
retention rules on both buckets to ensure consistent behavior across
|
||||
sites.
|
||||
|
||||
You must enable object locking during bucket creation as per S3 behavior.
|
||||
You can then configure object retention rules at any time.
|
||||
Object locking requires :ref:`versioning <minio-bucket-versioning>` and
|
||||
enables the feature implicitly.
|
||||
|
||||
Procedure
|
||||
---------
|
||||
|
||||
|
||||
@ -8,7 +8,7 @@ Enable One-Way Server-Side Bucket Replication
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 2
|
||||
:depth: 1
|
||||
|
||||
|
||||
The procedure on this page creates a new bucket replication rule for
|
||||
@ -45,35 +45,6 @@ one-way synchronization of objects between MinIO buckets.
|
||||
Requirements
|
||||
------------
|
||||
|
||||
MinIO Server-Side Replication Requires a MinIO Cluster as the Destination
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO server-side replication only works between MinIO clusters. Both the
|
||||
source and destination clusters *must* run MinIO.
|
||||
|
||||
To configure replication between arbitrary S3-compatible services,
|
||||
use :mc-cmd:`mc mirror`.
|
||||
|
||||
Enable Versioning on Source and Destination Buckets
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO relies on the immutability protections provided by versioning to
|
||||
synchronize objects between the source and replication target.
|
||||
|
||||
Use the :mc-cmd:`mc version suspend` command to enable versioning on
|
||||
*both* the source and destination bucket before starting this procedure:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc version ALIAS/PATH
|
||||
|
||||
- Replace :mc-cmd:`ALIAS <mc version ALIAS>` with the
|
||||
:mc:`alias <mc alias>` of the MinIO cluster.
|
||||
|
||||
- Replace :mc-cmd:`PATH <mc version ALIAS>` with the bucket on which
|
||||
to enable versioning.
|
||||
|
||||
Install and Configure ``mc`` with Access to Both Clusters.
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
@ -123,11 +94,6 @@ source and destination clusters:
|
||||
source cluster. You can restrict the user policy to specific buckets
|
||||
as-needed.
|
||||
|
||||
Use the :mc-cmd:`mc admin policy add` to add this policy to the
|
||||
source cluster. Use :mc-cmd:`mc admin user add` to create a user
|
||||
on the source cluster and :mc-cmd:`mc admin policy set` to associate
|
||||
the policy to that new user.
|
||||
|
||||
.. tab-item:: Replication Remote User
|
||||
|
||||
The following policy provides permissions for enabling synchronization of
|
||||
@ -149,16 +115,42 @@ source and destination clusters:
|
||||
cluster. To restrict the policy to specific buckets, specify those
|
||||
buckets as an element in the ``Resource`` array similar to
|
||||
``"arn:aws:s3:::bucketName/*"``.
|
||||
|
||||
Use the :mc-cmd:`mc admin policy add` to add this policy to the
|
||||
destination cluster. Use :mc-cmd:`mc admin user add` to create a user
|
||||
on the destination cluster and :mc-cmd:`mc admin policy set` to associate
|
||||
the policy to that new user.
|
||||
|
||||
MinIO strongly recommends creating users specifically for supporting
|
||||
bucket replication operations. See
|
||||
:mc:`mc admin user` and :mc:`mc admin policy` for more complete
|
||||
documentation on adding users and policies to a MinIO cluster.
|
||||
Use the reference policies above to create the necessary policies using :mc-cmd:`mc admin policy add`. After creating the necessary policy, MinIO strongly recommends creating a dedicated user or service account for supporting bucket replication operations.
|
||||
|
||||
- MinIO deployments configured for :ref:`Active Directory/LDAP <minio-external-identity-management-ad-ldap>` or :ref:`OpenID Connect <minio-external-identity-management-openid>` user management can create a dedicated :ref:`service account <minio-idp-service-account>` for bucket replication.
|
||||
|
||||
- MinIO deployments configured for :ref:`MinIO <minio-users>` user management can create either a dedicated user *or* service account for bucket replication.
|
||||
|
||||
See :mc:`mc admin user`, :mc:`mc admin user svcacct`, and :mc:`mc admin policy` for more complete documentation on adding users, service accounts, and policies to a MinIO cluster.
|
||||
|
||||
Replication Requires Matching Object Encryption Settings
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-encrypted-objects
|
||||
:end-before: end-replication-encrypted-objects
|
||||
|
||||
Replication Requires MinIO Deployments
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-minio-only
|
||||
:end-before: end-replication-minio-only
|
||||
|
||||
Replication Requires Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-versioning
|
||||
:end-before: end-replication-requires-versioning
|
||||
|
||||
Replication Requires Matching Object Locking State
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-object-locking
|
||||
:end-before: end-replication-requires-object-locking
|
||||
|
||||
Considerations
|
||||
--------------
|
||||
@ -166,11 +158,7 @@ Considerations
|
||||
Replication of Existing Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Starting with :mc:`mc` :minio-git:`RELEASE.2021-06-13T17-48-22Z
|
||||
<mc/releases/tag/RELEASE.2021-06-13T17-48-22Z>` and :mc:`minio`
|
||||
:minio-git:`RELEASE.2021-06-07T21-40-51Z
|
||||
<minio/releases/tag/RELEASE.2021-06-07T21-40-51Z>`, MinIO supports automatically
|
||||
replicating existing objects in a bucket.
|
||||
MinIO supports automatically replicating existing objects in a bucket.
|
||||
|
||||
MinIO requires explicitly enabling replication of existing objects using the
|
||||
:mc-cmd:`mc replicate add --replicate` or
|
||||
@ -203,35 +191,6 @@ application of :ref:`lifecycle management expiration rules
|
||||
|
||||
See :ref:`minio-replication-behavior-delete` for more complete documentation.
|
||||
|
||||
Replication of Encrypted Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO supports replicating objects encrypted with automatic
|
||||
Server-Side Encryption (SSE-S3). Both the source and destination buckets
|
||||
*must* have automatic SSE-S3 enabled for MinIO to replicate an encrypted object.
|
||||
|
||||
As part of the replication process, MinIO *decrypts* the object on the source
|
||||
bucket and transmits the unencrypted object. The destination MinIO cluster then
|
||||
re-encrypts the object using the destination bucket SSE-S3 configuration. MinIO
|
||||
*strongly recommends* :ref:`enabling TLS <minio-TLS>` on both source and
|
||||
destination clusters to ensure the safety of objects during transmission.
|
||||
|
||||
MinIO does *not* support replicating client-side encrypted objects
|
||||
(SSE-C).
|
||||
|
||||
Replication of Locked Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO supports replicating objects held under
|
||||
:ref:`WORM Locking <minio-object-locking>`. Both the source and destination
|
||||
buckets *must* have object locking enabled for MinIO to replicate the locked
|
||||
object.
|
||||
|
||||
You must enable object locking during bucket creation as per S3 behavior.
|
||||
You can then configure object retention rules at any time.
|
||||
Object locking requires :ref:`versioning <minio-bucket-versioning>` and
|
||||
enables the feature implicitly.
|
||||
|
||||
Multi-Site Replication
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
@ -8,7 +8,7 @@ Enable Two-Way Server-Side Bucket Replication
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 2
|
||||
:depth: 1
|
||||
|
||||
|
||||
The procedure on this page creates a new bucket replication rule for two-way
|
||||
@ -50,35 +50,6 @@ or more MinIO clusters, see :ref:`minio-bucket-replication-serverside-multi`
|
||||
Requirements
|
||||
------------
|
||||
|
||||
Replication Requires MinIO Remote Targets
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO server-side replication only works between MinIO clusters. Both the
|
||||
source and destination clusters *must* run MinIO.
|
||||
|
||||
To configure replication between arbitrary S3-compatible services,
|
||||
use :mc-cmd:`mc mirror`.
|
||||
|
||||
Replication Requires Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO relies on the immutability protections provided by versioning to
|
||||
synchronize objects between the source and replication target.
|
||||
|
||||
Use the :mc-cmd:`mc version enable` command to enable versioning on
|
||||
*both* the source and destination bucket before starting this procedure:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc version ALIAS/PATH
|
||||
|
||||
- Replace :mc-cmd:`ALIAS <mc version ALIAS>` with the
|
||||
:mc:`alias <mc alias>` of the MinIO cluster.
|
||||
|
||||
- Replace :mc-cmd:`PATH <mc version ALIAS>` with the bucket on which
|
||||
to enable versioning.
|
||||
|
||||
Install and Configure ``mc`` with Access to Both Clusters.
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
@ -128,11 +99,6 @@ destination clusters to configure and enable replication rules.
|
||||
source cluster. You can restrict the user policy to specific buckets
|
||||
as-needed.
|
||||
|
||||
Use the :mc-cmd:`mc admin policy add` to add this policy to *both*
|
||||
clusters. You can then create a user on both clusters using
|
||||
:mc-cmd:`mc admin user add` and associate the policy to those users
|
||||
with :mc-cmd:`mc admin policy set`.
|
||||
|
||||
.. tab-item:: Replication Remote User
|
||||
|
||||
The following policy provides permissions for enabling synchronization of
|
||||
@ -156,15 +122,41 @@ destination clusters to configure and enable replication rules.
|
||||
buckets as an element in the ``Resource`` array similar to
|
||||
``"arn:aws:s3:::bucketName/*"``.
|
||||
|
||||
Use the :mc-cmd:`mc admin policy add` to add this policy to *both*
|
||||
clusters. You can then create a user on both clusters using
|
||||
:mc-cmd:`mc admin user add` and associate the policy to those users
|
||||
with :mc-cmd:`mc admin policy set`.
|
||||
Use the reference policies above to create the necessary policies using :mc-cmd:`mc admin policy add`. After creating the necessary policy, MinIO strongly recommends creating a dedicated user or service account for supporting bucket replication operations.
|
||||
|
||||
MinIO strongly recommends creating users specifically for supporting
|
||||
bucket replication operations. See
|
||||
:mc:`mc admin user` and :mc:`mc admin policy` for more complete
|
||||
documentation on adding users and policies to a MinIO cluster.
|
||||
- MinIO deployments configured for :ref:`Active Directory/LDAP <minio-external-identity-management-ad-ldap>` or :ref:`OpenID Connect <minio-external-identity-management-openid>` user management can create a dedicated :ref:`service account <minio-idp-service-account>` for bucket replication.
|
||||
|
||||
- MinIO deployments configured for :ref:`MinIO <minio-users>` user management can create either a dedicated user *or* service account for bucket replication.
|
||||
|
||||
See :mc:`mc admin user`, :mc:`mc admin user svcacct`, and :mc:`mc admin policy` for more complete documentation on adding users, service accounts, and policies to a MinIO cluster.
|
||||
|
||||
Replication Requires Matching Object Encryption Settings
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-encrypted-objects
|
||||
:end-before: end-replication-encrypted-objects
|
||||
|
||||
Replication Requires MinIO Deployments
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-minio-only
|
||||
:end-before: end-replication-minio-only
|
||||
|
||||
Replication Requires Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-versioning
|
||||
:end-before: end-replication-requires-versioning
|
||||
|
||||
Replication Requires Matching Object Locking State
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-object-locking
|
||||
:end-before: end-replication-requires-object-locking
|
||||
|
||||
Considerations
|
||||
--------------
|
||||
@ -188,10 +180,7 @@ objects.
|
||||
Replication of Existing Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Starting with :mc:`mc` :minio-git:`RELEASE.2021-06-13T17-48-22Z
|
||||
<mc/releases/tag/RELEASE.2021-06-13T17-48-22Z>` and :mc:`minio`
|
||||
:minio-git:`RELEASE.2021-06-07T21-40-51Z
|
||||
<minio/releases/tag/RELEASE.2021-06-07T21-40-51Z>`, MinIO supports automatically
|
||||
MinIO supports automatically
|
||||
replicating existing objects in a bucket.
|
||||
|
||||
MinIO requires explicitly enabling replication of existing objects using the
|
||||
@ -227,37 +216,6 @@ of object expiration.
|
||||
|
||||
See :ref:`minio-replication-behavior-delete` for more complete documentation.
|
||||
|
||||
Replication of Encrypted Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO supports replicating objects encrypted with automatic
|
||||
Server-Side Encryption (SSE-S3). Both the source and destination buckets
|
||||
*must* have automatic SSE-S3 enabled for MinIO to replicate an encrypted object.
|
||||
|
||||
As part of the replication process, MinIO *decrypts* the object on the source
|
||||
bucket and transmits the unencrypted object. The destination MinIO cluster then
|
||||
re-encrypts the object using the destination bucket SSE-S3 configuration. MinIO
|
||||
*strongly recommends* :ref:`enabling TLS <minio-TLS>` on both source and
|
||||
destination clusters to ensure the safety of objects during transmission.
|
||||
|
||||
MinIO does *not* support replicating client-side encrypted objects
|
||||
(SSE-C).
|
||||
|
||||
Replication of Locked Objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
MinIO supports replicating objects held under
|
||||
:ref:`WORM Locking <minio-object-locking>`. Both replication buckets *must* have
|
||||
object locking enabled for MinIO to replicate the locked object. For
|
||||
active-active configuration, MinIO recommends using the *same*
|
||||
retention rules on both buckets to ensure consistent behavior across
|
||||
sites.
|
||||
|
||||
You must enable object locking during bucket creation as per S3 behavior.
|
||||
You can then configure object retention rules at any time.
|
||||
Object locking requires :ref:`versioning <minio-bucket-versioning>` and
|
||||
enables the feature implicitly.
|
||||
|
||||
Multi-Site Replication
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
@ -312,7 +312,7 @@ The replication process generally has one of the following flows:
|
||||
/replication/enable-server-side-one-way-bucket-replication
|
||||
/replication/enable-server-side-two-way-bucket-replication
|
||||
/replication/enable-server-side-multi-site-bucket-replication
|
||||
|
||||
/replication/server-side-replication-resynchronize-remote
|
||||
|
||||
.. _minio-bucket-replication-clientside:
|
||||
|
||||
|
||||
@ -0,0 +1,200 @@
|
||||
.. _minio-bucket-replication-resynchronize:
|
||||
|
||||
|
||||
========================================
|
||||
Resynchronize Bucket from Remote Replica
|
||||
========================================
|
||||
|
||||
.. default-domain:: minio
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
The procedure on this page resynchronizes the contents of a MinIO
|
||||
bucket using a healthy replication remote. Resynchronization supports
|
||||
recovery after partial or total loss of data on a MinIO deployment in a
|
||||
replica configuration.
|
||||
|
||||
For example, consider a MinIO active-active replication configuration similar
|
||||
to the following:
|
||||
|
||||
.. image:: /images/replication/active-active-twoway-replication.svg
|
||||
:width: 600px
|
||||
:alt: Active-Active Replication synchronizes data between two remote deployments.
|
||||
:align: center
|
||||
|
||||
Resynchronization allows using the healthy data on one of the participating
|
||||
MinIO deployments as the source for rebuilding the other deployment.
|
||||
|
||||
Resynchronization is a per-bucket process. You must repeat resynchronization
|
||||
for each bucket on the remote which suffered partial or total data loss.
|
||||
|
||||
.. admonition:: Professional Support during BC/DR Operations
|
||||
:class: important
|
||||
|
||||
`MinIO SUBNET <https://min.io/pricing?jmp=docs>`__ users can `log in <https://subnet.min.io/>`__ and create a new issue related to resynchronization. Coordination with MinIO Engineering via SUBNET can ensure successful resynchronization and restoration of normal operations, including performance testing and health diagnostics.
|
||||
|
||||
Community users can seek support on the `MinIO Community Slack <https://minio.slack.com>`__. Community Support is best-effort only and has no SLAs around responsiveness.
|
||||
|
||||
.. _minio-bucket-replication-serverside-resynchronize-requirements:
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
||||
MinIO Deployments Must Be Online
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Resynchronization requires both the source and remote deployments be online and
|
||||
able to accept read and write operations. The source *must* have
|
||||
complete network connectivity to the remote.
|
||||
|
||||
The remote deployment may be "unhealthy" in that it has suffered partial or
|
||||
total data loss. Resynchronization addresses the data loss as long as both
|
||||
source and destination maintain connectivity.
|
||||
|
||||
Resynchronization Requires Existing Replication Configuration
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Resynchronization requires the healthy source deployment have an existing replication configuration for the unhealthy target bucket. Additionally, resynchronization only applies to those replication rules created with the :ref:`existing object replication <minio-replication-behavior-existing-objects>` option.
|
||||
|
||||
- Use :mc-cmd:`mc admin bucket remote ls` to review the configured remote targets on the healthy source bucket.
|
||||
- Use :mc-cmd:`mc replicate ls` to review the configured replication rules on the healthy source bucket.
|
||||
|
||||
Replication Requires Matching Object Encryption Settings
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-encrypted-objects
|
||||
:end-before: end-replication-encrypted-objects
|
||||
|
||||
Replication Requires MinIO Deployments
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-minio-only
|
||||
:end-before: end-replication-minio-only
|
||||
|
||||
Replication Requires Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-versioning
|
||||
:end-before: end-replication-requires-versioning
|
||||
|
||||
Replication Requires Matching Object Locking State
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-object-locking
|
||||
:end-before: end-replication-requires-object-locking
|
||||
|
||||
Considerations
|
||||
--------------
|
||||
|
||||
Resynchronization Requires Time
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Resynchronization is a background processes that continually checks objects in
|
||||
the source MinIO bucket and copies them to the remote as-needed. The time
|
||||
required for replication to complete may vary depending on the number and size
|
||||
of objects, the throughput to the remote MinIO deployment, and the load on the
|
||||
source MinIO deployment. Total time for completion is generally not predictable
|
||||
due to these variables.
|
||||
|
||||
MinIO recommends configuring load balancers or proxies to direct traffic only
|
||||
to the healthy cluster until synchronization completes. The following commands
|
||||
can provide insight into the resynchronization status:
|
||||
|
||||
- :mc-cmd:`mc replicate status` on the source and remote to track total
|
||||
replicated data.
|
||||
|
||||
- Run ``mc ls -r --versions ALIAS/BUCKET | wc -l`` against both source and
|
||||
remote to validate the total number of objects and object versions on each.
|
||||
|
||||
Resynchronize Objects after Data Loss
|
||||
-------------------------------------
|
||||
|
||||
This procedure uses an existing :ref:`MinIO replication configuration <minio-bucket-replication-serverside>` to restore missing data to one of the MinIO deployments participating in that configuration. Specifically, a healthy MinIO deployment (the ``SOURCE``) synchronizes it's existing data to the unhealthy MinIO deployment (the ``TARGET``).
|
||||
|
||||
This procedure assumes an existing :ref:`alias <alias>` for the ``SOURCE`` that has the :ref:`necessary permissions <minio-bucket-replication-serverside-twoway-permissions>` for configuring replication.
|
||||
|
||||
You can repeat this procedure for each bucket that requires resynchronization. You can have no more than one replication job running per bucket.
|
||||
|
||||
1) List the Configured Replication Targets on the Healthy Source
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Run the :mc-cmd:`mc admin bucket remote ls` command to list the configured remote targets on the healthy ``SOURCE`` deployment for the ``BUCKET`` that requires resynchronization.
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc admin bucket remote ls SOURCE/BUCKET
|
||||
|
||||
- Replace ``SOURCE`` with the :ref:`alias <alias>` of the source MinIO deployment.
|
||||
|
||||
- Replace ``BUCKET`` with the name of the bucket to use as the source for resynchronization.
|
||||
|
||||
The output resembles the following:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
Remote URL Source->Target ARN SYNC PROXY
|
||||
https://minio-2.example.net:9000 data ->data arn:minio:replication::UUID:BUCKET proxy
|
||||
|
||||
|
||||
Identify the ARN associated to the ``BUCKET`` on the unhealthy ``TARGET`` MinIO deployment for use in the next step.
|
||||
|
||||
2) Start the Resynchronization Procedure
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Run the :mc-cmd:`mc replicate resync start` command to begin the resynchronization process:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc replicate resync start --remote-bucket "arn:minio:replication::UUID:BUCKET" SOURCE/BUCKET
|
||||
|
||||
- Replace the ``--remote-bucket`` value with the ARN of the unhealthy ``BUCKET`` on the ``TARGET`` MinIO deployment.
|
||||
|
||||
- Replaced ``SOURCE`` with the :ref:`alias <alias>` of the source MinIO deployment.
|
||||
|
||||
- Replace the ``BUCKET`` with the name of the bucket on the healthy ``SOURCE`` MinIO
|
||||
deployment.
|
||||
|
||||
The command returns a resynchronization job ID indicating that the process has
|
||||
begun.
|
||||
|
||||
3) Monitor Resynchronization
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Use the :mc-cmd:`mc replicate resync status` command on the source deployment to track the received replication data:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
mc replicate resync status ALIAS/BUCKET
|
||||
|
||||
The output resembles the following:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
mc replicate resync status /data
|
||||
Resync status summary:
|
||||
● arn:minio:replication::6593d572-4dc3-4bb9-8d90-7f79cc612f01:data
|
||||
Status: Ongoing
|
||||
Replication Status | Size (Bytes) | Count
|
||||
Replicated | 2.3 GiB | 18
|
||||
Failed | 0 B | 0
|
||||
|
||||
The :guilabel:`Status` updates to ``Completed`` once the resynchronization
|
||||
process completes.
|
||||
|
||||
4) Next Steps
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
- If the ``TARGET`` bucket damage extends to replication rules, you must recreate those rules to match the previous replication configuration. See :ref:`minio-bucket-replication-serverside-twoway` for additional guidance.
|
||||
|
||||
- Perform basic validation that all buckets in the replication configuration show similar results for commands such as :mc-cmd:`mc ls` and :mc-cmd:`mc stat`.
|
||||
|
||||
- After restoring any replication rules and verifying replication between sites, you can configure the reverse proxy, load balancer, or other network control plane managing connections to resume sending traffic to the resynchronized deployment.
|
||||
Reference in New Issue
Block a user