minio/docs
1
0
Fork 0
mirror of https://github.com/minio/docs.git synced 2025-07-30 07:03:26 +03:00
Code Activity

Creating site replication documentation.

Browse Source 
Adding replication overview and site replication overview docs.
Distinguising between bucket replication and site replication.
Adds mc admin replicate docs.
Including tutorials for how to do site replication.
This commit is contained in:
Daryl White
2022-04-12 20:23:53 -05:00
parent 3b5b5e4b46
commit 6a186df4a4
18 changed files with 1290 additions and 293 deletions
Download Patch File Download Diff File Expand all files Collapse all files

313
source/replication/bucket-replication-overview.rst Normal file
View File

@ -0,0 +1,313 @@
.. _minio-bucket-replication:
==================
Bucket Replication
==================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 2
MinIO supports server-side and client-side replication of objects between source
and destination buckets.
:ref:`Server-Side Bucket Replication <minio-bucket-replication-serverside>`
Configure per-bucket rules for automatically synchronizing objects between
buckets within the same MinIO cluster *or* between two independent MinIO
Clusters. MinIO applies rules as part of object write operations
(e.g. ``PUT``) and automatically synchronizes new objects *and* object
mutations, such as new object versions or changes to object metadata.
MinIO server-side bucket replication only supports MinIO clusters for the
remote replication target.
:ref:`Client-side Bucket Replication <minio-bucket-replication-clientside>`
Use The command process to synchronize objects between buckets
within the same S3-compatible cluster *or* between two independent
S3-compatible clusters. Client-side replication using :mc-cmd:`mc mirror`
supports MinIO-to-S3 and similar replication configurations.
.. _minio-bucket-replication-serverside:
Server-Side Bucket Replication
------------------------------
MinIO server-side bucket replication is an automatic bucket-level configuration
that synchronizes objects between a source and destination bucket. MinIO
server-side replication *requires* the source and destination bucket be two separate MinIO
clusters.
For each write operation to the bucket, MinIO checks all configured replication
rules for the bucket and applies the matching rule with highest configured
priority. MinIO synchronizes new objects *and* object mutations, such as
new object versions or changes to object metadata. This includes
metadata operations such as enabling or modifying object locking or
retention settings.
MinIO server-side bucket replication is functionally similar to Amazon S3
replication while adding the following MinIO-only features:
- Source and destination bucket names can match, supporting site-to-site
use cases such as Splunk or Veeam BC/DR.
- Simplified implementation than S3 bucket replication configuration, removing
the need to configure settings like AccessControlTranslation, Metrics, and
SourceSelectionCriteria.
- Active-Active (Two-Way) replication of objects between source and destination
buckets.
- Multi-Site replication of objects between three or more MinIO deployments
(New in :minio-release:`RELEASE.2021-09-23T04-46-24Z`).
.. _minio-replication-behavior-resync:
Resynchronization (Disaster Recovery)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Resynchronization primarily supports recovery after partial or total loss of the
data on a MinIO deployment using a healthy deployment in the replica
configuration. Use the :mc-cmd:`mc replicate resync` command completely
resynchronize the remote target (:mc-cmd:`mc admin bucket remote`) using the
specified source bucket.
The resynchronization process checks all objects in the source bucket against
all configured replication rules that include :ref:`existing object replication
<minio-replication-behavior-existing-objects>`. For each object which matches a
rule, the resynchronization process places the object into the replication
:ref:`queue <minio-replication-process>` regardless of the object's current
:ref:`replication status <minio-replication-process>`.
MinIO skips synchronizing those objects whose remote copy exactly match the
source, including object metadata. MinIO otherwise does not prioritize or modify
the queue with regards to the existing contents of the target.
:mc-cmd:`mc replicate resync` operates at the bucket level and does
*not* support prefix-level granularity. Initiating resynchronization on a large
bucket may result in a significant increase in replication-related load
and traffic. Use this command with caution and only when necessary.
For buckets with :ref:`object transition (Tiering)
<minio-lifecycle-management-tiering>` configured, replication resynchronization
restores objects in a non-transitioned state with no associated transition
metadata. Any data previously transitioned to the remote storage is therefore
permanently disconnected from the remote MinIO deployment. For tiering
configurations which specify an explicit human-readable prefix as part of the
remote configuration, you can safely purge the transitioned data in that prefix
to avoid costs associated to the "lost" data.
.. _minio-replication-behavior-delete:
Replication of Delete Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO supports replicating delete operations, where MinIO synchronizes
deleting specific object versions *and* new
:s3-docs:`delete markers <delete-marker-replication.html>`. Delete operation
replication uses the same :ref:`replication process <minio-replication-process>`
as all other replication operations.
MinIO requires explicitly enabling versioned deletes and delete marker
replication . Use the :mc-cmd:`mc replicate add --replicate` field to
specify both or either ``delete`` and ``delete-marker`` to enable versioned
deletes and delete marker replication respectively. To enable both, specify both
strings using a comma separator ``delete,delete-marker``.
For delete marker replication, MinIO begins the replication process after
a delete operation creates the delete marker. MinIO uses the
``X-Minio-Replication-DeleteMarker-Status`` metadata field for tracking
delete marker replication status. In
:ref:`active-active <minio-bucket-replication-serverside-twoway>`
replication configurations, MinIO may produce duplicate delete markers if
both clusters concurrently create a delete marker for an object *or*
if one or both clusters were down before the replication event synchronized.
For replicating the deletion of a specific object version, MinIO marks the
object version as ``PENDING`` until replication completes. Once the remote
target deletes that object version, MinIO deletes the object on the source.
While this process ensures near-synchronized version deletion, it may result
in listing operations returning the object version after the initial
delete operation. MinIO uses the ``X-Minio-Replication-Delete-Status`` for
tracking delete version replication status.
MinIO only replicates explicit client-driven delete operations. MinIO does *not*
replicate objects deleted from the application of
:ref:`lifecycle management expiration rules
<minio-lifecycle-management-expiration>`. For :ref:`active-active
<minio-bucket-replication-serverside-twoway>` configurations, set the same
expiration rules on *all* of of the replication buckets to ensure consistent
application of object expiration.
.. admonition:: MinIO Trims Empty Object Prefixes on Source and Remote Bucket
:class: note, dropdown
If a delete operation removes the last object in a bucket prefix, MinIO
recursively removes each empty part of the prefix up to the bucket root.
MinIO only applies the recursive removal to prefixes created *implicitly* as
part of object write operations - that is, the prefix was not created using
an explicit directory creation command such as :mc:`mc mb`.
If a replication rule enables replication delete operations, the replication
process *also* applies the implicit prefix trimming behavior on the
destination MinIO cluster.
For example, consider a bucket ``photos`` with the following object prefixes:
- ``photos/2021/january/myphoto.jpg``
- ``photos/2021/february/myotherphoto.jpg``
- ``photos/NYE21/NewYears.jpg``
``photos/NYE21`` is the *only* prefix explicitly created using :mc:`mc mb`.
All other prefixes were *implicitly* created as part of writing the object
located at that prefix.
- A command removes ``myphoto.jpg``. MinIO automatically trims the empty
``/janaury`` prefix.
- A command then removes the ``myotherphoto.jpg``. MinIO automatically
trims the ``/february`` prefix *and* the now-empty ``/2021`` prefix.
- A command removes the ``NewYears.jpg`` object. MinIO leaves the
``/NYE21`` prefix remains in place since it was *explicitly* created.
.. _minio-replication-behavior-existing-objects:
Replication of Existing Objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO by default does not enable existing object replication. Objects
created before replication was configured *or* while replication is
disabled are not synchronized to the target deployment.
Starting with :mc:`mc` :minio-git:`RELEASE.2021-06-13T17-48-22Z
<mc/releases/tag/RELEASE.2021-06-13T17-48-22Z>` and :mc:`minio`
:minio-git:`RELEASE.2021-06-07T21-40-51Z
<minio/releases/tag/RELEASE.2021-06-07T21-40-51Z>`, MinIO supports enabling
replication of existing objects in a bucket.
Enabling existing object replication marks all objects or object prefixes that
satisfy the replication rules as eligible for synchronization to the source
cluster, *even if* those objects were created prior to configuring or enabling
replication. You can enable existing object replication while configuring
or modifying a replication rule:
- For new replication rules, include ``"existing-objects"`` to the list of
replication features specified to :mc-cmd:`mc replicate add --replicate`.
- For existing replication rules, add ``"existing-objects"`` to the list of
existing replication features using
:mc-cmd:`mc replicate edit --replicate`. You must specify *all* desired
replication features when editing the replication rule.
Enabling existing object replication does not increase the priority of objects
pending replication. MinIO uses the same core
:ref:`replication scanner and queue system <minio-replication-process>` for
detecting and synchronizing objects regardless of the enabled replication
feature. The time required to fully synchronize a bucket depends on a number of
factors, including but not limited to the current cluster replication load,
overall cluster load, and the size of the namespace (all objects in the bucket).
If versioning was not previously enabled when configuring bucket replication,
existing objects have a ``versionid = null``. These objects do replicate.
MinIO existing object replication
implements functionality similar to
`AWS: Replicating existing objects between S3 buckets
<https://aws.amazon.com/blogs/storage/replicating-existing-objects-between-s3-buckets/>`__
without the overhead of contacting technical support.
Synchronous vs Asynchronous Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO supports specifying either asynchronous (default) or synchronous
replication for a given remote target.
With the default asynchronous replication, MinIO completes the originating
``PUT`` operation *before* placing the object into a :ref:`replication queue
<minio-replication-process>`. The originating client may therefore see a
successful ``PUT`` operation *before* the object is replicated. While
this may result in stale or missing objects on the remote, it mitigates
the risk of slow write operations due to replication load.
With synchronous replication, MinIO attempts to replicate the object *prior* to
completing the originating ``PUT`` operation. MinIO returns a successful ``PUT``
operation whether or not the replication attempts succeeds. While this may
result in more reliable synchronization between the source and remote target,
it may also increase the time of each write operation due to replication load.
You must explicitly enable synchronous replication when configuring the remote
target target using the :mc-cmd:`mc admin bucket remote add` command with the
:mc-cmd:`~mc admin bucket remote add` flag.
Replication Internals
~~~~~~~~~~~~~~~~~~~~~
This section documents internal replication behavior and is not critical to
using or implementing replication. This documentation is provided strictly
for learning and educational purposes.
.. _minio-replication-process:
Replication Process
+++++++++++++++++++
MinIO uses a replication queuing system with multiple concurrent replication
workers operating on that queue. MinIO continuously works to replicate and
remove objects from the queue while scanning for new unreplicated objects to
add to the queue.
MinIO sets the ``X-Amz-Replication-Status`` metadata field according to the
replication state of the object:
.. list-table::
:header-rows: 1
:width: 100%
:widths: 30 70
* - Replication State
- Description
* - ``PENDING``
- The object has not yet been replicated. MinIO applies this state
if the object meets one of the configured replication rules on the
bucket. MinIO continuously scans for ``PENDING`` objects not yet in the
replication queue and adds them to the queue as space is available.
For multi-site replication, objects remain
in the ``PENDING`` state until replicated to *all* configured
remotes for that bucket or bucket prefix.
* - ``COMPLETED``
- The object has successfully replicated to the remote cluster.
* - ``FAILED``
- The object failed to replicate to the remote cluster.
MinIO continuously scans for ``FAILED`` objects not yet in the
replication queue and adds them to the queue as space is available.
* - ``REPLICA``
- The object is itself a replica from a remote source.
The replication process generally has one of the following flows:
- ``PENDING -> COMPLETED``
- ``PENDING -> FAILED -> COMPLETED``
.. toctree::
:hidden:
:titlesonly:
/replication/enable-server-side-one-way-bucket-replication
/replication/enable-server-side-two-way-bucket-replication
/replication/enable-server-side-multi-site-bucket-replication
/replication/server-side-replication-resynchronize-remote
.. _minio-bucket-replication-clientside:
Client-Side Bucket Replication
------------------------------
The :mc:`mc` command :mc-cmd:`mc mirror` supports watching a source bucket
and automatically replicating objects to a destination bucket.

331
source/replication/replication-overview.rst
View File

@ -1,323 +1,70 @@
.. _minio-bucket-replication:
.. _minio-replication-overview:
==================
Bucket Replication
==================
====================
Replication Overview
====================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 2
:depth: 1
MinIO supports server-side and client-side replication of objects between source
and destination buckets.
Replication duplicates objects from one MinIO location to another MinIO location.
This provides a means for data backup and recovery or for geographic distribution across regions.
:ref:`Server-Side Bucket Replication <minio-bucket-replication-serverside>`
Configure per-bucket rules for automatically synchronizing objects between
buckets within the same MinIO cluster *or* between two independent MinIO
Clusters. MinIO applies rules as part of object write operations
(e.g. ``PUT``) and automatically synchronizes new objects *and* object
mutations, such as new object versions or changes to object metadata.
MinIO server-side bucket replication only supports MinIO clusters for the
remote replication target.
MinIO supports two types of replication:
:ref:`Client-side Bucket Replication <minio-bucket-replication-clientside>`
Use The command process to synchronize objects between buckets
within the same S3-compatible cluster *or* between two independent
S3-compatible clusters. Client-side replication using :mc-cmd:`mc mirror`
supports MinIO-to-S3 and similar replication configurations.
#. :ref:`Bucket Replication <minio-bucket-replication>`
#. :ref:`Site Replication <minio-site-replication-overview>`
.. _minio-bucket-replication-serverside:
You can use one or the other, but not both types of replication.
Server-Side Bucket Replication
------------------------------
.. note::
MinIO server-side bucket replication is an automatic bucket-level configuration
that synchronizes objects between a source and destination bucket. MinIO
server-side replication *requires* the source and destination bucket be MinIO
clusters. The source and destination bucket *may* be the same MinIO cluster *or*
two independent MinIO clusters.
MinIO also provides the :mc-cmd:`mc mirror` command.
``mc mirror`` is similar to bucket replication with the difference that bucket replication is between two MinIO locations.
``mc mirror`` duplicates objects between any two S3-compliant locations.
For each write operation to the bucket, MinIO checks all configured replication
rules for the bucket and applies the matching rule with highest configured
priority. MinIO synchronizes new objects *and* object mutations, such as
new object versions or changes to object metadata. This includes
metadata operations such as enabling or modifying object locking or
retention settings.
Bucket Replication
------------------
MinIO server-side bucket replication is functionally similar to Amazon S3
replication while adding the following MinIO-only features:
Bucket replication duplicates the content of a specific bucket from one MinIO location to another MinIO location.
The target location can be within a MinIO deployment or across deployments.
- Source and destination bucket names can match, supporting site-to-site
use cases such as Splunk or Veeam BC/DR.
Bucket replication can be either ``active-active`` or ``active-passive``.
- Simplified implementation than S3 bucket replication configuration, removing
the need to configure settings like AccessControlTranslation, Metrics, and
SourceSelectionCriteria.
Active-active bucket replication is two-way replication where changes on either bucket duplicate to the other bucket.
- Active-Active (Two-Way) replication of objects between source and destination
buckets.
:ref:`Active-passive bucket replication <minio-bucket-replication-serverside-oneway>` is one-way replication where only changes on the origin bucket are made to the target bucket.
With active-passive bucket repliction, changes made on the target bucket do not replicate to the origin bucket.
- Multi-Site replication of objects between three or more MinIO deployments
(New in :minio-release:`RELEASE.2021-09-23T04-46-24Z`).
.. _minio-replication-behavior-resync:
Site Replication
----------------
Resynchronization (Disaster Recovery)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Site replication expands the features of bucket replication to include IAM, security tokens, service accounts, and bucket features the same across all sites.
Resynchronization primarily supports recovery after partial or total loss of the
data on a MinIO deployment using a healthy deployment in the replica
configuration. Use the :mc-cmd:`mc replicate resync` command completely
resynchronize the remote target (:mc-cmd:`mc admin bucket remote`) using the
specified source bucket.
:ref:`Site replication <minio-site-replication-overview>` links multiple MinIO deployments together and keeps the buckets, objects, and Identify and Access Management (IAM) settings in sync across all connected sites.
The resynchronization process checks all objects in the source bucket against
all configured replication rules that include :ref:`existing object replication
<minio-replication-behavior-existing-objects>`. For each object which matches a
rule, the resynchronization process places the object into the replication
:ref:`queue <minio-replication-process>` regardless of the object's current
:ref:`replication status <minio-replication-process>`.
.. include:: /includes/common-replication.rst
:start-after: start-mc-admin-replicate-what-replicates
:end-before: end-mc-admin-replicate-what-replicates
MinIO skips synchronizing those objects whose remote copy exactly match the
source, including object metadata. MinIO otherwise does not prioritize or modify
the queue with regards to the existing contents of the target.
:mc-cmd:`mc replicate resync` operates at the bucket level and does
*not* support prefix-level granularity. Initiating resynchronization on a large
bucket may result in a significant increase in replication-related load
and traffic. Use this command with caution and only when necessary.
What Does Not Replicate?
~~~~~~~~~~~~~~~~~~~~~~~~
For buckets with :ref:`object transition (Tiering)
<minio-lifecycle-management-tiering>` configured, replication resynchronization
restores objects in a non-transitioned state with no associated transition
metadata. Any data previously transitioned to the remote storage is therefore
permanently disconnected from the remote MinIO deployment. For tiering
configurations which specify an explicit human-readable prefix as part of the
remote configuration, you can safely purge the transitioned data in that prefix
to avoid costs associated to the "lost" data.
Not everything replicates across sites.
.. _minio-replication-behavior-delete:
.. include:: /includes/common-replication.rst
:start-after: start-mc-admin-replicate-what-does-not-replicate
:end-before: end-mc-admin-replicate-what-does-not-replicate
Replication of Delete Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO supports replicating delete operations, where MinIO synchronizes
deleting specific object versions *and* new
:s3-docs:`delete markers <delete-marker-replication.html>`. Delete operation
replication uses the same :ref:`replication process <minio-replication-process>`
as all other replication operations.
MinIO requires explicitly enabling versioned deletes and delete marker
replication . Use the :mc-cmd:`mc replicate add --replicate` field to
specify both or either ``delete`` and ``delete-marker`` to enable versioned
deletes and delete marker replication respectively. To enable both, specify both
strings using a comma separator ``delete,delete-marker``.
For delete marker replication, MinIO begins the replication process after
a delete operation creates the delete marker. MinIO uses the
``X-Minio-Replication-DeleteMarker-Status`` metadata field for tracking
delete marker replication status. In
:ref:`active-active <minio-bucket-replication-serverside-twoway>`
replication configurations, MinIO may produce duplicate delete markers if
both clusters concurrently create a delete marker for an object *or*
if one or both clusters were down before the replication event synchronized.
For replicating the deletion of a specific object version, MinIO marks the
object version as ``PENDING`` until replication completes. Once the remote
target deletes that object version, MinIO deletes the object on the source.
While this process ensures near-synchronized version deletion, it may result
in listing operations returning the object version after the initial
delete operation. MinIO uses the ``X-Minio-Replication-Delete-Status`` for
tracking delete version replication status.
MinIO only replicates explicit client-driven delete operations. MinIO does *not*
replicate objects deleted from the application of
:ref:`lifecycle management expiration rules
<minio-lifecycle-management-expiration>`. For :ref:`active-active
<minio-bucket-replication-serverside-twoway>` configurations, set the same
expiration rules on *all* of of the replication buckets to ensure consistent
application of object expiration.
.. admonition:: MinIO Trims Empty Object Prefixes on Source and Remote Bucket
:class: note, dropdown
If a delete operation removes the last object in a bucket prefix, MinIO
recursively removes each empty part of the prefix up to the bucket root.
MinIO only applies the recursive removal to prefixes created *implicitly* as
part of object write operations - that is, the prefix was not created using
an explicit directory creation command such as :mc:`mc mb`.
If a replication rule enables replication delete operations, the replication
process *also* applies the implicit prefix trimming behavior on the
destination MinIO cluster.
For example, consider a bucket ``photos`` with the following object prefixes:
- ``photos/2021/january/myphoto.jpg``
- ``photos/2021/february/myotherphoto.jpg``
- ``photos/NYE21/NewYears.jpg``
``photos/NYE21`` is the *only* prefix explicitly created using :mc:`mc mb`.
All other prefixes were *implicitly* created as part of writing the object
located at that prefix.
- A command removes ``myphoto.jpg``. MinIO automatically trims the empty
``/janaury`` prefix.
- A command then removes the ``myotherphoto.jpg``. MinIO automatically
trims the ``/february`` prefix *and* the now-empty ``/2021`` prefix.
- A command removes the ``NewYears.jpg`` object. MinIO leaves the
``/NYE21`` prefix remains in place since it was *explicitly* created.
.. _minio-replication-behavior-existing-objects:
Replication of Existing Objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO by default does not enable existing object replication. Objects
created before replication was configured *or* while replication is
disabled are not synchronized to the target deployment.
Starting with :mc:`mc` :minio-git:`RELEASE.2021-06-13T17-48-22Z
<mc/releases/tag/RELEASE.2021-06-13T17-48-22Z>` and :mc:`minio`
:minio-git:`RELEASE.2021-06-07T21-40-51Z
<minio/releases/tag/RELEASE.2021-06-07T21-40-51Z>`, MinIO supports enabling
replication of existing objects in a bucket.
Enabling existing object replication marks all objects or object prefixes that
satisfy the replication rules as eligible for synchronization to the source
cluster, *even if* those objects were created prior to configuring or enabling
replication. You can enable existing object replication while configuring
or modifying a replication rule:
- For new replication rules, include ``"existing-objects"`` to the list of
replication features specified to :mc-cmd:`mc replicate add --replicate`.
- For existing replication rules, add ``"existing-objects"`` to the list of
existing replication features using
:mc-cmd:`mc replicate edit --replicate`. You must specify *all* desired
replication features when editing the replication rule.
Enabling existing object replication does not increase the priority of objects
pending replication. MinIO uses the same core
:ref:`replication scanner and queue system <minio-replication-process>` for
detecting and synchronizing objects regardless of the enabled replication
feature. The time required to fully synchronize a bucket depends on a number of
factors, including but not limited to the current cluster replication load,
overall cluster load, and the size of the namespace (all objects in the bucket).
MinIO does not synchronize existing unversioned objects. Specifically, the
bucket *must* have :ref:`versioning <minio-bucket-versioning>` enabled when the
object was created. You can use The command command to create a
"versioned" copy of that object. Once that object replicates successfully,
you can delete the unversioned object (versionid = ``null``).
MinIO existing object replication
implements functionality similar to
`AWS: Replicating existing objects between S3 buckets
<https://aws.amazon.com/blogs/storage/replicating-existing-objects-between-s3-buckets/>`__
without the overhead of contacting technical support.
Synchronous vs Asynchronous Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO supports specifying either asynchronous (default) or synchronous
replication for a given remote target.
With the default asynchronous replication, MinIO completes the originating
``PUT`` operation *before* placing the object into a :ref:`replication queue
<minio-replication-process>`. The originating client may therefore see a
successful ``PUT`` operation *before* the object is replicated. While
this may result in stale or missing objects on the remote, it mitigates
the risk of slow write operations due to replication load.
With synchronous replication, MinIO attempts to replicate the object *prior* to
completing the originating ``PUT`` operation. MinIO returns a successful ``PUT``
operation whether or not the replication attempts succeeds. While this may
result in more reliable synchronization between the source and remote target,
it may also increase the time of each write operation due to replication load.
You must explicitly enable synchronous replication when configuring the remote
target target using the :mc-cmd:`mc admin bucket remote add` command with the
:mc-cmd:`~mc admin bucket remote add` flag.
Replication Internals
~~~~~~~~~~~~~~~~~~~~~
This section documents internal replication behavior and is not critical to
using or implementing replication. This documentation is provided strictly
for learning and educational purposes.
.. _minio-replication-process:
Replication Process
+++++++++++++++++++
MinIO uses a replication queuing system with multiple concurrent replication
workers operating on that queue. MinIO continuously works to replicate and
remove objects from the queue while scanning for new unreplicated objects to
add to the queue.
MinIO sets the ``X-Amz-Replication-Status`` metadata field according to the
replication state of the object:
.. list-table::
:header-rows: 1
:width: 100%
:widths: 30 70
* - Replication State
- Description
* - ``PENDING``
- The object has not yet been replicated. MinIO applies this state
if the object meets one of the configured replication rules on the
bucket. MinIO continuously scans for ``PENDING`` objects not yet in the
replication queue and adds them to the queue as space is available.
For multi-site replication, objects remain
in the ``PENDING`` state until replicated to *all* configured
remotes for that bucket or bucket prefix.
* - ``COMPLETED``
- The object has successfully replicated to the remote cluster.
* - ``FAILED``
- The object failed to replicate to the remote cluster.
MinIO continuously scans for ``FAILED`` objects not yet in the
replication queue and adds them to the queue as space is available.
* - ``REPLICA``
- The object is itself a replica from a remote source.
MinIO ignores ``REPLICA`` objects, including any metadata-only changes
to that object. MinIO does not support
AWS `replica modification sync
<https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-s3-replication-adds-support-two-way-replication/>`__
at this time.
The replication process generally has one of the following flows:
- ``PENDING -> COMPLETED``
- ``PENDING -> FAILED -> COMPLETED``
.. toctree::
:hidden:
:titlesonly:
:hidden:
/replication/enable-server-side-one-way-bucket-replication
/replication/enable-server-side-two-way-bucket-replication
/replication/enable-server-side-multi-site-bucket-replication
/replication/server-side-replication-resynchronize-remote
.. _minio-bucket-replication-clientside:
Client-Side Bucket Replication
------------------------------
The :mc:`mc` command :mc-cmd:`mc mirror` supports watching a source bucket
and automatically replicating objects to a destination bucket.
/replication/bucket-replication-overview
/replication/site-replication-overview

524
source/replication/site-replication-overview.rst Normal file
View File

@ -0,0 +1,524 @@
.. _minio-site-replication-overview:
=========================
Site Replication Overview
=========================
.. default-domain:: minio
.. contents:: Table of Contents
:local:
:depth: 1
Site replication configures multiple independent MinIO deployments as a cluster of replicas called peer sites.
Site replication assumes the use of either the included MinIO identity provider (IDP) *or* an external IDP.
All configured deployments must use the same IDP.
Deployments using an external IDP must use the same configuration across sites.
Overview
--------
.. _minio-site-replication-what-replicates:
What Replicates Across All Sites
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common-replication.rst
:start-after: start-mc-admin-replicate-what-replicates
:end-before: end-mc-admin-replicate-what-replicates
What Does Not Replicate Across Sites
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common-replication.rst
:start-after: start-mc-admin-replicate-what-does-not-replicate
:end-before: end-mc-admin-replicate-what-does-not-replicate
Initial Site Replication Process
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
After enabling site replication, identity and access management (IAM) settings sync in the following order:
.. tab-set::
.. tab-item:: MinIO IDP
#. Policies
#. User accounts (for local users)
#. Groups
#. Service accounts
Service accounts for ``root`` do not sync.
#. Policy mapping for synced user accounts
#. Policy mapping for `Security Token Service (STS) users <https://docs.min.io/docs/minio-sts-quickstart-guide.html>`__
.. tab-item:: OIDC
#. Policies
#. Service accounts associated to OIDC accounts with a valid :ref:`MinIO Policy <minio-policy>`. ``root`` service accounts do not sync.
#. Policy mapping for synced user accounts
#. Policy mapping for `Security Token Service (STS) users <https://docs.min.io/docs/minio-sts-quickstart-guide.html>`__
.. tab-item:: LDAP
#. Policies
#. Groups
#. Service accounts associated to LDAP accounts with a valid :ref:`MinIO Policy <minio-policy>`. ``root`` service accounts do not sync.
#. Policy mapping for synced user accounts
#. Policy mapping for `Security Token Service (STS) users <https://docs.min.io/docs/minio-sts-quickstart-guide.html>`__
After the initial synchronization of data across peer sites, MinIO continually replicates and synchronizes :ref:`replicable data <minio-site-replication-what-replicates>` among all sites as they occur on any site.
Site Healing
~~~~~~~~~~~~
Any MinIO deployment in the site replication configuration can resynchronize damaged :ref:`replica-eligible data <minio-site-replication-what-replicates>` from the peer with the most updated ("latest") version of that data.
Prerequisites
-------------
One Site with Data at Setup
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Only *one* site can have data at the time of setup.
The other sites must be empty of buckets and objects.
After configuring site replication, any data on the first deployment replicates to the other sites.
All Sites Must Use the Same IDP
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All sites must use the same :ref:`Identity Provider <minio-authentication-and-identity-management>`.
Site replication supports the included MinIO IDP, OIDC, or LDAP.
Access to the Same Encryption Service
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For :ref:`SSE-S3 <minio-encryption-sse-s3>` or :ref:`SSE-KMS <minio-encryption-sse-kms>` encryption via Key Management Service (KMS), all sites must have access to a central KMS deployment.
You can achieve this with a central KES server or multiple KES servers (say one per site) connected via a central supported :ref:`key vault server <minio-sse>`.
Tutorials
---------
Configure Site Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. tab-set::
.. tab-item:: Console
#. :ref:`Deploy <deploy-minio-distributed>` two or more separate MinIO sites, using the same Identity Provider for each site
Only one site can have any buckets or objects on it.
The other site(s) must be empty.
#. In a browser, access the Console for the site with data (if any)
For example, ``https://<addressforsite>:9000``
Replace ``<addressforsite>`` with the IP address or URL for the MinIO deployment.
#. Select **Settings**, then **Site Replication**
.. image:: /images/minio-console/console-settings-site-replication.png
:width: 400px
:alt: MinIO Console menu with the Settings heading expanded to show Site Repilication
:align: center
#. Select :guilabel:`Add Sites +`
.. image:: /images/minio-console/console-settings-site-replication-add.png
:width: 600px
:alt: MinIO Console's Add Sites for Replication screen
:align: center
#. Complete the requested information for the site:
:Access Key: `(required)` The user name for ``root`` to use for signing in to the site.
:Secret Key: `(required)` The password for ``root`` to use for signing in to the site.
:Site Name: A name or other identifying text to associate to the site.
:Endpoint: `(required)` The URL or IP address and port to use to access the site.
To add additional sites beyond two, select the ``+`` button to the side of one of the Site entries.
To remove a site previously added, select the ``-`` button to the side of the site.
Site replication adds a :mc-cmd:`~mc admin user svcacct` under the ``root`` user to perform replication activities.
#. Select **Save**
#. Select :guilabel:`Replication Status` button to verify replication has completed across peer sites.
Any :ref:`replicable data <minio-site-replication-what-replicates>` that exists should show as successfully synced.
For more on reviewing site replication, see the :ref:`Site Replication Status tutorial <minio-site-replication-status-tutorial>`.
.. tab-item:: Command Line
The following steps create a new site replication configuration for three :ref:`distributed deployments <deploy-minio-distributed>`.
One of the sites contains :ref:`replicable data <minio-site-replication-what-replicates>`.
The three sites use aliases, ``minio1``, ``minio2``, and ``minio3``, and only ``minio1`` contains any data.
#. :ref:`Deploy <deploy-minio-distributed>` three or more separate MinIO sites, using the same :ref:`IDP <minio-authentication-and-identity-management>`
Start with empty sites *or* have no more than one site with any :ref:`replicable data <minio-site-replication-what-replicates>`.
#. Configure an alias for each site
For example, for three MinIO sites, you might create aliases ``minio1``, ``minio2``, and ``minio3``.
Use :mc-cmd:`mc alias set`
.. code-block:: shell
mc alias set minio1 https://minio1.example.com:9000 adminuser adminpassword
mc alias set minio2 https://minio2.example.com:9000 adminuser adminpassword
mc alias set minio3 https://minio3.example.com:9000 adminuser adminpassword
or define environment variables
.. code-block:: shell
export MC_HOST_minio1=https://adminuser:adminpassword@minio1.example.com
export MC_HOST_minio2=https://adminuser:adminpassword@minio2.example.com
export MC_HOST_minio3=https://adminuser:adminpassword@minio3.example.com
#. Add site replication configuration
.. code-block:: shell
mc admin replicate add minio1 minio2 minio3
If all sites are empty, the order of the aliases does not matter.
If one of the sites contains any :ref:`replicable data <minio-site-replication-what-replicates>`, you must list it first.
No more than one site can contain any replicable data.
#. Query the site replication configuration to verify
.. code-block:: shell
mc admin replicate info minio1
You can use the alias for any peer site in the site replication configuration.
#. Query the site replication status to confirm any initial data has replicated to all peer sites.
.. code-block:: shell
mc admin replicate status minio1
You can use the alias for any of the peer sites in the site replication configuration.
The output should say that all :ref:`replicable data <minio-site-replication-what-replicates>` is in sync.
The output could resemble the following:
.. code-block:: shell
Bucket replication status:
● 1/1 Buckets in sync
Policy replication status:
● 5/5 Policies in sync
User replication status:
No Users present
Group replication status:
No Groups present
For more on reviewing site replication, see the :ref:`Site Replication Status tutorial <minio-site-replication-status-tutorial>`.
Expand Site Replication
~~~~~~~~~~~~~~~~~~~~~~~
You can add more sites to an existing site replication configuration.
The new site must meet the following requirements:
- Site is fully deployed and accessible by hostname or IP
- Shares the IDP configuration as all other sites in the configuration
- Uses the same root user credentials as other configured sites
- Contains no bucket or object data
.. tab-set::
.. tab-item:: Console
#. Deploy a new, empty MinIO site
#. In a browser, access the Console for one of the exisitng replicated sites
For example, ``https://<addressforsite>:9000``
#. Select **Settings**, then **Site Replication**
.. image:: /images/minio-console/console-site-replication-list-of-sites.png
:width: 600px
:alt: MinIO Console Site Replication with three sites listed
:align: center
#. Select :guilabel:`Add Sites +`
.. image:: /images/minio-console/console-settings-site-replication-add.png
:width: 600px
:alt: MinIO Console's Add Sites for Replication screen
:align: center
#. Make the following entries:
:Access Key: `(required)` The user name to use for signing in to each site. Should be the same across all sites.
:Secret Key: `(required)` The password for the user name to use for signing in to each site. Should be the same across all sites.
:Site Name: An alias to use for the site name.
:Endpoint: `(required)` The URL or IP address and port to use to access the site.
To add additional sites beyond two, select the ``+`` button to the side of the last Site entry.
#. Select :guilabel:`Save`
.. tab-item:: Command Line
#. Deploy three or more separate MinIO sites, using the same external IDP
Only one site can have any buckets or objects on it.
The other sites must be empty.
#. Configure an alias for each site
To check the existing aliases, use :mc-cmd:`mc alias list`.
For example, for three MinIO sites, you might create aliases ``minio1``, ``minio2``, and ``minio3``.
Use :mc-cmd:`mc alias set`
.. code-block:: shell
mc alias set minio1 https://minio1.example.com:9000 adminuser adminpassword
mc alias set minio2 https://minio2.example.com:9000 adminuser adminpassword
mc alias set minio3 https://minio3.example.com:9000 adminuser adminpassword
or define environment variables
.. code-block:: shell
export MC_HOST_minio1=https://adminuser:adminpassword@minio1.example.com
export MC_HOST_minio2=https://adminuser:adminpassword@minio2.example.com
export MC_HOST_minio3=https://adminuser:adminpassword@minio3.example.com
#. Add site replication configuration
List all existing replicated sites first, then list the new site(s) to add.
In this example, ``minio1``, ``minio2``, and ``minio3`` are already configured for replication.
The command adds minio4 and minio5 as new sites to add to the replication.
``minio4`` and ``minio5`` must be empty.
.. code-block:: shell
mc admin replicate add minio1 minio2 minio3 minio4 minio5
#. Query the site replication configuration to verify
.. code-block:: shell
mc admin replicate info minio1
Modify a Site's Endpoint
~~~~~~~~~~~~~~~~~~~~~~~~
If a peer site changes its hostname, you can modify the replication configuration to reflect the new hostname.
.. tab-set::
.. tab-item:: Console
#. In a browser, access the Console for one of the replicated sites
For example, ``https://<addressforsite>:9000``
#. Select **Settings**, then **Site Replication**
#. Select the pencil **Edit** icon to the side of the site to update
.. image:: /images/minio-console/console-site-replication-edit-button.png
:width: 600px
:alt: MinIO Console's List of Replicated Sites screen with the edit buttons highlighted
:align: center
#. Make the following entries:
:New Endpoint: `(required)` The new endpoint address and port to use.
.. image:: /images/minio-console/console-settings-site-replication-edit-endpoint.png
:width: 600px
:alt: Example of the MinIO Console's Edit Replication Endpoint screen
:align: center
#. Select **Update**
.. tab-item:: Command Line
#. Obtain the site's Deployment ID with :mc-cmd:`mc admin replicate info`
.. code-block:: shell
mc admin replicate info <ALIAS>
#. Update the site's endpoint with :mc-cmd:`mc admin replicate edit`
.. code-block:: shell
mc admin replicate edit ALIAS --deployment-id [DEPLOYMENT-ID] --endpoint [NEW-ENDPOINT]
Replace [DEPLOYMENT-ID] with the deployment ID of the site to update.
Replace [NEW-ENDPOINT] with the new endpoint for the site.
Remove a Site from Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can remove a site from replication at any time.
You can re-add the site at a later date, but you must first completely wipe bucket and object data from the site.
.. tab-set::
.. tab-item:: Console
#. In a browser, access the Console for one of the replicated sites
For example, ``https://<addressforsite>:9000``
#. Select **Settings**, then **Site Replication**
#. Select the trash can Delete icon to the side of the site to update
.. image:: /images/minio-console/console-site-replication-delete-button.png
:width: 600px
:alt: MinIO Console's List of Replicated Sites screen with the delete buttons highlighted
:align: center
#. Confirm the site deletion at the prompt by selecting **Delete**
.. image:: /images/minio-console/console-settings-site-replication-confirm-delete.png
:width: 600px
:alt: Example of the MinIO Console's Edit Replication Endpoint screen
:align: center
.. tab-item:: Command Line
Use :mc-cmd:`mc admin replicate remove`
.. code-block:: shell
mc admin replicate remove <ALIAS> --all --force
The ``-all`` flag removes the site as a peer from all participating sites.
The ``--force`` flag is required to removes the site from the site replication configuration.
.. _minio-site-replication-status-tutorial:
Review Replication Status
~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO provides information on replication across the sites for users, groups, policies, or buckets.
The summary information includes the number of **Synced** and **Failed** items for each category.
.. tab-set::
.. tab-item:: Console
#. In a browser, access the Console for one of the replicated sites
For example, ``https://<addressforsite>:9000``
#. Select **Settings**, then **Site Replication**
#. Select :guilabel:`Replication Status`
.. image:: /images/minio-console/console-settings-site-replication-status-summary.png
:width: 600px
:alt: MinIO Console's Replication status from all Sites screen
:align: center
#. `(Optional)` View the replication status for a specific item
Select the type of item to view in the :guilabel:`View Replication Status for a:` dropdown
Specify the name of the specific Bucket, Group, Policy, or User to view
.. image:: /images/minio-console/console-settings-site-replication-status-item.png
:width: 600px
:alt: Example of replication status for a particular bucket item
:align: center
#. `(Optional)` Update the information by selecting :guilabel:`Refresh`
.. tab-item:: Command Line
Use :mc-cmd:`mc admin replicate status`
.. code-block:: shell
mc admin replicate status <ALIAS> --<flag> <value>
For example:
- ``mc admin replicate status minio3 --bucket images``
Displays the replication status for the ``images`` bucket on the ``minio3`` site.
The output resembles the following:
.. code-block::
● Bucket config replication summary for: images
Bucket | MINIO2 | MINIO3 | MINIO4
Tags | | |
Policy | | |
Quota | | |
Retention | | |
Encryption | | |
Replication | ✔ | ✔ | ✔
- ``mc admin replicate status minio3 --all``
Displays the replication status summary for all replication sites of which ``minio3`` is part.
The output resembles the following:
.. code-block::
Bucket replication status:
● 1/1 Buckets in sync
Policy replication status:
● 5/5 Policies in sync
User replication status:
● 1/1 Users in sync
Group replication status:
● 0/2 Groups in sync
Group | MINIO2 | MINIO3 | MINIO4
ittechs | ✗ in-sync | | ✗ in-sync
managers | ✗ in-sync | | ✗ in-sync