From 896cac0bc6f87dab553927ef540515b93b317eb8 Mon Sep 17 00:00:00 2001 From: ravindk89 Date: Wed, 24 Mar 2021 18:10:09 -0400 Subject: [PATCH] DOCS-330: Bucket Replication Initial Pass Poorna review, minor touchups --- source/concepts/erasure-coding.rst | 2 + source/index.rst | 1 + .../minio-mc-admin/mc-admin-bucket-remote.rst | 14 +- .../minio-cli/minio-mc/mc-replicate.rst | 788 +++++++++++++++++- ...server-side-one-way-bucket-replication.rst | 383 +++++++++ source/replication/replication-overview.rst | 80 ++ 6 files changed, 1258 insertions(+), 10 deletions(-) create mode 100644 source/replication/enable-server-side-one-way-bucket-replication.rst create mode 100644 source/replication/replication-overview.rst diff --git a/source/concepts/erasure-coding.rst b/source/concepts/erasure-coding.rst index 62a5cfe5..594c9aab 100644 --- a/source/concepts/erasure-coding.rst +++ b/source/concepts/erasure-coding.rst @@ -172,6 +172,8 @@ Write Quorum MinIO Write Quorum is ``(DRIVES - (EC:N)) + 1``. +.. _minio-ec-storage-class: + Storage Classes ~~~~~~~~~~~~~~~ diff --git a/source/index.rst b/source/index.rst index 8a543670..82cb6f16 100644 --- a/source/index.rst +++ b/source/index.rst @@ -21,6 +21,7 @@ Users deploying onto a Kubernetes cluster should start with our /introduction/minio-overview /concepts/feature-overview /tutorials/minio-installation + /replication/replication-overview /security/security-overview /monitoring/monitoring-overview /reference/minio-cli/minio-mc diff --git a/source/reference/minio-cli/minio-mc-admin/mc-admin-bucket-remote.rst b/source/reference/minio-cli/minio-mc-admin/mc-admin-bucket-remote.rst index e57b2d64..7c817efd 100644 --- a/source/reference/minio-cli/minio-mc-admin/mc-admin-bucket-remote.rst +++ b/source/reference/minio-cli/minio-mc-admin/mc-admin-bucket-remote.rst @@ -34,7 +34,7 @@ Add a New Replication Target ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Use :mc-cmd:`mc admin bucket remote add` to create a new replication target -ARN for use with :mc-cmd:`mc replicate`: +ARN for use with :mc-cmd:`mc replicate add`: .. code-block:: shell :class: copyable @@ -171,6 +171,18 @@ Syntax Mutually exclusive with :mc-cmd-option:`~mc admin bucket remote add region` + .. mc-cmd:: sync + :option: + + Enables synchronous replication, where MinIO attempts to replicate + the object *prior* to returning the PUT object response. Synchronous + replication may increase the time spent waiting for PUT operations + to return successfully. + + By default, :mc-cmd:`mc admin bucket remote add` operates in + asynchronous mode, where MinIO attempts replicating objects + *after* returning the PUT object response. + .. mc-cmd:: ls :fullpath: diff --git a/source/reference/minio-cli/minio-mc/mc-replicate.rst b/source/reference/minio-cli/minio-mc/mc-replicate.rst index cd0774d5..4c68cf33 100644 --- a/source/reference/minio-cli/minio-mc/mc-replicate.rst +++ b/source/reference/minio-cli/minio-mc/mc-replicate.rst @@ -15,16 +15,367 @@ Description .. start-mc-replicate-desc -The :mc:`mc replicate` command configures server-side bucket replication +The :mc:`mc replicate` command configures +:ref:`Server-Side Bucket Replication ` between MinIO deployments. .. end-mc-replicate-desc +Create Remote Target Before Configuring Replication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:mc:`mc replicate` depends on the ARN resource returned by +:mc:`mc admin bucket remote`. + +Server-Side Replication Requires MinIO Source and Destination +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO server-side replication only works between MinIO clusters. Both the +source and destination clusters *must* run MinIO. + +To configure replication between arbitrary S3-compatible services, +use :mc-cmd:`mc mirror`. + +Enable Versioning on Source and Destination Buckets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO relies on the immutability protections provided by versioning to +synchronize objects between the source and replication target. + +Use the :mc-cmd:`mc version enable` command to enable versioning on +*both* the source and destination bucket before starting this procedure: + +.. code-block:: shell + :class: copyable + + mc version enable ALIAS/PATH + +- Replace :mc-cmd:`ALIAS ` with the + :mc:`alias ` of the MinIO cluster. + +- Replace :mc-cmd:`PATH ` with the bucket on which + to enable versioning. + +Required Permissions +~~~~~~~~~~~~~~~~~~~~ + +Bucket Replication requires at minimum the following permissions on the +source and destination clusters: + +.. tabs:: + + .. tab:: Source Policy + + The source cluster *must* have a user with *at minimum* following attached + *or* inherited policy: + + .. code-block:: shell + :class: copyable + + { + "Version": "2012-10-17", + "Statement": [ + { + "Action": [ + "admin:SetBucketTarget", + "admin:GetBucketTarget" + ], + "Effect": "Allow", + "Sid": "" + }, + { + "Effect": "Allow", + "Action": [ + "s3:GetReplicationConfiguration", + "s3:ListBucket", + "s3:ListBucketMultipartUploads", + "s3:GetBucketLocation", + "s3:GetBucketVersioning" + ], + "Resource": [ + "arn:aws:s3:::SOURCEBUCKETNAME" + ] + } + ] + } + + Replace ``SOURCEBUCKETNAME`` with the name of the source bucket from which + MinIO replicates objects. + + Use the :mc-cmd:`mc admin policy set` command to associate the policy to + a user on the source MinIO cluster. + + .. tab:: Destination Policy + + The destination cluster *must* have a user with *at minimum* the + following attached *or* inherited policy: + + .. code-block:: shell + :class: copyable + + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:GetReplicationConfiguration", + "s3:ListBucket", + "s3:ListBucketMultipartUploads", + "s3:GetBucketLocation", + "s3:GetBucketVersioning", + "s3:GetBucketObjectLockConfiguration" + ], + "Resource": [ + "arn:aws:s3:::DESTINATIONBUCKETNAME" + ] + }, + { + "Effect": "Allow", + "Action": [ + "s3:GetReplicationConfiguration", + "s3:ReplicateTags", + "s3:AbortMultipartUpload", + "s3:GetObject", + "s3:GetObjectVersion", + "s3:GetObjectVersionTagging", + "s3:PutObject", + "s3:DeleteObject", + "s3:ReplicateObject", + "s3:ReplicateDelete" + ], + "Resource": [ + "arn:aws:s3:::DESTINATIONBUCKETNAME/*" + ] + } + ] + } + + Replace ``DESTINATIONBUCKETNAME`` with the name of the target bucket to + which MinIO replicates objects. + + Use the :mc-cmd:`mc admin policy set` command to associate the policy + to a user on the target MinIO cluster. + +MinIO strongly recommends creating users specifically for supporting +bucket replication operations. See +:mc:`mc admin user` and :mc:`mc admin policy` for more complete +documentation on adding users and policies to a MinIO cluster. + +Replication of Existing Objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO performs replication as part of writing an object (PUT operations). +MinIO does *not* apply replication rules to existing objects written +*before* enabling replication. + +For buckets with existing objects, consider using :mc:`mc mirror` or +:mc:`mc cp` to seed the destination bucket. Consider scheduling a maintenance +window during which applications stop writes to the bucket. Once +the :mc:`~mc mirror` or :mc:`~mc cp` fully sync the source and destination, +enable bucket replication and resume normal operations on the bucket. + +Replication of Delete Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO supports replicating delete operations onto the target bucket. +Specifically, MinIO can replicate both +:s3-docs:`Delete Markers ` *and* the deletion +of specific versioned objects: + +- For delete operations on an object, MinIO replication also creates the delete + marker on the target bucket. + +- For delete operations on versions of an object, + MinIO replication also deletes those versions on the target bucket. + +MinIO requires explicitly enabling replication of delete operations using the +:mc-cmd-option:`mc replicate add replicate` flag. This procedure includes the +required flags for enabling replication of delete operations and delete markers. + +.. note:: + + If a delete operation removes the last object in a bucket prefix, MinIO + recursively deletes any empty directories within that prefix. For example: + + ``play/mybucket/path/to/foo.txt`` + + The bucket prefix is ``/path/to/foo.txt``. If ``foo.txt`` is the last object + in prefix, MinIO deletes the entire prefix. If an object exists at ``/path``, + MinIO stops deleting the prefix at that point. + +.. admonition:: MinIO Trims Empty Object Prefixes + + If a delete operation removes the last object in a bucket prefix, MinIO + recursively removes each empty part of the prefix up to the bucket root. + MinIO only applies the recursive removal to prefixes created *implicitly* as + part of object write operations - that is, the prefix was not created using + an explicit directory creation command such as :mc:`mc mb`. + + If a replication rule enables replication delete operations, the replication + process *also* applies the implicit prefix trimming behavior on the + destination MinIO cluster. + + For example, consider a bucket ``photos`` with the following object prefixes: + + - ``photos/2021/january/myphoto.jpg`` + - ``photos/2021/february/myotherphoto.jpg`` + - ``photos/NYE21/NewYears.jpg`` + + ``photos/NYE21`` is the *only* prefix explicitly created using :mc:`mc mb`. + All other prefixes were *implicitly* created as part of writing the object + located at that prefix. If a command removes ``myphoto.jpg``, it also + automatically trims the empty ``/janaury`` prefix. If then removes + the ``myotherphoto.jpg``, it also automatically trims both the ``/february`` + prefix *and* the now-empty ``/2021`` prefix. If removes the + ``NewYears.jpg`` object, the ``/NYE21`` prefix remains in place since it was + *explicitly* created. + +Replication of Encrypted Objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO supports replicating objects encrypted with automatic +Server-Side Encryption (SSE-S3). Both the source and destination buckets +*must* have automatic SSE-S3 enabled for MinIO to replicate an encrypted object. + +As part of the replication process, MinIO *decrypts* the object on the source +bucket and transmits the unencrypted object. The destination MinIO cluster then +re-encrypts the object using the destination bucket SSE-S3 configuration. MinIO +*strongly recommends* :ref:`enabling TLS ` on both source and +destination clusters to ensure the safety of objects during transmission. + +MinIO does *not* support replicating client-side encrypted objects +(SSE-C). Examples -------- -TBD +See the following tutorials for more complete procedures on configuring +server-side replication with :mc:`mc replicate`: + +- :ref:`minio-bucket-replication-serverside-oneway` + +Add a New Replication Rule +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use :mc-cmd:`mc replicate add` to add a new replication rule to a bucket or bucket +prefix. :mc:`mc replicate` depends on the ARN resource returned by +:mc:`mc admin bucket remote`. + +.. code-block:: shell + :class: copyable + + mc replicate add ALIAS/PATH \ + --arn ARN \ + --remote-bucket BUCKET \ + [--FLAGS] + +- Replace :mc-cmd:`ALIAS ` with the + :mc:`alias ` of the MinIO cluster. + +- Replace :mc-cmd:`PATH ` with the path to the + bucket or bucket prefix on which to add the new rule. + +- Replace :mc-cmd:`ARN ` with the ARN of the + remote bucket target created by :mc:`mc admin bucket remote`. + +- Replace :mc-cmd:`BUCKET ` with the name of the + remote bucket target. The specified bucket name *must* match the ``ARN`` + bucket. + +Include all other optional flags. + +Modify an Existing Replication Rule +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use :mc-cmd:`mc replicate edit` to modify an existing replication rule. + +.. code-block:: shell + :class: copyable + + mc replicate edit ALIAS/PATH \ + --id ID \ + [--FLAGS] + +- Replace :mc-cmd:`ALIAS ` with the + :mc:`alias ` of the MinIO cluster. + +- Replace :mc-cmd:`PATH ` with the path to the + bucket or bucket prefix on which the rule exists. + +- Replace :mc-cmd:`ID ` with the unique identifier for the + rule to modify. Use :mc-cmd:`mc replicate ls` to retrieve the list of + replication rules on the bucket and their corresponding identifiers. + +.. important:: + + MinIO applies replication rules to objects as part of write operations. + Modifying a replication rule has no effect on existing objects in the + bucket. For example, enabling delete marker replication through the + :mc-cmd-option:`~mc replicate edit replicate` option does not automatically + replicate existing delete markers or deleted object versions. + +Disable or Enable an Existing Replication Rule +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use :mc-cmd:`mc replicate edit` with the +:mc-cmd-option:`~mc replicate edit state` flag to disable or enable a +replication rule. + +.. code-block:: shell + :class: copyable + + mc replicate edit ALIAS/PATH \ + --id ID \ + --state "disabled"|"enabled" + +- Replace :mc-cmd:`ALIAS ` with the + :mc:`alias ` of the MinIO cluster. + +- Replace :mc-cmd:`PATH ` with the path to the + bucket or bucket prefix on which the rule exists. + +- Replace :mc-cmd:`ID ` with the unique identifier for the + rule to modify. Use :mc-cmd:`mc replicate ls` to retrieve the list of + replication rules on the bucket and their corresponding identifiers. + +- Specify either ``"disabled"`` or ``"enabled"`` to the + :mc-cmd:`~mc replicate edit state` flag to disable or enable the replication + rule. + +.. important:: + + MinIO applies replication rules to objects as part of write operations. + Modifying a replication rule has no effect on existing objects in the + bucket. In context of enabling or disabling a replication rule, + objects written to a bucket with no enabled replication rules are + not automatically replicated if one or more rules are enabled later. + +Remove a Replication Rule +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use :mc-cmd:`mc replicate rm` to remove an existing replication rule: + +.. code-block:: shell + :class: copyable + + mc replicate rm ALIAS/PATH --id ID + +- Replace :mc-cmd:`ALIAS ` with the + :mc:`alias ` of the MinIO cluster. + +- Replace :mc-cmd:`PATH ` with the path to the + bucket or bucket prefix on which the rule exists. + +- Replace :mc-cmd:`ID ` with the unique identifier for the + rule to modify. Use :mc-cmd:`mc replicate ls` to retrieve the list of + replication rules on the bucket and their corresponding identifiers. + +.. important:: + + MinIO applies replication rules to objects as part of write operations. + Deleting a replication rule has no effect on objects replicated as + part of that rule. Syntax ------ @@ -32,29 +383,448 @@ Syntax .. mc-cmd:: add :fullpath: - Adds a new server-side replication configuration rule for a bucket. + Adds a new server-side replication configuration rule for a bucket. + Requires specifying the resource returned by + :mc:`mc admin bucket remote`. -.. mc-cmd:: set + :mc-cmd:`mc replicate add` has the following syntax: + + .. code-block:: shell + :class: copyable + + mc replicate add SOURCE \ + --arn ARN \ + --remote-bucket DESTINATION \ + --replicate OPTIONS \ + [FLAGS] + + :mc-cmd:`mc replicate add` supports the following arguments: + + .. mc-cmd:: SOURCE + + *Required* + + The full path to the bucket on which to add the bucket + replication configuration. Specify the + :mc:`alias ` of a configured MinIO service as the prefix to the + :mc-cmd:`~mc replicate add SOURCE` path. For example: + + .. code-block:: shell + + mc replicate add play/mybucket + + + .. mc-cmd:: arn + :option: + + *Required* + + Specify the ARN for the destination cluster and bucket. You can + retrieve the ARN using :mc-cmd:`mc admin bucket remote`: + + - Use the :mc-cmd:`mc admin bucket remote ls` to retrieve a list of + ARNs for the bucket on the destination cluster. + + - Use the :mc-cmd:`mc admin bucket remote add` to create an ARN for + the bucket on the destination cluster. + + The specified ARN bucket *must* match the value specified to + :mc-cmd-option:`~mc replicate add remote-bucket`. + + + .. mc-cmd:: remote-bucket + :option: + + *Required* + + Specify the name of the bucket on the destination cluster. The + name *must* match the ARN specified to + :mc-cmd-option:`~mc replicate add arn`. + + + .. mc-cmd:: replicate + :option: + + *Optional* + + Specify a comma-separated list of the following values to enable extended + replication features: + + - ``delete`` - Directs MinIO to replicate DELETE operations to the + destination bucket. + + - ``delete-marker`` - Directs MinIO to replicate delete markers to the + destination bucket. + + + .. mc-cmd:: tags + :option: + + *Optional* + + Specify one or more ampersand ``&`` separated key-value pair tags which + MinIO uses for filtering objects to replicate. For example: + + .. code-block:: shell + + --tags "TAG1=VALUE&TAG2=VALUE&TAG3=VALUE" + + MinIO applies the replication rule to any object whose tag set + contains the specified replication tags. + + + .. mc-cmd:: id + :option: + + *Optional* + + Specify a unique ID for the replication rule. MinIO automatically + generates an ID if one is not specified. + + + .. mc-cmd:: priority + :option: + + *Optional* + + Specify the integer priority of the replication rule. The value + *must* be unique among all other rules on the source bucket. + Higher values imply a *higher* priority than all other rules. + + The default value is ``0``. + + + .. mc-cmd:: storage-class + :option: + + *Optional* + + Specify the MinIO :ref:`storage class ` to + apply to replicated objects. + + + .. mc-cmd:: insecure + :option: + + *Optional* + + Disables verification of the destination cluster's TLS certificate. + This option may be required if the destination cluster uses a + self-signed certificate *or* a certificate signed by an unknown + Certificate Authority. + + .. mc-cmd:: disable + :option: + + *Optional* + + Creates the replication rule in the "disabled" state. MinIO + does not begin replicating objects using the rule until it + is enabled using :mc-cmd:`mc replicate edit`. + + +.. mc-cmd:: edit :fullpath: - Modifies an existing server-side replication configuration rule. + Modifies an existing server-side replication configuration rule for a bucket. + + :mc-cmd:`mc replicate edit` has the following syntax: + + .. code-block:: shell + :class: copyable + + mc replicate edit SOURCE --id IDENTIFIER [FLAGS] + + :mc-cmd:`mc replicate edit` supports the following arguments: + + .. mc-cmd:: SOURCE + + *Required* + + The full path to the bucket on which to edit the bucket + replication configuration. Specify the + :mc:`alias ` of a configured MinIO service as the prefix to the + :mc-cmd:`~mc replicate edit SOURCE` path. For example: + + .. code-block:: shell + + mc replicate edit play/mybucket + + .. mc-cmd:: id + :option: + + *Required* + + Specify the unique ID for a configured replication rule. + + .. mc-cmd:: remote-bucket + :option: + + *Optional* + + Specify the name of the bucket on the destination cluster. The + name *must* match the replication rule ARN. Use + :mc-cmd:`mc replicate ls` to validate the ARN for each configured + replication rule on the bucket. + + .. mc-cmd:: replicate + :option: + + *Optional* + + Specify a comma-separated list of the following values to enable extended + replication features: + + - ``delete`` - Directs MinIO to replicate DELETE operations to the + destination bucket. + + - ``delete-marker`` - Directs MinIO to replicate delete markers to the + destination bucket. + + MinIO does *not* apply the updated replication rules to existing + objects in the source bucket. For example, enabling delete marker + replication does not automatically replicate existing objects with + delete markers. + + + .. mc-cmd:: tags + :option: + + *Optional* + + Specify one or more ampersand ``&`` separated key-value pair tags which + MinIO uses for filtering objects to replicate. For example: + + .. code-block:: shell + + --tags "TAG1=VALUE&TAG2=VALUE&TAG3=VALUE" + + MinIO applies the replication rule to any object whose tag set + contains the specified replication tags. + + .. mc-cmd:: priority + :option: + + *Optional* + + Specify the integer priority of the replication rule. The value + *must* be unique among all other rules on the source bucket. + Higher values imply a *higher* priority than all other rules. + + + .. mc-cmd:: storage-class + :option: + + *Optional* + + Specify the MinIO :ref:`storage class ` to + apply to replicated objects. + + + .. mc-cmd:: insecure + :option: + + *Optional* + + Disables verification of the destination cluster's TLS certificate. + This option may be required if the destination cluster uses a + self-signed certificate *or* a certificate signed by an unknown + Certificate Authority. + + .. mc-cmd:: state + :option: + + *Optional* + + Enables or disables the replication rule. Specify one of the following + values: + + - ``"enable"`` - Enables the replication rule. MinIO begins replicating + objects written *after* enabling the rule. Existing objects require + manual migration to the destination bucket. + + - ``"disable"`` - Disables the replication rule. .. mc-cmd:: ls :fullpath: - Lists configured server-side replication configuration rules. + Lists the server-side replication configuration rules for a bucket. + + :mc-cmd:`mc replicate ls` has the following syntax: + + .. code-block:: shell + :class: copyable + + mc replicate ls SOURCE [FLAGS] + + :mc-cmd:`mc replicate ls` supports the following arguments: + + .. mc-cmd:: SOURCE + + *Required* + + The full path to the bucket on which to list the + replication configurations. Specify the + :mc:`alias ` of a configured MinIO service as the prefix to the + :mc-cmd:`~mc replicate add SOURCE` path. For example: + + .. code-block:: shell + + mc replicate add play/mybucket + + .. mc-cmd:: insecure + :option: + + *Optional* + + Disables verification of the destination cluster's TLS certificate. + This option may be required if the destination cluster uses a + self-signed certificate *or* a certificate signed by an unknown + Certificate Authority. + + .. mc-cmd:: status + :option: + + *Optional* + + Filter replication rules on the bucket based on their status. Specify + one of the following values: + + - ``enabled`` - Show only enabled replication rules. + - ``disabled`` - Show only disabled replication rules. + + If omitted, :mc-cmd:`mc replicate ls` defaults to showing all replication + rules. + .. mc-cmd:: export :fullpath: - Exports a server-side replication configuration rule. + Exports all server-side replication configuration rules for a bucket as a + JSON document. + + :mc-cmd:`mc replicate export` has the following syntax: + + .. code-block:: shell + :class: copyable + + mc replicate export SOURCE [FLAGS] + + :mc-cmd:`mc replicate export` supports the following arguments: + + .. mc-cmd:: SOURCE + + *Required* + + The full path to the bucket for which to export the + replication configurations. Specify the + :mc:`alias ` of a configured MinIO service as the prefix to the + :mc-cmd:`~mc replicate add SOURCE` path. For example: + + .. code-block:: shell + + mc replicate export play/mybucket + + .. mc-cmd:: insecure + :option: + + *Optional* + + Disables verification of the destination cluster's TLS certificate. + This option may be required if the destination cluster uses a + self-signed certificate *or* a certificate signed by an unknown + Certificate Authority. .. mc-cmd:: import :fullpath: - Imports a server-side replication configuration rule in JSON format. + Imports JSON-formatted server-side replication rules for a bucket through + ``STDIN``. + + :mc-cmd:`mc replicate import` has the following syntax: + + .. code-block:: shell + :class: copyable + + mc replicate import SOURCE [FLAGS] + + :mc-cmd:`mc replicate import` also supports input redirection for + specifying the path to the JSON-formatted rules: + + .. code-block:: shell + :class: copyable + + mc replicate import SOURCE [FLAGS] < /path/to/config + + :mc-cmd:`mc replicate import` supports the following arguments: + + .. mc-cmd:: SOURCE + + *Required* + + The full path to the bucket to which to import the + replication configurations. Specify the + :mc:`alias ` of a configured MinIO service as the prefix to the + :mc-cmd:`~mc replicate add SOURCE` path. For example: + + .. code-block:: shell + + mc replicate export play/mybucket + + .. mc-cmd:: insecure + :option: + + *Optional* + + Disables verification of the destination cluster's TLS certificate. + This option may be required if the destination cluster uses a + self-signed certificate *or* a certificate signed by an unknown + Certificate Authority. + .. mc-cmd:: rm :fullpath: - Removes a server-side replication rule. \ No newline at end of file + Removes one or more server-side replication rules on a bucket. + + :mc-cmd:`mc replicate rm` has the following syntax: + + .. code-block:: shell + :class: copyable + + mc replicate rm SOURCE --id ID [FLAGS] + + :mc-cmd:`mc replicate rm` supports the following arguments: + + .. mc-cmd:: SOURCE + + *Required* + + The full path to the bucket on which to remove the bucket + replication configuration. Specify the + :mc:`alias ` of a configured MinIO service as the prefix to the + :mc-cmd:`~mc replicate edit SOURCE` path. For example: + + .. code-block:: shell + + mc replicate edit play/mybucket + + .. mc-cmd:: id + :option: + + *Optional* + + Specify the unique ID for a configured replication rule. + + .. mc-cmd:: all + :option: + + Removes all replication rules on the specified bucket. Requires + specifying the :mc-cmd-option:`~mc replicate rm force` flag. + + .. mc-cmd:: force + :option: + + *Optional* + + Required if specifying :mc-cmd-option:`~mc replicate rm all` . \ No newline at end of file diff --git a/source/replication/enable-server-side-one-way-bucket-replication.rst b/source/replication/enable-server-side-one-way-bucket-replication.rst new file mode 100644 index 00000000..c01fd92e --- /dev/null +++ b/source/replication/enable-server-side-one-way-bucket-replication.rst @@ -0,0 +1,383 @@ +.. _minio-bucket-replication-serverside-oneway: + +============================================= +Enable One-Way Server-Side Bucket Replication +============================================= + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + + +MinIO Server-Side Bucket Replication supports automatically synchronizing +objects between a source and destination bucket. The destination bucket +can be on the same MinIO cluster as the source *or* an independent remote +MinIO cluster. + +The procedure on this page creates a new bucket replication rule for +one-way synchronization of objects from a source MinIO cluster to a destination +MinIO cluster. + +- Use the :mc-cmd:`mc replicate edit` command to modify an existing + replication rule. + +- Use the :mc-cmd-option:`mc replicate edit` command with the + :mc-cmd-option:`--state "disable" ` flag to + disable an existing replication rule. + +- Use the :mc-cmd:`mc replicate rm` command to remove an existing replication + rule. + + +.. todo: Diagram + +.. todo + This procedure specifically enables only one-way replication between the + source and destination buckets. For a procedure on two-way "active-active" + replication, see . + +.. _minio-bucket-replication-serverside-oneway-requirements: + +Requirements +------------ + +MinIO Server-Side Replication Requires a MinIO Cluster as the Destination +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO server-side replication only works between MinIO clusters. Both the +source and destination clusters *must* run MinIO. + +To configure replication between arbitrary S3-compatible services, +use :mc-cmd:`mc mirror`. + +Enable Versioning on Source and Destination Buckets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO relies on the immutability protections provided by versioning to +synchronize objects between the source and replication target. + +Use the :mc-cmd:`mc version enable` command to enable versioning on +*both* the source and destination bucket before starting this procedure: + +.. code-block:: shell + :class: copyable + + mc version enable ALIAS/PATH + +- Replace :mc-cmd:`ALIAS ` with the + :mc:`alias ` of the MinIO cluster. + +- Replace :mc-cmd:`PATH ` with the bucket on which + to enable versioning. + +``mc`` Command Line Interface +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This procedure uses :mc:`mc` for performing operations on both the source +and destination MinIO cluster. See the ``mc`` +:ref:`Installation Quickstart ` for instructions on downloading +and installing ``mc``. + +.. _minio-bucket-replication-serverside-oneway-permissions: + +Required Permissions +~~~~~~~~~~~~~~~~~~~~ + +Bucket Replication requires at minimum the following permissions on the +source and destination clusters: + +.. tabs:: + + .. tab:: Source Policy + + The source cluster *must* have a user with *at minimum* following attached + *or* inherited policy: + + .. code-block:: shell + :class: copyable + + { + "Version": "2012-10-17", + "Statement": [ + { + "Action": [ + "admin:SetBucketTarget", + "admin:GetBucketTarget" + ], + "Effect": "Allow", + "Sid": "" + }, + { + "Effect": "Allow", + "Action": [ + "s3:GetReplicationConfiguration", + "s3:ListBucket", + "s3:ListBucketMultipartUploads", + "s3:GetBucketLocation", + "s3:GetBucketVersioning" + ], + "Resource": [ + "arn:aws:s3:::SOURCEBUCKETNAME" + ] + } + ] + } + + Replace ``SOURCEBUCKETNAME`` with the name of the source bucket from which + MinIO replicates objects. + + Use the :mc-cmd:`mc admin policy set` command to associate the policy to + a user on the source MinIO cluster. + + .. tab:: Destination Policy + + The destination cluster *must* have a user with *at minimum* the + following attached *or* inherited policy: + + .. code-block:: shell + :class: copyable + + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:GetReplicationConfiguration", + "s3:ListBucket", + "s3:ListBucketMultipartUploads", + "s3:GetBucketLocation", + "s3:GetBucketVersioning", + "s3:GetBucketObjectLockConfiguration" + ], + "Resource": [ + "arn:aws:s3:::DESTINATIONBUCKETNAME" + ] + }, + { + "Effect": "Allow", + "Action": [ + "s3:GetReplicationConfiguration", + "s3:ReplicateTags", + "s3:AbortMultipartUpload", + "s3:GetObject", + "s3:GetObjectVersion", + "s3:GetObjectVersionTagging", + "s3:PutObject", + "s3:DeleteObject", + "s3:ReplicateObject", + "s3:ReplicateDelete" + ], + "Resource": [ + "arn:aws:s3:::DESTINATIONBUCKETNAME/*" + ] + } + ] + } + + Replace ``DESTINATIONBUCKETNAME`` with the name of the target bucket to + which MinIO replicates objects. + + Use the :mc-cmd:`mc admin policy set` command to associate the policy + to a user on the target MinIO cluster. + +MinIO strongly recommends creating users specifically for supporting +bucket replication operations. See +:mc:`mc admin user` and :mc:`mc admin policy` for more complete +documentation on adding users and policies to a MinIO cluster. + +Considerations +-------------- + +Replication of Existing Objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO performs replication as part of writing an object (PUT operations). MinIO +does *not* apply replication rules to existing objects in the bucket. Use +:mc:`mc cp` or :mc:`mc mirror` to migrate existing objects to the destination +bucket. + +Replication of Delete Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO supports replicating delete operations onto the target bucket. +Specifically, MinIO can replicate versioning +:s3-docs:`Delete Markers ` and the deletion +of specific versioned objects: + +- For delete operations on an object, MinIO replication also creates the delete + marker on the target bucket. + +- For delete operations on versions of an object, + MinIO replication also deletes those versions on the target bucket. + +MinIO requires explicitly enabling replication of delete operations using the +:mc-cmd-option:`mc replicate add replicate` or +:mc-cmd-option:`mc replicate edit replicate`. This procedure includes the +required flags for enabling replication of delete operations and delete markers. + +Replication of Encrypted Objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO supports replicating objects encrypted with automatic +Server-Side Encryption (SSE-S3). Both the source and destination buckets +*must* have automatic SSE-S3 enabled for MinIO to replicate an encrypted object. + +As part of the replication process, MinIO *decrypts* the object on the source +bucket and transmits the unencrypted object. The destination MinIO cluster then +re-encrypts the object using the destination bucket SSE-S3 configuration. MinIO +*strongly recommends* :ref:`enabling TLS ` on both source and +destination clusters to ensure the safety of objects during transmission. + +MinIO does *not* support replicating client-side encrypted objects +(SSE-C). + +Procedure +--------- + +1) Configure ``mc`` Access to the Source and Destination Cluster +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use the :mc-cmd:`mc alias set` command to add an alias for both source +and destination MinIO clusters. + +.. tabs:: + + .. tab: Source Cluster + + .. code-block:: shell + :class: copyable + + mc alias set SourceCluster HOSTNAME ACCESSKEY SECRETKEY + + - Replace :mc-cmd:`~mc alias set HOSTNAME` with the hostname or IP address + of a node in the source MinIO cluster. For distributed MinIO clusters + using a load balancer, specify the hostname or IP address of that load + balancer. + + - Replace :mc-cmd:`~mc alias set ACCESSKEY` and + :mc-cmd:`~mc alias set SECRETKEY` with the access and secret key for a + user with the :ref:`required permissions + ` on the source + MinIO cluster. + + .. tab:: DestinationCluster + + .. code-block:: shell + :class: copyable + + mc alias set DestinationCluster HOSTNAME ACCESSKEY SECRETKEY + + - Replace :mc-cmd:`~mc alias set HOSTNAME` with the hostname or IP address + of a node in the destination MinIO cluster. For distributed MinIO + clusters using a load balancer, specify the hostname or IP address of + that load balancer. + + - Replace :mc-cmd:`~mc alias set ACCESSKEY` and + :mc-cmd:`~mc alias set SECRETKEY` with the access and secret key for a + user with the :ref:`required permissions + ` on the + destination MinIO cluster. + +2) Create a Replication Target for the Destination Cluster +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use the :mc-cmd:`mc admin bucket remote` command to create a replication target +for the destination cluster. MinIO supports *one* remote target per destination +bucket. You cannot create multiple remote targets for the same destination +bucket. + +.. code-block:: shell + :class: copyable + + mc admin bucket remote add SourceCluster/SOURCEBUCKET \ + https://ACCESSKEY:SECRETKEY@HOSTNAME/DESTINATIONBUCKET + --service "replication" + [--sync] + +- Replace ``SOURCEBUCKET`` with the name of the bucket on the source cluster. + The name of the bucket *must* match the bucket name specified to the source + user's replication :ref:`policy + `. + +- Replace ``ACCESSKEY`` and ``SECRETKEY`` with the access and secret key for + a user with the :ref:`required permissions + ` on the destination + MinIO cluster. + +- Replace the ``HOSTNAME`` with the hostname or IP address of + a node in the MinIO cluster. For distributed MinIO clusters using a + load balancer, specify the hostname or IP address of that load balancer. + +- Replace the ``DESTINATIONBUCKET`` with the name of the bucket on the + destination cluster. The name of the bucket *must* match the bucket name + specified to the destination user's replication :ref:`policy + `. + +- MinIO defaults to using asynchronous object replication, where MinIO + replicates objects *after* returning the PUT object response. Specify the + :mc-cmd-option:`~mc admin bucket remote add sync` option to enable + synchronous replication, where MinIO attempts to replicate the object + *prior* to returning the PUT object response. + +The command returns an ARN similar to the following: + +.. code-block:: shell + + Role ARN = 'arn:minio:replication:::DESTINATIONBUCKET' + +Copy the ARN string for use in the next step. + +3) Create a New Bucket Replication Rule +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use the :mc-cmd:`mc replicate add` command to add the new server-side +replication rule to the source MinIO cluster. + +.. code-block:: shell + :class: copyable + + mc replicate add SourceCluster/SOURCEBUCKET \ + --remote-bucket DESTINATIONBUCKET \ + --arn 'arn:minio:replication:::DESTINATIONBUCKET' \ + --replicate "delete,delete-marker" + +- Replace ``SOURCEBUCKET`` with the name of the bucket on the source cluster. + +- Replace the ``DESTINATIONBUCKET`` with the name of the bucket on the + destination cluster. + +- Replace the ``--arn`` value with the ARN returned in the previous step. + +- Replace ``PRIORITY`` with the integer priority of this replication rule. + +- The ``--replicate "delete,delete-marker"`` flag enables replicating delete + markers and deletion of object versions. See + :mc-cmd-option:`mc replicate add replicate` for more complete documentation. + Omit this field to disable replication of delete operations. + +Specify any other supported optional arguments for +:mc-cmd:`mc replicate add`. + +4) Validate the Replication Configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use :mc-cmd:`mc cp` to copy a new object to the source bucket. + +.. code-block:: shell + :class: copyable + + mc cp ~/foo.txt SourceCluster/SOURCEBUCKET + +Use :mc-cmd:`mc ls` to verify the object exists on the destination bucket: + +.. code-block:: shell + :class: copyable + + mc ls DestinationCluster/DESTINATIONBUCKET + +If the remote target was configured *without* the +:mc-cmd-option:`~mc admin bucket remote add sync` option, the destination +bucket may have some delay before it receives the new object. \ No newline at end of file diff --git a/source/replication/replication-overview.rst b/source/replication/replication-overview.rst new file mode 100644 index 00000000..0e8d248c --- /dev/null +++ b/source/replication/replication-overview.rst @@ -0,0 +1,80 @@ +.. _minio-bucket-replication: + +================== +Bucket Replication +================== + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 1 + +MinIO supports server-side and client-side replication of objects between source +and destination buckets. MinIO offers both active-passive (one-way) and +active-active (two-way) flavors of the following replication types: + +:ref:`Server-Side Bucket Replication ` + Configure per-bucket rules for automatically synchronizing objects between + buckets within the same MinIO cluster *or* between two independent MinIO + Clusters. MinIO applies rules as part of object write operations + (e.g. ``PUT``) and automatically synchronizes new objects *and* object + mutations, such as new object versions or changes to object metadata. + + MinIO server-side bucket replication only supports MinIO clusters for the + remote replication target. + +:ref:`Client-side Bucket Replication ` + Use the :mc-cmd:`mc mirror` process to synchronize objects between buckets + within the same S3-compatible cluster *or* between two independent + S3-compatible clusters. Client-side replication using :mc-cmd:`mc mirror` + supports MinIO-to-S3 and similar replication configurations. + +.. _minio-bucket-replication-serverside: + +Server-Side Bucket Replication +------------------------------ + +MinIO server-side bucket replication is an automatic bucket-level configuration +that synchronizes objects between a source and destination bucket. MinIO +server-side replication *requires* the source and destination bucket be MinIO +clusters. The source and destination bucket *may* be the same MinIO cluster *or* +two independent MinIO clusters. + +For each write operation to the bucket, MinIO checks all configured replication +rules for the bucket and applies the matching rule with highest configured +priority. MinIO synchronizes new objects *and* object mutations, such as +new object versions or changes to object metadata. This includes +metadata operations such as enabling or modifying object locking or +retention settings. + +MinIO server-side bucket replication is functionally similar to Amazon S3 +replication while adding the following MinIO-only features: + +- Source and destination bucket names can match, supporting site-to-site + use cases such as Splunk or Veeam BC/DR. + +- Replicate object locking and retention settings from source to destination. + +- Simplified implementation than S3 bucket replication configuration, removing + the need to configure settings like AccessControlTranslation, Metrics, and + SourceSelectionCriteria. + +- Active-Active (Two-Way) replication of objects between source and destination + buckets. + +.. toctree:: + :hidden: + :titlesonly: + + /replication/enable-server-side-one-way-bucket-replication + + +.. _minio-bucket-replication-clientside: + +Client-Side Bucket Replication +------------------------------ + +The :mc:`mc` command :mc-cmd:`mc mirror` supports watching a source bucket +and automatically replicating objects to a destination bucket. +