From bf471f3e8ca06d0e1de20513aa74c135c8309a51 Mon Sep 17 00:00:00 2001 From: Daryl White <53910321+djwfyi@users.noreply.github.com> Date: Wed, 12 Oct 2022 11:21:36 -0500 Subject: [PATCH] Adds mc replicate diff command (#600) Closes #599 mc replicate diff was missing. This adds it. --- .../bucket-replication-requirements.rst | 2 +- ...server-side-two-way-bucket-replication.rst | 2 +- .../minio-mc-admin/mc-admin-bucket-remote.rst | 4 +- source/reference/minio-mc.rst | 34 ++-- .../reference/minio-mc/mc-replicate-diff.rst | 145 ++++++++++++++++++ 5 files changed, 168 insertions(+), 19 deletions(-) create mode 100644 source/reference/minio-mc/mc-replicate-diff.rst diff --git a/source/administration/bucket-replication/bucket-replication-requirements.rst b/source/administration/bucket-replication/bucket-replication-requirements.rst index a8faa9ee..29ffde7d 100644 --- a/source/administration/bucket-replication/bucket-replication-requirements.rst +++ b/source/administration/bucket-replication/bucket-replication-requirements.rst @@ -22,7 +22,7 @@ Replication can be done in any of the following ways: - :ref:`Active-Active ` Changes to eligible objects of either bucket replicate to the other bucket in a two-way direction. - :ref:`Multi-Site Active-Active ` - Changes to eligible objects on any bucket set up for bucket replication replicte to all of the other buckets. + Changes to eligible objects on any bucket set up for bucket replication replicate to all of the other buckets. Ensure you meet the following prerequisites before you set up any of these replication configurations. diff --git a/source/administration/bucket-replication/enable-server-side-two-way-bucket-replication.rst b/source/administration/bucket-replication/enable-server-side-two-way-bucket-replication.rst index 5cbafd67..8e5d797c 100644 --- a/source/administration/bucket-replication/enable-server-side-two-way-bucket-replication.rst +++ b/source/administration/bucket-replication/enable-server-side-two-way-bucket-replication.rst @@ -153,7 +153,7 @@ B) Create a replication rule from the second deployment back to the first deploy Configure Two-Way Bucket Replication Using the Command Line (:mc:`mc`) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This procecure creates two-way, active-active replication between two MinIO deployments. +This procedure creates two-way, active-active replication between two MinIO deployments. This procedure assumes you have already defined an alias for each deployment as a user with the :ref:`necessary replication permissions `. diff --git a/source/reference/minio-mc-admin/mc-admin-bucket-remote.rst b/source/reference/minio-mc-admin/mc-admin-bucket-remote.rst index 498b008e..d8df12fd 100644 --- a/source/reference/minio-mc-admin/mc-admin-bucket-remote.rst +++ b/source/reference/minio-mc-admin/mc-admin-bucket-remote.rst @@ -39,7 +39,7 @@ ARN for use with :mc:`mc replicate add`: .. code-block:: shell :class: copyable - mc admin bucket remote add SOURCE/BUCKET SOURCE/BUCKET + mc admin bucket remote add SOURCE/BUCKET DESTINATION/BUCKET - Replace :mc-cmd:`SOURCE ` with the :mc-cmd:`alias ` of the MinIO deployment to use as the replication @@ -73,6 +73,8 @@ bucket: Removing the target halts all in-progress :mc-cmd:`bucket replication ` to the target. +.. _minio-retrieve-remote-bucket-targets: + Retrieve Configured Replication Targets ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/reference/minio-mc.rst b/source/reference/minio-mc.rst index 821b5e20..fc4a919e 100644 --- a/source/reference/minio-mc.rst +++ b/source/reference/minio-mc.rst @@ -161,6 +161,19 @@ The following table lists :mc-cmd:`mc` commands: :mc-cmd:`mc` commands that operate on S3-compatible services *require* specifying an alias for that service. + * - | :mc:`mc anonymous set` + | :mc:`mc anonymous set-json` + | :mc:`mc anonymous get` + | :mc:`mc anonymous get-json` + | :mc:`mc anonymous list` + | :mc:`mc anonymous links` + + - The :mc:`mc anonymous` command supports setting or removing anonymous + :ref:`policies ` to a bucket and its contents. Buckets with + anonymous policies allow public access where clients can perform any + action granted by the policy without :ref:`authentication + `. + * - :mc:`mc cat` - .. include:: /reference/minio-mc/mc-cat.rst :start-after: start-mc-cat-desc @@ -254,19 +267,6 @@ The following table lists :mc-cmd:`mc` commands: :start-after: start-mc-pipe-desc :end-before: end-mc-pipe-desc - * - | :mc:`mc anonymous set` - | :mc:`mc anonymous set-json` - | :mc:`mc anonymous get` - | :mc:`mc anonymous get-json` - | :mc:`mc anonymous list` - | :mc:`mc anonymous links` - - - The :mc:`mc anonymous` command supports setting or removing anonymous - :ref:`policies ` to a bucket and its contents. Buckets with - anonymous policies allow public access where clients can perform any - action granted by the policy without :ref:`authentication - `. - * - :mc:`mc rb` - .. include:: /reference/minio-mc/mc-rb.rst :start-after: start-mc-rb-desc @@ -283,13 +283,14 @@ The following table lists :mc-cmd:`mc` commands: explicit object lock settings inherit the bucket default. * - | :mc:`mc replicate add` + | :mc:`mc replicate diff` | :mc:`mc replicate edit` - | :mc:`mc replicate ls` - | :mc:`mc replicate status` - | :mc:`mc replicate resync` | :mc:`mc replicate export` | :mc:`mc replicate import` + | :mc:`mc replicate ls` + | :mc:`mc replicate resync` | :mc:`mc replicate rm` + | :mc:`mc replicate status` - The :mc:`mc replicate ` command configures and manages the :ref:`Server-Side Bucket Replication @@ -496,6 +497,7 @@ All :ref:`commands ` support the following global options: /reference/minio-mc/mc-pipe /reference/minio-mc/mc-rb /reference/minio-mc/mc-replicate-add + /reference/minio-mc/mc-replicate-diff /reference/minio-mc/mc-replicate-edit /reference/minio-mc/mc-replicate-ls /reference/minio-mc/mc-replicate-resync diff --git a/source/reference/minio-mc/mc-replicate-diff.rst b/source/reference/minio-mc/mc-replicate-diff.rst new file mode 100644 index 00000000..6b60cd6e --- /dev/null +++ b/source/reference/minio-mc/mc-replicate-diff.rst @@ -0,0 +1,145 @@ +.. _minio-mc-replicate-diff: + +===================== +``mc replicate diff`` +===================== + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + +.. mc:: mc replicate diff + +Description +----------- + +.. start-mc-replicate-diff-desc + +The :mc:`mc replicate diff` shows a list of unreplicated new or deleted objects. + +.. end-mc-replicate-diff-desc + +You can list the replication status of objects for a particular remote target. +To do so, you must have the ARN of the remote target. +You can use :ref:`retrieve the remote targets configured for a bucket ` to find the ARN. + +Syntax +------ + +.. tab-set:: + + .. tab-item:: EXAMPLE + + The following command shows new or deleted objects in the ``notes`` bucket of the ``teamorange/projects`` prefix on the ``myminio`` alias that have not yet replicated to a specific remote target bucket. + The remote target's ARN is ``arn:minio:replication::3bb8c736-4014-42c5-b3cb-d64e3ebaa75e:notes``. + + .. code-block:: shell + :class: copyable + + mc replicate diff myminio/notes/teamorange/projects --arn arn:minio:replication::3bb8c736-4014-42c5-b3cb-d64e3ebaa75e:notes + + If any new or deleted objects have not yet replicated, the command outputs something similar to the following: + + .. code-block:: shell + + [0001-01-01 00:00:00 UTC] [2022-10-06 17:18:59 UTC] 478efe49-aa9d-46ab-8268-45b70cc4c341 PUT agenda.docx + [0001-01-01 00:00:00 UTC] [2022-10-06 17:18:15 UTC] b283bf43-319f-455a-a779-3c2e669fad88 PUT budget-meeting.docx + + In the output, ``PUT`` corresponds to a new object. + Deleted objects or versions would show ``DEL``. + + .. tab-item:: SYNTAX + + The command has the following syntax: + + .. code-block:: shell + :class: copyable + + mc [GLOBALFLAGS] replicate diff \ + [--arn "string"] \ + TARGET + + .. include:: /includes/common-minio-mc.rst + :start-after: start-minio-syntax + :end-before: end-minio-syntax + +Parameters +~~~~~~~~~~ + +.. mc-cmd:: TARGET + :required: + + The path to the alias, prefix, or object. + +.. mc-cmd:: arn + :optional: + + The ARN of the remote bucket to check for new or deleted objects that have not yet replicated. + + When specified, the command returns a list of any new or deleted objects that have not replicated to the remote target. + If not specified, the command returns a list of new or deleted objects on the source deployment that have not replicated to any remote target. + +Global Flags +~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-globals + :end-before: end-minio-mc-globals + +Examples +-------- + +View Unreplicated Versions of Objects at a Prefix +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Display unreplicated ``PUT`` and ``DELETE`` actions for a prefix: + +.. code-block:: shell + :class: copyable + + mc replicate diff myminio/mybucket/path/to/prefix + +- Replace ``myminio/mybucket`` with the :mc-cmd:`~mc replicate add ALIAS` and + full bucket path for which to create the replication configuration. + +- Replace ``path/to/prefix`` with the prefix or object to use for the request. + +If unreplicated objects exist, the output returns a list of the actions that created or removed objects at the prefix that have not replicated to a remote target: + +.. code-block:: shell + + [0001-01-01 00:00:00 UTC] [2022-10-06 17:18:59 UTC] 478efe49-aa9d-46ab-8268-45b70cc4c341 PUT agenda.docx + [0001-01-01 00:00:00 UTC] [2022-10-06 17:18:15 UTC] b283bf43-319f-455a-a779-3c2e669fad88 PUT budget-meeting.docx + +View Unreplicated Objects at a Specific Remote Target +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following :mc:`mc replicate diff` command shows unreplicated objects at an alias/bucket/prefix path for a specific remote target: + +.. code-block:: shell + :class: copyable + + mc replicate diff myminio/mybucket/path/to/prefix --arn + +- Replace ``myminio/mybucket`` with the :mc-cmd:`~mc replicate add ALIAS` and + full bucket path for which to show unreplicated objects. + +- Replace the ``path/to/prefix`` with the desired prefix or object path. + +- Replace ```` with the resource number for a specific remote target. + +If unreplicated objects exist, the output returns a list of the actions that created or removed objects that have not replicated to the remote target: + +.. code-block:: shell + + [0001-01-01 00:00:00 UTC] [2022-10-06 17:18:59 UTC] 478efe49-aa9d-46ab-8268-45b70cc4c341 PUT agenda.docx + [0001-01-01 00:00:00 UTC] [2022-10-06 17:18:15 UTC] b283bf43-319f-455a-a779-3c2e669fad88 PUT budget-meeting.docx + +S3 Compatibility +~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-s3-compatibility + :end-before: end-minio-mc-s3-compatibility