From 7a01ca72e6bbebddca942f257f068af6313d3f39 Mon Sep 17 00:00:00 2001 From: Ravind Kumar Date: Fri, 23 Jun 2023 13:19:45 -0400 Subject: [PATCH] DOCS-888: prefix exclusion of versioning prevents replication (#893) Closes #888 # Summary - MinIO requires versioning for replication - We support excluding prefixes/folders in buckets selectively - This prevents replication within those prefixes/folders Also included in this ticket - Added an image to the site replication page - Fixed a broken SVG - Minor tweaking of site replication page. --------- Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com> --- .../bucket-replication-requirements.rst | 4 +- .../object-management/object-versioning.rst | 15 ++++++- .../availability-multi-site-setup.svg | 19 +++++---- source/includes/common-replication.rst | 2 + source/operations/concepts/architecture.rst | 2 + .../multi-site-replication.rst | 41 ++++++++++++++----- 6 files changed, 61 insertions(+), 22 deletions(-) diff --git a/source/administration/bucket-replication/bucket-replication-requirements.rst b/source/administration/bucket-replication/bucket-replication-requirements.rst index 29ffde7d..3dde1753 100644 --- a/source/administration/bucket-replication/bucket-replication-requirements.rst +++ b/source/administration/bucket-replication/bucket-replication-requirements.rst @@ -47,8 +47,8 @@ Bucket Replication Requires MinIO Deployments :start-after: start-replication-minio-only :end-before: end-replication-minio-only -Versioning Objects for Bucket Replication ------------------------------------------ +Replication Requires Versioning +------------------------------- .. include:: /includes/common-replication.rst :start-after: start-replication-requires-versioning diff --git a/source/administration/object-management/object-versioning.rst b/source/administration/object-management/object-versioning.rst index e0517f63..df8c1a15 100644 --- a/source/administration/object-management/object-versioning.rst +++ b/source/administration/object-management/object-versioning.rst @@ -283,10 +283,13 @@ Exclude a Prefix From Versioning You can exclude certain :ref:`prefixes ` from versioning using the :ref:`MinIO Client `. This is useful for Spark/Hadoop workloads or others that initially create objects with temporary prefixes. -.. admonition:: Object locking +.. admonition:: Replication and Object Locking Require Versioning :class: note - Buckets with :ref:`object locking enabled ` require versioning and do not support excluding prefixes. + MinIO requires versioning to support :term:`replication`. + Objects in excluded prefixes do not replicate to any peer site or remote site. + + MinIO does not support excluding prefixes from versioning on buckets with :ref:`object locking enabled `. - Use :mc-cmd:`mc version enable` with the :mc-cmd:`~mc version --excluded-prefixes` option: @@ -351,6 +354,14 @@ Exclude Folders from Versioning You can exclude folders from versioning using the :ref:`MinIO Client `. +.. admonition:: Replication and Object Locking Require Versioning + :class: note + + MinIO requires versioning to support :term:`replication`. + Objects in excluded folders do not replicate to any peer site or remote site. + + MinIO does not support excluding folders from versioning on buckets with :ref:`object locking enabled `. + .. admonition:: Object locking :class: note diff --git a/source/images/availability/availability-multi-site-setup.svg b/source/images/availability/availability-multi-site-setup.svg index 7d9690f9..ba754220 100644 --- a/source/images/availability/availability-multi-site-setup.svg +++ b/source/images/availability/availability-multi-site-setup.svg @@ -21,14 +21,14 @@ showgrid="false" showguides="true" inkscape:zoom="3.2234344" - inkscape:cx="254.54217" - inkscape:cy="216.07389" + inkscape:cx="254.8524" + inkscape:cy="216.38412" inkscape:window-width="3840" - inkscape:window-height="2303" + inkscape:window-height="2123" inkscape:window-x="3840" - inkscape:window-y="2160" + inkscape:window-y="0" inkscape:window-maximized="1" - inkscape:current-layer="a"> + inkscape:current-layer="g9006"> + @@ -156,8 +161,8 @@ sodipodi:nodetypes="ccc" /> `. + + Overview -------- @@ -72,7 +83,6 @@ After enabling site replication, identity and access management (IAM) settings s After the initial synchronization of data across peer sites, MinIO continually replicates and synchronizes :ref:`replicable data ` among all sites as they occur on any site. - Site Healing ~~~~~~~~~~~~ @@ -88,6 +98,17 @@ Any MinIO deployment in the site replication configuration can resynchronize dam If one site loses data for any reason, resynchronize the data from another healthy site with :mc-cmd:`mc admin replicate resync`. This launches an active process that resynchronizes the data without waiting for the passive MinIO scanner to recognize the missing data. +Synchronous vs Asynchronous Replication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-replication.rst + :start-after: start-replication-sync-vs-async + :end-before: end-replication-sync-vs-async + +MinIO strongly recommends using the default asynchronous site replication. +Synchronous site replication performance depends strongly on latency between sites, where higher latency can result in lower PUT performance and replication lag. +To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option. + Proxy to Other Sites ~~~~~~~~~~~~~~~~~~~~ @@ -139,6 +160,14 @@ For :ref:`SSE-S3 ` or :ref:`SSE-KMS `. +Replication Requires Versioning +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Site replication *requires* :ref:`minio-bucket-versioning` and enables it for all created buckets automatically. +You cannot disable versioning in site replication deployments. + +MinIO cannot replicate objects in prefixes in the bucket that you excluded from versioning. + Load Balancers Installed on Each Multi-Node Site ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -158,16 +187,6 @@ Use :mc:`mc replicate rm` on the command line or the MinIO Console to remove buc Only one site can have data when setting up site replication. All other sites must be empty. -Synchronous vs Asynchronous Replication -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. include:: /includes/common-replication.rst - :start-after: start-replication-sync-vs-async - :end-before: end-replication-sync-vs-async - -MinIO strongly recommends using the default asynchronous site replication. -To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option. - Tutorials ---------