1
0
mirror of https://github.com/minio/docs.git synced 2025-07-30 07:03:26 +03:00

DOCS-888: prefix exclusion of versioning prevents replication (#893)

Closes #888 

# Summary

- MinIO requires versioning for replication
- We support excluding prefixes/folders in buckets selectively
- This prevents replication within those prefixes/folders

Also included in this ticket

- Added an image to the site replication page
- Fixed a broken SVG
- Minor tweaking of site replication page.

---------

Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com>
This commit is contained in:
Ravind Kumar
2023-06-23 13:19:45 -04:00
committed by GitHub
parent c7a9477286
commit 7a01ca72e6
6 changed files with 61 additions and 22 deletions

View File

@ -47,8 +47,8 @@ Bucket Replication Requires MinIO Deployments
:start-after: start-replication-minio-only
:end-before: end-replication-minio-only
Versioning Objects for Bucket Replication
-----------------------------------------
Replication Requires Versioning
-------------------------------
.. include:: /includes/common-replication.rst
:start-after: start-replication-requires-versioning

View File

@ -283,10 +283,13 @@ Exclude a Prefix From Versioning
You can exclude certain :ref:`prefixes <minio-admin-concepts-organize-objects>` from versioning using the :ref:`MinIO Client <minio-client>`.
This is useful for Spark/Hadoop workloads or others that initially create objects with temporary prefixes.
.. admonition:: Object locking
.. admonition:: Replication and Object Locking Require Versioning
:class: note
Buckets with :ref:`object locking enabled <minio-object-locking>` require versioning and do not support excluding prefixes.
MinIO requires versioning to support :term:`replication`.
Objects in excluded prefixes do not replicate to any peer site or remote site.
MinIO does not support excluding prefixes from versioning on buckets with :ref:`object locking enabled <minio-object-locking>`.
- Use :mc-cmd:`mc version enable` with the :mc-cmd:`~mc version --excluded-prefixes` option:
@ -351,6 +354,14 @@ Exclude Folders from Versioning
You can exclude folders from versioning using the :ref:`MinIO Client <minio-client>`.
.. admonition:: Replication and Object Locking Require Versioning
:class: note
MinIO requires versioning to support :term:`replication`.
Objects in excluded folders do not replicate to any peer site or remote site.
MinIO does not support excluding folders from versioning on buckets with :ref:`object locking enabled <minio-object-locking>`.
.. admonition:: Object locking
:class: note

View File

@ -21,14 +21,14 @@
showgrid="false"
showguides="true"
inkscape:zoom="3.2234344"
inkscape:cx="254.54217"
inkscape:cy="216.07389"
inkscape:cx="254.8524"
inkscape:cy="216.38412"
inkscape:window-width="3840"
inkscape:window-height="2303"
inkscape:window-height="2123"
inkscape:window-x="3840"
inkscape:window-y="2160"
inkscape:window-y="0"
inkscape:window-maximized="1"
inkscape:current-layer="a">
inkscape:current-layer="g9006">
<sodipodi:guide
position="332.87477,220.19"
orientation="0,-1"
@ -39,6 +39,11 @@
orientation="0,-1"
id="guide9033"
inkscape:locked="false" />
<sodipodi:guide
position="352.17997,282.13143"
orientation="0,-1"
id="guide1033"
inkscape:locked="false" />
</sodipodi:namedview>
<defs
id="defs4">
@ -156,8 +161,8 @@
sodipodi:nodetypes="ccc" />
<path
style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1.00235px;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:0, 0, 2.0047, 3.00705;stroke-dashoffset:0;stroke-opacity:1"
d="m 411.33994,131.29656 124.00343,52.08341"
id="path9990"
d="m 410.89652,130.84283 124.40492,-0.12056 v 52.95659"
id="path9988-3"
sodipodi:nodetypes="ccc" />
</g>
<g

Before

Width:  |  Height:  |  Size: 54 KiB

After

Width:  |  Height:  |  Size: 55 KiB

View File

@ -30,6 +30,8 @@ MinIO relies on the immutability protections provided by :ref:`versioning <minio
Use :mc-cmd:`mc version info` to validate the versioning status of both the source and remote buckets.
Use the :mc-cmd:`mc version enable` command to enable versioning as necessary.
If you exclude a prefix or folder from versioning within the source bucket, MinIO cannot replicate objects within that folder or prefix.
.. end-replication-requires-versioning
.. start-replication-requires-object-locking

View File

@ -121,6 +121,8 @@ Client applications can use any S3-compatible SDK or library to interact with th
In particular, the complexity of signature calculation typically makes interfacing via ``curl`` or similar REST clients impractical.
MinIO recommends using S3-compatible SDKs or libraries which perform the signature calculation automatically as part of operations.
.. _minio-deployment-architecture-replicated:
Replicated MinIO Deployments
----------------------------

View File

@ -12,10 +12,21 @@ Site Replication Overview
Site replication configures multiple independent MinIO deployments as a cluster of replicas called peer sites.
.. figure:: /images/architecture/architecture-load-balancer-multi-site.svg
:figwidth: 100%
:alt: Diagram of a site replication deployment with two sites
A site replication deployment with two peer sites.
A load balancer manages routing operations to either of the two sites.
Data written to one site automatically replicates to the other peer site.
Site replication assumes the use of either the included MinIO identity provider (IDP) *or* an external IDP.
All configured deployments must use the same IDP.
Deployments using an external IDP must use the same configuration across sites.
For more information on site replication architecture and deployment concepts, see :ref:`Deployment Architecture: Replicated MinIO Deployments <minio-deployment-architecture-replicated>`.
Overview
--------
@ -72,7 +83,6 @@ After enabling site replication, identity and access management (IAM) settings s
After the initial synchronization of data across peer sites, MinIO continually replicates and synchronizes :ref:`replicable data <minio-site-replication-what-replicates>` among all sites as they occur on any site.
Site Healing
~~~~~~~~~~~~
@ -88,6 +98,17 @@ Any MinIO deployment in the site replication configuration can resynchronize dam
If one site loses data for any reason, resynchronize the data from another healthy site with :mc-cmd:`mc admin replicate resync`.
This launches an active process that resynchronizes the data without waiting for the passive MinIO scanner to recognize the missing data.
Synchronous vs Asynchronous Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common-replication.rst
:start-after: start-replication-sync-vs-async
:end-before: end-replication-sync-vs-async
MinIO strongly recommends using the default asynchronous site replication.
Synchronous site replication performance depends strongly on latency between sites, where higher latency can result in lower PUT performance and replication lag.
To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option.
Proxy to Other Sites
~~~~~~~~~~~~~~~~~~~~
@ -139,6 +160,14 @@ For :ref:`SSE-S3 <minio-encryption-sse-s3>` or :ref:`SSE-KMS <minio-encryption-s
You can achieve this with a central KES server or multiple KES servers (say one per site) connected via a central supported :ref:`key vault server <minio-sse>`.
Replication Requires Versioning
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Site replication *requires* :ref:`minio-bucket-versioning` and enables it for all created buckets automatically.
You cannot disable versioning in site replication deployments.
MinIO cannot replicate objects in prefixes in the bucket that you excluded from versioning.
Load Balancers Installed on Each Multi-Node Site
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -158,16 +187,6 @@ Use :mc:`mc replicate rm` on the command line or the MinIO Console to remove buc
Only one site can have data when setting up site replication.
All other sites must be empty.
Synchronous vs Asynchronous Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: /includes/common-replication.rst
:start-after: start-replication-sync-vs-async
:end-before: end-replication-sync-vs-async
MinIO strongly recommends using the default asynchronous site replication.
To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option.
Tutorials
---------