mirror of
https://github.com/minio/docs.git
synced 2025-07-30 07:03:26 +03:00
DOCS-888: prefix exclusion of versioning prevents replication (#893)
Closes #888 # Summary - MinIO requires versioning for replication - We support excluding prefixes/folders in buckets selectively - This prevents replication within those prefixes/folders Also included in this ticket - Added an image to the site replication page - Fixed a broken SVG - Minor tweaking of site replication page. --------- Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com>
This commit is contained in:
@ -47,8 +47,8 @@ Bucket Replication Requires MinIO Deployments
|
||||
:start-after: start-replication-minio-only
|
||||
:end-before: end-replication-minio-only
|
||||
|
||||
Versioning Objects for Bucket Replication
|
||||
-----------------------------------------
|
||||
Replication Requires Versioning
|
||||
-------------------------------
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-requires-versioning
|
||||
|
@ -283,10 +283,13 @@ Exclude a Prefix From Versioning
|
||||
You can exclude certain :ref:`prefixes <minio-admin-concepts-organize-objects>` from versioning using the :ref:`MinIO Client <minio-client>`.
|
||||
This is useful for Spark/Hadoop workloads or others that initially create objects with temporary prefixes.
|
||||
|
||||
.. admonition:: Object locking
|
||||
.. admonition:: Replication and Object Locking Require Versioning
|
||||
:class: note
|
||||
|
||||
Buckets with :ref:`object locking enabled <minio-object-locking>` require versioning and do not support excluding prefixes.
|
||||
MinIO requires versioning to support :term:`replication`.
|
||||
Objects in excluded prefixes do not replicate to any peer site or remote site.
|
||||
|
||||
MinIO does not support excluding prefixes from versioning on buckets with :ref:`object locking enabled <minio-object-locking>`.
|
||||
|
||||
- Use :mc-cmd:`mc version enable` with the :mc-cmd:`~mc version --excluded-prefixes` option:
|
||||
|
||||
@ -351,6 +354,14 @@ Exclude Folders from Versioning
|
||||
|
||||
You can exclude folders from versioning using the :ref:`MinIO Client <minio-client>`.
|
||||
|
||||
.. admonition:: Replication and Object Locking Require Versioning
|
||||
:class: note
|
||||
|
||||
MinIO requires versioning to support :term:`replication`.
|
||||
Objects in excluded folders do not replicate to any peer site or remote site.
|
||||
|
||||
MinIO does not support excluding folders from versioning on buckets with :ref:`object locking enabled <minio-object-locking>`.
|
||||
|
||||
.. admonition:: Object locking
|
||||
:class: note
|
||||
|
||||
|
@ -21,14 +21,14 @@
|
||||
showgrid="false"
|
||||
showguides="true"
|
||||
inkscape:zoom="3.2234344"
|
||||
inkscape:cx="254.54217"
|
||||
inkscape:cy="216.07389"
|
||||
inkscape:cx="254.8524"
|
||||
inkscape:cy="216.38412"
|
||||
inkscape:window-width="3840"
|
||||
inkscape:window-height="2303"
|
||||
inkscape:window-height="2123"
|
||||
inkscape:window-x="3840"
|
||||
inkscape:window-y="2160"
|
||||
inkscape:window-y="0"
|
||||
inkscape:window-maximized="1"
|
||||
inkscape:current-layer="a">
|
||||
inkscape:current-layer="g9006">
|
||||
<sodipodi:guide
|
||||
position="332.87477,220.19"
|
||||
orientation="0,-1"
|
||||
@ -39,6 +39,11 @@
|
||||
orientation="0,-1"
|
||||
id="guide9033"
|
||||
inkscape:locked="false" />
|
||||
<sodipodi:guide
|
||||
position="352.17997,282.13143"
|
||||
orientation="0,-1"
|
||||
id="guide1033"
|
||||
inkscape:locked="false" />
|
||||
</sodipodi:namedview>
|
||||
<defs
|
||||
id="defs4">
|
||||
@ -156,8 +161,8 @@
|
||||
sodipodi:nodetypes="ccc" />
|
||||
<path
|
||||
style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1.00235px;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:0, 0, 2.0047, 3.00705;stroke-dashoffset:0;stroke-opacity:1"
|
||||
d="m 411.33994,131.29656 124.00343,52.08341"
|
||||
id="path9990"
|
||||
d="m 410.89652,130.84283 124.40492,-0.12056 v 52.95659"
|
||||
id="path9988-3"
|
||||
sodipodi:nodetypes="ccc" />
|
||||
</g>
|
||||
<g
|
||||
|
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 55 KiB |
@ -30,6 +30,8 @@ MinIO relies on the immutability protections provided by :ref:`versioning <minio
|
||||
Use :mc-cmd:`mc version info` to validate the versioning status of both the source and remote buckets.
|
||||
Use the :mc-cmd:`mc version enable` command to enable versioning as necessary.
|
||||
|
||||
If you exclude a prefix or folder from versioning within the source bucket, MinIO cannot replicate objects within that folder or prefix.
|
||||
|
||||
.. end-replication-requires-versioning
|
||||
|
||||
.. start-replication-requires-object-locking
|
||||
|
@ -121,6 +121,8 @@ Client applications can use any S3-compatible SDK or library to interact with th
|
||||
In particular, the complexity of signature calculation typically makes interfacing via ``curl`` or similar REST clients impractical.
|
||||
MinIO recommends using S3-compatible SDKs or libraries which perform the signature calculation automatically as part of operations.
|
||||
|
||||
.. _minio-deployment-architecture-replicated:
|
||||
|
||||
Replicated MinIO Deployments
|
||||
----------------------------
|
||||
|
||||
|
@ -12,10 +12,21 @@ Site Replication Overview
|
||||
|
||||
Site replication configures multiple independent MinIO deployments as a cluster of replicas called peer sites.
|
||||
|
||||
.. figure:: /images/architecture/architecture-load-balancer-multi-site.svg
|
||||
:figwidth: 100%
|
||||
:alt: Diagram of a site replication deployment with two sites
|
||||
|
||||
A site replication deployment with two peer sites.
|
||||
A load balancer manages routing operations to either of the two sites.
|
||||
Data written to one site automatically replicates to the other peer site.
|
||||
|
||||
Site replication assumes the use of either the included MinIO identity provider (IDP) *or* an external IDP.
|
||||
All configured deployments must use the same IDP.
|
||||
Deployments using an external IDP must use the same configuration across sites.
|
||||
|
||||
For more information on site replication architecture and deployment concepts, see :ref:`Deployment Architecture: Replicated MinIO Deployments <minio-deployment-architecture-replicated>`.
|
||||
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
@ -72,7 +83,6 @@ After enabling site replication, identity and access management (IAM) settings s
|
||||
|
||||
After the initial synchronization of data across peer sites, MinIO continually replicates and synchronizes :ref:`replicable data <minio-site-replication-what-replicates>` among all sites as they occur on any site.
|
||||
|
||||
|
||||
Site Healing
|
||||
~~~~~~~~~~~~
|
||||
|
||||
@ -88,6 +98,17 @@ Any MinIO deployment in the site replication configuration can resynchronize dam
|
||||
If one site loses data for any reason, resynchronize the data from another healthy site with :mc-cmd:`mc admin replicate resync`.
|
||||
This launches an active process that resynchronizes the data without waiting for the passive MinIO scanner to recognize the missing data.
|
||||
|
||||
Synchronous vs Asynchronous Replication
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-sync-vs-async
|
||||
:end-before: end-replication-sync-vs-async
|
||||
|
||||
MinIO strongly recommends using the default asynchronous site replication.
|
||||
Synchronous site replication performance depends strongly on latency between sites, where higher latency can result in lower PUT performance and replication lag.
|
||||
To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option.
|
||||
|
||||
Proxy to Other Sites
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
@ -139,6 +160,14 @@ For :ref:`SSE-S3 <minio-encryption-sse-s3>` or :ref:`SSE-KMS <minio-encryption-s
|
||||
|
||||
You can achieve this with a central KES server or multiple KES servers (say one per site) connected via a central supported :ref:`key vault server <minio-sse>`.
|
||||
|
||||
Replication Requires Versioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Site replication *requires* :ref:`minio-bucket-versioning` and enables it for all created buckets automatically.
|
||||
You cannot disable versioning in site replication deployments.
|
||||
|
||||
MinIO cannot replicate objects in prefixes in the bucket that you excluded from versioning.
|
||||
|
||||
Load Balancers Installed on Each Multi-Node Site
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
@ -158,16 +187,6 @@ Use :mc:`mc replicate rm` on the command line or the MinIO Console to remove buc
|
||||
Only one site can have data when setting up site replication.
|
||||
All other sites must be empty.
|
||||
|
||||
Synchronous vs Asynchronous Replication
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. include:: /includes/common-replication.rst
|
||||
:start-after: start-replication-sync-vs-async
|
||||
:end-before: end-replication-sync-vs-async
|
||||
|
||||
MinIO strongly recommends using the default asynchronous site replication.
|
||||
To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option.
|
||||
|
||||
Tutorials
|
||||
---------
|
||||
|
||||
|
Reference in New Issue
Block a user