mirror of
https://github.com/minio/docs.git
synced 2025-07-31 18:04:52 +03:00
DOCS-888: prefix exclusion of versioning prevents replication (#893)
Closes #888 # Summary - MinIO requires versioning for replication - We support excluding prefixes/folders in buckets selectively - This prevents replication within those prefixes/folders Also included in this ticket - Added an image to the site replication page - Fixed a broken SVG - Minor tweaking of site replication page. --------- Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com>
This commit is contained in:
@ -47,8 +47,8 @@ Bucket Replication Requires MinIO Deployments
|
|||||||
:start-after: start-replication-minio-only
|
:start-after: start-replication-minio-only
|
||||||
:end-before: end-replication-minio-only
|
:end-before: end-replication-minio-only
|
||||||
|
|
||||||
Versioning Objects for Bucket Replication
|
Replication Requires Versioning
|
||||||
-----------------------------------------
|
-------------------------------
|
||||||
|
|
||||||
.. include:: /includes/common-replication.rst
|
.. include:: /includes/common-replication.rst
|
||||||
:start-after: start-replication-requires-versioning
|
:start-after: start-replication-requires-versioning
|
||||||
|
@ -283,10 +283,13 @@ Exclude a Prefix From Versioning
|
|||||||
You can exclude certain :ref:`prefixes <minio-admin-concepts-organize-objects>` from versioning using the :ref:`MinIO Client <minio-client>`.
|
You can exclude certain :ref:`prefixes <minio-admin-concepts-organize-objects>` from versioning using the :ref:`MinIO Client <minio-client>`.
|
||||||
This is useful for Spark/Hadoop workloads or others that initially create objects with temporary prefixes.
|
This is useful for Spark/Hadoop workloads or others that initially create objects with temporary prefixes.
|
||||||
|
|
||||||
.. admonition:: Object locking
|
.. admonition:: Replication and Object Locking Require Versioning
|
||||||
:class: note
|
:class: note
|
||||||
|
|
||||||
Buckets with :ref:`object locking enabled <minio-object-locking>` require versioning and do not support excluding prefixes.
|
MinIO requires versioning to support :term:`replication`.
|
||||||
|
Objects in excluded prefixes do not replicate to any peer site or remote site.
|
||||||
|
|
||||||
|
MinIO does not support excluding prefixes from versioning on buckets with :ref:`object locking enabled <minio-object-locking>`.
|
||||||
|
|
||||||
- Use :mc-cmd:`mc version enable` with the :mc-cmd:`~mc version --excluded-prefixes` option:
|
- Use :mc-cmd:`mc version enable` with the :mc-cmd:`~mc version --excluded-prefixes` option:
|
||||||
|
|
||||||
@ -351,6 +354,14 @@ Exclude Folders from Versioning
|
|||||||
|
|
||||||
You can exclude folders from versioning using the :ref:`MinIO Client <minio-client>`.
|
You can exclude folders from versioning using the :ref:`MinIO Client <minio-client>`.
|
||||||
|
|
||||||
|
.. admonition:: Replication and Object Locking Require Versioning
|
||||||
|
:class: note
|
||||||
|
|
||||||
|
MinIO requires versioning to support :term:`replication`.
|
||||||
|
Objects in excluded folders do not replicate to any peer site or remote site.
|
||||||
|
|
||||||
|
MinIO does not support excluding folders from versioning on buckets with :ref:`object locking enabled <minio-object-locking>`.
|
||||||
|
|
||||||
.. admonition:: Object locking
|
.. admonition:: Object locking
|
||||||
:class: note
|
:class: note
|
||||||
|
|
||||||
|
@ -21,14 +21,14 @@
|
|||||||
showgrid="false"
|
showgrid="false"
|
||||||
showguides="true"
|
showguides="true"
|
||||||
inkscape:zoom="3.2234344"
|
inkscape:zoom="3.2234344"
|
||||||
inkscape:cx="254.54217"
|
inkscape:cx="254.8524"
|
||||||
inkscape:cy="216.07389"
|
inkscape:cy="216.38412"
|
||||||
inkscape:window-width="3840"
|
inkscape:window-width="3840"
|
||||||
inkscape:window-height="2303"
|
inkscape:window-height="2123"
|
||||||
inkscape:window-x="3840"
|
inkscape:window-x="3840"
|
||||||
inkscape:window-y="2160"
|
inkscape:window-y="0"
|
||||||
inkscape:window-maximized="1"
|
inkscape:window-maximized="1"
|
||||||
inkscape:current-layer="a">
|
inkscape:current-layer="g9006">
|
||||||
<sodipodi:guide
|
<sodipodi:guide
|
||||||
position="332.87477,220.19"
|
position="332.87477,220.19"
|
||||||
orientation="0,-1"
|
orientation="0,-1"
|
||||||
@ -39,6 +39,11 @@
|
|||||||
orientation="0,-1"
|
orientation="0,-1"
|
||||||
id="guide9033"
|
id="guide9033"
|
||||||
inkscape:locked="false" />
|
inkscape:locked="false" />
|
||||||
|
<sodipodi:guide
|
||||||
|
position="352.17997,282.13143"
|
||||||
|
orientation="0,-1"
|
||||||
|
id="guide1033"
|
||||||
|
inkscape:locked="false" />
|
||||||
</sodipodi:namedview>
|
</sodipodi:namedview>
|
||||||
<defs
|
<defs
|
||||||
id="defs4">
|
id="defs4">
|
||||||
@ -156,8 +161,8 @@
|
|||||||
sodipodi:nodetypes="ccc" />
|
sodipodi:nodetypes="ccc" />
|
||||||
<path
|
<path
|
||||||
style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1.00235px;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:0, 0, 2.0047, 3.00705;stroke-dashoffset:0;stroke-opacity:1"
|
style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1.00235px;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:0, 0, 2.0047, 3.00705;stroke-dashoffset:0;stroke-opacity:1"
|
||||||
d="m 411.33994,131.29656 124.00343,52.08341"
|
d="m 410.89652,130.84283 124.40492,-0.12056 v 52.95659"
|
||||||
id="path9990"
|
id="path9988-3"
|
||||||
sodipodi:nodetypes="ccc" />
|
sodipodi:nodetypes="ccc" />
|
||||||
</g>
|
</g>
|
||||||
<g
|
<g
|
||||||
|
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 55 KiB |
@ -30,6 +30,8 @@ MinIO relies on the immutability protections provided by :ref:`versioning <minio
|
|||||||
Use :mc-cmd:`mc version info` to validate the versioning status of both the source and remote buckets.
|
Use :mc-cmd:`mc version info` to validate the versioning status of both the source and remote buckets.
|
||||||
Use the :mc-cmd:`mc version enable` command to enable versioning as necessary.
|
Use the :mc-cmd:`mc version enable` command to enable versioning as necessary.
|
||||||
|
|
||||||
|
If you exclude a prefix or folder from versioning within the source bucket, MinIO cannot replicate objects within that folder or prefix.
|
||||||
|
|
||||||
.. end-replication-requires-versioning
|
.. end-replication-requires-versioning
|
||||||
|
|
||||||
.. start-replication-requires-object-locking
|
.. start-replication-requires-object-locking
|
||||||
|
@ -121,6 +121,8 @@ Client applications can use any S3-compatible SDK or library to interact with th
|
|||||||
In particular, the complexity of signature calculation typically makes interfacing via ``curl`` or similar REST clients impractical.
|
In particular, the complexity of signature calculation typically makes interfacing via ``curl`` or similar REST clients impractical.
|
||||||
MinIO recommends using S3-compatible SDKs or libraries which perform the signature calculation automatically as part of operations.
|
MinIO recommends using S3-compatible SDKs or libraries which perform the signature calculation automatically as part of operations.
|
||||||
|
|
||||||
|
.. _minio-deployment-architecture-replicated:
|
||||||
|
|
||||||
Replicated MinIO Deployments
|
Replicated MinIO Deployments
|
||||||
----------------------------
|
----------------------------
|
||||||
|
|
||||||
|
@ -12,10 +12,21 @@ Site Replication Overview
|
|||||||
|
|
||||||
Site replication configures multiple independent MinIO deployments as a cluster of replicas called peer sites.
|
Site replication configures multiple independent MinIO deployments as a cluster of replicas called peer sites.
|
||||||
|
|
||||||
|
.. figure:: /images/architecture/architecture-load-balancer-multi-site.svg
|
||||||
|
:figwidth: 100%
|
||||||
|
:alt: Diagram of a site replication deployment with two sites
|
||||||
|
|
||||||
|
A site replication deployment with two peer sites.
|
||||||
|
A load balancer manages routing operations to either of the two sites.
|
||||||
|
Data written to one site automatically replicates to the other peer site.
|
||||||
|
|
||||||
Site replication assumes the use of either the included MinIO identity provider (IDP) *or* an external IDP.
|
Site replication assumes the use of either the included MinIO identity provider (IDP) *or* an external IDP.
|
||||||
All configured deployments must use the same IDP.
|
All configured deployments must use the same IDP.
|
||||||
Deployments using an external IDP must use the same configuration across sites.
|
Deployments using an external IDP must use the same configuration across sites.
|
||||||
|
|
||||||
|
For more information on site replication architecture and deployment concepts, see :ref:`Deployment Architecture: Replicated MinIO Deployments <minio-deployment-architecture-replicated>`.
|
||||||
|
|
||||||
|
|
||||||
Overview
|
Overview
|
||||||
--------
|
--------
|
||||||
|
|
||||||
@ -72,7 +83,6 @@ After enabling site replication, identity and access management (IAM) settings s
|
|||||||
|
|
||||||
After the initial synchronization of data across peer sites, MinIO continually replicates and synchronizes :ref:`replicable data <minio-site-replication-what-replicates>` among all sites as they occur on any site.
|
After the initial synchronization of data across peer sites, MinIO continually replicates and synchronizes :ref:`replicable data <minio-site-replication-what-replicates>` among all sites as they occur on any site.
|
||||||
|
|
||||||
|
|
||||||
Site Healing
|
Site Healing
|
||||||
~~~~~~~~~~~~
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
@ -88,6 +98,17 @@ Any MinIO deployment in the site replication configuration can resynchronize dam
|
|||||||
If one site loses data for any reason, resynchronize the data from another healthy site with :mc-cmd:`mc admin replicate resync`.
|
If one site loses data for any reason, resynchronize the data from another healthy site with :mc-cmd:`mc admin replicate resync`.
|
||||||
This launches an active process that resynchronizes the data without waiting for the passive MinIO scanner to recognize the missing data.
|
This launches an active process that resynchronizes the data without waiting for the passive MinIO scanner to recognize the missing data.
|
||||||
|
|
||||||
|
Synchronous vs Asynchronous Replication
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
.. include:: /includes/common-replication.rst
|
||||||
|
:start-after: start-replication-sync-vs-async
|
||||||
|
:end-before: end-replication-sync-vs-async
|
||||||
|
|
||||||
|
MinIO strongly recommends using the default asynchronous site replication.
|
||||||
|
Synchronous site replication performance depends strongly on latency between sites, where higher latency can result in lower PUT performance and replication lag.
|
||||||
|
To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option.
|
||||||
|
|
||||||
Proxy to Other Sites
|
Proxy to Other Sites
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
@ -139,6 +160,14 @@ For :ref:`SSE-S3 <minio-encryption-sse-s3>` or :ref:`SSE-KMS <minio-encryption-s
|
|||||||
|
|
||||||
You can achieve this with a central KES server or multiple KES servers (say one per site) connected via a central supported :ref:`key vault server <minio-sse>`.
|
You can achieve this with a central KES server or multiple KES servers (say one per site) connected via a central supported :ref:`key vault server <minio-sse>`.
|
||||||
|
|
||||||
|
Replication Requires Versioning
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Site replication *requires* :ref:`minio-bucket-versioning` and enables it for all created buckets automatically.
|
||||||
|
You cannot disable versioning in site replication deployments.
|
||||||
|
|
||||||
|
MinIO cannot replicate objects in prefixes in the bucket that you excluded from versioning.
|
||||||
|
|
||||||
Load Balancers Installed on Each Multi-Node Site
|
Load Balancers Installed on Each Multi-Node Site
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
@ -158,16 +187,6 @@ Use :mc:`mc replicate rm` on the command line or the MinIO Console to remove buc
|
|||||||
Only one site can have data when setting up site replication.
|
Only one site can have data when setting up site replication.
|
||||||
All other sites must be empty.
|
All other sites must be empty.
|
||||||
|
|
||||||
Synchronous vs Asynchronous Replication
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
.. include:: /includes/common-replication.rst
|
|
||||||
:start-after: start-replication-sync-vs-async
|
|
||||||
:end-before: end-replication-sync-vs-async
|
|
||||||
|
|
||||||
MinIO strongly recommends using the default asynchronous site replication.
|
|
||||||
To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option.
|
|
||||||
|
|
||||||
Tutorials
|
Tutorials
|
||||||
---------
|
---------
|
||||||
|
|
||||||
|
Reference in New Issue
Block a user