1
0
mirror of https://github.com/minio/docs.git synced 2025-07-28 19:42:10 +03:00

docs- #377 Adds text to clarify overwrite, remove, and watch flags for mc mirror

Additional cleanup
This commit is contained in:
dwhitelf
2022-03-15 18:24:07 -05:00
committed by Daryl White
parent 64369f6b1d
commit b824e36e65

View File

@ -15,9 +15,8 @@ Syntax
.. start-mc-mirror-desc .. start-mc-mirror-desc
The :mc:`mc mirror` command synchronizes content to MinIO deployment, similar to The :mc:`mc mirror` command synchronizes content to MinIO deployment, similar to the ``rsync`` utility.
the ``rsync`` utility. :mc:`mc mirror` supports filesystems, MinIO deployments, :mc:`mc mirror` supports filesystems, MinIO deployments, and other S3-compatible hosts as the synchronization source.
and other S3-compatible hosts as the synchronization source.
.. end-mc-mirror-desc .. end-mc-mirror-desc
@ -25,16 +24,17 @@ and other S3-compatible hosts as the synchronization source.
.. tab-item:: EXAMPLE .. tab-item:: EXAMPLE
The following command synchronizes content from a local filesystem The following command synchronizes content from a local filesystem directory to the ``mydata`` bucket on the ``myminio`` MinIO deployment.
directory to the ``mydata`` bucket on the ``myminio`` MinIO deployment.
.. code-block:: shell .. code-block:: shell
:class: copyable :class: copyable
mc mirror --watch ~/mydata myminio/mydata mc mirror --watch ~/mydata myminio/mydata
The command "watches" the local filesystem and synchronizes any additional The command "watches" for files added or removed on the local filesystem and synchronizes those operations to MinIO until explicitly terminated.
operations to MinIO until explicitly terminated from the terminal:
:mc-cmd:`mc mirror --watch` updates files changed on the local filesystem to MinIO (see :mc-cmd:`~mc mirror --overwrite`).
``--watch`` does not remove other files from MinIO not present on the local filesystem (see :mc-cmd:`~mc mirror --remove`).
.. tab-item:: SYNTAX .. tab-item:: SYNTAX
@ -70,40 +70,31 @@ Parameters
.. mc-cmd:: SOURCE .. mc-cmd:: SOURCE
*REQUIRED* The file(s) or object(s) to synchronize to the *REQUIRED* The file(s) or object(s) to synchronize to the :mc-cmd:`~mc mirror TARGET` S3 host.
:mc-cmd:`~mc mirror TARGET` S3 host.
For objects on S3-compatible hosts, specify the path to the object as For objects on S3-compatible hosts, specify the path to the object as ``ALIAS/PATH``, where:
``ALIAS/PATH``, where:
- ``ALIAS`` is the :mc:`alias <mc alias>` of a configured S3-compatible host, - ``ALIAS`` is the :mc:`alias <mc alias>` of a configured S3-compatible host, *and*
*and*
- ``PATH`` is the path to the bucket or object. If specifying a bucket, - ``PATH`` is the path to the bucket or object. If specifying a bucket, :mc:`mc mirror` synchronizes all objects in the bucket.
:mc:`mc mirror` synchronizes all objects in the bucket.
.. code-block:: shell .. code-block:: shell
mc mirror [FLAGS] play/mybucket/ myminio/mybucket mc mirror [FLAGS] play/mybucket/ myminio/mybucket
For files on a filesystem, specify the full filesystem path to the file or For files on a filesystem, specify the full filesystem path to the file or directory :
directory :
.. code-block:: shell .. code-block:: shell
mc mirror [FLAGS] ~/data/ myminio/mybucket mc mirror [FLAGS] ~/data/ myminio/mybucket
If specifying a directory, :mc-cmd:`mc mirror` synchronizes all files in the If specifying a directory, :mc-cmd:`mc mirror` synchronizes all files in the directory.
directory.
.. mc-cmd:: TARGET .. mc-cmd:: TARGET
*REQUIRED* The full path to bucket in which :mc:`mc mirror` copies *REQUIRED* The full path to bucket in which :mc:`mc mirror` copies synchronized SOURCE objects. Specify the ``TARGET`` as ``ALIAS/PATH``, where:
synchronized SOURCE objects. Specify the ``TARGET`` as
``ALIAS/PATH``, where:
- ``ALIAS`` is the :mc:`alias <mc alias>` of a configured S3-compatible - ``ALIAS`` is the :mc:`alias <mc alias>` of a configured S3-compatible host, *and*
host, *and*
- ``PATH`` is the path to the bucket. - ``PATH`` is the path to the bucket.
@ -111,15 +102,13 @@ Parameters
mc mirror SOURCE play/mybucket mc mirror SOURCE play/mybucket
:mc:`mc mirror` uses the object or file names from the :mc:`mc mirror` uses the object or file names from the :mc-cmd:`~mc mirror SOURCE` when synchronizing to the ``TARGET`` bucket.
:mc-cmd:`~mc mirror SOURCE` when synchronizing to the ``TARGET`` bucket.
.. mc-cmd:: --attr .. mc-cmd:: --attr
Add custom metadata for mirrored objects. Specify key-value pairs as Add custom metadata for mirrored objects. Specify key-value pairs as ``KEY=VALUE\;``.
``KEY=VALUE\;``. For example, For example, ``--attr key1=value1\;key2=value2\;key3=value3``.
``--attr key1=value1\;key2=value2\;key3=value3``.
.. mc-cmd:: --disable-multipart .. mc-cmd:: --disable-multipart
@ -129,35 +118,27 @@ Parameters
.. mc-cmd:: --encrypt-key .. mc-cmd:: --encrypt-key
Encrypt or decrypt objects using server-side encryption with Encrypt or decrypt objects using server-side encryption with client-specified keys.
client-specified keys. Specify key-value pairs as ``KEY=VALUE``. Specify key-value pairs as ``KEY=VALUE``.
- Each ``KEY`` represents a bucket or object. - Each ``KEY`` represents a bucket or object.
- Each ``VALUE`` represents the data key to use for encrypting - Each ``VALUE`` represents the data key to use for encrypting object(s).
object(s).
Enclose the entire list of key-value pairs passed to Enclose the entire list of key-value pairs passed to :mc-cmd:``~mc mirror --encrypt-key`` in double quotes ``"``.
:mc-cmd:`~mc mirror --encrypt-key` in double quotes ``"``.
:mc-cmd:`~mc mirror --encrypt-key` can use the ``MC_ENCRYPT_KEY`` :mc-cmd:`~mc mirror --encrypt-key` can use the ``MC_ENCRYPT_KEY`` environment variable for retrieving a list of encryption key-value pairs as an alternative to specifying them on the command line.
environment variable for retrieving a list of encryption key-value pairs
as an alternative to specifying them on the command line.
You can only delete encrypted objects if you specify the correct You can only delete encrypted objects if you specify the correct :mc-cmd:`~mc mirror --encrypt-key` secret key.
:mc-cmd:`~mc mirror --encrypt-key` secret key.
.. mc-cmd:: --exclude .. mc-cmd:: --exclude
Exclude object(s) in the :mc-cmd:`~mc mirror SOURCE` path that Exclude object(s) in the :mc-cmd:`~mc mirror SOURCE` path that match the specified object name pattern.
match the specified object name pattern.
.. mc-cmd:: --fake .. mc-cmd:: --fake
Perform a fake mirror operation. Use this operation to perform Perform a fake mirror operation. Use this operation to perform validate that the :mc:`mc mirror` operation will only mirror the desired objects or buckets.
validate that the :mc:`mc mirror` operation will only
mirror the desired objects or buckets.
.. mc-cmd:: md5 .. mc-cmd:: md5
@ -167,22 +148,23 @@ Parameters
.. mc-cmd:: --monitoring-address .. mc-cmd:: --monitoring-address
Creates a `Prometheus <https://prometheus.io/>`__ endpoint for monitoring Creates a `Prometheus <https://prometheus.io/>`__ endpoint for monitoring mirroring activity.
mirroring activity. Specify the local network adapter and port address on Specify the local network adapter and port address on which to create the scraping endpoint.
which to create the scraping endpoint. Defaults to ``localhost:8081``). Defaults to ``localhost:8081``).
.. mc-cmd:: --newer-than .. mc-cmd:: --newer-than
Mirror object(s) newer than the specified number of days. Specify a Mirror object(s) newer than the specified number of days.
string in ``#d#hh#mm#ss`` format. For example: Specify a string in ``#d#hh#mm#ss`` format.
``--older-than 1d2hh3mm4ss``. For example: ``--older-than 1d2hh3mm4ss``.
.. mc-cmd:: --older-than .. mc-cmd:: --older-than
Mirror object(s) older than the specified time limit. Specify a string Mirror object(s) older than the specified time limit.
in ``#d#hh#mm#ss`` format. For example: ``--older-than 1d2hh3mm4ss``. Specify a string in ``#d#hh#mm#ss`` format.
For example: ``--older-than 1d2hh3mm4ss``.
Defaults to ``0`` (all objects). Defaults to ``0`` (all objects).
@ -191,44 +173,70 @@ Parameters
Overwrites object(s) on the :mc-cmd:`~mc mirror TARGET`. Overwrites object(s) on the :mc-cmd:`~mc mirror TARGET`.
For example, consider an active ``mc mirror --overwrite`` synchronizing content from Source to Destination.
If an object on Source changes, ``my mirror --overwrite`` synchronizes and overwrites any matching file on Destination.
Without ``--overwrite``, if an object already exists on the Destination, the mirror process fails to synchronize that object.
``mc mirror`` logs an error and continues to synchronize other objects.
.. mc-cmd:: --preserve, a .. mc-cmd:: --preserve, a
Preserve file system attributes and bucket policy rules of the Preserve file system attributes and bucket policy rules of the :mc-cmd:`~mc mirror SOURCE` on the :mc-cmd:`~mc mirror TARGET`.
:mc-cmd:`~mc mirror SOURCE` on the
:mc-cmd:`~mc mirror TARGET`.
.. mc-cmd:: --region .. mc-cmd:: --region
Specify the ``string`` region when creating new bucket(s) on the Specify the ``string`` region when creating new bucket(s) on the target.
target.
Defaults to ``"us-east-1"``. Defaults to ``"us-east-1"``.
.. mc-cmd:: --remove .. mc-cmd:: --remove
Removes extraneous object(s) on the target. Removes object(s) on the Target that do not exist on the Source.
Use the ``--remove`` flag to have the same list of objects on both Source and Target.
For example, objects A, B, and C exist on Source.
Objects C, D, and E exist on Target.
When running ``mc mirror --remove``, objects A and B copy to Target and objects D and E are removed from Target.
Since an object C already exists on both, nothing moves from Source to Target.
After the action, only objects A, B, and C exist on both the Source and the Target.
``mc mirror --remove`` does not verify that the contents of object C are the same on both Source and Target, only that an object called `C` exists on both.
To ensure objects on the Source and Target match both names `and` content, use :mc-cmd:`~mc mirror --overwrite` or :mc-cmd:`~mc mirror --watch`.
.. mc-cmd:: storage-class, sc .. mc-cmd:: storage-class, sc
Set the storage class for the new object(s) on the Set the storage class for the new object(s) on the :mc-cmd:`~mc mirror TARGET`.
:mc-cmd:`~mc mirror TARGET`.
See the Amazon documentation on See the Amazon documentation on :aws-docs:`Storage Classes <AmazonS3/latest/dev/storage-class-intro.html>` for more information on S3 storage classses.
:aws-docs:`Storage Classes <AmazonS3/latest/dev/storage-class-intro.html>`
for more information on S3 storage classses.
.. mc-cmd:: --watch, w .. mc-cmd:: --watch, w
Continuously monitor the :mc-cmd:`~mc mirror SOURCE` path and Use ``--watch`` flag to mirror objects from Source to Target, where the Target may also have additional objects not present on the Source.
synchronize changes.
- ``--watch`` continuously synchronizes files from Source to Target until explicitly terminated
- The Target may have files that do not exist on Source
- ``--watch`` overwrites objects on the Target if a match exists on Source, like the :mc-cmd:`~mc mirror --overwrite` flag
Defaults to ``0`` (all objects). Defaults to ``0`` (all objects).
For example, object A and B exist on the watched Source.
Objects A, B, and C exist on the watched Target.
A client writes object D to Source and removes object B.
After the operation, objects A and D exist on the Source.
Objects A, C, and D exist on the Target.
Global Flags Global Flags
~~~~~~~~~~~~ ~~~~~~~~~~~~
@ -249,52 +257,43 @@ Use :mc:`mc mirror` to mirror files from a filesystem to an S3 Host:
mc mirror FILEPATH ALIAS/PATH mc mirror FILEPATH ALIAS/PATH
- Replace :mc-cmd:`FILEPATH <mc mirror SOURCE>` with the full file path to the - Replace :mc-cmd:`FILEPATH <mc mirror SOURCE>` with the full file path to the directory to mirror.
directory to mirror.
- Replace :mc-cmd:`ALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>` - Replace :mc-cmd:`ALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
of a configured S3-compatible host.
- Replace :mc-cmd:`PATH <mc mirror TARGET>` with the destination bucket. - Replace :mc-cmd:`PATH <mc mirror TARGET>` with the destination bucket.
Continuously Mirror a Local Directory to an S3-Compatible Host Continuously Mirror a Local Directory to an S3-Compatible Host
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror files from a filesystem to an S3-compatible host where objects added to or deleted from the filesystem are added to or deleted from the host:
files from a filesystem to an S3-compatible host:
.. code-block:: .. code-block::
:class: copyable :class: copyable
mc mirror FILEPATH ALIAS/PATH mc mirror FILEPATH ALIAS/PATH
- Replace :mc-cmd:`FILEPATH <mc mirror SOURCE>` with the full file path to the - Replace :mc-cmd:`FILEPATH <mc mirror SOURCE>` with the full file path to the directory to mirror.
directory to mirror.
- Replace :mc-cmd:`ALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>` - Replace :mc-cmd:`ALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
of a configured S3-compatible host.
- Replace :mc-cmd:`PATH <mc mirror TARGET>` with the destination bucket. - Replace :mc-cmd:`PATH <mc mirror TARGET>` with the destination bucket.
Continuously Mirror S3 Bucket to an S3-Compatible Host Continuously Mirror S3 Bucket to an S3-Compatible Host
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror objects in a bucket on one S3-compatible host to another S3-compatible host where objects added to or deleted from the bucket are added to or deleted from the host.f
mirror objects in a bucket on one S3-compatible host to another S3-compatible
host.
.. code-block:: .. code-block::
:class: copyable :class: copyable
mc mirror --watch SRCALIAS/SRCPATH TGTALIAS/TGTPATH mc mirror --watch SRCALIAS/SRCPATH TGTALIAS/TGTPATH
- Replace :mc-cmd:`SRCALIAS <mc mirror SOURCE>` with :mc-cmd:`alias <mc alias>` - Replace :mc-cmd:`SRCALIAS <mc mirror SOURCE>` with :mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
of a configured S3-compatible host.
- Replace :mc-cmd:`SRCPATH <mc mirror SOURCE>` with the bucket to mirror. - Replace :mc-cmd:`SRCPATH <mc mirror SOURCE>` with the bucket to mirror.
- Replace :mc-cmd:`TGTALIAS <mc mirror TARGET>` with the - Replace :mc-cmd:`TGTALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
:mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
- Replace :mc-cmd:`TGTPATH <mc mirror TARGET>` with the destination bucket. - Replace :mc-cmd:`TGTPATH <mc mirror TARGET>` with the destination bucket.
@ -304,9 +303,11 @@ Behavior
MinIO Trims Empty Prefixes on Object Removal MinIO Trims Empty Prefixes on Object Removal
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The :mc-cmd:`mc mirror --watch` command continuously synchronizes the The :mc-cmd:`mc mirror --watch` command continuously synchronizes the source and destination targets for added and deleted objects.
source and destination targets. This includes automatically removing objects This includes automatically removing objects on the destination if they are removed on the source.
on the destination if they are removed on the source.
For objects updated on the source to also update on the target, use `--overwrite`.
To remove objects from the target that are not on the source, use `--remove`.
.. |command| replace:: :mc-cmd:`mc mirror --watch` .. |command| replace:: :mc-cmd:`mc mirror --watch`