mirror of
https://github.com/minio/docs.git
synced 2025-07-28 19:42:10 +03:00
docs- #377 Adds text to clarify overwrite, remove, and watch flags for mc mirror
Additional cleanup
This commit is contained in:
@ -15,9 +15,8 @@ Syntax
|
|||||||
|
|
||||||
.. start-mc-mirror-desc
|
.. start-mc-mirror-desc
|
||||||
|
|
||||||
The :mc:`mc mirror` command synchronizes content to MinIO deployment, similar to
|
The :mc:`mc mirror` command synchronizes content to MinIO deployment, similar to the ``rsync`` utility.
|
||||||
the ``rsync`` utility. :mc:`mc mirror` supports filesystems, MinIO deployments,
|
:mc:`mc mirror` supports filesystems, MinIO deployments, and other S3-compatible hosts as the synchronization source.
|
||||||
and other S3-compatible hosts as the synchronization source.
|
|
||||||
|
|
||||||
.. end-mc-mirror-desc
|
.. end-mc-mirror-desc
|
||||||
|
|
||||||
@ -25,16 +24,17 @@ and other S3-compatible hosts as the synchronization source.
|
|||||||
|
|
||||||
.. tab-item:: EXAMPLE
|
.. tab-item:: EXAMPLE
|
||||||
|
|
||||||
The following command synchronizes content from a local filesystem
|
The following command synchronizes content from a local filesystem directory to the ``mydata`` bucket on the ``myminio`` MinIO deployment.
|
||||||
directory to the ``mydata`` bucket on the ``myminio`` MinIO deployment.
|
|
||||||
|
|
||||||
.. code-block:: shell
|
.. code-block:: shell
|
||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc mirror --watch ~/mydata myminio/mydata
|
mc mirror --watch ~/mydata myminio/mydata
|
||||||
|
|
||||||
The command "watches" the local filesystem and synchronizes any additional
|
The command "watches" for files added or removed on the local filesystem and synchronizes those operations to MinIO until explicitly terminated.
|
||||||
operations to MinIO until explicitly terminated from the terminal:
|
|
||||||
|
:mc-cmd:`mc mirror --watch` updates files changed on the local filesystem to MinIO (see :mc-cmd:`~mc mirror --overwrite`).
|
||||||
|
``--watch`` does not remove other files from MinIO not present on the local filesystem (see :mc-cmd:`~mc mirror --remove`).
|
||||||
|
|
||||||
.. tab-item:: SYNTAX
|
.. tab-item:: SYNTAX
|
||||||
|
|
||||||
@ -70,40 +70,31 @@ Parameters
|
|||||||
|
|
||||||
.. mc-cmd:: SOURCE
|
.. mc-cmd:: SOURCE
|
||||||
|
|
||||||
*REQUIRED* The file(s) or object(s) to synchronize to the
|
*REQUIRED* The file(s) or object(s) to synchronize to the :mc-cmd:`~mc mirror TARGET` S3 host.
|
||||||
:mc-cmd:`~mc mirror TARGET` S3 host.
|
|
||||||
|
|
||||||
For objects on S3-compatible hosts, specify the path to the object as
|
For objects on S3-compatible hosts, specify the path to the object as ``ALIAS/PATH``, where:
|
||||||
``ALIAS/PATH``, where:
|
|
||||||
|
|
||||||
- ``ALIAS`` is the :mc:`alias <mc alias>` of a configured S3-compatible host,
|
- ``ALIAS`` is the :mc:`alias <mc alias>` of a configured S3-compatible host, *and*
|
||||||
*and*
|
|
||||||
|
|
||||||
- ``PATH`` is the path to the bucket or object. If specifying a bucket,
|
- ``PATH`` is the path to the bucket or object. If specifying a bucket, :mc:`mc mirror` synchronizes all objects in the bucket.
|
||||||
:mc:`mc mirror` synchronizes all objects in the bucket.
|
|
||||||
|
|
||||||
.. code-block:: shell
|
.. code-block:: shell
|
||||||
|
|
||||||
mc mirror [FLAGS] play/mybucket/ myminio/mybucket
|
mc mirror [FLAGS] play/mybucket/ myminio/mybucket
|
||||||
|
|
||||||
For files on a filesystem, specify the full filesystem path to the file or
|
For files on a filesystem, specify the full filesystem path to the file or directory :
|
||||||
directory :
|
|
||||||
|
|
||||||
.. code-block:: shell
|
.. code-block:: shell
|
||||||
|
|
||||||
mc mirror [FLAGS] ~/data/ myminio/mybucket
|
mc mirror [FLAGS] ~/data/ myminio/mybucket
|
||||||
|
|
||||||
If specifying a directory, :mc-cmd:`mc mirror` synchronizes all files in the
|
If specifying a directory, :mc-cmd:`mc mirror` synchronizes all files in the directory.
|
||||||
directory.
|
|
||||||
|
|
||||||
.. mc-cmd:: TARGET
|
.. mc-cmd:: TARGET
|
||||||
|
|
||||||
*REQUIRED* The full path to bucket in which :mc:`mc mirror` copies
|
*REQUIRED* The full path to bucket in which :mc:`mc mirror` copies synchronized SOURCE objects. Specify the ``TARGET`` as ``ALIAS/PATH``, where:
|
||||||
synchronized SOURCE objects. Specify the ``TARGET`` as
|
|
||||||
``ALIAS/PATH``, where:
|
|
||||||
|
|
||||||
- ``ALIAS`` is the :mc:`alias <mc alias>` of a configured S3-compatible
|
- ``ALIAS`` is the :mc:`alias <mc alias>` of a configured S3-compatible host, *and*
|
||||||
host, *and*
|
|
||||||
|
|
||||||
- ``PATH`` is the path to the bucket.
|
- ``PATH`` is the path to the bucket.
|
||||||
|
|
||||||
@ -111,15 +102,13 @@ Parameters
|
|||||||
|
|
||||||
mc mirror SOURCE play/mybucket
|
mc mirror SOURCE play/mybucket
|
||||||
|
|
||||||
:mc:`mc mirror` uses the object or file names from the
|
:mc:`mc mirror` uses the object or file names from the :mc-cmd:`~mc mirror SOURCE` when synchronizing to the ``TARGET`` bucket.
|
||||||
:mc-cmd:`~mc mirror SOURCE` when synchronizing to the ``TARGET`` bucket.
|
|
||||||
|
|
||||||
.. mc-cmd:: --attr
|
.. mc-cmd:: --attr
|
||||||
|
|
||||||
|
|
||||||
Add custom metadata for mirrored objects. Specify key-value pairs as
|
Add custom metadata for mirrored objects. Specify key-value pairs as ``KEY=VALUE\;``.
|
||||||
``KEY=VALUE\;``. For example,
|
For example, ``--attr key1=value1\;key2=value2\;key3=value3``.
|
||||||
``--attr key1=value1\;key2=value2\;key3=value3``.
|
|
||||||
|
|
||||||
.. mc-cmd:: --disable-multipart
|
.. mc-cmd:: --disable-multipart
|
||||||
|
|
||||||
@ -129,35 +118,27 @@ Parameters
|
|||||||
.. mc-cmd:: --encrypt-key
|
.. mc-cmd:: --encrypt-key
|
||||||
|
|
||||||
|
|
||||||
Encrypt or decrypt objects using server-side encryption with
|
Encrypt or decrypt objects using server-side encryption with client-specified keys.
|
||||||
client-specified keys. Specify key-value pairs as ``KEY=VALUE``.
|
Specify key-value pairs as ``KEY=VALUE``.
|
||||||
|
|
||||||
- Each ``KEY`` represents a bucket or object.
|
- Each ``KEY`` represents a bucket or object.
|
||||||
- Each ``VALUE`` represents the data key to use for encrypting
|
- Each ``VALUE`` represents the data key to use for encrypting object(s).
|
||||||
object(s).
|
|
||||||
|
|
||||||
Enclose the entire list of key-value pairs passed to
|
Enclose the entire list of key-value pairs passed to :mc-cmd:``~mc mirror --encrypt-key`` in double quotes ``"``.
|
||||||
:mc-cmd:`~mc mirror --encrypt-key` in double quotes ``"``.
|
|
||||||
|
|
||||||
:mc-cmd:`~mc mirror --encrypt-key` can use the ``MC_ENCRYPT_KEY``
|
:mc-cmd:`~mc mirror --encrypt-key` can use the ``MC_ENCRYPT_KEY`` environment variable for retrieving a list of encryption key-value pairs as an alternative to specifying them on the command line.
|
||||||
environment variable for retrieving a list of encryption key-value pairs
|
|
||||||
as an alternative to specifying them on the command line.
|
|
||||||
|
|
||||||
You can only delete encrypted objects if you specify the correct
|
You can only delete encrypted objects if you specify the correct :mc-cmd:`~mc mirror --encrypt-key` secret key.
|
||||||
:mc-cmd:`~mc mirror --encrypt-key` secret key.
|
|
||||||
|
|
||||||
.. mc-cmd:: --exclude
|
.. mc-cmd:: --exclude
|
||||||
|
|
||||||
|
|
||||||
Exclude object(s) in the :mc-cmd:`~mc mirror SOURCE` path that
|
Exclude object(s) in the :mc-cmd:`~mc mirror SOURCE` path that match the specified object name pattern.
|
||||||
match the specified object name pattern.
|
|
||||||
|
|
||||||
.. mc-cmd:: --fake
|
.. mc-cmd:: --fake
|
||||||
|
|
||||||
|
|
||||||
Perform a fake mirror operation. Use this operation to perform
|
Perform a fake mirror operation. Use this operation to perform validate that the :mc:`mc mirror` operation will only mirror the desired objects or buckets.
|
||||||
validate that the :mc:`mc mirror` operation will only
|
|
||||||
mirror the desired objects or buckets.
|
|
||||||
|
|
||||||
.. mc-cmd:: md5
|
.. mc-cmd:: md5
|
||||||
|
|
||||||
@ -167,22 +148,23 @@ Parameters
|
|||||||
.. mc-cmd:: --monitoring-address
|
.. mc-cmd:: --monitoring-address
|
||||||
|
|
||||||
|
|
||||||
Creates a `Prometheus <https://prometheus.io/>`__ endpoint for monitoring
|
Creates a `Prometheus <https://prometheus.io/>`__ endpoint for monitoring mirroring activity.
|
||||||
mirroring activity. Specify the local network adapter and port address on
|
Specify the local network adapter and port address on which to create the scraping endpoint.
|
||||||
which to create the scraping endpoint. Defaults to ``localhost:8081``).
|
Defaults to ``localhost:8081``).
|
||||||
|
|
||||||
.. mc-cmd:: --newer-than
|
.. mc-cmd:: --newer-than
|
||||||
|
|
||||||
|
|
||||||
Mirror object(s) newer than the specified number of days. Specify a
|
Mirror object(s) newer than the specified number of days.
|
||||||
string in ``#d#hh#mm#ss`` format. For example:
|
Specify a string in ``#d#hh#mm#ss`` format.
|
||||||
``--older-than 1d2hh3mm4ss``.
|
For example: ``--older-than 1d2hh3mm4ss``.
|
||||||
|
|
||||||
.. mc-cmd:: --older-than
|
.. mc-cmd:: --older-than
|
||||||
|
|
||||||
|
|
||||||
Mirror object(s) older than the specified time limit. Specify a string
|
Mirror object(s) older than the specified time limit.
|
||||||
in ``#d#hh#mm#ss`` format. For example: ``--older-than 1d2hh3mm4ss``.
|
Specify a string in ``#d#hh#mm#ss`` format.
|
||||||
|
For example: ``--older-than 1d2hh3mm4ss``.
|
||||||
|
|
||||||
Defaults to ``0`` (all objects).
|
Defaults to ``0`` (all objects).
|
||||||
|
|
||||||
@ -191,44 +173,70 @@ Parameters
|
|||||||
|
|
||||||
Overwrites object(s) on the :mc-cmd:`~mc mirror TARGET`.
|
Overwrites object(s) on the :mc-cmd:`~mc mirror TARGET`.
|
||||||
|
|
||||||
|
For example, consider an active ``mc mirror --overwrite`` synchronizing content from Source to Destination.
|
||||||
|
|
||||||
|
If an object on Source changes, ``my mirror --overwrite`` synchronizes and overwrites any matching file on Destination.
|
||||||
|
|
||||||
|
Without ``--overwrite``, if an object already exists on the Destination, the mirror process fails to synchronize that object.
|
||||||
|
``mc mirror`` logs an error and continues to synchronize other objects.
|
||||||
|
|
||||||
.. mc-cmd:: --preserve, a
|
.. mc-cmd:: --preserve, a
|
||||||
|
|
||||||
|
|
||||||
Preserve file system attributes and bucket policy rules of the
|
Preserve file system attributes and bucket policy rules of the :mc-cmd:`~mc mirror SOURCE` on the :mc-cmd:`~mc mirror TARGET`.
|
||||||
:mc-cmd:`~mc mirror SOURCE` on the
|
|
||||||
:mc-cmd:`~mc mirror TARGET`.
|
|
||||||
|
|
||||||
.. mc-cmd:: --region
|
.. mc-cmd:: --region
|
||||||
|
|
||||||
|
|
||||||
Specify the ``string`` region when creating new bucket(s) on the
|
Specify the ``string`` region when creating new bucket(s) on the target.
|
||||||
target.
|
|
||||||
|
|
||||||
Defaults to ``"us-east-1"``.
|
Defaults to ``"us-east-1"``.
|
||||||
|
|
||||||
.. mc-cmd:: --remove
|
.. mc-cmd:: --remove
|
||||||
|
|
||||||
|
|
||||||
Removes extraneous object(s) on the target.
|
Removes object(s) on the Target that do not exist on the Source.
|
||||||
|
|
||||||
|
Use the ``--remove`` flag to have the same list of objects on both Source and Target.
|
||||||
|
|
||||||
|
For example, objects A, B, and C exist on Source.
|
||||||
|
Objects C, D, and E exist on Target.
|
||||||
|
|
||||||
|
When running ``mc mirror --remove``, objects A and B copy to Target and objects D and E are removed from Target.
|
||||||
|
Since an object C already exists on both, nothing moves from Source to Target.
|
||||||
|
|
||||||
|
After the action, only objects A, B, and C exist on both the Source and the Target.
|
||||||
|
|
||||||
|
``mc mirror --remove`` does not verify that the contents of object C are the same on both Source and Target, only that an object called `C` exists on both.
|
||||||
|
To ensure objects on the Source and Target match both names `and` content, use :mc-cmd:`~mc mirror --overwrite` or :mc-cmd:`~mc mirror --watch`.
|
||||||
|
|
||||||
.. mc-cmd:: storage-class, sc
|
.. mc-cmd:: storage-class, sc
|
||||||
|
|
||||||
|
|
||||||
Set the storage class for the new object(s) on the
|
Set the storage class for the new object(s) on the :mc-cmd:`~mc mirror TARGET`.
|
||||||
:mc-cmd:`~mc mirror TARGET`.
|
|
||||||
|
|
||||||
See the Amazon documentation on
|
See the Amazon documentation on :aws-docs:`Storage Classes <AmazonS3/latest/dev/storage-class-intro.html>` for more information on S3 storage classses.
|
||||||
:aws-docs:`Storage Classes <AmazonS3/latest/dev/storage-class-intro.html>`
|
|
||||||
for more information on S3 storage classses.
|
|
||||||
|
|
||||||
.. mc-cmd:: --watch, w
|
.. mc-cmd:: --watch, w
|
||||||
|
|
||||||
|
|
||||||
Continuously monitor the :mc-cmd:`~mc mirror SOURCE` path and
|
Use ``--watch`` flag to mirror objects from Source to Target, where the Target may also have additional objects not present on the Source.
|
||||||
synchronize changes.
|
|
||||||
|
- ``--watch`` continuously synchronizes files from Source to Target until explicitly terminated
|
||||||
|
- The Target may have files that do not exist on Source
|
||||||
|
- ``--watch`` overwrites objects on the Target if a match exists on Source, like the :mc-cmd:`~mc mirror --overwrite` flag
|
||||||
|
|
||||||
Defaults to ``0`` (all objects).
|
Defaults to ``0`` (all objects).
|
||||||
|
|
||||||
|
For example, object A and B exist on the watched Source.
|
||||||
|
Objects A, B, and C exist on the watched Target.
|
||||||
|
|
||||||
|
A client writes object D to Source and removes object B.
|
||||||
|
|
||||||
|
After the operation, objects A and D exist on the Source.
|
||||||
|
Objects A, C, and D exist on the Target.
|
||||||
|
|
||||||
|
|
||||||
Global Flags
|
Global Flags
|
||||||
~~~~~~~~~~~~
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
@ -249,52 +257,43 @@ Use :mc:`mc mirror` to mirror files from a filesystem to an S3 Host:
|
|||||||
|
|
||||||
mc mirror FILEPATH ALIAS/PATH
|
mc mirror FILEPATH ALIAS/PATH
|
||||||
|
|
||||||
- Replace :mc-cmd:`FILEPATH <mc mirror SOURCE>` with the full file path to the
|
- Replace :mc-cmd:`FILEPATH <mc mirror SOURCE>` with the full file path to the directory to mirror.
|
||||||
directory to mirror.
|
|
||||||
|
|
||||||
- Replace :mc-cmd:`ALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>`
|
- Replace :mc-cmd:`ALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
|
||||||
of a configured S3-compatible host.
|
|
||||||
|
|
||||||
- Replace :mc-cmd:`PATH <mc mirror TARGET>` with the destination bucket.
|
- Replace :mc-cmd:`PATH <mc mirror TARGET>` with the destination bucket.
|
||||||
|
|
||||||
Continuously Mirror a Local Directory to an S3-Compatible Host
|
Continuously Mirror a Local Directory to an S3-Compatible Host
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror
|
Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror files from a filesystem to an S3-compatible host where objects added to or deleted from the filesystem are added to or deleted from the host:
|
||||||
files from a filesystem to an S3-compatible host:
|
|
||||||
|
|
||||||
.. code-block::
|
.. code-block::
|
||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc mirror FILEPATH ALIAS/PATH
|
mc mirror FILEPATH ALIAS/PATH
|
||||||
|
|
||||||
- Replace :mc-cmd:`FILEPATH <mc mirror SOURCE>` with the full file path to the
|
- Replace :mc-cmd:`FILEPATH <mc mirror SOURCE>` with the full file path to the directory to mirror.
|
||||||
directory to mirror.
|
|
||||||
|
|
||||||
- Replace :mc-cmd:`ALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>`
|
- Replace :mc-cmd:`ALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
|
||||||
of a configured S3-compatible host.
|
|
||||||
|
|
||||||
- Replace :mc-cmd:`PATH <mc mirror TARGET>` with the destination bucket.
|
- Replace :mc-cmd:`PATH <mc mirror TARGET>` with the destination bucket.
|
||||||
|
|
||||||
Continuously Mirror S3 Bucket to an S3-Compatible Host
|
Continuously Mirror S3 Bucket to an S3-Compatible Host
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously
|
Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror objects in a bucket on one S3-compatible host to another S3-compatible host where objects added to or deleted from the bucket are added to or deleted from the host.f
|
||||||
mirror objects in a bucket on one S3-compatible host to another S3-compatible
|
|
||||||
host.
|
|
||||||
|
|
||||||
.. code-block::
|
.. code-block::
|
||||||
:class: copyable
|
:class: copyable
|
||||||
|
|
||||||
mc mirror --watch SRCALIAS/SRCPATH TGTALIAS/TGTPATH
|
mc mirror --watch SRCALIAS/SRCPATH TGTALIAS/TGTPATH
|
||||||
|
|
||||||
- Replace :mc-cmd:`SRCALIAS <mc mirror SOURCE>` with :mc-cmd:`alias <mc alias>`
|
- Replace :mc-cmd:`SRCALIAS <mc mirror SOURCE>` with :mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
|
||||||
of a configured S3-compatible host.
|
|
||||||
|
|
||||||
- Replace :mc-cmd:`SRCPATH <mc mirror SOURCE>` with the bucket to mirror.
|
- Replace :mc-cmd:`SRCPATH <mc mirror SOURCE>` with the bucket to mirror.
|
||||||
|
|
||||||
- Replace :mc-cmd:`TGTALIAS <mc mirror TARGET>` with the
|
- Replace :mc-cmd:`TGTALIAS <mc mirror TARGET>` with the :mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
|
||||||
:mc-cmd:`alias <mc alias>` of a configured S3-compatible host.
|
|
||||||
|
|
||||||
- Replace :mc-cmd:`TGTPATH <mc mirror TARGET>` with the destination bucket.
|
- Replace :mc-cmd:`TGTPATH <mc mirror TARGET>` with the destination bucket.
|
||||||
|
|
||||||
@ -304,9 +303,11 @@ Behavior
|
|||||||
MinIO Trims Empty Prefixes on Object Removal
|
MinIO Trims Empty Prefixes on Object Removal
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The :mc-cmd:`mc mirror --watch` command continuously synchronizes the
|
The :mc-cmd:`mc mirror --watch` command continuously synchronizes the source and destination targets for added and deleted objects.
|
||||||
source and destination targets. This includes automatically removing objects
|
This includes automatically removing objects on the destination if they are removed on the source.
|
||||||
on the destination if they are removed on the source.
|
|
||||||
|
For objects updated on the source to also update on the target, use `--overwrite`.
|
||||||
|
To remove objects from the target that are not on the source, use `--remove`.
|
||||||
|
|
||||||
.. |command| replace:: :mc-cmd:`mc mirror --watch`
|
.. |command| replace:: :mc-cmd:`mc mirror --watch`
|
||||||
|
|
||||||
|
Reference in New Issue
Block a user