diff --git a/source/reference/minio-mc/mc-mirror.rst b/source/reference/minio-mc/mc-mirror.rst index aed21485..4b36c4c8 100644 --- a/source/reference/minio-mc/mc-mirror.rst +++ b/source/reference/minio-mc/mc-mirror.rst @@ -15,9 +15,8 @@ Syntax .. start-mc-mirror-desc -The :mc:`mc mirror` command synchronizes content to MinIO deployment, similar to -the ``rsync`` utility. :mc:`mc mirror` supports filesystems, MinIO deployments, -and other S3-compatible hosts as the synchronization source. +The :mc:`mc mirror` command synchronizes content to MinIO deployment, similar to the ``rsync`` utility. +:mc:`mc mirror` supports filesystems, MinIO deployments, and other S3-compatible hosts as the synchronization source. .. end-mc-mirror-desc @@ -25,16 +24,17 @@ and other S3-compatible hosts as the synchronization source. .. tab-item:: EXAMPLE - The following command synchronizes content from a local filesystem - directory to the ``mydata`` bucket on the ``myminio`` MinIO deployment. + The following command synchronizes content from a local filesystem directory to the ``mydata`` bucket on the ``myminio`` MinIO deployment. .. code-block:: shell :class: copyable mc mirror --watch ~/mydata myminio/mydata - The command "watches" the local filesystem and synchronizes any additional - operations to MinIO until explicitly terminated from the terminal: + The command "watches" for files added or removed on the local filesystem and synchronizes those operations to MinIO until explicitly terminated. + + :mc-cmd:`mc mirror --watch` updates files changed on the local filesystem to MinIO (see :mc-cmd:`~mc mirror --overwrite`). + ``--watch`` does not remove other files from MinIO not present on the local filesystem (see :mc-cmd:`~mc mirror --remove`). .. tab-item:: SYNTAX @@ -70,40 +70,31 @@ Parameters .. mc-cmd:: SOURCE - *REQUIRED* The file(s) or object(s) to synchronize to the - :mc-cmd:`~mc mirror TARGET` S3 host. + *REQUIRED* The file(s) or object(s) to synchronize to the :mc-cmd:`~mc mirror TARGET` S3 host. - For objects on S3-compatible hosts, specify the path to the object as - ``ALIAS/PATH``, where: + For objects on S3-compatible hosts, specify the path to the object as ``ALIAS/PATH``, where: - - ``ALIAS`` is the :mc:`alias ` of a configured S3-compatible host, - *and* + - ``ALIAS`` is the :mc:`alias ` of a configured S3-compatible host, *and* - - ``PATH`` is the path to the bucket or object. If specifying a bucket, - :mc:`mc mirror` synchronizes all objects in the bucket. + - ``PATH`` is the path to the bucket or object. If specifying a bucket, :mc:`mc mirror` synchronizes all objects in the bucket. .. code-block:: shell mc mirror [FLAGS] play/mybucket/ myminio/mybucket - For files on a filesystem, specify the full filesystem path to the file or - directory : + For files on a filesystem, specify the full filesystem path to the file or directory : .. code-block:: shell mc mirror [FLAGS] ~/data/ myminio/mybucket - If specifying a directory, :mc-cmd:`mc mirror` synchronizes all files in the - directory. + If specifying a directory, :mc-cmd:`mc mirror` synchronizes all files in the directory. .. mc-cmd:: TARGET - *REQUIRED* The full path to bucket in which :mc:`mc mirror` copies - synchronized SOURCE objects. Specify the ``TARGET`` as - ``ALIAS/PATH``, where: + *REQUIRED* The full path to bucket in which :mc:`mc mirror` copies synchronized SOURCE objects. Specify the ``TARGET`` as ``ALIAS/PATH``, where: - - ``ALIAS`` is the :mc:`alias ` of a configured S3-compatible - host, *and* + - ``ALIAS`` is the :mc:`alias ` of a configured S3-compatible host, *and* - ``PATH`` is the path to the bucket. @@ -111,15 +102,13 @@ Parameters mc mirror SOURCE play/mybucket - :mc:`mc mirror` uses the object or file names from the - :mc-cmd:`~mc mirror SOURCE` when synchronizing to the ``TARGET`` bucket. + :mc:`mc mirror` uses the object or file names from the :mc-cmd:`~mc mirror SOURCE` when synchronizing to the ``TARGET`` bucket. .. mc-cmd:: --attr - Add custom metadata for mirrored objects. Specify key-value pairs as - ``KEY=VALUE\;``. For example, - ``--attr key1=value1\;key2=value2\;key3=value3``. + Add custom metadata for mirrored objects. Specify key-value pairs as ``KEY=VALUE\;``. + For example, ``--attr key1=value1\;key2=value2\;key3=value3``. .. mc-cmd:: --disable-multipart @@ -129,35 +118,27 @@ Parameters .. mc-cmd:: --encrypt-key - Encrypt or decrypt objects using server-side encryption with - client-specified keys. Specify key-value pairs as ``KEY=VALUE``. + Encrypt or decrypt objects using server-side encryption with client-specified keys. + Specify key-value pairs as ``KEY=VALUE``. - Each ``KEY`` represents a bucket or object. - - Each ``VALUE`` represents the data key to use for encrypting - object(s). + - Each ``VALUE`` represents the data key to use for encrypting object(s). - Enclose the entire list of key-value pairs passed to - :mc-cmd:`~mc mirror --encrypt-key` in double quotes ``"``. + Enclose the entire list of key-value pairs passed to :mc-cmd:``~mc mirror --encrypt-key`` in double quotes ``"``. - :mc-cmd:`~mc mirror --encrypt-key` can use the ``MC_ENCRYPT_KEY`` - environment variable for retrieving a list of encryption key-value pairs - as an alternative to specifying them on the command line. + :mc-cmd:`~mc mirror --encrypt-key` can use the ``MC_ENCRYPT_KEY`` environment variable for retrieving a list of encryption key-value pairs as an alternative to specifying them on the command line. - You can only delete encrypted objects if you specify the correct - :mc-cmd:`~mc mirror --encrypt-key` secret key. + You can only delete encrypted objects if you specify the correct :mc-cmd:`~mc mirror --encrypt-key` secret key. .. mc-cmd:: --exclude - Exclude object(s) in the :mc-cmd:`~mc mirror SOURCE` path that - match the specified object name pattern. + Exclude object(s) in the :mc-cmd:`~mc mirror SOURCE` path that match the specified object name pattern. .. mc-cmd:: --fake - Perform a fake mirror operation. Use this operation to perform - validate that the :mc:`mc mirror` operation will only - mirror the desired objects or buckets. + Perform a fake mirror operation. Use this operation to perform validate that the :mc:`mc mirror` operation will only mirror the desired objects or buckets. .. mc-cmd:: md5 @@ -167,22 +148,23 @@ Parameters .. mc-cmd:: --monitoring-address - Creates a `Prometheus `__ endpoint for monitoring - mirroring activity. Specify the local network adapter and port address on - which to create the scraping endpoint. Defaults to ``localhost:8081``). + Creates a `Prometheus `__ endpoint for monitoring mirroring activity. + Specify the local network adapter and port address on which to create the scraping endpoint. + Defaults to ``localhost:8081``). .. mc-cmd:: --newer-than - Mirror object(s) newer than the specified number of days. Specify a - string in ``#d#hh#mm#ss`` format. For example: - ``--older-than 1d2hh3mm4ss``. + Mirror object(s) newer than the specified number of days. + Specify a string in ``#d#hh#mm#ss`` format. + For example: ``--older-than 1d2hh3mm4ss``. .. mc-cmd:: --older-than - Mirror object(s) older than the specified time limit. Specify a string - in ``#d#hh#mm#ss`` format. For example: ``--older-than 1d2hh3mm4ss``. + Mirror object(s) older than the specified time limit. + Specify a string in ``#d#hh#mm#ss`` format. + For example: ``--older-than 1d2hh3mm4ss``. Defaults to ``0`` (all objects). @@ -191,44 +173,70 @@ Parameters Overwrites object(s) on the :mc-cmd:`~mc mirror TARGET`. + For example, consider an active ``mc mirror --overwrite`` synchronizing content from Source to Destination. + + If an object on Source changes, ``my mirror --overwrite`` synchronizes and overwrites any matching file on Destination. + + Without ``--overwrite``, if an object already exists on the Destination, the mirror process fails to synchronize that object. + ``mc mirror`` logs an error and continues to synchronize other objects. + .. mc-cmd:: --preserve, a - Preserve file system attributes and bucket policy rules of the - :mc-cmd:`~mc mirror SOURCE` on the - :mc-cmd:`~mc mirror TARGET`. + Preserve file system attributes and bucket policy rules of the :mc-cmd:`~mc mirror SOURCE` on the :mc-cmd:`~mc mirror TARGET`. .. mc-cmd:: --region - Specify the ``string`` region when creating new bucket(s) on the - target. + Specify the ``string`` region when creating new bucket(s) on the target. Defaults to ``"us-east-1"``. .. mc-cmd:: --remove - Removes extraneous object(s) on the target. + Removes object(s) on the Target that do not exist on the Source. + + Use the ``--remove`` flag to have the same list of objects on both Source and Target. + + For example, objects A, B, and C exist on Source. + Objects C, D, and E exist on Target. + + When running ``mc mirror --remove``, objects A and B copy to Target and objects D and E are removed from Target. + Since an object C already exists on both, nothing moves from Source to Target. + + After the action, only objects A, B, and C exist on both the Source and the Target. + + ``mc mirror --remove`` does not verify that the contents of object C are the same on both Source and Target, only that an object called `C` exists on both. + To ensure objects on the Source and Target match both names `and` content, use :mc-cmd:`~mc mirror --overwrite` or :mc-cmd:`~mc mirror --watch`. .. mc-cmd:: storage-class, sc - Set the storage class for the new object(s) on the - :mc-cmd:`~mc mirror TARGET`. + Set the storage class for the new object(s) on the :mc-cmd:`~mc mirror TARGET`. - See the Amazon documentation on - :aws-docs:`Storage Classes ` - for more information on S3 storage classses. + See the Amazon documentation on :aws-docs:`Storage Classes ` for more information on S3 storage classses. .. mc-cmd:: --watch, w - Continuously monitor the :mc-cmd:`~mc mirror SOURCE` path and - synchronize changes. + Use ``--watch`` flag to mirror objects from Source to Target, where the Target may also have additional objects not present on the Source. + + - ``--watch`` continuously synchronizes files from Source to Target until explicitly terminated + - The Target may have files that do not exist on Source + - ``--watch`` overwrites objects on the Target if a match exists on Source, like the :mc-cmd:`~mc mirror --overwrite` flag Defaults to ``0`` (all objects). + For example, object A and B exist on the watched Source. + Objects A, B, and C exist on the watched Target. + + A client writes object D to Source and removes object B. + + After the operation, objects A and D exist on the Source. + Objects A, C, and D exist on the Target. + + Global Flags ~~~~~~~~~~~~ @@ -249,52 +257,43 @@ Use :mc:`mc mirror` to mirror files from a filesystem to an S3 Host: mc mirror FILEPATH ALIAS/PATH -- Replace :mc-cmd:`FILEPATH ` with the full file path to the - directory to mirror. +- Replace :mc-cmd:`FILEPATH ` with the full file path to the directory to mirror. -- Replace :mc-cmd:`ALIAS ` with the :mc-cmd:`alias ` - of a configured S3-compatible host. +- Replace :mc-cmd:`ALIAS ` with the :mc-cmd:`alias ` of a configured S3-compatible host. - Replace :mc-cmd:`PATH ` with the destination bucket. Continuously Mirror a Local Directory to an S3-Compatible Host ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror -files from a filesystem to an S3-compatible host: +Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror files from a filesystem to an S3-compatible host where objects added to or deleted from the filesystem are added to or deleted from the host: .. code-block:: :class: copyable mc mirror FILEPATH ALIAS/PATH -- Replace :mc-cmd:`FILEPATH ` with the full file path to the - directory to mirror. +- Replace :mc-cmd:`FILEPATH ` with the full file path to the directory to mirror. -- Replace :mc-cmd:`ALIAS ` with the :mc-cmd:`alias ` - of a configured S3-compatible host. +- Replace :mc-cmd:`ALIAS ` with the :mc-cmd:`alias ` of a configured S3-compatible host. - Replace :mc-cmd:`PATH ` with the destination bucket. Continuously Mirror S3 Bucket to an S3-Compatible Host ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously -mirror objects in a bucket on one S3-compatible host to another S3-compatible -host. +Use :mc:`mc mirror` with :mc-cmd:`~mc mirror --watch` to continuously mirror objects in a bucket on one S3-compatible host to another S3-compatible host where objects added to or deleted from the bucket are added to or deleted from the host.f .. code-block:: :class: copyable mc mirror --watch SRCALIAS/SRCPATH TGTALIAS/TGTPATH -- Replace :mc-cmd:`SRCALIAS ` with :mc-cmd:`alias ` - of a configured S3-compatible host. +- Replace :mc-cmd:`SRCALIAS ` with :mc-cmd:`alias ` of a configured S3-compatible host. - Replace :mc-cmd:`SRCPATH ` with the bucket to mirror. -- Replace :mc-cmd:`TGTALIAS ` with the - :mc-cmd:`alias ` of a configured S3-compatible host. +- Replace :mc-cmd:`TGTALIAS ` with the :mc-cmd:`alias ` of a configured S3-compatible host. - Replace :mc-cmd:`TGTPATH ` with the destination bucket. @@ -304,9 +303,11 @@ Behavior MinIO Trims Empty Prefixes on Object Removal ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The :mc-cmd:`mc mirror --watch` command continuously synchronizes the -source and destination targets. This includes automatically removing objects -on the destination if they are removed on the source. +The :mc-cmd:`mc mirror --watch` command continuously synchronizes the source and destination targets for added and deleted objects. +This includes automatically removing objects on the destination if they are removed on the source. + +For objects updated on the source to also update on the target, use `--overwrite`. +To remove objects from the target that are not on the source, use `--remove`. .. |command| replace:: :mc-cmd:`mc mirror --watch`