diff --git a/source/administration/batch-framework.rst b/source/administration/batch-framework.rst new file mode 100644 index 00000000..37e4750d --- /dev/null +++ b/source/administration/batch-framework.rst @@ -0,0 +1,158 @@ +.. _minio-batch-framework: + +=============== +Batch Framework +=============== + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + +.. versionadded:: MinIO RELEASE.2022-10-08T20-11-00Z + + The Batch Framework was introduced with the ``replicate`` job type in the :mc:`mc` RELEASES.2022-10-08T20-11-00Z. + +Overview +-------- + +The MinIO Batch Framework allows you to create, manage, monitor, and execute jobs using a YAML-formatted job definition file (a "batch file"). +The batch jobs run directly on the MinIO deployment to take advantage of the server-side processing power without constraints of the local machine where you run the :ref:`MinIO Client `. + +A batch file can define any number of job tasks. +MinIO does not limit on the number of job tasks that you can define in a batch file. + +Once started, MinIO starts processing the job. +Time to completion depends on the resources available to the deployment. + +If any portion of the job fails, MinIO retries the job up to the number of times defined in the job definition. + +The MinIO Batch Framework supports the following job types: + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + :width: 100% + + * - Job Type + - Description + + * - ``replicate`` + - Perform a one-time replication procedure from one MinIO location to another MinIO location. + +MinIO Batch CLI +--------------- + +- Install the :ref:`MinIO Client ` +- Define an :mc-cmd:`alias ` for the MinIO deployment + +The :mc:`mc batch` commands include + +.. list-table:: + :widths: 30 70 + :width: 90% + + * - :mc:`mc batch generate` + - .. include:: /reference/minio-mc/mc-batch-generate.rst + :start-after: start-mc-batch-generate-desc + :end-before: end-mc-batch-generate-desc + * - :mc:`mc batch start` + - .. include:: /reference/minio-mc/mc-batch-start.rst + :start-after: start-mc-batch-start-desc + :end-before: end-mc-batch-start-desc + * - :mc:`mc batch list` + - .. include:: /reference/minio-mc/mc-batch-list.rst + :start-after: start-mc-batch-list-desc + :end-before: end-mc-batch-list-desc + * - :mc:`mc batch status` + - .. include:: /reference/minio-mc/mc-batch-status.rst + :start-after: start-mc-batch-status-desc + :end-before: end-mc-batch-status-desc + * - :mc:`mc batch describe` + - .. include:: /reference/minio-mc/mc-batch-describe.rst + :start-after: start-mc-batch-describe-desc + :end-before: end-mc-batch-describe-desc + +Access to ``mc batch`` +---------------------- + +You can use MinIO's :ref:`Policy Based Access Control ` and the :ref:`administrative policy actions ` to restrict who can start a batch job, retrieve a list of running jobs, or describe a running job. + +Job Types +--------- + +``replicate`` +~~~~~~~~~~~~~ + +The ``replicate`` job type performs a single-run replication of objects from one MinIO deployment to another. +The definition file can limit the replication by bucket, prefix, and/or filters to only replicate certain objects. + +For example, you can use a batch job to perform a one-time replication sync of objects from ``minio-alpha/invoices/`` to ``minio-baker/invoices``. + +The advantages of Batch Replication over :mc:`mc mirror` include: + +- Removes the client to cluster network as a potential bottleneck +- A user only needs access to starting a batch job with no other permissions, as the job runs entirely server side on the cluster +- The job provides for retry attempts in event that objects do not replicate +- Batch jobs are one-time, curated processes allowing for fine control replication + +Sample YAML Description File for a ``replicate`` Job Type ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +Create a basic ``replicate`` job definition file you can edit with :mc-cmd:`mc batch generate`. + +.. code-block:: yaml + + replicate: + apiVersion: v1 + # source of the objects to be replicated + source: + type: TYPE # valid values are "s3" + bucket: BUCKET + prefix: PREFIX + # NOTE: if source is remote then target must be "local" + # endpoint: ENDPOINT + # credentials: + # accessKey: ACCESS-KEY + # secretKey: SECRET-KEY + # sessionToken: SESSION-TOKEN # Available when rotating credentials are used + + # target where the objects must be replicated + target: + type: TYPE # valid values are "s3" + bucket: BUCKET + prefix: PREFIX + # NOTE: if target is remote then source must be "local" + # endpoint: ENDPOINT + # credentials: + # accessKey: ACCESS-KEY + # secretKey: SECRET-KEY + # sessionToken: SESSION-TOKEN # Available when rotating credentials are used + + # optional flags based filtering criteria + # for all source objects + flags: + filter: + newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s) + olderThan: "7d" # match objects older than this value (e.g. 7d10h31s) + createdAfter: "date" # match objects created after "date" + createdBefore: "date" # match objects created before "date" + + ## NOTE: tags are not supported when "source" is remote. + # tags: + # - key: "name" + # value: "pick*" # match objects with tag 'name', with all values starting with 'pick' + + ## NOTE: metadata filter not supported when "source" is non MinIO. + # metadata: + # - key: "content-type" + # value: "image/*" # match objects with 'content-type', with all values starting with 'image/' + + notify: + endpoint: "https://notify.endpoint" # notification endpoint to receive job status events + token: "Bearer xxxxx" # optional authentication token for the notification endpoint + + retry: + attempts: 10 # number of retries for the job before giving up + delay: "500ms" # least amount of delay between each retry \ No newline at end of file diff --git a/source/administration/identity-access-management/policy-based-access-control.rst b/source/administration/identity-access-management/policy-based-access-control.rst index 876eb211..e794ce3b 100644 --- a/source/administration/identity-access-management/policy-based-access-control.rst +++ b/source/administration/identity-access-management/policy-based-access-control.rst @@ -761,6 +761,18 @@ services: Allows access to MinIO :ref:`metrics `. Only required if MinIO requires authentication for scraping metrics. +.. policy-action:: admin:ListBatchJobs + + Allows access to list the active batch jobs. + +.. policy-action:: admin:DescribeBatchJobs + + Allows access to the see the definition details of a running batch job. + +.. policy-action:: admin:StartBatchJob + + Allows user to begin a batch job run. + ``mc admin`` Policy Condition Keys ---------------------------------- diff --git a/source/index.rst b/source/index.rst index c07a71bb..00d5325b 100644 --- a/source/index.rst +++ b/source/index.rst @@ -94,6 +94,7 @@ and non-protected. /administration/identity-access-management /administration/server-side-encryption /administration/bucket-replication + /administration/batch-framework /administration/concepts @@ -119,6 +120,7 @@ and non-protected. /administration/identity-access-management /administration/server-side-encryption /administration/bucket-replication + /administration/batch-framework /administration/concepts .. cond:: k8s or openshift or container or macos or windows diff --git a/source/reference/minio-mc.rst b/source/reference/minio-mc.rst index 60ae846b..cdb97baa 100644 --- a/source/reference/minio-mc.rst +++ b/source/reference/minio-mc.rst @@ -174,6 +174,16 @@ The following table lists :mc-cmd:`mc` commands: action granted by the policy without :ref:`authentication `. + * - | :mc:`mc batch generate` + | :mc:`mc batch start` + | :mc:`mc batch list` + | :mc:`mc batch status` + | :mc:`mc batch describe` + - The ``mc batch`` commands allow you to run one or more job tasks on a MinIO deployment. + Specify the jobs to run in a YAML-formatted file you can start generating with :mc:`mc batch generate`. + Use :mc:`mc batch start` to when you are ready to begin the job(s) defined in the file. + Display current jobs with :mc:`mc batch list`, see current status with :mc:`mc batch status`, and output the full definition for a current job with :mc:`mc batch describe`. + * - :mc:`mc cat` - .. include:: /reference/minio-mc/mc-cat.rst :start-after: start-mc-cat-desc @@ -471,6 +481,11 @@ All :ref:`commands ` support the following global options: /reference/minio-mc/mc-anonymous-links /reference/minio-mc/mc-anonymous-get-json /reference/minio-mc/mc-anonymous-set-json + /reference/minio-mc/mc-batch-describe + /reference/minio-mc/mc-batch-generate + /reference/minio-mc/mc-batch-list + /reference/minio-mc/mc-batch-start + /reference/minio-mc/mc-batch-status /reference/minio-mc/mc-cat /reference/minio-mc/mc-cp /reference/minio-mc/mc-diff diff --git a/source/reference/minio-mc/mc-batch-describe.rst b/source/reference/minio-mc/mc-batch-describe.rst new file mode 100644 index 00000000..6e2130c3 --- /dev/null +++ b/source/reference/minio-mc/mc-batch-describe.rst @@ -0,0 +1,108 @@ +.. _minio-mc-batch-describe: + +===================== +``mc batch describe`` +===================== + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + +.. mc:: mc batch describe + +.. versionchanged:: MinIO RELEASE.2022-10-08T20-11-00Z or later + +Syntax +------ + +.. start-mc-batch-describe-desc + +The :mc:`mc batch describe` command outputs the job definition for a specified job ID. + +.. end-mc-batch-describe-desc + +You must specify the job ID. +To find the job ID, use :mc-cmd:`mc batch list`. + +.. tab-set:: + + .. tab-item:: EXAMPLE + + The following command outputs the job definition for the job identified as ``KwSysDpxcBU9FNhGkn2dCf``. + + .. code-block:: shell + :class: copyable + + mc batch describe myminio KwSysDpxcBU9FNhGkn2dCf + + .. tab-item:: SYNTAX + + The command has the following syntax: + + .. code-block:: shell + :class: copyable + + mc [GLOBALFLAGS] batch describe TARGET \ + JOBID + + .. include:: /includes/common-minio-mc.rst + :start-after: start-minio-syntax + :end-before: end-minio-syntax + +Parameters +~~~~~~~~~~ + +.. mc-cmd:: TARGET + :required: + + The :ref:`alias ` for the MinIO deployment to look for the Job ID. + +.. mc-cmd:: JOBID + :required: + + The unique identifier of a job to describe. + To find the ID of a job, use :mc:`mc batch list`. + +Global Flags +~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-globals + :end-before: end-minio-mc-globals + +Example +------- + +Show the Definition of an In Progress Batch Job +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following command provides the full job definition of a specific job at :mc:`alias ` ``myminio``: + +.. code-block:: shell + :class: copyable + + mc batch describe myminio KwSysDpxcBU9FNhGkn2dCf + +- Replace ``myminio`` with the :mc:`alias ` of the MinIO deployment that should run the job. +- Replace ``KwSysDpxcBU9FNhGkn2dCf`` with the ID of the job to define. + +The output of the above command is similar to the following: + +.. code-block:: shell + + mc batch describe myminio KwSysDpxcBU9FNhGkn2dCf + replicate: + apiVersion: v1 + ... + +Note, this example is truncated. +The output is the full job definition for the specified job. + +S3 Compatibility +~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-s3-compatibility + :end-before: end-minio-mc-s3-compatibility diff --git a/source/reference/minio-mc/mc-batch-generate.rst b/source/reference/minio-mc/mc-batch-generate.rst new file mode 100644 index 00000000..00b2c831 --- /dev/null +++ b/source/reference/minio-mc/mc-batch-generate.rst @@ -0,0 +1,309 @@ +.. _minio-mc-batch-generate: + +===================== +``mc batch generate`` +===================== + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + +.. mc:: mc batch + +.. mc:: mc batch generate + +.. versionchanged:: MinIO RELEASE.2022-10-08T20-11-00Z or later + +Syntax +------ + +.. start-mc-batch-generate-desc + +The :mc:`mc batch generate` command creates a basic YAML-formatted template file for the specified job type. + +.. end-mc-batch-generate-desc + +After MinIO creates the file, open it in your preferred text editor tool to further customize. + +- You can add multiple job task definitions to a batch file. +- MinIO imposes no limit to the number of job task definitions per batch file. + +See :ref:`job types ` for the supported jobs you can generate. + +.. tab-set:: + + .. tab-item:: EXAMPLE + + The following command creates a basic YAML file for a replicate job on the ``mybucket`` bucket of the ``myminio`` alias. + + .. code-block:: shell + :class: copyable + + mc batch generate myminio/mybucket replicate + + .. tab-item:: SYNTAX + + The command has the following syntax: + + .. code-block:: shell + :class: copyable + + mc [GLOBALFLAGS] batch generate \ + TARGET \ + JOBTYPE + + .. include:: /includes/common-minio-mc.rst + :start-after: start-minio-syntax + :end-before: end-minio-syntax + +Parameters +~~~~~~~~~~ + +.. mc-cmd:: TARGET + :required: + + The :ref:`alias ` and bucket on the MinIO deployment to which to add the object lifecycle management rule. + + For example: + + .. code-block:: none + + mc batch generate myminio/mydata replicate + + You can use just an alias without specifying a bucket to perform the job at the deployment's root. + +.. mc-cmd:: JOBTYPE + :required: + + The type of job to generate a YAML document for. + + Currently, :mc:`mc batch` only supports the ``replicate`` job type. + + +Global Flags +~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-globals + :end-before: end-minio-mc-globals + +Examples +-------- + +Generate a ``yaml`` File for a Replicate Job Type +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following command generates a YAML blueprint for a replicate type batch job and names the file ``replicate`` with the ``.yaml`` extension: + +.. code-block:: shell + :class: copyable + + mc batch generate alias/ replicate + +- Replace ``alias`` with the :mc:`alias ` and (optionally) bucket of the MinIO deployment that is the source of the replication job. + +- Replace ```` with the bucket on the alias where the batch should run. + The bucket can be left blank to perform the action on the alias deployment's root location. + +- Replace ``replicate`` with the type of job to generate a yaml file for. + + At the time of release, :mc:``mc batch`` only supports the ``replicate`` job type. + + +S3 Compatibility +~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-s3-compatibility + :end-before: end-minio-mc-s3-compatibility + +.. _minio-batch-job-types: + +Job Types +--------- + +:mc:`mc batch` currently supports the following job task types: + +- ``replicate`` + + Replicate objects between two MinIO deployments. + Provides similar functionality to :ref:`bucket replication ` as a batch job rather than continual scanning function. + +MinIO may add more job types in the future. + +``replicate`` +~~~~~~~~~~~~~ + +Use the ``replicate`` job type to create a batch job that performs replicates objects between two MinIO deployments. + +The YAML **must** define the source and target deployments. +Optionally, the YAML can also define flags to filter which objects replicate, send notifications for the job, or define retry attempts for the job. + +For the **source deployment** + +- Required information + + .. list-table:: + :widths: 25 75 + :width: 100% + + * - ``type:`` + - Must be ``s3``. + * - ``bucket:`` + - The bucket on the deployment. + +- Optional information + + .. list-table:: + :widths: 25 75 + :width: 100% + + * - ``prefix:`` + - The prefix on the object(s) that should replicate. + + * - ``endpoint:`` + - | Location of the source deployment. + | If the location is not remote, use ``local``. + + * - ``credentials:`` + - The ``accesskey:`` and ``secretKey:`` or the ``sessionToken:`` that grants access to the object(s). + +For the **target deployment** + +- Required information + + .. list-table:: + :widths: 25 75 + :width: 100% + + * - ``type:`` + - Must be ``s3``. + * - ``bucket:`` + - The bucket on the deployment. + +- Optional information + + .. list-table:: + :widths: 25 75 + :width: 100% + + * - ``prefix:`` + - The prefix on the object(s) to replicate. + + * - ``endpoint:`` + - | The location of the source deployment. + | If the location is not remote, use ``local``. + + * - ``credentials:`` + - The ``accesskey`` and ``secretKey`` or the ``sessionToken`` that grants access to the object(s). + +For **filters** + +.. list-table:: + :widths: 25 75 + :width: 100% + + * - ``newerThan:`` + - A string representing a length of time in ``#d#h#s`` format. + + Only objects newer than the specified length of time replicate. + For example, ``7d``, ``24h``, ``5d12h30s`` are valid strings. + * - ``olderThan:`` + - A string representing a length of time in ``#d#h#s`` format. + + Only objects older than the specified length of time replicate. + * - ``createdAfter:`` + - A date in ``YYYY-MM-DD`` format. + + Only objects created after the date replicate. + * - ``createdBefore:`` + - A date in ``YYYY-MM-DD`` format. + + Only objects created prior to the date replicate. + +For **notifications** + +.. list-table:: + :widths: 25 75 + :width: 100% + + * - ``endpoint:`` + - The predefined endpoint to send events for notifications. + * - ``token:`` + - An optional :abbr:`JWT ` to access the ``endpoint``. + +For **retry attempts** + +If something interrupts the job, you can define how many attempts to retry the job batch. +For each retry, you can also define how long to wait between attempts. + +.. list-table:: + :widths: 25 75 + :width: 100% + + * - ``attempts:`` + - Number of tries to complete the batch job before giving up. + * - ``delay:`` + - The least amount of time to wait between each attempt. + + +Sample YAML ++++++++++++ + +.. code-block:: yaml + + replicate: + apiVersion: v1 + # source of the objects to be replicated + source: + type: TYPE # valid values are "s3" + bucket: BUCKET + prefix: PREFIX + # NOTE: if source is remote then target must be "local" + # endpoint: ENDPOINT + # credentials: + # accessKey: ACCESS-KEY + # secretKey: SECRET-KEY + # sessionToken: SESSION-TOKEN # Available when rotating credentials are used + + # target where the objects must be replicated + target: + type: TYPE # valid values are "s3" + bucket: BUCKET + prefix: PREFIX + # NOTE: if target is remote then source must be "local" + # endpoint: ENDPOINT + # credentials: + # accessKey: ACCESS-KEY + # secretKey: SECRET-KEY + # sessionToken: SESSION-TOKEN # Available when rotating credentials are used + + # optional flags based filtering criteria + # for all source objects + flags: + filter: + newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s) + olderThan: "7d" # match objects older than this value (e.g. 7d10h31s) + createdAfter: "date" # match objects created after "date" + createdBefore: "date" # match objects created before "date" + + ## NOTE: tags are not supported when "source" is remote. + # tags: + # - key: "name" + # value: "pick*" # match objects with tag 'name', with all values starting with 'pick' + + ## NOTE: metadata filter not supported when "source" is non MinIO. + # metadata: + # - key: "content-type" + # value: "image/*" # match objects with 'content-type', with all values starting with 'image/' + + notify: + endpoint: "https://notify.endpoint" # notification endpoint to receive job status events + token: "Bearer xxxxx" # optional authentication token for the notification endpoint + + retry: + attempts: 10 # number of retries for the job before giving up + delay: "500ms" # least amount of delay between each retry + \ No newline at end of file diff --git a/source/reference/minio-mc/mc-batch-list.rst b/source/reference/minio-mc/mc-batch-list.rst new file mode 100644 index 00000000..4360428d --- /dev/null +++ b/source/reference/minio-mc/mc-batch-list.rst @@ -0,0 +1,103 @@ +.. _minio-mc-batch-list: + +================= +``mc batch list`` +================= + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + +.. mc:: mc batch list + +.. versionchanged:: MinIO RELEASE.2022-10-08T20-11-00Z or later + +Syntax +------ + +.. start-mc-batch-list-desc + +The :mc:`mc batch list` command outputs a list of the batch jobs currently in progress on a deployment. + +.. end-mc-batch-list-desc + + +.. tab-set:: + + .. tab-item:: EXAMPLE + + The following command outputs a list of all jobs currently in progress on the ``myminio`` alias. + + .. code-block:: shell + :class: copyable + + mc batch list myminio + + .. tab-item:: SYNTAX + + The command has the following syntax: + + .. code-block:: shell + :class: copyable + + mc [GLOBALFLAGS] batch list TARGET \ + --type "string" + + .. include:: /includes/common-minio-mc.rst + :start-after: start-minio-syntax + :end-before: end-minio-syntax + +Parameters +~~~~~~~~~~ + +.. mc-cmd:: TARGET + :required: + + The :ref:`alias ` and bucket on the MinIO deployment to which to add the object lifecycle management rule. + +.. mc-cmd:: --type + :optional: + + List batch jobs only of a certain type. + +Global Flags +~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-globals + :end-before: end-minio-mc-globals + +Example +------- + +List all ``replicate`` type batch jobs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following command starts the batch of job(s) defined in the file ``replication.yaml`` on the deployment at :mc:`alias ` ``myminio``: + +.. code-block:: shell + :class: copyable + + mc batch list myminio --type "replicate" + +- Replace ``myminio`` with the :mc:`alias ` of the MinIO deployment that should run the job. + +- Replace ``replicate`` with the job type to output. + + Currently, :mc:`mc batch` only supports the ``replicate`` job type. + +The output of the above command is similar to the following: + +.. code-block:: shell + + ID TYPE USER STARTED + E24HH4nNMcgY5taynaPfxu replicate minioadmin 1 minute ago + +S3 Compatibility +~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-s3-compatibility + :end-before: end-minio-mc-s3-compatibility diff --git a/source/reference/minio-mc/mc-batch-start.rst b/source/reference/minio-mc/mc-batch-start.rst new file mode 100644 index 00000000..3640fadb --- /dev/null +++ b/source/reference/minio-mc/mc-batch-start.rst @@ -0,0 +1,119 @@ +.. _minio-mc-batch-start: + +================== +``mc batch start`` +================== + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + +.. mc:: mc batch start + +.. versionchanged:: MinIO RELEASE.2022-10-08T20-11-00Z or later + +Syntax +------ + +.. start-mc-batch-start-desc + +The :mc:`mc batch start` command launches a batch job from a job batch YAML file. + +.. end-mc-batch-start-desc + +The batch job runs to completion (or up to the number of retries specified in the file) one time. +To run the batch job again after completion, you must start it again. + +.. tab-set:: + + .. tab-item:: EXAMPLE + + The following command creates a basic YAML file for a replicate job on the ``mybucket`` bucket of the ``myminio`` alias. + + .. code-block:: shell + :class: copyable + + mc batch start myminio/mybucket jobfile.yaml + + The output of the above command is something similar to: + + .. code-block:: shell + + Successfully start 'replicate' job `B34HHqnNMcg1taynaPfxu` on '2022-10-24 17:19:06.296974771 -0700 PDT' + + .. tab-item:: SYNTAX + + The command has the following syntax: + + .. code-block:: shell + :class: copyable + + mc [GLOBALFLAGS] batch start \ + TARGET \ + JOBFILE + + .. include:: /includes/common-minio-mc.rst + :start-after: start-minio-syntax + :end-before: end-minio-syntax + +Parameters +~~~~~~~~~~ + +.. mc-cmd:: TARGET + :required: + + The :ref:`alias ` and bucket on the MinIO deployment to which to add the object lifecycle management rule. + + For example: + + .. code-block:: none + + mc batch start myminio/mydata replicate.yaml + + You can use just an alias without specifying a bucket to perform the job at the deployment's root. + +.. mc-cmd:: JOBFILE + :required: + + A YAML-defined batch job. + The job may have as many tasks as desired; there is no predefined limit. + +Global Flags +~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-globals + :end-before: end-minio-mc-globals + +Example +------- + +Start a Batch Job +~~~~~~~~~~~~~~~~~ + +The following command starts the batch of job(s) defined in the file ``replication.yaml`` on the deployment at :mc:`alias ` ``myminio``: + +.. code-block:: shell + :class: copyable + + mc batch start myminio ./replication.yaml + +- Replace ``myminio`` with the :mc:`alias ` of the MinIO deployment that should run the job. + +- Replace ``./replication.yaml`` with the yaml-formatted file that describes the batch job. + Use the file path relative to your current location. + +The output of the above command is similar to the following: + +.. code-block:: shell + + Successfully start 'replicate' job `E24HH4nNMcgY5taynaPfxu` on '2022-09-26 17:19:06.296974771 -0700 PDT' + +S3 Compatibility +~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-s3-compatibility + :end-before: end-minio-mc-s3-compatibility diff --git a/source/reference/minio-mc/mc-batch-status.rst b/source/reference/minio-mc/mc-batch-status.rst new file mode 100644 index 00000000..701259ee --- /dev/null +++ b/source/reference/minio-mc/mc-batch-status.rst @@ -0,0 +1,105 @@ +.. _minio-mc-batch-status: + +=================== +``mc batch status`` +=================== + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + +.. mc:: mc batch status + +.. versionchanged:: MinIO RELEASE.2022-10-08T20-11-00Z or later + +Syntax +------ + +.. start-mc-batch-status-desc + +The :mc:`mc batch status` command outputs real-time summaries of job events on a MinIO server. + +.. end-mc-batch-status-desc + + +.. tab-set:: + + .. tab-item:: EXAMPLE + + The following command outputs a list of all jobs currently in progress on the ``myminio`` alias. + + .. code-block:: shell + :class: copyable + + mc batch status myminio KwSysDpxcBU9FNhGkn2dCf + + .. tab-item:: SYNTAX + + The command has the following syntax: + + .. code-block:: shell + :class: copyable + + mc [GLOBALFLAGS] batch list TARGET \ + JOBID + + .. include:: /includes/common-minio-mc.rst + :start-after: start-minio-syntax + :end-before: end-minio-syntax + +Parameters +~~~~~~~~~~ + +.. mc-cmd:: TARGET + :required: + + The :ref:`alias ` and bucket on the MinIO deployment to which to add the object lifecycle management rule. + +.. mc-cmd:: JOBID + :optional: + + The unique identifier of a job to summarize. + To find the ID of a job, use :mc:`mc batch list`. + +Global Flags +~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-globals + :end-before: end-minio-mc-globals + +Example +------- + +Summary the Events of a Replicate Job +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following command provides the real-time summary of an active job on the deployment at :mc:`alias ` ``myminio``: + +.. code-block:: shell + :class: copyable + + mc batch status myminio KwSysDpxcBU9FNhGkn2dCf + +- Replace ``myminio`` with the :mc:`alias ` of the MinIO deployment that should run the job. + +The output of the above command is similar to the following: + +.. code-block:: shell + + ●∙∙ + Objects: 28766 + Versions: 28766 + Throughput: 3.0 MiB/s + Transferred: 406 MiB + Elapsed: 2m14.227222868s + CurrObjName: share/doc/xml-core/examples/foo.xmlcatalogs + +S3 Compatibility +~~~~~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-s3-compatibility + :end-before: end-minio-mc-s3-compatibility