Clean up of batch framework and mc batch docs (#749)

- Removes bad copy paste info about ilm from batch command docs.
- Clarifies that batch files are one task only.
- Fixes source type from `s3` to `minio`.
- Cleans up indentation errors in replicate yaml example.
- Replicate task is local to remote only.

Closes #748

source/administration/batch-framework.rst

@@ -20,8 +20,7 @@ Overview
 The MinIO Batch Framework allows you to create, manage, monitor, and execute jobs using a YAML-formatted job definition file (a "batch file").
 The batch jobs run directly on the MinIO deployment to take advantage of the server-side processing power without constraints of the local machine where you run the :ref:`MinIO Client <minio-client>`.
 
-A batch file can define any number of job tasks.
+A batch file defines one job task.
-MinIO does not limit on the number of job tasks that you can define in a batch file.
 
 Once started, MinIO starts processing the job.
 Time to completion depends on the resources available to the deployment.
@@ -82,10 +81,10 @@ You can use MinIO's :ref:`Policy Based Access Control <minio-policy>` and the :r
 Job Types
 ---------
 
-``replicate``
+Replicate
-~~~~~~~~~~~~~
+~~~~~~~~~
 
-The ``replicate`` job type performs a single-run replication of objects from one MinIO deployment to another.
+Use the ``replicate`` job type to create a batch job that replicates objects from the local MinIO deployment to another MinIO location.
 The definition file can limit the replication by bucket, prefix, and/or filters to only replicate certain objects.
 
 For example, you can use a batch job to perform a one-time replication sync of objects from ``minio-alpha/invoices/`` to ``minio-baker/invoices``.
@@ -106,57 +105,5 @@ Sample YAML Description File for a ``replicate`` Job Type
 
 Create a basic ``replicate`` job definition file you can edit with :mc:`mc batch generate`.
 
-.. code-block:: yaml
+.. literalinclude:: /includes/code/replicate.yaml
-
+   :language: yaml
-   replicate:
-     apiVersion: v1
-     # source of the objects to be replicated
-     source:
-         type: TYPE # valid values are "s3"
-   	   bucket: BUCKET
-	      prefix: PREFIX
-	      # NOTE: if source is remote then target must be "local"
-	      # endpoint: ENDPOINT
-	      # credentials:
-         #   accessKey: ACCESS-KEY
-         #   secretKey: SECRET-KEY
-         #   sessionToken: SESSION-TOKEN # Available when rotating credentials are used
-
-     # target where the objects must be replicated
-     target:
-	      type: TYPE # valid values are "s3"
-	      bucket: BUCKET
-	      prefix: PREFIX
-	      # NOTE: if target is remote then source must be "local"
-	      # endpoint: ENDPOINT
-	      # credentials:
-         #   accessKey: ACCESS-KEY
-         #   secretKey: SECRET-KEY
-         #   sessionToken: SESSION-TOKEN # Available when rotating credentials are used
-
-     # optional flags based filtering criteria
-     # for all source objects
-     flags:
-	      filter:
-	         newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
-	         olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
-	         createdAfter: "date" # match objects created after "date"
-	         createdBefore: "date" # match objects created before "date"
-
-      	   ## NOTE: tags are not supported when "source" is remote.
-	         # tags:
-      	   #   - key: "name"
-      	   #     value: "pick*" # match objects with tag 'name', with all values starting with 'pick'
-
-      	   ## NOTE: metadata filter not supported when "source" is non MinIO.
-	         # metadata:
-      	   #   - key: "content-type"
-      	   #     value: "image/*" # match objects with 'content-type', with all values starting with 'image/'
-
-	      notify:
-	         endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
-	         token: "Bearer xxxxx" # optional authentication token for the notification endpoint
-
-	      retry:
-	         attempts: 10 # number of retries for the job before giving up
-	         delay: "500ms" # least amount of delay between each retry

source/includes/code/replicate.yaml

@@ -0,0 +1,49 @@
+replicate:
+  apiVersion: v1
+  # source of the objects to be replicated
+  source:
+    type: TYPE # valid values are "s3"
+    bucket: BUCKET
+    prefix: PREFIX
+    # endpoint: ENDPOINT
+    # credentials:
+    #   accessKey: ACCESS-KEY
+    #   secretKey: SECRET-KEY
+    #   sessionToken: SESSION-TOKEN # Available when rotating credentials are used
+
+  # target where the objects must be replicated
+  target:
+    type: TYPE # valid values are "s3"
+    bucket: BUCKET
+    prefix: PREFIX
+    # endpoint: ENDPOINT
+    # credentials:
+    #   accessKey: ACCESS-KEY
+    #   secretKey: SECRET-KEY
+    #   sessionToken: SESSION-TOKEN # Available when rotating credentials are used
+
+  # optional flags based filtering criteria
+  # for all source objects
+  flags:
+    filter:
+      newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
+      olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
+      createdAfter: "date" # match objects created after "date"
+      createdBefore: "date" # match objects created before "date"
+
+      # tags:
+      #   - key: "name"
+      #     value: "pick*" # match objects with tag 'name', with all values starting with 'pick'
+
+      ## NOTE: metadata filter not supported when "source" is non MinIO.
+      # metadata:
+      #   - key: "content-type"
+      #     value: "image/*" # match objects with 'content-type', with all values starting with 'image/'
+
+  notify:
+    endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
+    token: "Bearer xxxxx" # optional authentication token for the notification endpoint
+
+  retry:
+    attempts: 10 # number of retries for the job before giving up
+    delay: "500ms" # least amount of delay between each retry

source/reference/minio-mc/mc-batch-describe.rst

@@ -106,3 +106,8 @@ S3 Compatibility
 .. include:: /includes/common-minio-mc.rst
    :start-after: start-minio-mc-s3-compatibility
    :end-before: end-minio-mc-s3-compatibility
+
+Permissions
+-----------
+
+You must have the :policy-action:`admin:DescribeBatchJobs` permission to describe jobs on the deployment. 

source/reference/minio-mc/mc-batch-generate.rst

@@ -24,9 +24,7 @@ The :mc:`mc batch generate` command creates a basic YAML-formatted template file
 .. end-mc-batch-generate-desc
 
 After MinIO creates the file, open it in your preferred text editor tool to further customize.
-
+You can define one job task definition per batch file.
-- You can add multiple job task definitions to a batch file.
-- MinIO imposes no limit to the number of job task definitions per batch file.
 
 See :ref:`job types <minio-batch-job-types>` for the supported jobs you can generate.
 
@@ -62,15 +60,14 @@ Parameters
 .. mc-cmd:: TARGET
    :required:
    
-   The :ref:`alias <alias>` and bucket on the MinIO deployment to which to add the object lifecycle management rule. 
+   The :ref:`alias <alias>` used to generate the YAML template file.
+   The specified ``alias`` does not restrict the deployment(s) where you can use the generated file.
    
    For example:
 
    .. code-block:: none
 
-      mc batch generate myminio/mydata replicate
+      mc batch generate myminio replicate
-
-   You can use just an alias without specifying a bucket to perform the job at the deployment's root.
 
 .. mc-cmd:: JOBTYPE
    :required:
@@ -98,12 +95,9 @@ The following command generates a YAML blueprint for a replicate type batch job
 .. code-block:: shell
    :class: copyable
 
-   mc batch generate alias/<bucket> replicate
+   mc batch generate alias replicate > replicate.yaml
 
-- Replace ``alias`` with the :mc:`alias <mc alias>` and (optionally) bucket of the MinIO deployment that is the source of the replication job.
+- Replace ``alias`` with the :mc:`alias <mc alias>` to use to generate the yaml file.
-
-- Replace ``<bucket>`` with the bucket on the alias where the batch should run.
-  The bucket can be left blank to perform the action on the alias deployment's root location.
 
 - Replace ``replicate`` with the type of job to generate a yaml file for.
  
@@ -134,7 +128,7 @@ MinIO may add more job types in the future.
 ``replicate``
 ~~~~~~~~~~~~~
 
-Use the ``replicate`` job type to create a batch job that performs replicates objects between two MinIO deployments.
+Use the ``replicate`` job type to create a batch job that replicates objects from the local MinIO deployment to another MinIO location.
 
 The YAML **must** define the source and target deployments.
 Optionally, the YAML can also define flags to filter which objects replicate, send notifications for the job, or define retry attempts for the job.
@@ -148,7 +142,7 @@ For the **source deployment**
      :width: 100%
 
      * - ``type:``
-       - Must be ``s3``.
+       - Must be ``minio``.
      * - ``bucket:`` 
        - The bucket on the deployment.
 
@@ -162,8 +156,7 @@ For the **source deployment**
        - The prefix on the object(s) that should replicate.
 
      * - ``endpoint:`` 
-       - | Location of the source deployment.
+       - | Location of the source deployment, must be ``local``.
-         | If the location is not remote, use ``local``.
 
      * - ``credentials:`` 
        - The ``accesskey:`` and ``secretKey:`` or the ``sessionToken:`` that grants access to the object(s).
@@ -177,7 +170,7 @@ For the **target deployment**
      :width: 100%
 
      * - ``type:`` 
-       - Must be ``s3``.
+       - Must be ``minio``.
      * - ``bucket:`` 
        - The bucket on the deployment.
 
@@ -250,58 +243,5 @@ For each retry, you can also define how long to wait between attempts.
 Sample YAML
 +++++++++++
 
-.. code-block:: yaml
+.. literalinclude:: /includes/code/replicate.yaml
-
+   :language: yaml
-   replicate:
-     apiVersion: v1
-     # source of the objects to be replicated
-     source:
-         type: TYPE # valid values are "s3"
-   	   bucket: BUCKET
-	      prefix: PREFIX
-	      # NOTE: if source is remote then target must be "local"
-	      # endpoint: ENDPOINT
-	      # credentials:
-         #   accessKey: ACCESS-KEY
-         #   secretKey: SECRET-KEY
-         #   sessionToken: SESSION-TOKEN # Available when rotating credentials are used
-
-     # target where the objects must be replicated
-     target:
-	      type: TYPE # valid values are "s3"
-	      bucket: BUCKET
-	      prefix: PREFIX
-	      # NOTE: if target is remote then source must be "local"
-	      # endpoint: ENDPOINT
-	      # credentials:
-         #   accessKey: ACCESS-KEY
-         #   secretKey: SECRET-KEY
-         #   sessionToken: SESSION-TOKEN # Available when rotating credentials are used
-
-     # optional flags based filtering criteria
-     # for all source objects
-     flags:
-	      filter:
-	         newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
-	         olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
-	         createdAfter: "date" # match objects created after "date"
-	         createdBefore: "date" # match objects created before "date"
-
-      	   ## NOTE: tags are not supported when "source" is remote.
-	         # tags:
-      	   #   - key: "name"
-      	   #     value: "pick*" # match objects with tag 'name', with all values starting with 'pick'
-
-      	   ## NOTE: metadata filter not supported when "source" is non MinIO.
-	         # metadata:
-      	   #   - key: "content-type"
-      	   #     value: "image/*" # match objects with 'content-type', with all values starting with 'image/'
-
-	      notify:
-	         endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
-	         token: "Bearer xxxxx" # optional authentication token for the notification endpoint
-
-	      retry:
-	         attempts: 10 # number of retries for the job before giving up
-	         delay: "500ms" # least amount of delay between each retry
-            

source/reference/minio-mc/mc-batch-list.rst

@@ -55,7 +55,7 @@ Parameters
 .. mc-cmd:: TARGET
    :required:
    
-   The :ref:`alias <alias>` and bucket on the MinIO deployment to which to add the object lifecycle management rule. 
+   The :ref:`alias <alias>` of the deployment for which you want to list jobs in progress. 
    
 .. mc-cmd:: --type
    :optional:
@@ -75,7 +75,7 @@ Example
 List all ``replicate`` type batch jobs
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The following command starts the batch of job(s) defined in the file ``replication.yaml`` on the deployment at :mc:`alias <mc alias>` ``myminio``:
+The following command lists the ``replicate``` type job(s) on the deployment at :mc:`alias <mc alias>` ``myminio``:
 
 .. code-block:: shell
    :class: copyable
@@ -101,3 +101,8 @@ S3 Compatibility
 .. include:: /includes/common-minio-mc.rst
    :start-after: start-minio-mc-s3-compatibility
    :end-before: end-minio-mc-s3-compatibility
+
+Permissions
+-----------
+
+You must have the :policy-action:`admin:ListBatchJobs` permission to list jobs on the deployment. 

source/reference/minio-mc/mc-batch-start.rst

@@ -64,15 +64,13 @@ Parameters
 .. mc-cmd:: TARGET
    :required:
    
-   The :ref:`alias <alias>` and bucket on the MinIO deployment to which to add the object lifecycle management rule. 
+   The :ref:`alias <alias>` on which to start the batch job.
    
    For example:
 
    .. code-block:: none
 
-      mc batch start myminio/mydata replicate.yaml
+      mc batch start myminio replicate.yaml
-
-   You can use just an alias without specifying a bucket to perform the job at the deployment's root.
 
 .. mc-cmd:: JOBFILE
    :required:
@@ -117,3 +115,9 @@ S3 Compatibility
 .. include:: /includes/common-minio-mc.rst
    :start-after: start-minio-mc-s3-compatibility
    :end-before: end-minio-mc-s3-compatibility
+
+
+Permissions
+-----------
+
+You must have the :policy-action:`admin:StartBatchJob` permission on the deployment to start jobs. 

source/reference/minio-mc/mc-batch-status.rst

@@ -55,7 +55,7 @@ Parameters
 .. mc-cmd:: TARGET
    :required:
    
-   The :ref:`alias <alias>` and bucket on the MinIO deployment to which to add the object lifecycle management rule. 
+   The :ref:`alias <alias>` for which to display batch job statuses. 
 
 .. mc-cmd:: JOBID
    :optional: