docs/source/reference/minio-mc/mc-batch-generate.rst

mc batch generate

minio

Table of Contents

mc batch generate

MinIO RELEASE.2022-10-08T20-11-00Z or later

Syntax

The mc batch generate command creates a basic YAML-formatted template file for the specified job type.

After MinIO creates the file, open it in your preferred text editor tool to further customize. You can define one job task definition per batch file.

See job types <minio-batch-job-types> for the supported jobs you can generate.

EXAMPLE

The following command creates a basic YAML file for a replicate job on the mybucket bucket of the myminio alias.

mc batch generate myminio/mybucket replicate

SYNTAX

The command has the following syntax:

mc [GLOBALFLAGS] batch generate \
                       TARGET   \
                       JOBTYPE

Parameters

TARGET

The alias <alias> used to generate the YAML template file. The specified alias does not restrict the deployment(s) where you can use the generated file.

For example:

mc batch generate myminio replicate

JOBTYPE

The type of job to generate a YAML document for.

Currently, mc batch supports the replicate and keyrotate job types.

Global Flags

Examples

Generate a yaml File for a Replicate Job Type

The following command generates a YAML blueprint for a replicate type batch job and names the file replicate with the .yaml extension:

mc batch generate alias replicate > replicate.yaml

• Replace alias with the alias <mc alias> to use to generate the yaml file.

• Replace replicate with the type of job to generate a yaml file for.

  `mc batch supports the replicate and keyrotate` job types.

S3 Compatibility

Job Types

mc batch currently supports the following job task types:

• replicate

  Replicate objects between two MinIO deployments. Provides similar functionality to bucket replication <minio-bucket-replication> as a batch job rather than continual scanning function.

• keyrotate

  MinIO RELEASE.2023-04-07T05-28-58Z

  Rotate the sse-s3 or sse-kms keys for objects at rest on a MinIO deployment.

replicate

Use the replicate job type to create a batch job that replicates objects from the local MinIO deployment to another MinIO location.

The YAML must define the source and target deployments. If the _source deployment is remote, then the _target deployment must be local. Optionally, the YAML can also define flags to filter which objects replicate, send notifications for the job, or define retry attempts for the job.

MinIO RELEASE.2023-04-07T05-28-58Z

You can replicate from a remote MinIO deployment to the local deployment that runs the batch job.

For the source deployment

• Required information

  type:	Must be minio.
  bucket:	The bucket on the deployment.

• Optional information

  prefix:	The prefix on the object(s) that should replicate.
  endpoint:	Location of the source deployment. If the location is not remote, use local.
  credentials:	The accesskey: and secretKey: or the sessionToken: that grants access to the object(s).

For the target deployment

• Required information

  type:	Must be minio.
  bucket:	The bucket on the deployment.

• Optional information

  prefix:	The prefix on the object(s) to replicate.
  endpoint:	The location of the source deployment. If the location is not remote, use local. If the location of the source is remote, the source for target must be local.
  credentials:	The accesskey and secretKey or the sessionToken that grants access to the object(s).

For filters

newerThan:	A string representing a length of time in #d#h#s format. Only objects newer than the specified length of time replicate. For example, 7d, 24h, 5d12h30s are valid strings.
olderThan:	A string representing a length of time in #d#h#s format. Only objects older than the specified length of time replicate.
createdAfter:	A date in YYYY-MM-DD format. Only objects created after the date replicate.
createdBefore:	A date in YYYY-MM-DD format. Only objects created prior to the date replicate.

For notifications

endpoint:	The predefined endpoint to send events for notifications.
token:	An optional JWT <JSON Web Token> to access the endpoint.

For retry attempts

If something interrupts the job, you can define how many attempts to retry the job batch. For each retry, you can also define how long to wait between attempts.

attempts:	Number of tries to complete the batch job before giving up.
delay:	The least amount of time to wait between each attempt.

Sample YAML

/includes/code/replicate.yaml

keyrotate

MinIO RELEASE.2023-04-07T05-28-58Z

Use the keyrotate job type to create a batch job that cycles the sse-s3 or sse-kms keys <minio-sse-data-encryption> for encrypted objects.

Required information

type:	Either sse-s3 or sse-kms.
key:	Only for use with the sse-kms type. The key to use to unseal the key vault.
context:	Only for use with the sse-kms type. The context within which to perform actions.

Optional information

For flag based filters

newerThan:	A string representing a length of time in #d#h#s format. Keys rotate only for objects newer than the specified length of time. For example, 7d, 24h, 5d12h30s are valid strings.
olderThan:	A string representing a length of time in #d#h#s format. Keys rotate only for objects older than the specified length of time.
createdAfter:	A date in YYYY-MM-DD format. Keys rotate only for objects created after the date.
createdBefore:	A date in YYYY-MM-DD format. Keys rotate only for objects created prior to the date.
tags:	Rotate keys only for objects with tags that match the specified key: and value:.
metadtaa:	Rotate keys only for objects with metadata that match the specified key: and value:.
kmskey:	Rotate keys only for objects with a KMS key-id that match the specified value. This is only applicable for the sse-kms type.

For notifications

endpoint:	The predefined endpoint to send events for notifications.
token:	An optional JWT <JSON Web Token> to access the endpoint.

For retry attempts

If something interrupts the job, you can define a maximum number of retry attempts. For each retry, you can also define how long to wait between attempts.

attempts:	Number of tries to complete the batch job before giving up.
delay:	The amount of time to wait between each attempt.

Sample YAML

/includes/code/keyrotate.yaml