a065b7a29f
# Summary Closes #1083 Also reorganizes the batch framework pages. Could use additional refinement, but considering that out of scope for now. --------- Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com>
4.2 KiB
Batch Framework
minio
Table of Contents
Overview
The MinIO Batch Framework allows you to create, manage, monitor, and
execute jobs using a YAML-formatted job definition file (a "batch
file"). The batch jobs run directly on the MinIO deployment to take
advantage of the server-side processing power without constraints of the
local machine where you run the MinIO Client <minio-client>.
A batch file defines one job task.
Once started, MinIO starts processing the job. Time to completion depends on the resources available to the deployment.
If any portion of the job fails, MinIO retries the job up to the number of times defined in the job definition.
The MinIO Batch Framework supports the following job types:
| Job Type | Description |
|---|---|
replicate <minio-batch-framework-replicate-job> |
Perform a one-time replication procedure from one MinIO location to another MinIO location. |
keyrotate <minio-batch-framework-keyrotate-job> |
Perform a one-time process to cycle the sse-s3 or sse-kms <minio-sse-data-encryption>
cryptographic keys on objects. |
expire <minio-batch-framework-expire-job> |
Perform a one-time immediate expiration of objects in a bucket. |
MinIO Batch CLI
- Install the
MinIO Client <minio-client> - Define an
alias <mc alias set>for the MinIO deployment
The mc batch commands
include
mc batch generate |
|
mc batch start |
|
mc batch list |
|
mc batch status |
|
mc batch describe |
Access to
mc batch
Each batch job executes using the credentials specified in the batch
definition. The success of a given batch job depends on those
credentials having the appropriate permissions <minio-policy> to perform all
requested actions.
The user executing the batch job must have the following permissions. You can alternatively restrict users from accessing these functions by blocking or limiting access to these actions:
admin:ListBatchJobs-
Grants the user the ability to see batch jobs currently in process.
admin:DescribeBatchJobs-
Grants the user the ability to see the definition details of batch job currently in process.
admin:StartBatchJob-
Grants the user the ability to start a batch job. The job may be further restricted by the credentials the job uses to access either the source or target deployments.
admin:CancelBatchJob-
Allows the user to stop a batch job currently in progress.
You can assign any of these actions to users independently or in any combination.
The built-in ConsoleAdmin policy includes sufficient
access to perform all of these types of batch job actions.
Local Deployment
You run a batch job against a particular deployment by passing an
alias to the mc batch command. The deployment you specify in the
command becomes the local deployment within the context of
that batch job.
/administration/batch-framework-job-replicate /administration/batch-framework-job-keyrotate /administration/batch-framework-job-expire