minio/docs
1
0
Fork 0
mirror of https://github.com/minio/docs.git synced 2025-06-05 19:57:02 +03:00
Code

Adding site recovery content (#540)

Browse Source 
Fills out content for the site recovery stub.
Partially addresses #492 .

Also minor fixes for spelling and formatting in the site-replication
doc.
This commit is contained in:
Daryl White 2022-09-19 16:25:52 -05:00 committed by GitHub
parent 4b2683ca2b
commit aa1c677ecc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 138 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
source/developers/haskell/API.md
View File

@ -20,15 +20,33 @@ awsCI { connectAccesskey = "your-access-key"
```
| Bucket operations | Object Operations | Presigned Operations |
|:--------------------------------------------------|:----------------------------------------------|:--------------------------------------------------|
| [`listBuckets`](#listBuckets) | [`getObject`](#getObject) | [`presignedGetObjectUrl`](#presignedGetObjectUrl) |
| [`makeBucket`](#makeBucket) | [`putObject`](#putObject) | [`presignedPutObjectUrl`](#presignedPutObjectUrl) |
| [`removeBucket`](#removeBucket) | [`fGetObject`](#fGetObject) | [`presignedPostPolicy`](#presignedPostPolicy) |
| [`listObjects`](#listObjects) | [`fPutObject`](#fPutObject) | |
| [`listObjectsV1`](#listObjectsV1) | [`copyObject`](#copyObject) | |
| [`listIncompleteUploads`](#listIncompleteUploads) | [`removeObject`](#removeObject) | |
| [`bucketExists`](#bucketExists) | [`selectObjectContent`](#selectObjectContent) | |
| Bucket operations | Object operations | Presigned operations | Bucket Policy & Notification operations | Custom Settings | |
|-------------------------------------------------------|-----------------------------------------------------|-----------------------------------------------|---------------------------------------------------------------|-------------------------------------------------------|-----|
| [`makeBucket`](#makeBucket) | [`getObject`](#getObject) | [`presignedUrl`](#presignedUrl) | [`getBucketNotification`](#getBucketNotification) | [`setS3TransferAccelerate`](#setS3TransferAccelerate) | |
| [`listBuckets`](#listBuckets) | [`getPartialObject`](#getPartialObject) | [`presignedGetObject`](#presignedGetObject) | [`setBucketNotification`](#setBucketNotification) | | |
| [`bucketExists`](#bucketExists) | [`fGetObject`](#fGetObject) | [`presignedPutObject`](#presignedPutObject) | [`removeAllBucketNotification`](#removeAllBucketNotification) | | |
| [`removeBucket`](#removeBucket) | [`putObject`](#putObject) | [`presignedPostPolicy`](#presignedPostPolicy) | [`getBucketPolicy`](#getBucketPolicy) | | |
| [`listObjects`](#listObjects) | [`fPutObject`](#fPutObject) | | [`setBucketPolicy`](#setBucketPolicy) | | |
| [`listObjectsV2`](#listObjectsV2) | [`copyObject`](#copyObject) | | [`listenBucketNotification`](#listenBucketNotification) | | |
| [`listIncompleteUploads`](#listIncompleteUploads) | [`statObject`](#statObject) | | | | |
| [`getBucketVersioning`](#getBucketVersioning) | [`removeObject`](#removeObject) | | | | |
| [`setBucketVersioning`](#setBucketVersioning) | [`removeObjects`](#removeObjects) | | | | |
| [`getBucketTagging`](#getBucketTagging) | [`removeIncompleteUpload`](#removeIncompleteUpload) | | | | |
| [`setBucketTagging`](#setBucketTagging) | [`putObjectRetention`](#putObjectRetention) | | | | |
| [`removeBucketTagging`](#removeBucketTagging) | [`getObjectRetention`](#getObjectRetention) | | | | |
| [`setBucketLifecycle`](#setBucketLifecycle) | [`putObjectTagging`](#putObjectTagging) | | | | |
| [`getBucketLifecycle`](#getBucketLifecycle) | [`removeObjectTagging`](#removeObjectTagging) | | | | |
| [`removeBucketLifecycle`](#removeBucketLifecycle) | [`getObjectTagging`](#getObjectTagging) | | | | |
| [`setObjectLockConfig`](#setObjectLockConfig) | [`getObjectLegalHold`](#getObjectLegalHold) | | | | |
| [`getObjectLockConfig`](#getObjectLockConfig) | [`setObjectLegalHold`](#setObjectLegalHold) | | | | |
| [`getBucketEncryption`](#getBucketEncryption) | [`composeObject`](#composeObject) | | | | |
| [`setBucketEncryption`](#setBucketEncryption) | [`selectObjectContent`](#selectObjectContent) | | | | |
| [`removeBucketEncryption`](#removeBucketEncryption) | | | | | |
| [`setBucketReplication`](#setBucketReplication) | | | | | |
| [`getBucketReplication`](#getBucketReplication) | | | | | |
| [`removeBucketReplication`](#removeBucketReplication) | | | | | |
## 1. Constructor
## 1. Connecting and running operations on the storage service

95
source/operations/data-recovery/recover-after-site-failure.rst
View File

@ -1,8 +1,8 @@
.. _minio-restore-hardware-failure-site:
==========================
Recover after Site Failure
==========================
=====================
Site Failure Recovery
=====================
.. default-domain:: minio
@ -10,4 +10,91 @@ Recover after Site Failure
:local:
:depth: 1
ToDo- site replication recovery procedure
MinIO can make the loss of an entire site, while significant, a relatively minor incident.
Site recovery depends on the replication option you use for the site.
.. list-table::
:widths: 25 75
:width: 100%
* - Site Replication
- Total restoration of IAM configurations, bucket configurations, and data from the healthy peer site(s)
* - Bucket Replication
- Data restoration of objects and metadata from a healthy remote location for each bucket configured for replication
* - :mc-cmd:`mc mirror`
- Data restoration of objects only from a healthy remote location with no versioning
Site Replication
----------------
:ref:`Site replication <minio-site-replication-overview>` keeps two or more MinIO deployments in sync with IAM policies, buckets, bucket configurations, objects, and object metadata.
If a peer site fails, such as due to a major disaster or long power outage, you can use the remaining healthy site(s) to restore the :ref:`replicable data <minio-site-replication-what-replicates>`.
The following procedure can restore data in scenarios where :ref:`site replication <minio-site-replication-overview>` was active prior to the site loss.
1. Deploy a new MinIO site using the same ``root`` credentials as used on other deployments in the site replication configuration
You can use the original hardware, if still available, but you must first wipe any remaining data before creating the new site.
2. Configure the new site with the same Identity Provider (IDp) as the other site(s)
3. :ref:`Expand the existing site replication <minio-expand-site-replication>` by adding the newly deployed site
4. Remove the lost site from the site replication configuration
Site replication healing automatically adds IAM settings, buckets, bucket configurations, and objects from the existing site(s) to the new site with no further action required.
You cannot configure site replication if any bucket replication rules remain in place on other healthy sites.
Bucket replication is mutually exclusive with site replication.
If you are switching from using bucket replication to using site replication, you must first remove all bucket replication rules from the healthy site prior to setting up site replication.
Active Bucket Replication Resynchronization
-------------------------------------------
For scenarios where :ref:`bucket replication <minio-bucket-replication>` was in place prior to the failure, you can use :mc-cmd:`mc replicate resync` to restore data to a new site.
Create a new site to replace the failed deployment, then synchronize the data from an existing, healthy, bucket replication-enabled deployment to the new site.
1. Deploy a new MinIO site
2. Set up IAM and users as needed
3. On the site with data, create a new ``remote target`` using the :mc-cmd:`mc admin bucket remote add` command and record the ARN from the output
4. From the site with the data, use the :mc-cmd:`mc replicate resync start` command with the ARN from the previous command to rebuild the bucket on the new site
5. Wait for re-synchronization to complete (us :mc-cmd:`mc replicate resync status` to check)
6. Set up bucket replication rule(s) from the new MinIO site to the existing target bucket(s)
7. `(Optional)` Delete the bucket replication rules from the target deployment(s) to restore an active-passive replication scenario
Passive Bucket Replication Resynchronization
--------------------------------------------
:ref:`Bucket replication <minio-bucket-replication>` can directly restore the site contents by performing a replication from the target bucket(s) to a new MinIO site.
As a passive process, bucket replication may not perform as quickly as desired for a site recovery scenario.
Using bucket replication relies on the standard replication scanner queue, which does not take priority over other processes.
For recovery procedures with stricter SLA/SLO, use the active bucket replication process with :mc-cmd:`mc replicate resync` command as described above.
Bucket replication rules copy the object, its version ID, versions, and other metadata to the target bucket.
MinIO can restore the object with all of these attributes to a new MinIO site if bucket replication had already been in use prior to the site loss.
1. Deploy a new MinIO site
2. Set up IAM and users as needed
3. On the remaining target bucket deployment(s), create bucket replication rule(s) for each bucket to the new MinIO site
4. Wait for replication to complete
5. Set up bucket replication rule(s) from the new MinIO site to the existing target bucket(s)
6. `(Optional)` Delete the bucket replication rules from the target deployment(s) to restore an active-passive replication scenario
Do not delete the bucket replication rules from the deployments used to recover data if you prefer to keep an active-active replication between the buckets.
In active-active replication, changes to the objects at either location affect the objects at the other location.
Mirroring
---------
MinIO's mirroring copies an object from any S3 compatible storage system.
Mirroring only copies the latest version of each object and does not include versioning metadata, regardless of the source.
You cannot restore those attributes with this method.
Use :mc-cmd:`mc mirror` in situations where you need to restore only the latest version of an object.
Use bucket replication or site replication where those methods were already in use if you are copying from another MinIO deployment and wish to restore the object's version history and version metadata.
1. Deploy a new MinIO site
2. Set up IAM and users as needed
3. Create buckets on the new site
4. Use the :mc-cmd:`mc cp` CLI command to copy the contents from the mirror location to the new MinIO site

22
source/operations/install-deploy-manage/multi-site-replication.rst
View File

@ -104,9 +104,23 @@ For :ref:`SSE-S3 <minio-encryption-sse-s3>` or :ref:`SSE-KMS <minio-encryption-s
You can achieve this with a central KES server or multiple KES servers (say one per site) connected via a central supported :ref:`key vault server <minio-sse>`.
Switch to Site Replication from Bucket Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:ref:`Bucket replication <minio-bucket-replication>` and multi-site replication are mutually exclusive.
You cannot use both replication methods on the same deployments.
If you previously set up bucket replication and wish to now use site replication, you must first delete all of the bucket replication rules on the deployment that has data when initializing site replication.
Use :mc-cmd:`mc replicate rm` on the command line or the MinIO Console to remove bucket replication rules.
Only one site can have data when setting up site replication.
All other sites must be empty.
Tutorials
---------
.. _minio-configure-site-replication:
Configure Site Replication
~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -129,7 +143,7 @@ Configure Site Replication
.. image:: /images/minio-console/console-settings-site-replication.png
:width: 400px
:alt: MinIO Console menu with the Settings heading expanded to show Site Repilication
:alt: MinIO Console menu with the Settings heading expanded to show Site Replication
:align: center
#. Select :guilabel:`Add Sites +`
@ -241,6 +255,8 @@ Configure Site Replication
For more on reviewing site replication, see the :ref:`Site Replication Status tutorial <minio-site-replication-status-tutorial>`.
.. _minio-expand-site-replication:
Expand Site Replication
~~~~~~~~~~~~~~~~~~~~~~~
@ -259,7 +275,7 @@ The new site must meet the following requirements:
#. Deploy a new, empty MinIO site
#. In a browser, access the Console for one of the exisitng replicated sites
#. In a browser, access the Console for one of the existing replicated sites
For example, ``https://<addressforsite>:9000``
@ -324,7 +340,7 @@ The new site must meet the following requirements:
List all existing replicated sites first, then list the new site(s) to add.
In this example, ``minio1``, ``minio2``, and ``minio3`` are already configured for replication.
The command adds minio4 and minio5 as new sites to add to the replication.
The command adds ``minio4`` and ``minio5`` as new sites to add to the replication.
``minio4`` and ``minio5`` must be empty.
.. code-block:: shell