Initial work for git-managed redirects (#556)

This is the initial work for what should be a git-managed set of nginx
configs we can include into the Nginx Docs server.

The idea here is we manage redirects in git, and either chron-copy them
to a location we can include via the sites-available/<file>, *or* we
just include them from the git path (if nginx tolerates that)

I can't really _test_ these right now since they are using live URLs. if
we move forward with the soft-launch, we'll have a few opportunities to
maybe test a subset of these URLs and see what happens.

Please let me know if there is anything *obviously* wrong here.

ToDo: The Legacy Docs redirects

redirects/container.nginx

@@ -0,0 +1,3 @@
+location /minio/baremetal/quickstart/container.html {
+      return 301 https://min.io/minio/docs/container/index.html?ref=docs-redirect;
+}

redirects/old/container.nginx

@@ -0,0 +1,3 @@
+location /minio/baremetal/quickstart/container.html {
+      return 301 https://min.io/minio/docs/container/operations/index.html?ref=docs-redirect;
+}

redirects/old/k8s.nginx

@@ -0,0 +1,70 @@
+# This file contains all redirects for the Kubernetes Documentation for MinIO
+# This includes redirecting links from legacy docs and the Docs V1 projects
+
+# Use this as a template
+#  location <path> {
+#       return 301 https://min.io/docs/minio/kubernetes/upstream/<filepath>.html?ref=docs-redirect;
+#  }
+
+
+location /minio/baremetal/quickstart/k8s.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/operations/index.html?ref=docs-redirect;
+}
+
+location /minio/k8s/deployment/deploy-minio-operator.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/operations/installation.html?ref=docs-redirect;
+}
+
+
+location /minio/k8s/tenant-management/deploy-minio-tenant.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/operations/install-deploy-manage/deploy-minio-tenant.html?ref=docs-redirect;
+}
+
+location /minio/k8s/core-concepts/core-concepts.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/operations/concepts.html?ref=docs-redirect;
+}
+
+location /minio/k8s/tenant-management/manage-minio-tenant.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/operations/deploy-manage-tenants.html?ref=docs-redirect;
+}
+
+location /minio/k8s/tutorials/user-management.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/administration/identity-access-management/minio-user-management.html?ref=docs-redirect;
+}
+
+location /minio/k8s/tutorials/group-management.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/administration/identity-access-management/minio-group-management.html?ref=docs-redirect;
+}
+
+location /minio/k8s/tutorials/policy-management.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/administration/identity-access-management/policy-based-access-control.html?ref=docs-redirect;
+}
+
+location /minio/k8s/reference/production-recommendations.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/operations/checklists.html?ref=docs-redirect;
+}
+
+location /minio/k8s/operator-console/operator-console.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/operations/install-deploy-manage/minio-operator-console.html?ref=docs-redirect;
+}
+
+
+# OpenShift Specific 
+# Will need to change these when we have dedicated OS platform docs
+# -----------------------------------------------------------------
+
+location /minio/k8s/openshift/deploy-minio-on-openshift.html {
+     return 302 https://min.io/docs/minio/kubernetes/upstream/operations/installation.html?ref=docs-redirect;
+}
+
+location /minio/k8s/openshift/deploy-minio-tenant.html {
+     return 302 https://min.io/docs/minio/kubernetes/upstream/operations/install-deploy-manage/deploy-minio-tenant.html?ref=docs-redirect;
+}
+
+# -----------------------------------------------------------------
+
+# This one *must* be last in the order so as to not override other specific pages
+
+location /minio/k8s {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/index.html?ref=docs-redirect;
+}

redirects/old/legacy.nginx

@@ -0,0 +1,191 @@
+location /docs/minio-gateway-for-nas.html {
+     return 301 https://blog.min.io/deprecation-of-the-minio-gateway?ref=docs-redirect;
+}
+
+location /docs/minio-gateway-for-s3.html {
+     return 301 https://blog.min.io/deprecation-of-the-minio-gateway?ref=docs-redirect;
+}
+
+location /docs/minio-vault-legacy.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption/configure-minio-kes-hashicorp.html?ref=docs-redirect;
+}
+
+location /docs/minio-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/index.html?ref=docs-redirect;
+}
+
+location /docs/minio-docker-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/container/index.html?ref=docs-redirect;
+}
+
+location /docs/minio-erasure-code-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/operations/concepts.html#how-does-minio-provide-availability-redundancy-and-reliability?ref=docs-redirect;
+}
+
+location /docs/distributed-minio-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html?ref=docs-redirect;
+}
+
+location /docs/how-to-secure-access-to-minio-server-with-tls.html {
+     return 301 https://min.io/docs/minio/linux/operations/network-encryption.html?ref=docs-redirect;
+}
+
+location /docs/minio-security-overview.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption.html?ref=docs-redirect;
+}
+
+location /docs/minio-server-limits-per-tenant.html {
+     return 301 https://min.io/docs/minio/linux/operations/checklists.html?ref=docs-redirect;
+}
+
+location /docs/minio-server-configuration-guide.html {
+     return 301 https://min.io/docs/minio/linux/reference/minio-mc-admin/mc-admin-config.html?ref=docs-redirect;
+}
+
+location /docs/minio-monitoring-guide.html {
+     return 301 https://min.io/docs/minio/linux/operations/monitoring.html?ref=docs-redirect;
+}
+
+location /docs/how-to-monitor-minio-using-prometheus.html {
+     return 301 https://min.io/docs/minio/linux/operations/monitoring/collect-minio-metrics-using-prometheus.html?ref=docs-redirect;
+}
+
+location /docs/minio-kms-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/administration/server-side-encryption/server-side-encryption-sse-kms.html?ref=docs-redirect;
+}
+
+location /docs/minio-multi-user-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/administration/identity-access-management.html?ref=docs-redirect;
+}
+
+location /docs/minio-sts-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/developers/security-token-service.html?ref=docs-redirect;
+}
+
+location /docs/minio-bucket-versioning-guide.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/object-versioning.html?ref=docs-redirect;
+}
+
+location /docs/minio-bucket-object-lock-guide.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/object-retention.html?ref=docs-redirect;
+}
+
+location /docs/minio-bucket-replication-guide.html {
+     return 301 https://min.io/docs/minio/linux/administration/bucket-replication.html?ref=docs-redirect;
+}
+
+location /docs/minio-bucket-notification-guide.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring.html?ref=docs-redirect;
+}
+
+location /docs/minio-bucket-lifecycle-guide.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/object-lifecycle-management.html?ref=docs-redirect;
+}
+
+location /docs/minio-select-api-quickstart-guide.html {
+     return 301 https://blog.min.io/tag/s3-select/?ref=docs-redirect;
+}
+
+location /docs/using-minio-with-veeam.html {
+     return 301 https://min.io/docs/minio/linux/integrations/using-minio-with-veeam.html?ref=docs-redirect;
+}
+
+location /docs/minio-deployment-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/index.html?ref=docs-redirect;
+}
+
+location /docs/deploy-minio-on-kubernetes.html {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/index.html?ref=docs-redirect;
+}
+
+location /docs/deploy-minio-on-docker-compose.html {
+     return 301 https://min.io/docs/minio/container/index.html?ref=docs-redirect;
+}
+
+location /docs/minio-client-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/reference/minio-mc.html?ref=docs-redirect;
+}
+
+location /docs/minio-client-complete-guide.html {
+     return 301 https://min.io/docs/minio/linux/reference/minio-mc.html?ref=docs-redirect;
+}
+
+location /docs/minio-admin-complete-guide.html {
+     return 301 https://min.io/docs/minio/linux/reference/minio-mc-admin.html?ref=docs-redirect;
+}
+
+location /docs/java-client-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/developers/java/minio-java.html?ref=docs-redirect;
+}
+
+location /docs/java-client-api-reference.html {
+     return 301 https://min.io/docs/minio/linux/developers/java/API.html?ref=docs-redirect;
+}
+
+location /docs/golang-client-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/developers/go/minio-go.html?ref=docs-redirect;
+}
+
+location /docs/golang-client-api-reference.html {
+     return 301 https://min.io/docs/minio/linux/developers/go/API.html
+}
+
+location /docs/python-client-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/developers/python/minio-py.html?ref=docs-redirect;
+}
+
+location /docs/python-client-api-reference.html {
+     return 301 https://min.io/docs/minio/linux/developers/python/API.html?ref=docs-redirect;
+}
+
+location /docs/javascript-client-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/developers/javascript/minio-javascript.html?ref=docs-redirect;
+}
+
+location /docs/javascript-client-api-reference.html {
+     return 301 https://min.io/docs/minio/linux/developers/javascript/API.html?ref=docs-redirect;
+}
+
+location /docs/dotnet-client-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/developers/dotnet/minio-dotnet.html?ref=docs-redirect;
+}
+
+location /docs/dotnet-client-api-reference.html {
+     return 301 https://min.io/docs/minio/linux/developers/dotnet/API.html?ref=docs-redirect;
+}
+
+location /docs/haskell-client-quickstart-guide.html {
+     return 301 https://min.io/docs/minio/linux/developers/haskell/minio-haskell.html?ref=docs-redirect;
+}
+
+location /docs/haskell-client-api-reference.html {
+     return 301 https://min.io/docs/minio/linux/developers/haskell/API.html?ref=docs-redirect;
+}
+
+location /docs/disaggregated-spark-and-hadoop-hive-with-minio.html {
+     return 301 https://min.io/docs/minio/linux/integrations/disaggregated-spark-and-hadoop-hive-with-minio.html?ref=docs-redirect;
+}
+
+location /docs/aws-cli-with-minio.html {
+     return 301 https://min.io/docs/minio/linux/integrations/aws-cli-with-minio.html?ref=docs-redirect;
+}
+
+location /docs/setup-nginx-proxy-with-minio.html {
+     return 301 https://min.io/docs/minio/linux/integrations/setup-nginx-proxy-with-minio.html?ref=docs-redirect;
+}
+
+location /docs/upload-files-from-browser-using-pre-signed-urls.html {
+     return 301 https://min.io/docs/minio/linux/integrations/upload-files-from-browser-using-pre-signed-urls.html?ref=docs-redirect;
+}
+
+location /docs/how-to-use-minio-s-server-side-encryption-with-aws-cli.html {
+     return 301 https://min.io/docs/minio/linux/integrations/how-to-use-minio-s-server-side-encryption-with-aws-cli.html?ref=docs-redirect;
+}
+
+location /docs/how-to-run-minio-in-freenas.html {
+     return 301 https://min.io/docs/minio/linux/?ref=docs-redirect;
+}
+
+location /docs {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/index.html?ref=docs-redirect;
+}

redirects/old/linux.nginx

@@ -0,0 +1,344 @@
+# Top level redirects from NuDocs 1 to NuDocs 2
+# =============================================
+
+location /minio/baremetal/introduction/minio-overview.html {
+     return 301 https://min.io/docs/minio/linux/operations/concepts.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/concepts/erasure-coding.html {
+     return 301 https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/quickstart/linux.html {
+     return 301 https://min.io/docs/minio/linux/operations/index.html?ref=docs-redirect;
+}
+
+
+# Deploy/Manage Redirects from NuDocs 1 to NuDocs 2
+# =================================================
+
+location /minio/baremetal/installation/deployment-and-management.html {
+     return 301 https://min.io/docs/minio/linux/operations/installation.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/installation/deploy-minio-distributed.html {
+     return 301 https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/installation/deploy-minio-single-node-single-drive.html {
+     return 301 https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-single-node-single-drive.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/installation/expand-minio-distributed.html {
+     return 301 https://min.io/docs/minio/linux/operations/install-deploy-manage/expand-minio-deployment.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/installation/upgrade-minio.html {
+     return 301 https://min.io/docs/minio/linux/operations/install-deploy-manage/upgrade-minio-deployment.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/installation/restore-minio.html {
+     return 301 https://min.io/docs/minio/linux/operations/data-recovery.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/installation/decommission-pool.html {
+     return 301 https://min.io/docs/minio/linux/operations/install-deploy-manage/decommission-server-pool.html?ref=docs-redirect;
+}       
+
+
+# Security Redirects from NuDocs 1 to NuDocs 2
+# ============================================
+
+location /minio/baremetal/security/IAM/identity-access-management.html {
+     return 301 https://min.io/docs/minio/linux/administration/identity-access-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/security-overview.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/encryption/encryption-key-management.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/minio-identity-management/basic-authentication-with-minio-identity-provider.html {
+     return 301 https://min.io/docs/minio/linux/administration/identity-access-management/minio-identity-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/minio-identity-management/user-management.html {
+     return 301 https://min.io/docs/minio/linux/administration/identity-access-management/minio-user-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/minio-identity-management/group-management.html {
+     return 301 https://min.io/docs/minio/linux/administration/identity-access-management/minio-group-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/minio-identity-management/policy-based-access-control.html {
+     return 301 https://min.io/docs/minio/linux/administration/identity-access-management/policy-based-access-control.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/openid-external-identity-management/external-authentication-with-openid-identity-provider.html {
+     return 301 https://min.io/docs/minio/linux/administration/identity-access-management/oidc-access-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/openid-external-identity-management/configure-openid-external-identity-management.html {
+     return 301 https://min.io/docs/minio/linux/operations/external-iam/configure-openid-external-identity-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/openid-external-identity-management/AssumeRoleWithWebIdentity.html {
+     return 301 https://min.io/docs/minio/linux/developers/security-token-service/AssumeRoleWithWebIdentity.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/ad-ldap-external-identity-management/external-authentication-with-ad-ldap-identity-provider.html {
+     return 301 https://min.io/docs/minio/linux/administration/identity-access-management/ad-ldap-access-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/ad-ldap-external-identity-management/configure-openid-external-identity-management.html {
+     return 301 https://min.io/docs/minio/linux/operations/external-iam/configure-ad-ldap-external-identity-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/ad-ldap-external-identity-management/AssumeRoleWithWebIdentity.html {
+     return 301 https://min.io/docs/minio/linux/developers/security-token-service/AssumeRoleWithLDAPIdentity.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/encryption-overview.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/server-side-encryption/minio-server-side-encryption.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/server-side-encryption/configure-minio-kes-hashicorp.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption/configure-minio-kes-hashicorp.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/server-side-encryption/configure-minio-kes-aws.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption/configure-minio-kes-aws.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/server-side-encryption/configure-minio-kes-gcp.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption/configure-minio-kes-gcp.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/server-side-encryption/configure-minio-kes-azure.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption/configure-minio-kes-azure.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/server-side-encryption/server-side-encryption-sse-kms.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption/server-side-encryption-sse-kms.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/server-side-encryption/server-side-encryption-sse-s3.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption/server-side-encryption-sse-s3.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/server-side-encryption/server-side-encryption-sse-c.html {
+     return 301 https://min.io/docs/minio/linux/operations/server-side-encryption/server-side-encryption-sse-c.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/security/network-encryption/minio-tls.html {
+     return 301 https://min.io/docs/minio/linux/operations/network-encryption.html?ref=docs-redirect;
+}
+
+# Object Management Redirects from NuDocs 1 to NuDocs 2
+# =====================================================
+
+location /minio/baremetal/object-retention/minio-object-retention.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/object-retention/bucket-versioning.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/object-versioning.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/object-retention/minio-object-locking.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/object-retention.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/lifecycle-management/lifecycle-management-overview.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/object-lifecycle-management.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/lifecycle-management/transition-objects-to-s3.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/transition-objects-to-s3.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/lifecycle-management/transition-objects-to-gcs.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/transition-objects-to-gcs.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/lifecycle-management/transition-objects-to-azure.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/transition-objects-to-azure.html?ref=docs-redirect;
+}
+
+# Replication Redirects from NuDocs 1 to NuDocs 2
+# ===============================================
+
+location /minio/baremetal/lifecycle-management/create-lifecycle-management-expiration-rule.html {
+     return 301 https://min.io/docs/minio/linux/administration/object-management/create-lifecycle-management-expiration-rule.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/replication/replication-overview.html {
+     return 301 https://min.io/docs/minio/linux/administration/bucket-replication.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/bucket-replication-overview.html {
+     return 301 https://min.io/docs/minio/linux/administration/bucket-replication.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/replication/enable-server-side-one-way-bucket-replication.html {
+     return 301 https://min.io/docs/minio/linux/administration/bucket-replication/enable-server-side-one-way-bucket-replication.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/replication/enable-server-side-two-way-bucket-replication.html {
+     return 301 https://min.io/docs/minio/linux/administration/bucket-replication/enable-server-side-two-way-bucket-replication.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/replication/enable-server-side-multi-site-bucket-replication.html {
+     return 301 https://min.io/docs/minio/linux/administration/bucket-replication/enable-server-side-multi-site-bucket-replication.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/replication/server-side-replication-resynchronize-remote.html {
+     return 301 https://min.io/docs/minio/linux/administration/bucket-replication/server-side-replication-resynchronize-remote.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/replication/site-replication-overview.html {
+     return 301 https://min.io/docs/minio/linux/operations/install-deploy-manage/multi-site-replication.html?ref=docs-redirect;
+}
+
+# Monitoring Redirects from NuDocs 1 to NuDocs 2
+# ==============================================
+
+location /minio/baremetal/monitoring/monitoring-overview.html {
+     return 301 https://min.io/docs/minio/linux/operations/monitoring.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/metrics-alerts/minio-metrics-and-alerts.html {
+     return 301 https://min.io/docs/minio/linux/operations/monitoring.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/metrics-alerts/collect-minio-metrics-using-prometheus.html {
+     return 301 https://min.io/docs/minio/linux/operations/monitoring/collect-minio-metrics-using-prometheus.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/logging/minio-logging.html { 
+     return 301 https://min.io/docs/minio/linux/monitoring/minio-logging.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/healthcheck-probe.html {
+     return 301 https://min.io/docs/minio/linux/operations/monitoring/healthcheck-probe.html?ref=docs-redirect;
+}
+
+# Bucket Notification Redirects from NuDocs 1 to NuDocs 2
+# =======================================================
+
+location /minio/baremetal/monitoring/bucket-notifications/bucket-notifications.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/bucket-notifications.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-amqp.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-amqp.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-mqtt.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-mqtt.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-nats.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-nats.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-nsq.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-nsq.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-elasticsearch.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-elasticsearch.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-kafka.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-kafka.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-mysql.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-mysql.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-postgresql.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-postgresql.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-redis.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-redis.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/monitoring/bucket-notifications/publish-events-to-webhook.html {
+     return 301 https://min.io/docs/minio/linux/administration/monitoring/publish-events-to-webhook.html?ref=docs-redirect;
+}
+
+
+
+
+# Reference Redirects from NuDocs 1 to NuDocs 2
+# =============================================
+
+
+location /minio/baremetal/reference/minio-mc.html {
+     return 301 https://min.io/docs/minio/linux/reference/minio-mc.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/reference/minio-mc/ {
+     rewrite /minio/baremetal/reference/minio-mc/(.*) https://min.io/docs/minio/linux/reference/minio-mc/$1 permanent;
+}
+
+location /minio/baremetal/reference/minio-mc-admin.html {
+     return 301 https://min.io/docs/minio/linux/reference/minio-mc.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/reference/minio-mc-admin/ {
+     rewrite /minio/baremetal/reference/minio-mc-admin/(.*) https://min.io/docs/minio/linux/reference/minio-mc-admin/$1 permanent;
+}
+
+location /minio/baremetal/reference/minio-server.html {
+     return 301 https://min.io/docs/minio/linux/reference/minio-server.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/console/minio-console.html {
+     return 301 https://min.io/docs/minio/linux/administration/minio-console.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/sdk/minio-drivers.html {
+     return 301 https://min.io/docs/minio/linux/developers/minio-drivers.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/sdk/java/minio-java.html {
+     return 301 https://min.io/docs/minio/linux/developers/java/minio-java.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/sdk/python/minio-python.html {
+     return 301 https://min.io/docs/minio/linux/developers/python/minio-python.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/sdk/go/minio-go.html {
+     return 301 https://min.io/docs/minio/linux/developers/go/minio-go.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/sdk/dotnet/minio-dotnet.html {
+     return 301 https://min.io/docs/minio/linux/developers/dotnet/minio-dotnet.html?ref=docs-redirect;
+}
+
+# Miscellaneous Redirects from NuDocs 1 to NuDocs 2
+# =================================================
+
+location /minio/baremetal/support/support-overview.html {
+     return 301 https://min.io/docs/minio/linux/operations/troubleshooting.html?ref=docs-redirect;
+}
+
+location /minio/baremetal/support/encrypting-files.html {
+     return 301 https://min.io/docs/minio/linux/operations/troubleshooting/encrypting-files.html?ref=docs-redirect;
+}
+
+location /minio/baremetal {
+     return 301 https://min.io/docs/minio/linux/index.html?ref=docs-redirect;
+}

redirects/old/macos.nginx

@@ -0,0 +1,3 @@
+location /minio/baremetal/quickstart/macos.html {
+     return 301 https://min.io/docs/minio/macos/index.html?ref=docs-redirect;
+}

redirects/old/vmware.nginx

@@ -0,0 +1,3 @@
+location /minio/vsphere {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/index.html?ref=docs-redirect;
+}

redirects/old/windows.nginx

@@ -0,0 +1,3 @@
+location /minio/baremetal/quickstart/windows.html {
+     return 301 https://min.io/minio/docs/windows/operations/index.html?ref=docs-redirect;
+}

redirects/redirects.nginx

@@ -0,0 +1,11 @@
+location = /docs {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/index.html;
+}
+
+location = /docs/minio {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/index.html;
+}
+
+location = /docs/minio/kubernetes {
+     return 301 https://min.io/docs/minio/kubernetes/upstream/index.html;
+}

redirects/windows.nginx

@@ -0,0 +1,3 @@
+location /minio/baremetal/quickstart/windows.html {
+     return 301 https://min.io/minio/docs/windows/index.html?ref=docs-redirect;
+}

source/default-conf.py

@@ -142,7 +142,8 @@ elif tags.has("macos"):
         'reference/kubectl-minio-plugin*',
         'reference/minio-server*',
         'reference/minio-mc*',
-        'developers/*'
+        'developers/*',
+        'integrations/*'
     ]
 elif tags.has("windows"):
     # html_baseurl is used for generating the sitemap.xml for each platform. These are combined in a sitemapindex.xml.
@@ -159,7 +160,8 @@ elif tags.has("windows"):
         'reference/kubectl-minio-plugin*',
         'reference/minio-server*',
         'reference/minio-mc*',
-        'developers/*'
+        'developers/*',
+        'integrations/*'
     ]
 elif tags.has("container"):
     html_baseurl = 'https://min.io/docs/minio/container/'
@@ -177,7 +179,8 @@ elif tags.has("container"):
         'reference/kubectl-minio-plugin*',
         'reference/minio-server*',
         'reference/minio-mc*',
-        'developers/*'
+        'developers/*',
+        'integrations/*'
     ]
 elif tags.has("k8s"):
     html_baseurl = 'https://min.io/docs/minio/kubernetes/upstream/'
@@ -191,7 +194,8 @@ elif tags.has("k8s"):
         'operations/manage-existing-deployments.rst',
         'reference/minio-server*',
         'reference/minio-mc*',
-        'developers/*'
+        'developers/*',
+        'integrations/*'
 
     ]
 else:

source/index.rst

@@ -35,7 +35,7 @@ MinIO is built to deploy anywhere - public or private cloud, baremetal infrastru
 This documentation targets the latest stable version of MinIO: |minio-tag|.
 
 MinIO is released under `GNU Affero General Public License v3.0 
-<https://www.gnu.org/licenses/agpl-3.0.en.html>`__. 
+<https://www.gnu.org/licenses/agpl-3.0.en.html?ref=docs>`__. 
 
 You can get started exploring MinIO features using our ``play`` server at
 https://play.min.io. ``play`` is a *public* MinIO cluster running the latest
@@ -140,10 +140,11 @@ without validating their usage do so at their own risk.
       :titlesonly:
       :hidden:
 
-      Software Development Kits (SDK) <https://min.io/docs/minio/linux/developers/minio-drivers.html>
-      Security Token Service (STS) <https://min.io/docs/minio/linux/developers/security-token-service.html>
-      MinIO Client <https://min.io/docs/minio/linux/reference/minio-mc.html>
-      MinIO Admin Client <https://min.io/docs/minio/linux/reference/minio-mc-admin.html>
+      Software Development Kits (SDK) <https://min.io/docs/minio/linux/developers/minio-drivers.html?ref=docs>
+      Security Token Service (STS) <https://min.io/docs/minio/linux/developers/security-token-service.html?ref=docs>
+      MinIO Client <https://min.io/docs/minio/linux/reference/minio-mc.html?ref=docs>
+      MinIO Admin Client <https://min.io/docs/minio/linux/reference/minio-mc-admin.html?ref=docs>
+      Integrations <https://min.io/docs/minio/linux/integrations/integrations.html?ref=docs>
 
 .. cond:: linux
 
@@ -156,6 +157,7 @@ without validating their usage do so at their own risk.
       /reference/minio-mc
       /reference/minio-mc-admin
       /reference/minio-server/minio-server
+      /integrations/integrations
 
 .. cond:: k8s
 

source/integrations/aws-cli-with-minio.md

@@ -0,0 +1,83 @@
+# AWS CLI with MinIO Server [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
+
+AWS CLI is a unified tool to manage AWS services. It is frequently the tool used to transfer data in and out of AWS S3. It works with any S3 compatible cloud storage service.
+
+In this recipe we will learn how to configure and use AWS CLI to manage data with MinIO Server.
+
+## 1. Prerequisites
+
+Install MinIO Server from [here](https://docs.min.io).
+
+## 2. Installation
+
+Install AWS CLI from <https://aws.amazon.com/cli/>
+
+## 3. Configuration
+
+To configure AWS CLI, type `aws configure` and specify the MinIO key information.
+
+Access credentials shown in this example belong to <https://play.min.io:9000>.
+These credentials are open to public. Feel free to use this service for testing and development. Replace with your own MinIO keys in deployment.
+
+```sh
+aws configure
+AWS Access Key ID [None]: Q3AM3UQ867SPQQA43P2F
+AWS Secret Access Key [None]: zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG
+Default region name [None]: us-east-1
+Default output format [None]: ENTER
+```
+
+Additionally enable AWS Signature Version '4' for MinIO server.
+
+```sh
+aws configure set default.s3.signature_version s3v4
+```
+
+## 4. Commands
+
+### To list your buckets
+
+```sh
+aws --endpoint-url https://play.min.io:9000 s3 ls
+2016-03-27 02:06:30 deebucket
+2016-03-28 21:53:49 guestbucket
+2016-03-29 13:34:34 mbtest
+2016-03-26 22:01:36 mybucket
+2016-03-26 15:37:02 testbucket
+```
+
+### To list contents inside bucket
+
+```sh
+aws --endpoint-url https://play.min.io:9000 s3 ls s3://mybucket
+2016-03-30 00:26:53      69297 argparse-1.2.1.tar.gz
+2016-03-30 00:35:37      67250 simplejson-3.3.0.tar.gz
+```
+
+### To make a bucket
+
+```sh
+aws --endpoint-url https://play.min.io:9000 s3 mb s3://mybucket
+make_bucket: s3://mybucket/
+```
+
+### To add an object to a bucket
+
+```sh
+aws --endpoint-url https://play.min.io:9000 s3 cp simplejson-3.3.0.tar.gz s3://mybucket
+upload: ./simplejson-3.3.0.tar.gz to s3://mybucket/simplejson-3.3.0.tar.gz
+```
+
+### To delete an object from a bucket
+
+```sh
+aws --endpoint-url https://play.min.io:9000 s3 rm s3://mybucket/argparse-1.2.1.tar.gz
+delete: s3://mybucket/argparse-1.2.1.tar.gz
+```
+
+### To remove a bucket
+
+```sh
+aws --endpoint-url https://play.min.io:9000 s3 rb s3://mybucket
+remove_bucket: s3://mybucket/
+```

source/integrations/disaggregated-spark-and-hadoop-hive-with-minio.md

@@ -0,0 +1,306 @@
+# Disaggregated HDP Spark and Hive with MinIO
+
+## **1. Cloud-native Architecture**
+
+![cloud-native](/images/integrations/spark/image1.png "cloud native architecture")
+
+Kubernetes manages stateless Spark and Hive containers elastically on the compute nodes. Spark has native scheduler integration with Kubernetes. Hive, for legacy reasons, uses YARN scheduler on top of Kubernetes.
+
+All access to MinIO object storage is via S3/SQL SELECT API. In addition to the compute nodes, MinIO containers are also managed by Kubernetes as stateful containers with local storage (JBOD/JBOF) mapped as persistent local volumes. This architecture enables multi-tenant MinIO, allowing isolation of data between customers.
+
+MinIO also supports multi-cluster, multi-site federation similar to AWS regions and tiers. Using MinIO Information Lifecycle Management (ILM), you can configure data to be tiered between NVMe based hot storage, and HDD based warm storage. All data is encrypted with per-object key. Access Control and Identity Management between the tenants are managed by MinIO using OpenID Connect or Kerberos/LDAP/AD.
+
+## **2. Prerequisites**
+
+- Install Hortonworks Distribution using this [guide.](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/ch_Installing_Ambari.html)
+  - [Setup Ambari](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/set_up_the_ambari_server.html) which automatically sets up YARN
+  - [Installing Spark](https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/installing-spark/content/installing_spark.html)
+- Install MinIO Distributed Server using one of the guides below.
+  - [Deployment based on Kubernetes](https://docs.min.io/docs/deploy-minio-on-kubernetes.html#minio-distributed-server-deployment)
+  - [Deployment based on MinIO Helm Chart](https://github.com/helm/charts/tree/master/stable/minio)
+
+## **3. Configure Hadoop, Spark, Hive to use MinIO**
+
+After successful installation navigate to the Ambari UI `http://<ambari-server>:8080/` and login using the default credentials: [**_username: admin, password: admin_**]
+
+![ambari-login](/images/integrations/spark/image3.png "ambari login")
+
+### **3.1 Configure Hadoop**
+
+Navigate to **Services** -> **HDFS** -> **CONFIGS** -> **ADVANCED** as shown below
+
+![hdfs-configs](/images/integrations/spark/image2.png "hdfs advanced configs")
+
+Navigate to **Custom core-site** to configure MinIO parameters for `_s3a_` connector
+
+![s3a-config](/images/integrations/spark/image5.png "custom core-site")
+
+```
+sudo pip install yq
+alias kv-pairify='yq ".configuration[]" | jq ".[]" | jq -r ".name + \"=\" + .value"'
+```
+
+Let's take for example a set of 12 compute nodes with an aggregate memory of _1.2TiB_, we need to do following settings for optimal results. Add the following optimal entries for _core-site.xml_ to configure _s3a_ with **MinIO**. Most important options here are
+
+```
+cat ${HADOOP_CONF_DIR}/core-site.xml | kv-pairify | grep "mapred"
+
+mapred.maxthreads.generate.mapoutput=2 # Num threads to write map outputs
+mapred.maxthreads.partition.closer=0 # Asynchronous map flushers
+mapreduce.fileoutputcommitter.algorithm.version=2 # Use the latest committer version
+mapreduce.job.reduce.slowstart.completedmaps=0.99 # 99% map, then reduce
+mapreduce.reduce.shuffle.input.buffer.percent=0.9 # Min % buffer in RAM
+mapreduce.reduce.shuffle.merge.percent=0.9 # Minimum % merges in RAM
+mapreduce.reduce.speculative=false # Disable speculation for reducing
+mapreduce.task.io.sort.factor=999 # Threshold before writing to disk
+mapreduce.task.sort.spill.percent=0.9 # Minimum % before spilling to disk
+```
+
+S3A is the connector to use S3 and other S3-compatible object stores such as MinIO. MapReduce workloads typically interact with object stores in the same way they do with HDFS. These workloads rely on HDFS atomic rename functionality to complete writing data to the datastore. Object storage operations are atomic by nature and they do not require/implement rename API. The default S3A committer emulates renames through copy and delete APIs. This interaction pattern causes significant loss of performance because of the write amplification. _Netflix_, for example, developed two new staging committers - the Directory staging committer and the Partitioned staging committer - to take full advantage of native object storage operations. These committers do not require rename operation. The two staging committers were evaluated, along with another new addition called the Magic committer for benchmarking.
+
+It was found that the directory staging committer was the fastest among the three, S3A connector should be configured with the following parameters for optimal results:
+
+```
+cat ${HADOOP_CONF_DIR}/core-site.xml | kv-pairify | grep "s3a"
+
+fs.s3a.access.key=minio
+fs.s3a.secret.key=minio123
+fs.s3a.path.style.access=true
+fs.s3a.block.size=512M
+fs.s3a.buffer.dir=${hadoop.tmp.dir}/s3a
+fs.s3a.committer.magic.enabled=false
+fs.s3a.committer.name=directory
+fs.s3a.committer.staging.abort.pending.uploads=true
+fs.s3a.committer.staging.conflict-mode=append
+fs.s3a.committer.staging.tmp.path=/tmp/staging
+fs.s3a.committer.staging.unique-filenames=true
+fs.s3a.connection.establish.timeout=5000
+fs.s3a.connection.ssl.enabled=false
+fs.s3a.connection.timeout=200000
+fs.s3a.endpoint=http://minio:9000
+fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem
+
+fs.s3a.committer.threads=2048 # Number of threads writing to MinIO
+fs.s3a.connection.maximum=8192 # Maximum number of concurrent conns
+fs.s3a.fast.upload.active.blocks=2048 # Number of parallel uploads
+fs.s3a.fast.upload.buffer=disk # Use disk as the buffer for uploads
+fs.s3a.fast.upload=true # Turn on fast upload mode
+fs.s3a.max.total.tasks=2048 # Maximum number of parallel tasks
+fs.s3a.multipart.size=512M # Size of each multipart chunk
+fs.s3a.multipart.threshold=512M # Size before using multipart uploads
+fs.s3a.socket.recv.buffer=65536 # Read socket buffer hint
+fs.s3a.socket.send.buffer=65536 # Write socket buffer hint
+fs.s3a.threads.max=2048 # Maximum number of threads for S3A
+```
+
+The rest of the other optimization options are discussed in the links below
+
+- [https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html](https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html)
+- [https://hadoop.apache.org/docs/r3.1.1/hadoop-aws/tools/hadoop-aws/committers.html](https://hadoop.apache.org/docs/r3.1.1/hadoop-aws/tools/hadoop-aws/committers.html)
+
+Once the config changes are applied, proceed to restart **Hadoop** services.
+
+![hdfs-services](/images/integrations/spark/image7.png "hdfs restart services")
+
+### **3.2 Configure Spark2**
+
+Navigate to **Services** -> **Spark2** -> **CONFIGS** as shown below
+
+![spark-config](/images/integrations/spark/image6.png "spark config")
+
+Navigate to “**Custom spark-defaults**” to configure MinIO parameters for `_s3a_` connector
+
+![spark-config](/images/integrations/spark/image9.png "spark defaults")
+
+Add the following optimal entries for _spark-defaults.conf_ to configure Spark with **MinIO**.
+
+```
+spark.hadoop.fs.s3a.access.key minio
+spark.hadoop.fs.s3a.secret.key minio123
+spark.hadoop.fs.s3a.path.style.access true
+spark.hadoop.fs.s3a.block.size 512M
+spark.hadoop.fs.s3a.buffer.dir ${hadoop.tmp.dir}/s3a
+spark.hadoop.fs.s3a.committer.magic.enabled false
+spark.hadoop.fs.s3a.committer.name directory
+spark.hadoop.fs.s3a.committer.staging.abort.pending.uploads true
+spark.hadoop.fs.s3a.committer.staging.conflict-mode append
+spark.hadoop.fs.s3a.committer.staging.tmp.path /tmp/staging
+spark.hadoop.fs.s3a.committer.staging.unique-filenames true
+spark.hadoop.fs.s3a.committer.threads 2048 # number of threads writing to MinIO
+spark.hadoop.fs.s3a.connection.establish.timeout 5000
+spark.hadoop.fs.s3a.connection.maximum 8192 # maximum number of concurrent conns
+spark.hadoop.fs.s3a.connection.ssl.enabled false
+spark.hadoop.fs.s3a.connection.timeout 200000
+spark.hadoop.fs.s3a.endpoint http://minio:9000
+spark.hadoop.fs.s3a.fast.upload.active.blocks 2048 # number of parallel uploads
+spark.hadoop.fs.s3a.fast.upload.buffer disk # use disk as the buffer for uploads
+spark.hadoop.fs.s3a.fast.upload true # turn on fast upload mode
+spark.hadoop.fs.s3a.impl org.apache.hadoop.spark.hadoop.fs.s3a.S3AFileSystem
+spark.hadoop.fs.s3a.max.total.tasks 2048 # maximum number of parallel tasks
+spark.hadoop.fs.s3a.multipart.size 512M # size of each multipart chunk
+spark.hadoop.fs.s3a.multipart.threshold 512M # size before using multipart uploads
+spark.hadoop.fs.s3a.socket.recv.buffer 65536 # read socket buffer hint
+spark.hadoop.fs.s3a.socket.send.buffer 65536 # write socket buffer hint
+spark.hadoop.fs.s3a.threads.max 2048 # maximum number of threads for S3A
+```
+
+Once the config changes are applied, proceed to restart **Spark** services.
+
+![spark-config](/images/integrations/spark/image12.png "spark restart services")
+
+### **3.3 Configure Hive**
+
+Navigate to **Services** -> **Hive** -> **CONFIGS**-> **ADVANCED** as shown below
+
+![hive-config](/images/integrations/spark/image10.png "hive advanced config")
+
+Navigate to “**Custom hive-site**” to configure MinIO parameters for `_s3a_` connector
+
+![hive-config](/images/integrations/spark/image11.png "hive advanced config")
+
+Add the following optimal entries for `hive-site.xml` to configure Hive with **MinIO**.
+
+```
+hive.blobstore.use.blobstore.as.scratchdir=true
+hive.exec.input.listing.max.threads=50
+hive.load.dynamic.partitions.thread=25
+hive.metastore.fshandler.threads=50
+hive.mv.files.threads=40
+mapreduce.input.fileinputformat.list-status.num-threads=50
+```
+
+For more information about these options please visit [https://www.cloudera.com/documentation/enterprise/5-11-x/topics/admin_hive_on_s3_tuning.html](https://www.cloudera.com/documentation/enterprise/5-11-x/topics/admin_hive_on_s3_tuning.html)
+
+![hive-config](/images/integrations/spark/image13.png "hive advanced custom config")
+
+Once the config changes are applied, proceed to restart all Hive services.
+
+![hive-config](/images/integrations/spark/image14.png "restart hive services")
+
+## **4. Run Sample Applications**
+
+After installing Hive, Hadoop and Spark successfully, we can now proceed to run some sample applications to see if they are configured appropriately.  We can use Spark Pi and Spark WordCount programs to validate our Spark installation. We can also explore how to run Spark jobs from the command line and Spark shell.
+
+### **4.1 Spark Pi**
+
+Test the Spark installation by running the following compute intensive example, which calculates pi by “throwing darts” at a circle. The program generates points in the unit square ((0,0) to (1,1)) and counts how many points fall within the unit circle within the square. The result approximates pi.
+
+Follow these steps to run the Spark Pi example:
+
+- Login as user **‘spark’**.
+- When the job runs, the library can now use **MinIO** during intermediate processing.
+- Navigate to a node with the Spark client and access the spark2-client directory:
+
+```
+cd /usr/hdp/current/spark2-client
+su spark
+```
+
+- Run the Apache Spark Pi job in yarn-client mode, using code from **org.apache.spark**:
+
+```
+./bin/spark-submit --class org.apache.spark.examples.SparkPi \
+    --master yarn-client \
+    --num-executors 1 \
+    --driver-memory 512m \
+    --executor-memory 512m \
+    --executor-cores 1 \
+    examples/jars/spark-examples*.jar 10
+```
+
+The job should produce an output as shown below. Note the value of pi in the output.
+
+```
+17/03/22 23:21:10 INFO DAGScheduler: Job 0 finished: reduce at SparkPi.scala:38, took 1.302805 s
+Pi is roughly 3.1445191445191445
+```
+
+Job status can also be viewed in a browser by navigating to the YARN ResourceManager Web UI and clicking on job history server information.
+
+### **4.2 WordCount**
+
+WordCount is a simple program that counts how often a word occurs in a text file. The code builds a dataset of (String, Int) pairs called counts, and saves the dataset to a file.
+
+The following example submits WordCount code to the Scala shell. Select an input file for the Spark WordCount example. We can use any text file as input.
+
+- Login as user **‘spark’**.
+- When the job runs, the library can now use **MinIO** during intermediate processing.
+- Navigate to a node with Spark client and access the spark2-client directory:
+
+```
+cd /usr/hdp/current/spark2-client
+su spark
+```
+
+The following example uses _log4j.properties_ as the input file:
+
+#### **4.2.1 Upload the input file to HDFS:**
+
+```
+hadoop fs -copyFromLocal /etc/hadoop/conf/log4j.properties
+          s3a://testbucket/testdata
+```
+
+#### **4.2.2  Run the Spark shell:**
+
+```
+./bin/spark-shell --master yarn-client --driver-memory 512m --executor-memory 512m
+```
+
+The command should produce an output as shown below. (with additional status messages):
+
+```
+Spark context Web UI available at http://172.26.236.247:4041
+Spark context available as 'sc' (master = yarn, app id = application_1490217230866_0002).
+Spark session available as 'spark'.
+Welcome to
+
+
+      ____              __
+     / __/__  ___ _____/ /__
+    _\ \/ _ \/ _ `/ __/  '_/
+   /___/ .__/\_,_/_/ /_/\_\   version 2.1.0.2.6.0.0-598
+      /_/
+
+Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112)
+Type in expressions to have them evaluated.
+Type :help for more information.
+
+scala>
+```
+
+- At the _scala>_ prompt, submit the job by typing the following commands, Replace node names, file name, and file location with your values:
+
+```
+scala> val file = sc.textFile("s3a://testbucket/testdata")
+file: org.apache.spark.rdd.RDD[String] = s3a://testbucket/testdata MapPartitionsRDD[1] at textFile at <console>:24
+
+scala> val counts = file.flatMap(line => line.split(" ")).map(word => (word, 1)).reduceByKey(_ + _)
+counts: org.apache.spark.rdd.RDD[(String, Int)] = ShuffledRDD[4] at reduceByKey at <console>:25
+
+scala> counts.saveAsTextFile("s3a://testbucket/wordcount")
+```
+
+Use one of the following approaches to view job output:
+
+View output in the Scala shell:
+
+```
+scala> counts.count()
+364
+```
+
+To view the output from MinIO exit the Scala shell. View WordCount job status:
+
+```
+hadoop fs -ls s3a://testbucket/wordcount
+```
+
+The output should be similar to the following:
+
+```
+Found 3 items
+-rw-rw-rw-   1 spark spark          0 2019-05-04 01:36 s3a://testbucket/wordcount/_SUCCESS
+-rw-rw-rw-   1 spark spark       4956 2019-05-04 01:36 s3a://testbucket/wordcount/part-00000
+-rw-rw-rw-   1 spark spark       5616 2019-05-04 01:36 s3a://testbucket/wordcount/part-00001
+```

source/integrations/generate-lets-encrypt-certificate-using-certbot-for-minio.md

@@ -0,0 +1,72 @@
+# Generate Let's Encrypt certificate using Certbot for MinIO [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
+[Let’s Encrypt](https://letsencrypt.org/) is a new free, automated, and open source, Certificate Authority.
+
+[Certbot](https://certbot.eff.org/) is a console based certificate generation tool for Let's Encrypt.
+
+In this recipe, we will generate a Let's Encypt certificate using Certbot. This certificate will then be deployed for use in the MinIO server.
+
+## 1. Prerequisites
+- Install MinIO Server from [here](https://docs.min.io/docs/minio-quickstart-guide).
+- Install Certbot from [here](https://certbot.eff.org/)
+
+## 2. Dependencies
+- Port 443 for https needs to be open and available at time of executing `certbot`.
+- Certbot needs root access while executing because only root is allowed to bind to any port below 1024.
+- We will be using our own domain ``myminio.com``  as an example in this recipe. Replace with your own domain under your setup.
+
+## 3. Recipe Steps
+
+### Step 1: Install Certbot
+Install Certbot by following the documentation at https://certbot.eff.org/
+
+### Step 2: Generate Let's Encrypt cert
+```sh
+# certbot certonly --standalone -d myminio.com --staple-ocsp -m test@yourdomain.io --agree-tos
+```
+
+### Step 3: Verify Certificates
+List your certs saved in `/etc/letsencrypt/live/myminio.com` directory.
+```sh
+$ ls -l /etc/letsencrypt/live/myminio.com
+total 4
+lrwxrwxrwx 1 root root  37 Aug  2 09:58 cert.pem -> ../../archive/myminio.com/cert4.pem
+lrwxrwxrwx 1 root root  38 Aug  2 09:58 chain.pem -> ../../archive/myminio.com/chain4.pem
+lrwxrwxrwx 1 root root  42 Aug  2 09:58 fullchain.pem -> ../../archive/myminio.com/fullchain4.pem
+lrwxrwxrwx 1 root root  40 Aug  2 09:58 privkey.pem -> ../../archive/myminio.com/privkey4.pem
+-rw-r--r-- 1 root root 543 May 10 22:07 README
+```
+
+### Step 4: Set up SSL on MinIO Server with the certificates.
+The certificate and key generated via Certbot needs to be placed inside user's home directory.
+```sh
+$ cp /etc/letsencrypt/live/myminio.com/fullchain.pem /home/user/.minio/certs/public.crt
+$ cp /etc/letsencrypt/live/myminio.com/privkey.pem /home/user/.minio/certs/private.key
+```
+
+### Step 5: Change ownership of certificates.
+```sh
+$ sudo chown user:user /home/user/.minio/certs/private.key
+$ sudo chown user:user /home/user/.minio/certs/public.crt
+```
+
+### Step 6: Start MinIO Server using HTTPS.
+
+If you are not going to run MinIO with `root` privileges, you will need to give MinIO the capability of listening on ports less than 1024 using the following command:
+
+```sh
+sudo setcap 'cap_net_bind_service=+ep' ./minio
+```
+
+Now, you can start MinIO Server on port "443".
+
+```sh
+$ ./minio server --address ":443" /mnt/data
+```
+
+If you are using dockerized version of MinIO then you would need to
+```sh
+$ sudo docker run -p 443:443 -v /home/user/.minio:/root/.minio/ -v /home/user/data:/data minio/minio server --address ":443" /data
+```
+
+### Step 7: Visit <https://myminio.com> in the browser.
+![Letsencrypt](https://github.com/minio/cookbook/blob/master/docs/screenshots/letsencrypt-certbot-minio.jpg?raw=true)

source/integrations/integrations.rst

@@ -0,0 +1,25 @@
+============
+Integrations
+============
+
+.. default-domain:: minio
+
+.. contents:: Table of Contents
+   :local:
+   :depth: 2
+
+The following pages provide integration guides between MinIO and select third-party software.
+The guides on this page were previously hosted under the :guilabel:`Cookbook` heading of our legacy documentation.
+
+All provided guides assume familiarity with the third-party integration software, and do not replace the official documentation for that software.
+
+.. toctree::
+   :titlesonly:
+   :hidden:
+
+   /integrations/using-minio-with-veeam.md
+   /integrations/disaggregated-spark-and-hadoop-hive-with-minio.md
+   /integrations/aws-cli-with-minio.md
+   /integrations/setup-nginx-proxy-with-minio.md
+   /integrations/presigned-put-upload-via-browser.md
+   /integrations/generate-lets-encrypt-certificate-using-certbot-for-minio.md

source/integrations/presigned-put-upload-via-browser.md

@@ -0,0 +1,105 @@
+# Upload Files Using Pre-signed URLs [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
+
+Using pre-signed URLs, a client can upload files directly to an S3-compatible cloud storage server (S3) without exposing the S3 credentials to the user. 
+
+This guide describes how to use the [`presignedPutObject`](https://docs.min.io/docs/javascript-client-api-reference#presignedPutObject) API from the [MinIO JavaScript Library](https://github.com/minio/minio-js) to generate a pre-signed URL. This is demonstrated through a JavaScript example in which an Express Node.js server exposes an endpoint to generate a pre-signed URL and a client-side web application uploads a file to MinIO Server using that URL.
+
+1. [Create the Server](#createserver) 
+2. [Create the Client-side Web Application](#createclient)
+
+## <a name="createserver"></a>1. Create the Server
+The server consists of an [Express](https://expressjs.com) Node.js server that exposes an endpoint called `/presignedUrl`. This endpoint uses a `Minio.Client` object to generate a short-lived, pre-signed URL that can be used to upload a file to Mino Server.
+
+```js
+// In order to use the MinIO JavaScript API to generate the pre-signed URL, begin by instantiating
+// a `Minio.Client` object and pass in the values for your server.
+// The example below uses values for play.min.io:9000
+
+const Minio = require('minio')
+
+var client = new Minio.Client({
+    endPoint: 'play.min.io',
+    port: 9000,
+    useSSL: true,
+    accessKey: 'Q3AM3UQ867SPQQA43P2F',
+    secretKey: 'zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG'
+})
+
+// Instantiate an `express` server and expose an endpoint called `/presignedUrl` as a `GET` request that
+// accepts a filename through a query parameter called `name`. For the implementation of this endpoint,
+// invoke [`presignedPutObject`](https://docs.min.io/docs/javascript-client-api-reference#presignedPutObject) 
+// on the `Minio.Client` instance to generate a pre-signed URL, and return that URL in the response:
+
+// express is a small HTTP server wrapper, but this works with any HTTP server
+const server = require('express')()
+
+server.get('/presignedUrl', (req, res) => {
+    client.presignedPutObject('uploads', req.query.name, (err, url) => {
+        if (err) throw err
+        res.end(url)
+    })
+})
+
+server.get('/', (req, res) => {
+    res.sendFile(__dirname + '/index.html');
+})
+
+server.listen(8080)
+```
+
+## <a name="createclient"></a>2. Create the Client-side Web Application
+The client-side web application's user interface contains a selector field that allows the user to select files for upload, as well as a button that invokes an `onclick` handler called `upload`:
+
+```html
+<input type="file" id="selector" multiple>
+<button onclick="upload()">Upload</button>
+
+<div id="status">No uploads</div>
+
+<script type="text/javascript">
+  // `upload` iterates through all files selected and invokes a helper function called `retrieveNewURL`.
+  function upload() {
+        // Get selected files from the input element.
+        var files = document.querySelector("#selector").files;
+        for (var i = 0; i < files.length; i++) {
+            var file = files[i];
+            // Retrieve a URL from our server.
+            retrieveNewURL(file, (file, url) => {
+                // Upload the file to the server.
+                uploadFile(file, url);
+            });
+        }
+    }
+
+    // `retrieveNewURL` accepts the name of the current file and invokes the `/presignedUrl` endpoint to
+    // generate a pre-signed URL for use in uploading that file: 
+    function retrieveNewURL(file, cb) {
+        fetch(`/presignedUrl?name=${file.name}`).then((response) => {
+            response.text().then((url) => {
+                cb(file, url);
+            });
+        }).catch((e) => {
+            console.error(e);
+        });
+    }
+
+    // ``uploadFile` accepts the current filename and the pre-signed URL. It then uses `Fetch API`
+    // to upload this file to S3 at `play.min.io:9000` using the URL:
+    function uploadFile(file, url) {
+        if (document.querySelector('#status').innerText === 'No uploads') {
+            document.querySelector('#status').innerHTML = '';
+        }
+        fetch(url, {
+            method: 'PUT',
+            body: file
+        }).then(() => {
+            // If multiple files are uploaded, append upload status on the next line.
+            document.querySelector('#status').innerHTML += `<br>Uploaded ${file.name}.`;
+        }).catch((e) => {
+            console.error(e);
+        });
+    }
+</script>
+```
+
+**Note:** This uses the [File API](https://developer.mozilla.org/en-US/docs/Web/API/File), [QuerySelector API](https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelector), [fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) & [Promise API](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise).

source/integrations/setup-nginx-proxy-with-minio.md

@@ -0,0 +1,113 @@
+# Set up Nginx proxy with MinIO Server [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
+
+Nginx is an open source Web server and a reverse proxy server.
+
+In this recipe we will learn how to set up Nginx proxy with MinIO Server.
+
+## 1. Prerequisites
+
+Install MinIO Server from [here](https://docs.min.io/docs/minio-quickstart-guide).
+
+## 2. Installation
+
+Install Nginx from [here](http://nginx.org/en/download.html).
+
+## 3. Configuration
+
+### Proxy all requests
+Add the following content as a file ``/etc/nginx/sites-enabled``, e.g. ``/etc/nginx/sites-enables/minio``  and also remove the existing ``default`` file in same directory.
+
+```sh
+server {
+ listen 80;
+ server_name example.com;
+
+ # To allow special characters in headers
+ ignore_invalid_headers off;
+ # Allow any size file to be uploaded.
+ # Set to a value such as 1000m; to restrict file size to a specific value
+ client_max_body_size 0;
+ # To disable buffering
+ proxy_buffering off;
+
+ location / {
+   proxy_set_header X-Real-IP $remote_addr;
+   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+   proxy_set_header X-Forwarded-Proto $scheme;
+   proxy_set_header Host $http_host;
+
+   proxy_connect_timeout 300;
+   # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
+   proxy_http_version 1.1;
+   proxy_set_header Connection "";
+   chunked_transfer_encoding off;
+
+   proxy_pass http://localhost:9000; # If you are using docker-compose this would be the hostname i.e. minio
+   # Health Check endpoint might go here. See https://www.nginx.com/resources/wiki/modules/healthcheck/
+   # /minio/health/live;
+ }
+}
+```
+
+Note:
+
+* Replace example.com with your own hostname.
+* Replace ``http://localhost:9000``  with your own server name.
+* Add ``client_max_body_size 1000m;`` in the ``http`` context in order to be able to upload large files — simply adjust the value accordingly. The default value is `1m` which is far too low for most scenarios. To disable checking of client request body size, set ``client_max_body_size`` to `0`.
+* Nginx buffers responses by default. To disable Nginx from buffering MinIO response to temp file, set `proxy_buffering off;`. This will improve time-to-first-byte for client requests.
+* Nginx disallows special characters by default.  Set ``ignore_invalid_headers off;`` to allow headers with special characters.
+
+### Proxy requests based on the bucket
+If you want to serve web-application and MinIO from the same nginx port then you can proxy the MinIO requests based on the bucket name using path based routing. For nginx this uses the `location` directive, which also supports object key pattern-match based proxy splitting.
+
+```sh
+ # Proxy requests to the bucket "photos" to MinIO server running on port 9000
+ location /photos/ {
+   proxy_set_header X-Real-IP $remote_addr;
+   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+   proxy_set_header X-Forwarded-Proto $scheme;
+   proxy_set_header Host $http_host;
+
+   proxy_connect_timeout 300;
+   # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
+   proxy_http_version 1.1;
+   proxy_set_header Connection "";
+   chunked_transfer_encoding off;
+
+   proxy_pass http://localhost:9000;
+ }
+
+ # Proxy any other request to the application server running on port 9001
+ location / {
+   proxy_set_header X-Real-IP $remote_addr;
+   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+   proxy_set_header X-Forwarded-Proto $scheme;
+   proxy_set_header Host $http_host;
+
+   proxy_connect_timeout 300;
+   # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
+   proxy_http_version 1.1;
+   proxy_set_header Connection "";
+   chunked_transfer_encoding off;
+
+   proxy_pass http://localhost:9001;
+ }
+```
+
+## 4. Recipe Steps
+
+### Step 1: Start MinIO server.
+
+```sh
+minio server /mydatadir
+```
+
+### Step 2: Restart Nginx server.
+
+```sh
+sudo service nginx restart
+```
+
+## Explore Further
+
+Refer [this blog post](https://www.nginx.com/blog/enterprise-grade-cloud-storage-nginx-plus-minio/) for various MinIO and Nginx configuration options.

source/integrations/using-minio-with-veeam.md

@@ -0,0 +1,93 @@
+# Using MinIO with Veeam
+
+When using Veeam Backup and Replication, you can use S3 compatible object storage such as MinIO as a capacity tier for backups.  This disaggregates storage for the Veeam infrastructure and allows you to retain control of your data. With the ease of use of setup and administration of MinIO, it allows a Veeam backup admin to easily deploy their own object store for capacity tiering.
+
+## Prerequisites
+
+- One or both of Veeam Backup and Replication with support for S3 compatible object store (e.g. 9.5.4) and Veeam Backup for Office365 (VBO)
+- MinIO object storage set up per <https://docs.min.io/docs/minio-quickstart-guide.html>
+- Veeam requires TLS connections to the object storage.  This can be configured per <https://docs.minio.io/docs/how-to-secure-access-to-minio-server-with-tls.html>
+- The S3 bucket, Access Key and Secret Key have to be created before and outside of Veeam.
+- Configure the minio client for the Veeam MinIO endpoint - <https://docs.min.io/docs/minio-client-quickstart-guide.html>
+
+## Setting up an S3 compatible object store for Veeam Backup and Replication
+
+### Create a bucket for Veeam backups
+
+Create a bucket for Veeam Backup, e.g.,
+
+```
+mc mb myminio/veeambackup
+```
+
+> NOTE: For Veeam Backup with Immutability, create the bucket with object lock enabled, e.g.,
+
+```
+mc mb -l myminio/veeambackup
+```
+
+> Object locking requires erasure coding enabled on the minio server. For more information see <https://docs.minio.io/docs/minio-erasure-code-quickstart-guide.html>.
+
+### Add MinIO as an object store for Veeam
+
+Follow the steps from the Veeam documentation for adding MinIO as an object store - <https://helpcenter.veeam.com/docs/backup/vsphere/adding_s3c_object_storage.html?ver=100>
+
+For Veeam Backup with Immutability, choose the amount of days you want to make backups immutable for
+
+![Choose Immutability Days for Object Store](/images/integrations/veeam/object_store_immutable_days.png)
+
+### Creating the Scale-out Backup Repository
+
+- Under the Backup Infrastructure view, click on Scale-out Repositories and click the Add Scale-out Repository button on the ribbon.
+
+- Follow the on screen wizard
+
+- On the Capacity Tier screen, check the box to Extend scale-out backup repository capacity with object storage checkbox and select the object storage. If you want to be able to test backup data immediately after a job is run, under the object storage selection, check the "Copy" box and uncheck the "Move" box.
+
+### Create a backup job
+
+#### Backup Virtual Machines with Veeam Backup and Replication
+
+- Under Home > Jobs > Backup in Navigation Pane, click on Backup Job button in the ribbon and choose Virtual Machine. Follow the on screen wizard.
+
+- On the Storage screen, choose the Scale-out Backup Repository that was configured previously.
+
+- Continue with the backup job creation.  On the Summary screen, check the Run the Job when I click Finish checkbox and click the Finish button. The backup job will start immediately.  This will create an Active Full backup of the VMs within the backup job.
+
+- Since we selected Copy mode when creating the SOBR, the backup will be copied to the capacity tier as soon as it is created on the performance tier.
+
+- For Veeam Backup with Immutability, you can choose a number of restore points or days to make backups immutable.
+
+![Choose Immutability Options for Backups](/images/integrations/veeam/backup_job_immutable_days.png)
+
+#### Backup Office 365 with VBO
+
+- Create a new bucket for VBO backups
+
+```
+mc mb -l myminio/vbo
+```
+
+- Under Backup Infrastructure, right click on Object Storage Repositories and choose "Add object storage"
+
+![Adding Object Storage to VBO Step 1](/images/integrations/veeam/1_add_object_store.png)
+
+- Follow through the wizard as above for Veeam Backup and Replication as the steps are the same between both products
+
+- Under Backup Infrastructure -> Backup Repositories, right click and "Add Backup Repository"
+
+- Follow the wizard.  Under the "Object Storage Backup Repository" section, choose the MinIO object storage you created above
+
+![Adding Object Storage to VBO Backup Repository](/images/integrations/veeam/6_add_sobr_with_object_store.png)
+
+- When you create your backup job, choose the backup repository you created above.
+
+## Test the setup
+
+The next time the backup job runs, you can use the  `mc admin trace myminio` command and verify traffic is flowing to the MinIO nodes. For Veeam Backup and Replication you will need to wait for the backup to complete to the performance tier before it migrates data to the capacity tier (i.e., MinIO).
+
+```
+20:09:10.216 [200 OK] s3.GetObject veeam-minio01:9000/vbo/Veeam/Backup365/vbotest/Organizations/6571606ecbc4455dbfe23b83f6f45597/Webs/ca2d0986229b4ec88e3a217ef8f04a1d/Items/efaa67764b304e77badb213d131beab6/f4f0cf600f494c3eb702d8eafe0fabcc.aac07493e6cd4c71845d2495a4e1e19b 139.178.68.158    9.789ms      ↑ 90 B ↓ 8.5 KiB
+20:09:10.244 [200 OK] s3.GetObject veeam-minio01:9000/vbo/Veeam/Backup365/vbotest/RepositoryLock/cad99aceb50c49ecb9e07246c3b9fadc_bfd985e5deec4cebaf481847f2c34797 139.178.68.158    16.21ms      ↑ 90 B ↓ 402 B
+20:09:10.283 [200 OK] s3.PutObject veeam-minio01:9000/vbo/Veeam/Backup365/vbotest/CommonInfo/WebRestorePoints/18f1aba8f55f4ac6b805c4de653eb781 139.178.68.158    29.787ms     ↑ 1005 B ↓ 296 B
+```