minio/docs
1
0
Fork 0
mirror of https://github.com/minio/docs.git synced 2025-07-30 07:03:26 +03:00
Code Activity

Multiple SERVER Releases (#1141)

Browse Source 
MinIO SERVER RELEASE.2024-01-05T22-17-24Z - added new metrics to
github.com/minio/minio for later sync

MinIO SERVER RELEASE.2024-01-28T22-35-53Z - MinIO preallocates memory,
mc update compresses binary in transit MinIO SERVER
RELEASE.2024-02-06T21-36-22Z -

MinIO adds condition key for restricting STS AssumeRoleWithWebIdentity
duration at policy level

Closes #1124 , 

Partially addresses #1116 
Partially Addresses #1105


---------

Co-authored-by: Daryl White <53910321+djwfyi@users.noreply.github.com>
This commit is contained in:
Ravind Kumar
2024-03-11 10:06:46 -04:00
committed by GitHub
parent 54584b290c
commit 67e81ce2f8
11 changed files with 179 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
source/administration/identity-access-management/policy-based-access-control.rst
View File

@ -675,6 +675,40 @@ MinIO supports the following condition keys for all supported
For additional keys supported by a specific S3 action, see the reference documentation for that action.
MinIO Extended Condition Keys
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO extends the S3 standard condition keys with the following extended key:
``sts:DurationSeconds``
.. versionadded:: MinIO SERVER RELEASE.2024-02-06T21-36-22Z
Specify a time in seconds to limit the duration of *all* Security Token Service credentials generated by :ref:`minio-sts-assumerolewithwebidentity`.
This value overrides the ``DurationSeconds`` field specified to the client.
For example:
.. code-block:: json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRoleWithWebIdentity"
],
"Condition": {
"NumericLessThanEquals": {
"sts:DurationSeconds": "300"
}
}
}
]
}
.. _minio-policy-mc-admin-actions:
``mc admin`` Policy Action Keys

4
source/operations/checklists/hardware.rst
View File

@ -189,6 +189,10 @@ The following table provides general guidelines for allocating memory for use by
* - More than 1 Pebibyte (Pi)
- 128GiB
.. important::
Starting with :minio-release:`RELEASE.2024-01-28T22-35-53Z`, MinIO preallocates 2GiB of memory per node in distributed setups and 1GiB of memory for a single-node setup.
.. _minio-hardware-checklist-storage:
Storage

2
source/operations/data-recovery/recover-after-site-failure.rst
View File

@ -92,7 +92,7 @@ This procedure assumes a *total loss* of one or more peer sites versus replicati
All healthy peers in the site replication configuration update for the new peer automatically.
You can use the :mc-cmd:`mc admin replicate info` command to verify the new site replication configuration.
#. Resynchronize the new peer with :mc:`mc admin replicate resync`.
#. Resynchronize the new peer with :mc-cmd:`mc admin replicate resync`.
.. code-block:: shell
:class: copyable

10
source/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.rst
View File

@ -104,6 +104,16 @@ Storage Requirements
:start-after: start-storage-requirements-desc
:end-before: end-storage-requirements-desc
Memory Requirements
~~~~~~~~~~~~~~~~~~~
.. versionchanged:: RELEASE.2024-01-28T22-35-53Z
MinIO pre-allocates 2GiB of system memory at startup.
MinIO recommends a *minimum* of 32GiB of memory per host.
See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.
Time Synchronization
~~~~~~~~~~~~~~~~~~~~

10
source/operations/install-deploy-manage/deploy-minio-single-node-multi-drive.rst
View File

@ -33,6 +33,16 @@ Storage Requirements
:start-after: start-storage-requirements-desc
:end-before: end-storage-requirements-desc
Memory Requirements
~~~~~~~~~~~~~~~~~~~
.. versionchanged:: RELEASE.2024-01-28T22-35-53Z
MinIO pre-allocates 2GiB of system memory at startup.
MinIO recommends a *minimum* of 32GiB of memory per host.
See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.
.. _deploy-minio-standalone-multidrive:
Deploy Single-Node Multi-Drive MinIO

44
source/operations/install-deploy-manage/deploy-minio-single-node-single-drive.rst
View File

@ -11,14 +11,8 @@ Deploy MinIO: Single-Node Single-Drive
:depth: 2
The procedures on this page cover deploying MinIO in a Single-Node Single-Drive (SNSD) configuration for early development and evaluation.
|SNSD| deployments provide no added reliability or availability beyond what the underlying storage volume implements (RAID, LVM, ZFS, etc.).
Starting with :minio-release:`RELEASE.2022-06-02T02-11-04Z`, MinIO implements a zero-parity erasure coded backend for single-node single-drive deployments.
This feature allows access to :ref:`erasure coding dependent features <minio-erasure-coding>` without the requirement of multiple drives.
This mode **requires** accessing stored objects through the S3 API, and does **not** support direct access to objects through the filesystem/POSIX interface.
MinIO only starts in |SNSD| mode if the storage volume or path is empty *or* only contain files generated by a previous |SNSD| deployment.
See the documentation on :ref:`SNSD behavior with pre-existing data <minio-snsd-pre-existing-data>` for more information.
|SNSD| deployments use a zero-parity erasure coded backend that provides no added reliability or availability beyond what the underlying storage volume implements.
These deployments are best suited for local testing and evaluation, or for small-scale data workloads that do not have availability or performance requirements.
.. cond:: container
@ -67,6 +61,37 @@ The following table lists the possible storage volume states and MinIO behavior:
.. versionchanged:: RELEASE.2022-10-29T06-21-33Z
Prerequisites
-------------
Storage Requirements
~~~~~~~~~~~~~~~~~~~~
The following requirements summarize the :ref:`minio-hardware-checklist-storage` section of MinIO's hardware recommendations:
Use Local Storage
Direct-Attached Storage (DAS) has significant performance and consistency advantages over networked storage (:abbr:`NAS (Network Attached Storage)`, :abbr:`SAN (Storage Area Network)`, :abbr:`NFS (Network File Storage)`).
MinIO strongly recommends flash storage (NVMe, SSD) for primary or "hot" data.
Use XFS-Formatting for Drives
MinIO strongly recommends provisioning XFS formatted drives for storage.
MinIO uses XFS as part of internal testing and validation suites, providing additional confidence in performance and behavior at all scales.
Persist Drive Mounting and Mapping Across Reboots
Use ``/etc/fstab`` to ensure consistent drive-to-mount mapping across node reboots.
Non-Linux Operating Systems should use the equivalent drive mount management tool.
Memory Requirements
~~~~~~~~~~~~~~~~~~~
.. versionchanged:: RELEASE.2024-01-28T22-35-53Z
MinIO pre-allocates 2GiB of system memory at startup.
MinIO recommends a *minimum* of 32GiB of memory per host.
See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.
.. _deploy-minio-standalone:
Deploy Single-Node Single-Drive MinIO
@ -83,9 +108,6 @@ The following procedure deploys MinIO consisting of a single MinIO server and a
MinIO cannot provide consistency guarantees if the underlying storage
volumes are NFS or a similar network-attached storage volume.
For deployments that *require* using network-attached storage, use
NFSv4 for best results.
.. cond:: linux
.. include:: /includes/linux/steps-deploy-minio-single-node-single-drive.rst

1
source/operations/install-deploy-manage/deploy-minio-tenant.rst
View File

@ -362,6 +362,7 @@ Settings marked with an asterisk :guilabel:`*` are *required*:
* - :guilabel:`Memory per Node [Gi]`
- Specify the total amount of memory (RAM) to allocate per MinIO server pod.
See :ref:`minio-hardware-checklist-memory` for guidance on setting this value.
MinIO **requires** a minimum of 2GiB of memory per worker.
The Kubernetes cluster *must* have worker nodes with sufficient free RAM to match the pod request.

20
source/operations/monitoring/metrics-and-alerts.rst
View File

@ -46,6 +46,20 @@ MinIO provides scraping endpoints for the following metric groups:
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
.. tab-item:: Resource Metrics
.. versionadded:: RELEASE.2023-10-07T15-07-38Z
You can scrape :ref:`resource metrics <minio-available-resource-metrics>` using the following URL endpoint:
.. code-block:: shell
:class: copyable
http://HOSTNAME:PORT/minio/v2/metrics/resource
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
MinIO by default requires authentication for scraping the metrics endpoints.
Use the :mc-cmd:`mc admin prometheus generate` command to generate the necessary bearer tokens.
@ -110,6 +124,7 @@ Each metric includes a label for the MinIO server which generated that metric.
- :ref:`Cluster Metrics <minio-available-cluster-metrics>`
- :ref:`Bucket Metrics <minio-available-bucket-metrics>`
- :ref:`Resource Metrics <minio-available-resource-metrics>`
.. _minio-available-cluster-metrics:
@ -121,6 +136,11 @@ Each metric includes a label for the MinIO server which generated that metric.
.. include:: /includes/common-metrics-bucket.md
:parser: myst_parser.sphinx_
.. _minio-available-resource-metrics:
.. include:: /includes/common-metrics-resource.md
:parser: myst_parser.sphinx_
.. toctree::
:titlesonly:
:hidden:

77
source/operations/monitoring/minio-logging.rst
View File

@ -320,51 +320,66 @@ MinIO audit logs resemble the following JSON document:
{
"version": "1",
"deploymentid": "bc0e4d1e-bacc-42eb-91ad-2d7f3eacfa8d",
"time": "2019-08-12T21:34:37.187817748Z",
"deploymentid": "8ca2b7ad-20cf-4d07-9efb-28b2f519f4a5",
"time": "2024-02-29T19:39:25.744431903Z",
"event": "",
"trigger": "incoming",
"api": {
"name": "PutObject",
"bucket": "testbucket",
"object": "hosts",
"name": "CompleteMultipartUpload",
"bucket": "data",
"object": "test-data.csv",
"status": "OK",
"statusCode": 200,
"timeToFirstByte": "366333ns",
"timeToResponse": "16438202ns"
"rx": 267,
"tx": 358,
"txHeaders": 387,
"timeToFirstByte": "2096989ns",
"timeToFirstByteInNS": "2096989",
"timeToResponse": "2111986ns",
"timeToResponseInNS": "2111986"
},
"remotehost": "127.0.0.1",
"requestID": "15BA4A72C0C70AFC",
"userAgent": "MinIO (linux; amd64) minio-go/v6.0.32 mc/2019-08-12T18:27:13Z",
"requestID": "17B86CB0ED88EBE9",
"userAgent": "MinIO (linux; amd64) minio-go/v7.0.67 mc/RELEASE.2024-02-24T01-33-20Z",
"requestPath": "/data/test-data.csv",
"requestHost": "minio.example.net:9000",
"requestQuery": {
"uploadId": "OGNhMmI3YWQtMjBjZi00ZDA3LTllZmItMjhiMmY1MTlmNGE1LmU3MjNlNWI4LTNiYWYtNDYyNy1hNzI3LWMyNDE3NTVjMmMzNw"
},
"requestHeader": {
"Authorization": "AWS4-HMAC-SHA256 Credential=minio/20190812/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length,Signature=d3f02a6aeddeb29b06e1773b6a8422112890981269f2463a26f307b60423177c",
"Content-Length": "686",
"Accept-Encoding": "zstd,gzip",
"Authorization": "AWS4-HMAC-SHA256 Credential=minioadmin/20240229/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=ccb3acdc1763509a88a7e4a3d7fe431ef0ee5ca3f66ccb430d5a09326e87e893",
"Content-Length": "267",
"Content-Type": "application/octet-stream",
"User-Agent": "MinIO (linux; amd64) minio-go/v6.0.32 mc/2019-08-12T18:27:13Z",
"X-Amz-Content-Sha256": "STREAMING-AWS4-HMAC-SHA256-PAYLOAD",
"X-Amz-Date": "20190812T213437Z",
"X-Amz-Decoded-Content-Length": "512"
"User-Agent": "MinIO (linux; amd64) minio-go/v7.0.67 mc/RELEASE.2024-02-24T01-33-20Z",
"X-Amz-Content-Sha256": "d61969719ee94f43c4e87044229b7a13b54cab320131e9a77259ad0c9344f6d3",
"X-Amz-Date": "20240229T193925Z"
},
"responseHeader": {
"Accept-Ranges": "bytes",
"Content-Length": "0",
"Content-Security-Policy": "block-all-mixed-content",
"ETag": "a414c889dc276457bd7175f974332cb0-1",
"Server": "MinIO/DEVELOPMENT.2019-08-12T21-28-07Z",
"Vary": "Origin",
"X-Amz-Request-Id": "15BA4A72C0C70AFC",
"Content-Length": "358",
"Content-Type": "application/xml",
"ETag": "1d9fdc88af5e74f5eac0a3dd750ce58e-2",
"Server": "MinIO",
"Strict-Transport-Security": "max-age=31536000; includeSubDomains",
"Vary": "Origin,Accept-Encoding",
"X-Amz-Id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
"X-Amz-Request-Id": "17B86CB0ED88EBE9",
"X-Content-Type-Options": "nosniff",
"X-Xss-Protection": "1; mode=block"
},
"tags": {
"objectErasureMap": {
"object": {
"objectLocation": {
"name": "Mousepad Template-v03final.jpg",
"poolId": 1,
"setId": 10,
"setId": 1,
"disks": [
"http://server01/mnt/pool1/disk01",
"http://server02/mnt/pool1/disk02",
"http://server03/mnt/pool1/disk03",
"http://server04/mnt/pool1/disk04"
"/mnt/drive-1",
"/mnt/drive-2",
"/mnt/drive-3",
"/mnt/drive-4"
]
}
}
}
}
},
"accessKey": "minioadmin"
}

12
source/reference/minio-mc-admin/mc-admin-update.rst
View File

@ -88,3 +88,15 @@ Syntax
:optional:
Pass this flag to confirm the update and bypass the confirmation prompt.
Behavior
--------
Binary Compression
~~~~~~~~~~~~~~~~~~
.. versionchanged:: RELEASE.2024-01-28T22-35-53Z
:mc-cmd:`mc admin update` compresses the binary before sending to all nodes in the deployment.
This feature does not apply to :ref:`systemctl managed deployments <minio-upgrade-systemctl>`.

9
sync-minio-server-docs.sh
View File

@ -6,7 +6,14 @@ set -x
function main() {
curl --retry 10 -Ls https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/list.md | csplit - /"# Bucket Metrics"/
mv xx00 source/includes/common-metrics-cluster.md
mv xx01 source/includes/common-metrics-bucket.md
# Kludgy. Does csplit again on the Bucket Metrics file fragment
# Tried to get smart using `grep '^# [A-Za-z]` to get line numbers but got stuck
cat xx01 | csplit - /"# Resource Metrics"/
mv xx00 source/includes/common-metrics-bucket.md
mv xx01 source/includes/common-metrics-resource.md
}
main "$@"