From ea489b4a30a11c80d4cedb01d2a26d2cb040664d Mon Sep 17 00:00:00 2001 From: Daryl White <53910321+djwfyi@users.noreply.github.com> Date: Thu, 12 Jan 2023 13:57:36 -0600 Subject: [PATCH] Updates for minio Dec 2022 releases (#701) Two releases for MinIO in December 2022 and Jan 2023 affected the docs. Updates for Dec 2022 minio updates - Adds 500,000 as the recommended but not hard limit to number of buckets - Adds info on encrypting list of file names in zip archives Closes #684 Updates for Jan 2 2023 release - Adds info about setting server to listen on multiple IP addresses - Updates list of available metrics Closes #689 --- source/administration/minio-console.rst | 3 +- .../monitoring/metrics-and-alerts.rst | 138 +++++++++++++++++- source/reference/minio-mc/mc-mb.rst | 6 + .../reference/minio-mc/mc-support-inspect.rst | 4 + .../reference/minio-server/minio-server.rst | 42 ++++-- 5 files changed, 175 insertions(+), 18 deletions(-) diff --git a/source/administration/minio-console.rst b/source/administration/minio-console.rst index 7e252c26..7d0a8bc0 100644 --- a/source/administration/minio-console.rst +++ b/source/administration/minio-console.rst @@ -158,7 +158,8 @@ To see the rules for bucket names, select :guilabel:`View Bucket Naming Rules`. While creating a bucket, you can enable :ref:`versioning `, :ref:`object locking `, bucket size (quota) limits, and :ref:`retention rules ` (which require versioning). -MinIO recommends no more than 500,000 buckets per deployment. +MinIO does not limit the total number of buckets allowed on a deployment. +However, MinIO recommends no more than 500,000 buckets per deployment as a general guideline. Each bucket has :guilabel:`Manage` and :guilabel:`Browse` buttons. diff --git a/source/operations/monitoring/metrics-and-alerts.rst b/source/operations/monitoring/metrics-and-alerts.rst index c8ff0bdc..fea3711b 100644 --- a/source/operations/monitoring/metrics-and-alerts.rst +++ b/source/operations/monitoring/metrics-and-alerts.rst @@ -113,6 +113,10 @@ These metrics are only populated for MinIO clusters with Total number of bytes that failed at least once to replicate for a given bucket. You can identify the bucket using the ``{ bucket="STRING" }`` label +.. metric:: minio_bucket_replication_latency + + Replication latency in milliseconds. + .. metric:: minio_bucket_replication_pending_bytes Total number of bytes pending to replicate for a given bucket. @@ -164,7 +168,7 @@ Capacity Metrics Total storage available on a specific drive for a node in the MinIO deployment. You can identify the drive and node using the ``{ disk="/path/to/disk",server="STRING"}`` labels respectively. -.. metric:: minio_node_disk_total_bytes +.. metric:: minio_node_disk_total_bytes Total storage on a specific drive for a node in the MinIO deployment. You can identify the drive and node using the ``{ disk="/path/to/disk",server="STRING"}`` labels respectively. @@ -183,7 +187,6 @@ Lifecycle Management Metrics Total number of bytes transitioned using :ref:`tiering/transition lifecycle management rules ` - .. metric:: minio_cluster_ilm_transitioned_objects Total number of objects transitioned using :ref:`tiering/transition lifecycle management rules ` @@ -196,6 +199,10 @@ Lifecycle Management Metrics Total number of pending :ref:`object transition ` tasks +.. metric:: minio_node_ilm_transition_active_tasks + + Number of active ILM transition tasks + .. metric:: minio_node_ilm_expiry_pending_tasks Total number of pending :ref:`object expiration ` tasks @@ -204,6 +211,10 @@ Lifecycle Management Metrics Total number of active :ref:`object expiration ` tasks +.. metric:: minio_node_ilm_versions_scanned + + Total number of object versions checked for ilm actions since server start + Node and Drive Health Metrics ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -227,7 +238,27 @@ Node and Drive Health Metrics Total number of MinIO nodes online. -.. metric:: minio_heal_objects_error_total +.. metric:: minio_node_disk_free_inodes + + Total free inodes. + +.. metric:: minio_node_disk_latency_us + + Average last minute latency in µs for drive API storage operations. + +.. metric:: minio_node_disk_offline_total + + Total drives offline. + +.. metric:: minio_node_disk_online_total + + Total drives online. + +.. metric:: minio_node_disk_total + + Total drives. + +.. metric:: minio_heal_objects_errors_total Objects for which healing failed in current self healing run @@ -300,6 +331,11 @@ Scanner Metrics Total number of write SysCalls to the kernel. ``/proc/[pid]/io syscw`` +.. metric:: minio_usage_last_activity_nano_seconds + + Time elapsed since last scan activity. + This is set to ``0`` until first scan cycle. + S3 Metrics ~~~~~~~~~~ @@ -313,6 +349,14 @@ S3 Metrics Total number of bytes of S3 traffic received per bucket. You can identify the bucket using the ``{ bucket="STRING" }`` label. +.. metric:: minio_s3_requests_incoming_total + + Volatile number of total incoming S3 requests. + +.. metric:: minio_s3_requests_canceled_total + + Total number S3 requests that were canceled from the client while processing. + .. metric:: minio_s3_requests_inflight_total Total number of S3 requests currently in flight. @@ -321,6 +365,26 @@ S3 Metrics Total number of S3 requests. +.. metric:: minio_s3_requests_rejected_auth_total + + Total number S3 requests rejected for auth failure. + +.. metric:: minio_s3_requests_rejected_header_total + + Total number S3 requests rejected for invalid header. + +.. metric:: minio_s3_requests_rejected_invalid_total + + Total number S3 invalid requests. + +.. metric:: minio_s3_requests_rejected_timestamp_total + + Total number S3 requests rejected for invalid timestamp. + +.. metric:: minio_s3_requests_waiting_total + + Number of S3 requests in the waiting queue. + .. metric:: minio_s3_time_ttfb_seconds_distribution Distribution of the time to first byte across API calls. @@ -345,6 +409,27 @@ S3 Metrics Total number of S3 requests with 5xx errors. +IAM Metrics +~~~~~~~~~~~ + +.. metric:: minio_node_iam_last_sync_duration_millis + + Last successful IAM data sync duration in milliseconds. + +.. metric:: minio_node_iam_since_last_sync_millis + + Time (in milliseconds) since last successful IAM data sync. + + This value starts at zero and only increments after the the first sync after server start. + +.. metric:: minio_node_iam_sync_failures + + Number of failed IAM data syncs since server start. + +.. metric:: minio_node_iam_sync_successes + + Number of successful IAM data syncs since server start. + Internal Metrics ~~~~~~~~~~~~~~~~ @@ -356,6 +441,18 @@ Internal Metrics Total number of bytes sent to the other peer nodes. +.. metric:: minio_inter_node_traffic_dial_avg_time + + Average time of internodes TCP dial calls. + +.. metric:: minio_inter_node_traffic_dial_errors + + Total number of internode TCP dial timeouts and errors. + +.. metric:: minio_inter_node_traffic_errors_total + + Total number of failed internode calls. + .. metric:: minio_node_file_descriptor_limit_total Limit on total number of open file descriptors for the MinIO Server process. @@ -384,6 +481,29 @@ Internal Metrics Total bytes written by the process to the underlying storage system, ``/proc/[pid]/io write_bytes`` +Key Management System (KMS) Metrics +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. metric:: minio_cluster_kms_online + + Reports whether the KMS is online (1) or offline (0). + +.. metric:: minio_cluster_kms_request_error + + Number of KMS requests that failed due to some error. (HTTP 4xx status code). + +.. metric:: minio_cluster_kms_request_failure + + Number of KMS requests that failed due to some internal failure. (HTTP 5xx status code). + +.. metric:: minio_cluster_kms_request_success + + Number of KMS requests that succeeded. + +.. metric:: minio_cluster_kms_uptime + + The time the KMS has been up and running in seconds. + Software and Process Metrics ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -395,6 +515,10 @@ Software and Process Metrics MinIO Release tag for the server +.. metric:: minio_node_go_routine_total + + Total number of go routines running. + .. metric:: minio_node_process_starttime_seconds Start time for MinIO process per node, time in seconds since Unix epoch. @@ -403,6 +527,14 @@ Software and Process Metrics Uptime for MinIO process per node in seconds. +.. metric:: minio_node_process_cpu_total_seconds + + Total user and system CPU time spent in seconds. + +.. metric:: minio_node_process_resident_memory_bytes + + Resident memory size in bytes. + .. toctree:: :titlesonly: :hidden: diff --git a/source/reference/minio-mc/mc-mb.rst b/source/reference/minio-mc/mc-mb.rst index 512b2184..66c34d6b 100644 --- a/source/reference/minio-mc/mc-mb.rst +++ b/source/reference/minio-mc/mc-mb.rst @@ -163,6 +163,12 @@ The new bucket enables :ref:`object versioning ` for al Behavior -------- +Bucket Limits Per Deployment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO does not limit the number of buckets you can create on a deployment. +However, MinIO recommends no more than 500,000 buckets per deployment as a general guideline. + Bucket Limits for Non-MinIO S3 Services ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/reference/minio-mc/mc-support-inspect.rst b/source/reference/minio-mc/mc-support-inspect.rst index 37626e9d..d95a49f0 100644 --- a/source/reference/minio-mc/mc-support-inspect.rst +++ b/source/reference/minio-mc/mc-support-inspect.rst @@ -22,6 +22,10 @@ MinIO assembles this data from each backend drive storing an :ref:`erasure shard The command produces an encrypted zip file that includes all matching files with their respective *host+drive+path*. +.. versionchanged:: RELEASE.2022-12-12T19-27-27Z + + When writing the zip archive, MinIO also encrypts the zip index of file names included in the archive. + The resulting report is intended for use by MinIO Engineering via |SUBNET| and may contain internal or private data points associated to the object. Exercise caution before sending a report to a third party or posting the report in a public forum. diff --git a/source/reference/minio-server/minio-server.rst b/source/reference/minio-server/minio-server.rst index 42fd44bd..b8314fa3 100644 --- a/source/reference/minio-server/minio-server.rst +++ b/source/reference/minio-server/minio-server.rst @@ -94,6 +94,7 @@ The command accepts the following arguments: information on Server Pools, see :ref:`Server Pool `. .. mc-cmd:: DIRECTORIES + :required: The directories or drives the :mc:`minio server` process uses as the storage backend. @@ -132,9 +133,9 @@ The command accepts the following arguments: .. mc-cmd:: --address - + :optional: - *Optional* Binds the :mc:`minio ` server process to a + Binds the :mc:`minio ` server process to a specific network address and port number. Specify the address and port as ``ADDRESS:PORT``, where ``ADDRESS`` is an IP address or hostname and ``PORT`` is a valid and open port on the host system. @@ -143,13 +144,27 @@ The command accepts the following arguments: on the host machine, specify ``:PORT`` where ``PORT`` is a valid and open port on the host. + .. versionchanged:: RELEASE.2023-01-02T09-40-09Z + + You can configure your hosts file to have MinIO only listen on specific IPs. + For example, if the machine's `/etc/hosts` file contains the following: + + .. code-block:: shell + + 127.0.1.1 minioip + 127.0.1.2 minioip + + A command like the following would listen for API calls on port ``9000`` on both configured IP addresses. + + .. code-block:: shell + + minio server --address "minioip:9000" ~/miniodirectory + If omitted, :mc:`minio ` binds to port ``9000`` on all configured IP addresses or hostnames on the host machine. .. mc-cmd:: --console-address - - - *Optional* + :optional: Specifies a static port for the embedded MinIO Console. @@ -157,9 +172,9 @@ The command accepts the following arguments: MinIO server outputs the port to the system log. .. mc-cmd:: --certs-dir, -S - + :optional: - *Optional* Specifies the path to the folder containing certificates the + Specifies the path to the folder containing certificates the :mc:`minio` process uses for configuring TLS/SSL connectivity. Omit to use the default directory paths: @@ -170,20 +185,19 @@ The command accepts the following arguments: See :ref:`minio-TLS` for more information on TLS/SSL connectivity. .. mc-cmd:: --quiet - + :optional: - *Optional* Disables startup information. + Disables startup information. .. mc-cmd:: --anonymous - + :optional: - *Optional* Hides sensitive information from logging. + Hides sensitive information from logging. .. mc-cmd:: --json - + :optional: - *Optional* Outputs server logs and startup information in ``JSON`` - format. + Outputs server logs and startup information in ``JSON`` format. .. _minio-server-environment-variables: