mirror of
https://github.com/minio/docs.git
synced 2025-07-02 05:02:40 +03:00
Metrics v3 (#1289)
Add metrics v3 to the Metrics and Alerts page and other relevant locations, deprecate metrics v2. - Updated the metrics extract script to parse the [list of v3 metrics from GitHub](https://github.com/minio/minio/blob/master/docs/metrics/v3.md). - Remove references to MinIO Console metrics. - Make `:mc-release:` show friendly release name text like `:minio-release:` does. Staged: Linux http://192.241.195.202:9000/staging/DOCS-1266/linux/operations/monitoring/metrics-and-alerts.html k8s http://192.241.195.202:9000/staging/DOCS-1266/k8s/operations/monitoring/metrics-and-alerts.html fixes https://github.com/minio/docs/issues/1266
This commit is contained in:
Andrea Longo
GitHub
@ -62,7 +62,7 @@ extlinks = {
|
||||
's3-api' : ('https://docs.aws.amazon.com/AmazonS3/latest/API/%s', None),
|
||||
'iam-docs' : ('https://docs.aws.amazon.com/IAM/latest/UserGuide/%s', None),
|
||||
'minio-release' : ('https://github.com/minio/minio/releases/tag/%s', '%s'),
|
||||
'mc-release' : ('https://github.com/minio/mc/releases/tag/%s', None),
|
||||
'mc-release' : ('https://github.com/minio/mc/releases/tag/%s', '%s'),
|
||||
'prometheus-docs' : ('https://prometheus.io/docs/%s', None),
|
||||
'podman-docs' : ('https://docs.podman.io/en/latest/%s', None),
|
||||
'podman-git' : ('https://github.com/containers/podman/%s', None),
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 160 KiB |
@ -375,23 +375,6 @@ In the Prometheus configuration, specify the path to the alert file in the ``rul
|
||||
|
||||
Once triggered, Prometheus sends the alert to the configured AlertManager service.
|
||||
|
||||
5) (Optional) Configure MinIO Console to Query Prometheus
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The Console also supports displaying time-series and historical data by querying a :prometheus-docs:`Prometheus <prometheus/latest/getting_started/>` service configured to scrape data from the MinIO deployment.
|
||||
|
||||
.. image:: /images/minio-console/console-metrics.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console displaying Prometheus-backed Monitoring Data
|
||||
:align: center
|
||||
|
||||
To enable historical data visualization in MinIO Console, set the following environment variables on each node in the MinIO deployment:
|
||||
|
||||
- Set :envvar:`MINIO_PROMETHEUS_URL` to the URL of the Prometheus service
|
||||
- Set :envvar:`MINIO_PROMETHEUS_JOB_ID` to the unique job ID assigned to the collected metrics
|
||||
|
||||
Restart the MinIO deployment and visit the :ref:`Monitoring <minio-console-monitoring>` pane to see the historical data views.
|
||||
|
||||
Dashboards
|
||||
----------
|
||||
|
||||
|
||||
@ -10,85 +10,95 @@ Metrics and Alerts
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 3
|
||||
:depth: 2
|
||||
|
||||
.. admonition:: Metrics Version 2 Deprecated
|
||||
:class: note
|
||||
|
||||
Starting with MinIO Server :minio-release:`RELEASE.2024-07-15T19-02-30Z` and MinIO Client :mc-release:`RELEASE.2024-07-11T18-01-28Z`, metrics version 3 replaces the deprecated :ref:`metrics version 2 <minio-metrics-v2>`.
|
||||
|
||||
MinIO publishes cluster and node metrics using the :prometheus-docs:`Prometheus Data Model <concepts/data_model/>`.
|
||||
You can use any scraping tool to pull metrics data from MinIO for further analysis and alerting.
|
||||
|
||||
MinIO provides scraping endpoints for the following metric groups:
|
||||
For metrics version 3, all metrics are available under the base ``/minio/metrics/v3`` endpoint by appending an additional path for each category.
|
||||
|
||||
.. tab-set::
|
||||
For example, the following endpoint returns audit metrics:
|
||||
|
||||
.. tab-item:: Cluster Metrics
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
You can scrape :ref:`cluster-level metrics <minio-available-cluster-metrics>` using the following URL endpoint:
|
||||
http://HOSTNAME:PORT/minio/metrics/v3/audit
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
|
||||
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
|
||||
|
||||
http://HOSTNAME:PORT/minio/v2/metrics/cluster
|
||||
By default, MinIO requires authentication to scrape the metrics endpoints.
|
||||
To generate the needed bearer tokens, use :mc:`mc admin prometheus generate`.
|
||||
You can also disable metrics endpoint authentication by setting :envvar:`MINIO_PROMETHEUS_AUTH_TYPE` to ``public``.
|
||||
|
||||
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
|
||||
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
|
||||
MinIO provides the following scraping endpoints, relative to the base URL:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 30 70
|
||||
:width: 100%
|
||||
|
||||
.. tab-item:: Bucket Metrics
|
||||
* - Category
|
||||
- Path
|
||||
|
||||
.. versionchanged:: RELEASE.2023-08-31T15-31-16Z
|
||||
* - API
|
||||
- ``/api/requests``
|
||||
|
||||
You can scrape :ref:`bucket-level metrics <minio-available-bucket-metrics>` using the following URL endpoint:
|
||||
``/bucket/api``
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
* - Audit
|
||||
- ``/audit``
|
||||
|
||||
http://HOSTNAME:PORT/minio/v2/metrics/bucket
|
||||
* - Cluster
|
||||
- ``/cluster/config``
|
||||
|
||||
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
|
||||
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
|
||||
``/cluster/erasure-set``
|
||||
|
||||
.. tab-item:: Resource Metrics
|
||||
``/cluster/health``
|
||||
|
||||
.. versionadded:: RELEASE.2023-10-07T15-07-38Z
|
||||
``/cluster/iam``
|
||||
|
||||
You can scrape :ref:`resource metrics <minio-available-resource-metrics>` using the following URL endpoint:
|
||||
``/cluster/usage/buckets``
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
``/cluster/usage/objects``
|
||||
|
||||
http://HOSTNAME:PORT/minio/v2/metrics/resource
|
||||
* - Debug
|
||||
- ``/debug/go``
|
||||
|
||||
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
|
||||
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
|
||||
* - ILM
|
||||
- ``/ilm``
|
||||
|
||||
* - Logger webhook
|
||||
- ``/logger/webhook``
|
||||
|
||||
MinIO by default requires authentication for scraping the metrics endpoints.
|
||||
Use the :mc:`mc admin prometheus generate` command to generate the necessary bearer tokens.
|
||||
You can alternatively disable metrics endpoint authentication by setting :envvar:`MINIO_PROMETHEUS_AUTH_TYPE` to ``public``.
|
||||
* - Notification
|
||||
- ``/notification``
|
||||
|
||||
.. _minio-console-metrics:
|
||||
* - Replication
|
||||
- ``/replication``
|
||||
|
||||
MinIO Console Metrics Dashboard
|
||||
-------------------------------
|
||||
``/bucket/replication``
|
||||
|
||||
The :ref:`MinIO Console <minio-console-monitoring>` provides a point-in-time metrics dashboard by default:
|
||||
* - Scanner
|
||||
- ``/scanner``
|
||||
|
||||
.. image:: /images/minio-console/console-metrics-simple.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console with Point-In-Time Metrics
|
||||
:align: center
|
||||
* - System
|
||||
- ``/system/drive``
|
||||
|
||||
The Console also supports displaying time-series and historical data by querying a :prometheus-docs:`Prometheus <prometheus/latest/getting_started/>` service configured to scrape data from the MinIO deployment.
|
||||
Specifically, the MinIO Console uses :prometheus-docs:`Prometheus query API <prometheus/latest/querying/api/>` to retrieve stored metrics data and display the following visualizations:
|
||||
``/system/memory``
|
||||
|
||||
- :guilabel:`Usage` - provides historical and on-demand visualization of overall usage and status
|
||||
- :guilabel:`Traffic` - provides historical and on-demand visualization of network traffic
|
||||
- :guilabel:`Resources` - provides historical and on-demand visualization of resources (compute and storage)
|
||||
- :guilabel:`Info` - provides point-in-time status of the deployment
|
||||
``/system/cpu``
|
||||
|
||||
.. image:: /images/minio-console/console-metrics.png
|
||||
:width: 600px
|
||||
:alt: MinIO Console displaying Prometheus-backed Monitoring Data
|
||||
:align: center
|
||||
``/system/network/internode``
|
||||
|
||||
``/system/process``
|
||||
|
||||
For a complete list of metrics for each endpoint, see :ref:`Available Metrics <minio-metrics-and-alerts-available-metrics>`.
|
||||
|
||||
.. cond:: k8s
|
||||
|
||||
@ -118,29 +128,69 @@ Available Metrics
|
||||
MinIO publishes a number of metrics at the cluster, node, or bucket levels.
|
||||
Each metric includes a label for the MinIO server which generated that metric.
|
||||
|
||||
.. versionchanged:: MinIO RELEASE.2023-07-21T21-12-44Z
|
||||
- :ref:`API Metrics <minio-available-v3-api-metrics>`
|
||||
- :ref:`Audit Metrics <minio-available-v3-audit-metrics>`
|
||||
- :ref:`Cluster Metrics <minio-available-v3-cluster-metrics>`
|
||||
- :ref:`Debug Metrics <minio-available-v3-debug-metrics>`
|
||||
- :ref:`ILM Metrics <minio-available-v3-ilm-metrics>`
|
||||
- :ref:`Logger webhook Metrics <minio-available-v3-logger-webhook-metrics>`
|
||||
- :ref:`Notification Metrics <minio-available-v3-notification-metrics>`
|
||||
- :ref:`Replication Metrics <minio-available-v3-replication-metrics>`
|
||||
- :ref:`Scanner Metrics <minio-available-v3-scanner-metrics>`
|
||||
- :ref:`System Metrics <minio-available-v3-system-metrics>`
|
||||
|
||||
Bucket metrics have moved to use their own, separate endpoint.
|
||||
|
||||
- :ref:`Cluster Metrics <minio-available-cluster-metrics>`
|
||||
- :ref:`Bucket Metrics <minio-available-bucket-metrics>`
|
||||
- :ref:`Resource Metrics <minio-available-resource-metrics>`
|
||||
.. _minio-available-v3-api-metrics:
|
||||
|
||||
.. _minio-available-cluster-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-cluster.md
|
||||
.. include:: /includes/common-metrics-v3-api.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-bucket-metrics:
|
||||
.. _minio-available-v3-audit-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-bucket.md
|
||||
.. include:: /includes/common-metrics-v3-audit.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-resource-metrics:
|
||||
.. _minio-available-v3-cluster-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-resource.md
|
||||
.. include:: /includes/common-metrics-v3-cluster.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-v3-debug-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-v3-debug.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-v3-ilm-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-v3-ilm.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-v3-logger-webhook-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-v3-logger-webhook.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-v3-notification-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-v3-notification.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-v3-replication-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-v3-replication.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-v3-scanner-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-v3-scanner.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-v3-system-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-v3-system.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
|
||||
.. toctree::
|
||||
:titlesonly:
|
||||
:hidden:
|
||||
|
||||
88
source/operations/monitoring/metrics-v2-deprecated.rst
Normal file
88
source/operations/monitoring/metrics-v2-deprecated.rst
Normal file
@ -0,0 +1,88 @@
|
||||
:orphan:
|
||||
.. _minio-metrics-v2:
|
||||
|
||||
=================
|
||||
Metrics Version 2
|
||||
=================
|
||||
|
||||
.. default-domain:: minio
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 3
|
||||
|
||||
.. admonition:: Metrics Version 2 Deprecated
|
||||
:class: note
|
||||
|
||||
Starting with MinIO Server :minio-release:`RELEASE.2024-07-15T19-02-30Z` and MinIO Client :mc-release:`RELEASE.2024-07-11T18-01-28Z`, :ref:`metrics version 3 <minio-metrics-and-alerts>` replaces the deprecated metrics version 2.
|
||||
|
||||
The following sections describe the deprecated endpoints and metrics.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Cluster Metrics
|
||||
|
||||
You can scrape :ref:`cluster-level metrics <minio-available-cluster-metrics>` using the following URL endpoint:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
http://HOSTNAME:PORT/minio/v2/metrics/cluster
|
||||
|
||||
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
|
||||
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
|
||||
|
||||
|
||||
.. tab-item:: Bucket Metrics
|
||||
|
||||
.. versionchanged:: MinIO RELEASE.2023-07-21T21-12-44Z
|
||||
|
||||
Bucket metrics have moved to use their own, separate endpoint.
|
||||
|
||||
.. versionchanged:: RELEASE.2023-08-31T15-31-16Z
|
||||
|
||||
You can scrape :ref:`bucket-level metrics <minio-available-bucket-metrics>` using the following URL endpoint:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
http://HOSTNAME:PORT/minio/v2/metrics/bucket
|
||||
|
||||
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
|
||||
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
|
||||
|
||||
|
||||
.. tab-item:: Resource Metrics
|
||||
|
||||
.. versionadded:: RELEASE.2023-10-07T15-07-38Z
|
||||
|
||||
You can scrape :ref:`resource metrics <minio-available-resource-metrics>` using the following URL endpoint:
|
||||
|
||||
.. code-block:: shell
|
||||
:class: copyable
|
||||
|
||||
http://HOSTNAME:PORT/minio/v2/metrics/resource
|
||||
|
||||
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
|
||||
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.
|
||||
|
||||
|
||||
- :ref:`Cluster Metrics <minio-available-cluster-metrics>`
|
||||
- :ref:`Bucket Metrics <minio-available-bucket-metrics>`
|
||||
- :ref:`Resource Metrics <minio-available-resource-metrics>`
|
||||
|
||||
.. _minio-available-cluster-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-cluster.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-bucket-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-bucket.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
.. _minio-available-resource-metrics:
|
||||
|
||||
.. include:: /includes/common-metrics-resource.md
|
||||
:parser: myst_parser.sphinx_
|
||||
|
||||
@ -4,6 +4,9 @@ set -e
|
||||
set -x
|
||||
|
||||
function main() {
|
||||
|
||||
# Extract metrics v2 list
|
||||
|
||||
curl --retry 10 -Ls https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/list.md | csplit - /"# Bucket Metrics"/
|
||||
mv xx00 source/includes/common-metrics-cluster.md
|
||||
|
||||
@ -14,6 +17,58 @@ function main() {
|
||||
|
||||
mv xx00 source/includes/common-metrics-bucket.md
|
||||
mv xx01 source/includes/common-metrics-resource.md
|
||||
|
||||
# Extract metrics v3 list
|
||||
|
||||
# Get the full list
|
||||
|
||||
curl --retry 10 -Ls https://raw.githubusercontent.com/minio/minio/master/docs/metrics/v3.md | csplit - /"## Metric Categories"/
|
||||
|
||||
# Ignore xx00, contains intro text
|
||||
# Overwritten in second csplit anyway
|
||||
|
||||
# Split remainder of file into categories by H3
|
||||
# Number of times to match must be exact
|
||||
cat xx01 | csplit - /"^### "/ '{9}'
|
||||
|
||||
# Copy each category into an include file
|
||||
# Order below must match section order in source file
|
||||
|
||||
# Ignore xx00 again, more intro text
|
||||
rm xx00
|
||||
|
||||
# API (Renamed from Request)
|
||||
# Hopefully ex works the same on both Mac and Linux, unlike sed
|
||||
ex -sc '%s/### Request metrics/### API metrics/' -c 'x' xx01
|
||||
mv xx01 source/includes/common-metrics-v3-api.md
|
||||
|
||||
# Audit
|
||||
mv xx02 source/includes/common-metrics-v3-audit.md
|
||||
|
||||
# Cluster
|
||||
mv xx03 source/includes/common-metrics-v3-cluster.md
|
||||
|
||||
# Debug
|
||||
mv xx04 source/includes/common-metrics-v3-debug.md
|
||||
|
||||
# ILM
|
||||
mv xx05 source/includes/common-metrics-v3-ilm.md
|
||||
|
||||
# Logger Webhook
|
||||
mv xx06 source/includes/common-metrics-v3-logger-webhook.md
|
||||
|
||||
# Notification
|
||||
mv xx07 source/includes/common-metrics-v3-notification.md
|
||||
|
||||
# Replication
|
||||
mv xx08 source/includes/common-metrics-v3-replication.md
|
||||
|
||||
# Scanner
|
||||
mv xx09 source/includes/common-metrics-v3-scanner.md
|
||||
|
||||
# System
|
||||
mv xx10 source/includes/common-metrics-v3-system.md
|
||||
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user