From 34cf55934a495a862ec166f4f1e9e76e8e5ed55c Mon Sep 17 00:00:00 2001 From: Daryl White <53910321+djwfyi@users.noreply.github.com> Date: Tue, 22 Aug 2023 13:54:51 -0500 Subject: [PATCH] Updates for several July 2023 mc releases (#964) mc RELEASE.2023-07-18T21-05-38Z - Replaces mc replicate diff with mc replicate backlog - Adds mc support top net mc RELEASE.2023-07-07T05-25-51Z - Adds mc support perf site-replication - Deprecates mc admin replicate --sync flag in favor of new mc admihn replicate --mode flag - Updates output example of mc admin user info to show authentication info Partially addresses #930 --- .../multi-site-replication.rst | 2 +- source/operations/monitoring/grafana.rst | 4 - .../minio-mc-admin/mc-admin-replicate.rst | 18 ++++- .../minio-mc-admin/mc-admin-user-info.rst | 20 ++++- source/reference/minio-mc-deprecated.rst | 4 + source/reference/minio-mc.rst | 2 +- ...cate-diff.rst => mc-replicate-backlog.rst} | 31 +++++--- source/reference/minio-mc/mc-replicate.rst | 10 +-- .../minio-mc/mc-support-callhome.rst | 4 + source/reference/minio-mc/mc-support-perf.rst | 33 +++++++- .../reference/minio-mc/mc-support-profile.rst | 4 + .../reference/minio-mc/mc-support-top-api.rst | 4 + .../minio-mc/mc-support-top-disk.rst | 4 + .../minio-mc/mc-support-top-locks.rst | 4 + .../reference/minio-mc/mc-support-top-net.rst | 75 +++++++++++++++++++ source/reference/minio-mc/mc-support-top.rst | 8 +- 16 files changed, 196 insertions(+), 31 deletions(-) rename source/reference/minio-mc/{mc-replicate-diff.rst => mc-replicate-backlog.rst} (81%) create mode 100644 source/reference/minio-mc/mc-support-top-net.rst diff --git a/source/operations/install-deploy-manage/multi-site-replication.rst b/source/operations/install-deploy-manage/multi-site-replication.rst index dbf4bb34..1a6dcc6c 100644 --- a/source/operations/install-deploy-manage/multi-site-replication.rst +++ b/source/operations/install-deploy-manage/multi-site-replication.rst @@ -114,7 +114,7 @@ Synchronous vs Asynchronous Replication MinIO strongly recommends using the default asynchronous site replication. Synchronous site replication performance depends strongly on latency between sites, where higher latency can result in lower PUT performance and replication lag. -To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --sync` option. +To configure synchronous site replication use :mc-cmd:`mc admin replicate update` with the :mc-cmd:`~mc admin replicate update --mode` option. Proxy to Other Sites ~~~~~~~~~~~~~~~~~~~~ diff --git a/source/operations/monitoring/grafana.rst b/source/operations/monitoring/grafana.rst index 307cf386..6444e711 100644 --- a/source/operations/monitoring/grafana.rst +++ b/source/operations/monitoring/grafana.rst @@ -32,10 +32,6 @@ To track changes to the Grafana dashboard, introspect the JSON files for the `se .. _minio-server-grafana-metrics: -To track changes to the Grafana dashboard, introspect the JSON files for the `server `__ or `bucket `__ dashboards in the MinIO Server GitHub repository. - -.. _minio-server-grafana-metrics: - MinIO Server Metrics Dashboard ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/reference/minio-mc-admin/mc-admin-replicate.rst b/source/reference/minio-mc-admin/mc-admin-replicate.rst index 5a1eaeb1..6ec6d5b3 100644 --- a/source/reference/minio-mc-admin/mc-admin-replicate.rst +++ b/source/reference/minio-mc-admin/mc-admin-replicate.rst @@ -82,7 +82,8 @@ Syntax Create or expand a site replication configuration. The configuration uses asynchronous site replication by default, as MinIO recommends. - If your circumstances require synchronous site replication, complete the configuration, then use :mc-cmd:`mc admin replicate update --sync`. + To enable synchronous site replication, create the replication using this command *first*. + Then use :mc-cmd:`mc admin replicate update --mode sync ` to update the configuration. .. tab-set:: @@ -160,7 +161,7 @@ Syntax ALIAS \ --deployment-id [deploymentID] \ --endpoint [newEndpoint] \ - --sync ["enable" | "disable"] + --mode ["sync" | "async"] .. mc-cmd:: ALIAS :required: @@ -179,9 +180,22 @@ Syntax The new endpoint or URL to associate with the peer site. + .. mc-cmd:: --mode + :optional: + + Specify whether MinIO performs replication operations to the peer synchronously or asynchronously. + Available values are ``sync`` and ``async``. + + Defaults to ``async``. + .. mc-cmd:: --sync :optional: + .. important:: + + The ``--sync`` flag has been deprecated as of ``RELEASE.2023-07-07T05-25-51Z``. + Use :mc-cmd:`~mc admin replicate update --mode` instead. + Enable or disable synchronous site replication. Available values are ``enable`` and ``disable``. If not defined, MInIO uses asynchronous site replication. diff --git a/source/reference/minio-mc-admin/mc-admin-user-info.rst b/source/reference/minio-mc-admin/mc-admin-user-info.rst index 2bd18f47..a2fd70e3 100644 --- a/source/reference/minio-mc-admin/mc-admin-user-info.rst +++ b/source/reference/minio-mc-admin/mc-admin-user-info.rst @@ -91,14 +91,25 @@ Use :mc-cmd:`mc admin user info` to view detailed user information for a user on - Replace :mc-cmd:`USERNAME ` with the username of the user to display information for. -The output resembles the following: +For the :ref:`MinIO internal IDentity Provider (IDP) `, the output resembles the following: .. code-block:: shell - AccessKey: myuser + AccessKey: miniouser Status: enabled - PolicyName: readwrite - MemberOf: + PolicyName: + MemberOf: [] + Authentication: builtin (miniouser) + +For a :ref:`third-party ` identity service such as LDAP, the output resembles the following: + +.. code-block:: shell + + AccessKey: uid=dillon,ou=people,ou=swengg,dc=min,dc=io + Status: + PolicyName: consoleAdmin + MemberOf: [] + Authentication: ldap/localhost:1389 (uid=dillon,ou=people,ou=swengg,dc=min,dc=io) View Policies from Group Membership ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -129,6 +140,7 @@ The output resembles the following: "policies": [ "testingGroupPolicy" ] + "authentication": builtin (myuser) } ] } diff --git a/source/reference/minio-mc-deprecated.rst b/source/reference/minio-mc-deprecated.rst index e978d80a..c19a6859 100644 --- a/source/reference/minio-mc-deprecated.rst +++ b/source/reference/minio-mc-deprecated.rst @@ -27,6 +27,10 @@ Table of Deprecated Commands - Replacement Command - Version of Change + * - ``mc replicate diff`` + - :mc-cmd:`mc replicate backlog` + - mc RELEASE.2023-07-18T21-05-38Z + * - ``mc ilm add`` - :mc-cmd:`mc ilm rule add` - mc RELEASE.2022-12-24T15-21-38Z diff --git a/source/reference/minio-mc.rst b/source/reference/minio-mc.rst index 4f25e11d..187e74cb 100644 --- a/source/reference/minio-mc.rst +++ b/source/reference/minio-mc.rst @@ -341,7 +341,7 @@ The following table lists :mc-cmd:`mc` commands: :end-before: end-mc-rb-desc * - | :mc:`mc replicate add` - | :mc:`mc replicate diff` + | :mc:`mc replicate backlog` | :mc:`mc replicate export` | :mc:`mc replicate import` | :mc:`mc replicate ls` diff --git a/source/reference/minio-mc/mc-replicate-diff.rst b/source/reference/minio-mc/mc-replicate-backlog.rst similarity index 81% rename from source/reference/minio-mc/mc-replicate-diff.rst rename to source/reference/minio-mc/mc-replicate-backlog.rst index 6b60cd6e..835afe0e 100644 --- a/source/reference/minio-mc/mc-replicate-diff.rst +++ b/source/reference/minio-mc/mc-replicate-backlog.rst @@ -1,8 +1,9 @@ .. _minio-mc-replicate-diff: +.. _minio-mc-replicate-backlog: -===================== -``mc replicate diff`` -===================== +======================== +``mc replicate backlog`` +======================== .. default-domain:: minio @@ -11,15 +12,21 @@ :depth: 2 .. mc:: mc replicate diff +.. mc:: mc replicate backlog + +.. versionchanged:: mc.RELEASE.2023-07-18T21-05-38Z + + ``mc replicate diff`` has been renamed ``mc replicate backlog``. + No functionality has changed. Description ----------- -.. start-mc-replicate-diff-desc +.. start-mc-replicate-backlog-desc -The :mc:`mc replicate diff` shows a list of unreplicated new or deleted objects. +The :mc:`mc replicate backlog` shows a list of unreplicated new or deleted objects. -.. end-mc-replicate-diff-desc +.. end-mc-replicate-backlog-desc You can list the replication status of objects for a particular remote target. To do so, you must have the ARN of the remote target. @@ -38,7 +45,7 @@ Syntax .. code-block:: shell :class: copyable - mc replicate diff myminio/notes/teamorange/projects --arn arn:minio:replication::3bb8c736-4014-42c5-b3cb-d64e3ebaa75e:notes + mc replicate backlog myminio/notes/teamorange/projects --arn arn:minio:replication::3bb8c736-4014-42c5-b3cb-d64e3ebaa75e:notes If any new or deleted objects have not yet replicated, the command outputs something similar to the following: @@ -57,8 +64,8 @@ Syntax .. code-block:: shell :class: copyable - mc [GLOBALFLAGS] replicate diff \ - [--arn "string"] \ + mc [GLOBALFLAGS] replicate backlog \ + [--arn "string"] \ TARGET .. include:: /includes/common-minio-mc.rst @@ -99,7 +106,7 @@ Display unreplicated ``PUT`` and ``DELETE`` actions for a prefix: .. code-block:: shell :class: copyable - mc replicate diff myminio/mybucket/path/to/prefix + mc replicate backlog myminio/mybucket/path/to/prefix - Replace ``myminio/mybucket`` with the :mc-cmd:`~mc replicate add ALIAS` and full bucket path for which to create the replication configuration. @@ -116,12 +123,12 @@ If unreplicated objects exist, the output returns a list of the actions that cre View Unreplicated Objects at a Specific Remote Target ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The following :mc:`mc replicate diff` command shows unreplicated objects at an alias/bucket/prefix path for a specific remote target: +The following :mc:`mc replicate backlog` command shows unreplicated objects at an alias/bucket/prefix path for a specific remote target: .. code-block:: shell :class: copyable - mc replicate diff myminio/mybucket/path/to/prefix --arn + mc replicate backlog myminio/mybucket/path/to/prefix --arn - Replace ``myminio/mybucket`` with the :mc-cmd:`~mc replicate add ALIAS` and full bucket path for which to show unreplicated objects. diff --git a/source/reference/minio-mc/mc-replicate.rst b/source/reference/minio-mc/mc-replicate.rst index a406c3c2..b852b340 100644 --- a/source/reference/minio-mc/mc-replicate.rst +++ b/source/reference/minio-mc/mc-replicate.rst @@ -42,10 +42,10 @@ Subcommands :start-after: start-mc-replicate-add-desc :end-before: end-mc-replicate-add-desc - * - :mc:`~mc replicate diff` - - .. include:: /reference/minio-mc/mc-replicate-diff.rst - :start-after: start-mc-replicate-diff-desc - :end-before: end-mc-replicate-diff-desc + * - :mc:`~mc replicate backlog` + - .. include:: /reference/minio-mc/mc-replicate-backlog.rst + :start-after: start-mc-replicate-backlog-desc + :end-before: end-mc-replicate-backlog-desc * - :mc:`~mc replicate export` - .. include:: /reference/minio-mc/mc-replicate-export.rst @@ -87,7 +87,7 @@ Subcommands :hidden: /reference/minio-mc/mc-replicate-add - /reference/minio-mc/mc-replicate-diff + /reference/minio-mc/mc-replicate-backlog /reference/minio-mc/mc-replicate-ls /reference/minio-mc/mc-replicate-update /reference/minio-mc/mc-replicate-resync diff --git a/source/reference/minio-mc/mc-support-callhome.rst b/source/reference/minio-mc/mc-support-callhome.rst index f0a1401c..9ec83ca8 100644 --- a/source/reference/minio-mc/mc-support-callhome.rst +++ b/source/reference/minio-mc/mc-support-callhome.rst @@ -30,6 +30,10 @@ When enabled, MinIO can send logs to SUBNET in real time, diagnostics every 24 h MinIO disables this functionality by default, regardless of registration status. You must explicitly enable the ``callhome`` function to begin real time log upload. +.. include:: /includes/common-mc-support.rst + :start-after: start-minio-only + :end-before: end-minio-only + Syntax ------ diff --git a/source/reference/minio-mc/mc-support-perf.rst b/source/reference/minio-mc/mc-support-perf.rst index 1d5ad259..e2658c2c 100644 --- a/source/reference/minio-mc/mc-support-perf.rst +++ b/source/reference/minio-mc/mc-support-perf.rst @@ -14,6 +14,10 @@ ``mc support perf`` replaces the ``mc admin speedtest`` command. +.. include:: /includes/common-mc-support.rst + :start-after: start-minio-only + :end-before: end-minio-only + Description ----------- @@ -26,7 +30,7 @@ Use the :mc:`mc support perf` command to review the performance of the S3 API (r The resulting tests can provide general guidance of deployment performance under S3 ``GET`` and ``PUT`` requests and identify any potential bottlenecks. For more complete performance testing, consider using a combination of load-testing using your staging application environments and the MinIO `WARP `_ S3 benchmarking tool. -:mc:`mc support perf` has three subcommands +:mc:`mc support perf` has the following subcommands #. :mc-cmd:`~mc support perf drive` @@ -44,6 +48,10 @@ For more complete performance testing, consider using a combination of load-test Measure the network throughput to a client. +#. :mc-cmd:`~mc support perf site-replication` + + Measure the speed of site replication operations. + .. include:: /includes/common-mc-support.rst :start-after: start-minio-only :end-before: end-minio-only @@ -105,6 +113,16 @@ Run a network throughput test on a cluster with alias ``minio1``. mc support perf net minio1 +Test Site Replication Speed +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Run a test on the speed of site replication operations from the ``minio1`` site to other configured peers. + +.. code-block:: shell + :class: copyable + + mc support perf site-replication minio1 + Syntax ------ @@ -165,6 +183,19 @@ Syntax [--airgap] \ ALIAS +.. mc-cmd:: site-replication + :fullpath: + + Measure the speed of site replication operations from the specified ``ALIAS`` to other configured peers. + + .. code-block:: shell + + mc [GLOBAL FLAGS] support perf site-replication \ + --duration \ + [--verbose, -v] \ + ALIAS + + Parameters ~~~~~~~~~~ diff --git a/source/reference/minio-mc/mc-support-profile.rst b/source/reference/minio-mc/mc-support-profile.rst index f0e454f5..23e670dc 100644 --- a/source/reference/minio-mc/mc-support-profile.rst +++ b/source/reference/minio-mc/mc-support-profile.rst @@ -10,6 +10,10 @@ .. mc:: mc support profile +.. include:: /includes/common-mc-support.rst + :start-after: start-minio-only + :end-before: end-minio-only + Description ----------- diff --git a/source/reference/minio-mc/mc-support-top-api.rst b/source/reference/minio-mc/mc-support-top-api.rst index b5555c6a..38e05ff4 100644 --- a/source/reference/minio-mc/mc-support-top-api.rst +++ b/source/reference/minio-mc/mc-support-top-api.rst @@ -10,6 +10,10 @@ .. mc:: mc support top api +.. include:: /includes/common-mc-support.rst + :start-after: start-minio-only + :end-before: end-minio-only + Syntax ------ diff --git a/source/reference/minio-mc/mc-support-top-disk.rst b/source/reference/minio-mc/mc-support-top-disk.rst index a0b64886..f670144f 100644 --- a/source/reference/minio-mc/mc-support-top-disk.rst +++ b/source/reference/minio-mc/mc-support-top-disk.rst @@ -10,6 +10,10 @@ .. mc:: mc support top disk +.. include:: /includes/common-mc-support.rst + :start-after: start-minio-only + :end-before: end-minio-only + Syntax ------ diff --git a/source/reference/minio-mc/mc-support-top-locks.rst b/source/reference/minio-mc/mc-support-top-locks.rst index d1723c35..23fcd9f6 100644 --- a/source/reference/minio-mc/mc-support-top-locks.rst +++ b/source/reference/minio-mc/mc-support-top-locks.rst @@ -10,6 +10,10 @@ .. mc:: mc support top locks +.. include:: /includes/common-mc-support.rst + :start-after: start-minio-only + :end-before: end-minio-only + Syntax ------ diff --git a/source/reference/minio-mc/mc-support-top-net.rst b/source/reference/minio-mc/mc-support-top-net.rst new file mode 100644 index 00000000..4e853e02 --- /dev/null +++ b/source/reference/minio-mc/mc-support-top-net.rst @@ -0,0 +1,75 @@ +======================= +``mc support top net`` +======================= + +.. default-domain:: minio + +.. contents:: Table of Contents + :local: + :depth: 2 + +.. mc:: mc support top net + +.. include:: /includes/common-mc-support.rst + :start-after: start-minio-only + :end-before: end-minio-only + +Syntax +------ + +.. start-mc-support-top-net-desc + +The :mc:`mc support top net` command displays realtime network metrics. + +.. end-mc-support-top-net-desc + +.. tab-set:: + + .. tab-item:: EXAMPLE + + The following command displays the current realtime network metrics for the :term:`alias` ``myminio`` deployment. + + .. code-block:: shell + :class: copyable + + mc support top net myminio/ + + The output returns information such as the server URL, network interface, receive rate, transmit rate, and system messages. + + .. tab-item:: SYNTAX + + The command has the following syntax: + + .. code-block:: shell + :class: copyable + + mc [GLOBALFLAGS] support top disk \ + [--interval value] \ + TARGET + + .. include:: /includes/common-minio-mc.rst + :start-after: start-minio-syntax + :end-before: end-minio-syntax + +Parameters +~~~~~~~~~~ + +.. mc-cmd:: TARGET + :required: + + The full path to the :ref:`alias ` or :term:`prefix` where the command should run. + +.. mc-cmd:: --interval + :optional: + + The interval in seconds between metric requests. + + By default, the command requests metrics every second. + +Global Flags +~~~~~~~~~~~~ + +.. include:: /includes/common-minio-mc.rst + :start-after: start-minio-mc-globals + :end-before: end-minio-mc-globals + diff --git a/source/reference/minio-mc/mc-support-top.rst b/source/reference/minio-mc/mc-support-top.rst index a4a2074f..9073b85c 100644 --- a/source/reference/minio-mc/mc-support-top.rst +++ b/source/reference/minio-mc/mc-support-top.rst @@ -16,6 +16,10 @@ ``mc support top`` replaces the ``mc admin top`` command. +.. include:: /includes/common-mc-support.rst + :start-after: start-minio-only + :end-before: end-minio-only + Description ----------- @@ -35,6 +39,7 @@ MinIO deployments, similar to the output of the ``top`` command in a shell. - :mc-cmd:`~mc support top api` - :mc-cmd:`~mc support top locks` - :mc-cmd:`~mc support top disk` +- :mc-cmd:`~mc support top net` Refer to the pages linked above for each subcommand for details. @@ -53,4 +58,5 @@ The command has the following syntax: /reference/minio-mc/mc-support-top-api /reference/minio-mc/mc-support-top-locks - /reference/minio-mc/mc-support-top-disk \ No newline at end of file + /reference/minio-mc/mc-support-top-disk + /reference/minio-mc/mc-support-top-net \ No newline at end of file