minio/docs
1
0
Fork 0
mirror of https://github.com/minio/docs.git synced 2025-07-28 19:42:10 +03:00
Code Activity

Fixing multiple issues (#906)

Browse Source 
DOCS-902: Add "M' and "Mi" units to --limit-upload and --limit-download
DOCS-895: Take Bucket/IAM snapshot before site replication config
DOCS-886: Add audit_kafka config settings and envvars.
This commit is contained in:
Ravind Kumar
2023-06-30 16:06:48 -04:00
committed by GitHub
parent 9302e4fd86
commit c6b87ee0ee
10 changed files with 432 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

101
source/includes/common-mc-admin-config.rst
View File

@ -1022,3 +1022,104 @@ Ensure you have at least one other admin user, such as one with the :userpolicy:
If you do not have another admin user, disabling the root account locks administrative access to the deployment.
.. end-minio-root-api-access
.. kafka audit settings
.. start-minio-kafka-audit-logging-brokers-desc
A comma-separated list of Kafka broker addresses:
.. code-block:: shell
brokers="https://kafka-1.example.net:9092,https://kafka-2.example.net:9092"
At least one broker must be online and reachable by the MinIO server to initialize and send audit log events.
MinIO checks each specified broker in order of specification.
.. end-minio-kafka-audit-logging-brokers-desc
.. start-minio-kafka-audit-logging-topic-desc
The name of the Kafka topic to associate to MinIO audit log events.
.. end-minio-kafka-audit-logging-topic-desc
.. start-minio-kafka-audit-logging-tls-desc
Set to ``"on"`` to enable TLS connectivity to the specified Kafka brokers.
Defaults to ``"off"``.
.. end-minio-kafka-audit-logging-tls-desc
.. start-minio-kafka-audit-logging-tls-skip-verify-desc
Set to ``"on"`` to direct MinIO to skip verification of the Kafka broker TLS certificates.
You can use this option for enabling connectivity to Kafka brokers using TLS certificates signed by unknown parties, such as self-signed or corporate-internal Certificate Authorities (CA).
MinIO by default uses the system trust store *and* the contents of the MinIO :ref:`CA directory <minio-tls>` for verifying remote client TLS certificates.
Defaults to ``"off"`` for strict verification of TLS certificates.
.. end-minio-kafka-audit-logging-tls-skip-verify-desc
.. start-minio-kafka-audit-logging-tls-client-auth-desc
Set to ``"on"`` to direct MinIO to use mTLS to authenticate against the Kafka brokers.
.. end-minio-kafka-audit-logging-tls-client-auth-desc
.. start-minio-kafka-audit-logging-client-tls-cert-desc
The path to the TLS client certificate to use for mTLS authentication.
.. end-minio-kafka-audit-logging-client-tls-cert-desc
.. start-minio-kafka-audit-logging-client-tls-key-desc
The path to the TLS client private key to use for mTLS authentication.
.. end-minio-kafka-audit-logging-client-tls-key-desc
.. start-minio-kafka-audit-logging-sasl-desc
Set to ``"on"`` to direct MinIO to use SASL to authenticate against the Kafka brokers.
.. end-minio-kafka-audit-logging-sasl-desc
.. start-minio-kafka-audit-logging-sasl-username-desc
The SASL username MinIO uses for authentication against the Kafka brokers.
.. end-minio-kafka-audit-logging-sasl-username-desc
.. start-minio-kafka-audit-logging-sasl-password-desc
The SASL password MinIO uses for authentication against the Kafka brokers.
.. end-minio-kafka-audit-logging-sasl-password-desc
.. start-minio-kafka-audit-logging-sasl-mechanism-desc
The SASL mechanism MinIO uses for authentication against the Kafka brokers.
Defaults to ``plain``.
.. end-minio-kafka-audit-logging-sasl-mechanism-desc
.. start-minio-kafka-audit-logging-version-desc
The version of the Kafka broker MinIO expects at the specified endpoints.
MinIO returns an error if the Kakfa broker verison does not match those specified to this setting.
.. end-minio-kafka-audit-logging-version-desc
.. start-minio-kafka-audit-logging-comment-desc
A comment to associate with the configuration.
.. end-minio-kafka-audit-logging-comment-desc

4
source/includes/linux/minio-client.rst
View File

@ -9,9 +9,11 @@
- ``B`` for bytes
- ``K`` for kilobytes
- ``M`` for megabytes
- ``G`` for gigabytes
- ``T`` for terabytes
- ``Ki`` for kibibytes
- ``Mi`` for mibibytes
- ``Gi`` for gibibytes
- ``Ti`` for tebibytes
@ -32,9 +34,11 @@
- ``B`` for bytes
- ``K`` for kilobytes
- ``M`` for megabytes
- ``G`` for gigabytes
- ``T`` for terabytes
- ``Ki`` for kibibytes
- ``Mi`` for mibibytes
- ``Gi`` for gibibytes
- ``Ti`` for tebibytes

10
source/operations/install-deploy-manage/decommission-server-pool.rst
View File

@ -136,15 +136,6 @@ Complete any planned :ref:`hardware expansion <expand-minio-distributed>` prior
Decommissioning requires that a cluster's topology remain stable throughout the pool draining process.
Do **not** attempt to perform expansion and decommission changes in a single step.
Decommissioning Ignores Delete Markers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MinIO does *not* migrate objects whose only remaining version is a
:ref:`delete markers <minio-bucket-versioning-delete>`. This avoids creating
empty metadata on the remaining server pools for objects already considered
fully deleted.
Decommissioning is Resumable
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -171,6 +162,7 @@ Decommissioning Ignores Expired Objects and Trailing ``DeleteMarker``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Starting with :minio-release:`RELEASE.2023-05-27T05-56-19Z`, decommissioning ignores objects where the only remaining version is a ``DeleteMarker``.
This avoids creating empty metadata on the remaining server pool(s) for objects that are effectively fully deleted.
Starting with :minio-release:`minio-lifecycle-management-scanner`, decommissioning also ignores object versions which have expired based on the configured :ref:`lifecycle rules <minio-lifecycle-management-expiration>` for the parent bucket.

6
source/operations/install-deploy-manage/multi-site-replication.rst
View File

@ -131,6 +131,12 @@ MinIO does not proxy ``LIST``, ``DELETE``, and ``PUT`` operations.
Prerequisites
-------------
Back Up Cluster Settings First
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :mc:`mc admin cluster bucket export` and :mc:`mc admin cluster iam export` commands to take a snapshot of the bucket metadata and IAM configurations respectively prior to configuring Site Replication.
You can use these snapshots to restore bucket/IAM settings in the event of misconfiguration during site replication configuration.
One Site with Data at Setup
~~~~~~~~~~~~~~~~~~~~~~~~~~~

161
source/reference/minio-mc-admin/mc-admin-config.rst
View File

@ -316,6 +316,167 @@ HTTP Webhook Audit Log Target
This setting corresponds to the :envvar:`MINIO_AUDIT_WEBHOOK_QUEUE_SIZE` environment variable.
.. _minio-server-config-logging-kafka-audit:
Kafka Audit Log Target
~~~~~~~~~~~~~~~~~~~~~~
.. mc-conf:: audit_kafka
The top-level configuration key for defining a Kafka broker target for publishing :ref:`MinIO audit logs <minio-logging>`.
Use :mc-cmd:`mc admin config set` to set or update a Kafka audit target.
Specify additional optional arguments as a whitespace (``" "``)-delimited list.
.. code-block:: shell
:class: copyable
mc admin config set audit_kafka \
brokers="https://kafka-endpoint.example.net:9092" [ARGUMENTS=VALUE ...]
The :mc-conf:`audit_kafka` configuration key accepts the following arguments:
.. mc-conf:: brokers
:required:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-brokers-desc
:end-before: end-minio-kafka-audit-logging-brokers-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_BROKERS` environment variable.
.. mc-conf:: topic
:required:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-topic-desc
:end-before: end-minio-kafka-audit-logging-topic-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_TOPIC` environment variable.
.. mc-conf:: tls
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-tls-desc
:end-before: end-minio-kafka-audit-logging-tls-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_TLS` environment variable.
.. mc-conf:: tls_skip_verify
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-tls-skip-verify-desc
:end-before: end-minio-kafka-audit-logging-tls-skip-verify-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_TLS_SKIP_VERIFY` environment variable.
.. mc-conf:: tls_client_auth
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-tls-client-auth-desc
:end-before: end-minio-kafka-audit-logging-tls-client-auth-desc
Requires specifying :mc-conf:`~audit_kafka.client_tls_cert` and :mc-conf:`~audit_kafka.client_tls_key`.
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_TLS_CLIENT_AUTH` environment variable.
.. mc-conf:: client_tls_cert
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-client-tls-cert-desc
:end-before: end-minio-kafka-audit-logging-client-tls-cert-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_CLIENT_TLS_CERT` environment variable.
.. mc-conf:: client_tls_key
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-client-tls-key-desc
:end-before: end-minio-kafka-audit-logging-client-tls-key-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_CLIENT_TLS_KEY` environment variable.
.. mc-conf:: sasl
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-sasl-desc
:end-before: end-minio-kafka-audit-logging-sasl-desc
Requires specifying :mc-conf:`~audit_kafka.sasl_username` and :mc-conf:`~audit_kafka.sasl_password`.
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_SASL` environment variable.
.. mc-conf:: sasl_username
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-sasl-username-desc
:end-before: end-minio-kafka-audit-logging-sasl-username-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_SASL_USERNAME` environment variable.
.. mc-conf:: sasl_password
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-sasl-password-desc
:end-before: end-minio-kafka-audit-logging-sasl-password-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_SASL_PASSWORD` environment variable.
.. mc-conf:: sasl_mechanism
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-sasl-mechanism-desc
:end-before: end-minio-kafka-audit-logging-sasl-mechanism-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_SASL_MECHANISM` environment variable.
.. important::
The ``PLAIN`` authentication mechanism sends credentials in plain text over the network.
Use :mc-conf:`~audit_kafka.tls` to enable TLS connectivity to the Kafka brokers and ensure secure transmission of SASL credentials.
.. mc-conf:: version
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-version-desc
:end-before: end-minio-kafka-audit-logging-version-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_VERSION` environment variable.
.. mc-conf:: comment
:optional:
:delimiter: " "
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-comment-desc
:end-before: end-minio-kafka-audit-logging-comment-desc
This setting corresponds to the :envvar:`MINIO_AUDIT_KAFKA_COMMENT` environment variable.
.. _minio-server-config-bucket-notification-amqp:
AMQP Service for Bucket Notifications

4
source/reference/minio-mc-admin/mc-admin-rebalance.rst
View File

@ -168,7 +168,9 @@ Rebalancing Ignores Expired Objects and Trailing ``DeleteMarker``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Starting with :minio-release:`RELEASE.2023-06-23T20-26-00Z`, rebalancing ignores object versions which have expired based on the configured :ref:`lifecycle rules <minio-lifecycle-management-expiration>` for the parent bucket.
Rebalancing also ignores objects where the only remaining version is a ``DeleteMarker``.
Rebalancing also ignores objects where the only remaining version is a :ref:`delete marker <minio-bucket-versioning-delete>`.
This avoids inter-pool :abbr:`I/O (Input/Output)` for objects already considered fully deleted.
MinIO relies on the :ref:`scanner <minio-lifecycle-management-scanner>` to capture and remove those expired objects or trailing ``DeleteMarker`` objects.

4
source/reference/minio-mc/mc-batch-generate.rst
View File

@ -167,7 +167,7 @@ For the **source deployment**
* - ``endpoint:``
- | Location of the source deployment.
|
| If the source is the :ref:`alias <alias>`` specified to the command, you can omit this and the ``credentials`` fields.
| If the source is the :ref:`alias <alias>` specified to the command, you can omit this and the ``credentials`` fields.
| If the source is "local", the target *must* specify the remote deployment with ``endpoint`` and ``credentials``.
* - ``path:``
@ -207,7 +207,7 @@ For the **target deployment**
* - ``endpoint:``
- | The location of the target deployment.
|
| If the target is the :ref:`alias <alias>`` specified to the command, you can omit this and the ``credentials`` fields.
| If the target is the :ref:`alias <alias>` specified to the command, you can omit this and the ``credentials`` fields.
| If the target is "local", the source *must* specify the remote deployment with ``endpoint`` and ``credentials``.

2
source/reference/minio-mc/mc-mirror.rst
View File

@ -149,6 +149,8 @@ Parameters
Perform a mock mirror operation.
Use this operation to test that the :mc:`mc mirror` operation will only mirror the desired objects or buckets.
.. --limit-download and --limit-upload included here
.. include:: /includes/linux/minio-client.rst
:start-after: start-mc-limit-flags-desc
:end-before: end-mc-limit-flags-desc

145
source/reference/minio-server/minio-server.rst
View File

@ -592,6 +592,7 @@ documentation.
- :ref:`minio-sever-envvar-logging-regular`
- :ref:`minio-sever-envvar-logging-audit`
- :ref:`minio-sever-envvar-logging-audit-kafka`
.. _minio-sever-envvar-logging-regular:
@ -701,8 +702,8 @@ server logs webhook endpoints:
.. _minio-sever-envvar-logging-audit:
Audit Logs
++++++++++
Webhook Audit Logs
++++++++++++++++++
The following section documents environment variables for configuring MinIO to
publish audit logs to an HTTP webhook endpoint. See
@ -800,6 +801,146 @@ audit log webhook endpoints:
This variable corresponds to the :mc-conf:`audit_webhook queue_size <audit_webhook.queue_size>` configuration setting.
.. _minio-sever-envvar-logging-audit-kafka:
Kafka Audit Logs
++++++++++++++++
The following section documents environment variables for configuring MinIO to publish audit logs to a Kafka broker.
.. envvar:: MINIO_AUDIT_KAFKA_ENABLE
:required:
Set to ``"on"`` to enable the target.
Set to ``"off"`` to disable the target.
.. envvar:: MINIO_AUDIT_KAFKA_BROKERS
:required:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-brokers-desc
:end-before: end-minio-kafka-audit-logging-brokers-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.brokers` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_TOPIC
:required:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-topic-desc
:end-before: end-minio-kafka-audit-logging-topic-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.topic` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_TLS
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-tls-desc
:end-before: end-minio-kafka-audit-logging-tls-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.tls` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_TLS_SKIP_VERIFY
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-tls-skip-verify-desc
:end-before: end-minio-kafka-audit-logging-tls-skip-verify-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.tls_skip_verify` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_SASL
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-sasl-desc
:end-before: end-minio-kafka-audit-logging-sasl-desc
Requires specifying :envvar:`MINIO_AUDIT_KAFKA_SASL_USERNAME` and :envvar:`MINIO_AUDIT_KAFKA_SASL_PASSWORD`.
This environment variable corresponds to the :mc-conf:`audit_kafka.sasl` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_SASL_USERNAME
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-sasl-username-desc
:end-before: end-minio-kafka-audit-logging-sasl-username-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.sasl_username` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_SASL_PASSWORD
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-sasl-password-desc
:end-before: end-minio-kafka-audit-logging-sasl-password-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.sasl_password` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_SASL_MECHANISM
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-sasl-mechanism-desc
:end-before: end-minio-kafka-audit-logging-sasl-mechanism-desc
.. important::
The ``PLAIN`` authentication mechanism sends credentials in plain text over the network.
Use :envvar:`MINIO_AUDIT_KAFKA_TLS` to enable TLS connectivity to the Kafka brokers and ensure secure transmission of SASL credentials.
This environment variable corresponds to the :mc-conf:`audit_kafka.sasl_mechanism` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_TLS_CLIENT_AUTH
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-tls-client-auth-desc
:end-before: end-minio-kafka-audit-logging-tls-client-auth-desc
Requires specifying :envvar:`MINIO_AUDIT_KAFKA_CLIENT_TLS_CERT` and :envvar:`MINIO_AUDIT_KAFKA_CLIENT_TLS_KEY`.
This environment variable corresponds to the :mc-conf:`audit_kafka.tls_client_auth` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_CLIENT_TLS_CERT
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-client-tls-cert-desc
:end-before: end-minio-kafka-audit-logging-client-tls-cert-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.client_tls_cert` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_CLIENT_TLS_KEY
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-client-tls-key-desc
:end-before: end-minio-kafka-audit-logging-client-tls-key-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.client_tls_key` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_VERSION
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-version-desc
:end-before: end-minio-kafka-audit-logging-version-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.version` configuration setting.
.. envvar:: MINIO_AUDIT_KAFKA_COMMENT
:optional:
.. include:: /includes/common-mc-admin-config.rst
:start-after: start-minio-kafka-audit-logging-comment-desc
:end-before: end-minio-kafka-audit-logging-comment-desc
This environment variable corresponds to the :mc-conf:`audit_kafka.comment` configuration setting.
Bucket Notifications
~~~~~~~~~~~~~~~~~~~~