1
0
mirror of https://github.com/minio/docs.git synced 2025-07-03 16:02:47 +03:00
Files
Daryl White eb94513104 Breaking up envvars and config settings into separate settings pages (#1028)
- Creates a slew of folders and pages nested under the mc-server page to store settings.
- Breaks up all of the environment variable options formerly in the mc-server page into these separate pages.
- Moves the config settings formerly in the mc admin config page to the appropriate new settings pages.
- Adds a MiniO Client settings page and document `mc-host-<alias>`

Closes #1017
2023-11-03 07:55:22 -04:00

9.8 KiB

Kafka Notification Settings

minio

Table of Contents

This page documents settings for configuring an Kafka service as a target for Bucket Notifications <minio-bucket-notifications>. See minio-bucket-notifications-publish-kafka for a tutorial on using these settings.

Multiple Kafka Targets

You can specify multiple Kafka service endpoints by appending a unique identifier _ID for each set of related Kafka settings on to the top level key.

Examples

For example, the following commands set two distinct Kafka service endpoints as PRIMARY and SECONDARY respectively:

Environment Variable

set MINIO_NOTIFY_KAFKA_ENABLE_PRIMARY="on"
set MINIO_NOTIFY_KAFKA_BROKERS_PRIMARY="https://kafka1.example.net:9200, https://kafka2.example.net:9200"

set MINIO_NOTIFY_KAFKA_ENABLE_SECONDARY="on"
set MINIO_NOTIFY_KAFKA_BROKERS_SECONDARY="https://kafka1.example.net:9200, https://kafka2.example.net:9200"

Configuration Setting

mc admin config set notify_kafka:primary \ 
   brokers="https://kafka1.example.net:9200, https://kafka2.example.net:9200"
   [ARGUMENT=VALUE ...]

mc admin config set notify_kafka:secondary \
   brokers="https://kafka1.example.net:9200, https://kafka2.example.net:9200"
   [ARGUMENT=VALUE ...]

Notice that for configuration settings, the unique identifier appends to notify_kafka only, not to each individual argument.

Settings

Enable

Required

Environment Variable

MINIO_NOTIFY_KAFKA_ENABLE

Specify on to enable publishing bucket notifications to a Kafka service endpoint.

Defaults to off.

Configuration Setting

notify_kafka

The top-level configuration key for defining an Kafka service endpoint for use with MinIO bucket notifications <minio-bucket-notifications>.

Use mc admin config set to set or update an Kafka service endpoint. The ~notify_kafka.brokers argument is required for each target. Specify additional optional arguments as a whitespace (" ")-delimited list.

mc admin config set notify_kafka \ 
  brokers="https://kafka1.example.net:9200, https://kafka2.example.net:9200"
  [ARGUMENT="VALUE"] ... \

Brokers

Required

Environment Variable

MINIO_NOTIFY_KAFKA_BROKERS

Configuration Setting

notify_kafka brokers

Specify a comma-separated list of Kafka broker addresses. For example:

"kafka1.example.com:2021,kafka2.example.com:2021"

Topic

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_TOPIC

Configuration Setting

notify_kafka topic

Specify the name of the Kafka topic to which MinIO publishes bucket events.

SASL

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_SASL

Configuration Setting

notify_kafka sasl

Specify on to enable SASL authentication.

SASL Username

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_SASL_USERNAME

Configuration Setting

notify_kafka sasl_username

Specify the username for performing SASL/PLAIN or SASL/SCRAM authentication to the Kafka broker(s).

SASL Password

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_SASL_PASSWORD

Configuration Setting

notify_kafka sasl_password

Specify the password for performing SASL/PLAIN or SASL/SCRAM authentication to the Kafka broker(s).

RELEASE.2023-06-23T20-26-00Z

MinIO redacts this value when returned as part of mc admin config get.

SASL Mechanism

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_SASL_MECHANISM

Configuration Setting

notify_kafka sasl_mechanism

Specify the SASL mechanism to use for authenticating to the Kafka broker(s). MinIO supports the following mechanisms:

  • PLAIN (Default)
  • SHA256
  • SHA512

TLS Client Auth

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_TLS_CLIENT_AUTH

Configuration Setting

notify_kafka tls_client_auth

Specify the client authentication type of the Kafka broker(s). The following table lists the supported values and their mappings

Value Authentication Type
0 NoClientCert
1 RequestClientCert
2 RequireAnyClientCert
3 VerifyClientCertIfGiven
4 RequireAndVerifyClientCert

See ClientAuthType for more information on each client auth type.

TLS

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_TLS

Configuration Setting

notify_kafka tls

Specify on to enable TLS connectivity to the Kafka broker(s).

TLS Skip Verify

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_TLS_SKIP_VERIFY

Configuration Setting

notify_kafka tls_skip_verify

Enables or disables TLS verification of the NATS service endpoint TLS certificates.

  • Specify on to disable TLS verification (Default).
  • Specify off to enable TLS verification.

Client TLS Cert

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_CLIENT_TLS_CERT

Configuration Setting

notify_kafka client_tls_cert

Specify the path to the client certificate to use for performing mTLS authentication to the Kafka broker(s).

Client TLS Key

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_CLIENT_TLS_KEY

Configuration Setting

notify_kafka client_tls_key

Specify the path to the client private key to use for performing mTLS authentication to the Kafka broker(s).

Version

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_VERSION

Configuration Setting

notify_kafka version

Specify the version of the Kafka cluster to assume when performing operations against that cluster. See the sarama reference documentation for more information on this field's behavior.

Queue Directory

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_QUEUE_DIR

Configuration Setting

notify_kafka queue_dir

Specify the directory path to enable MinIO's persistent event store for undelivered messages, such as /opt/minio/events.

MinIO stores undelivered events in the specified store while the Kafka server/broker is offline and replays the stored events when connectivity resumes.

Queue Limit

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_QUEUE_LIMIT

Configuration Setting

notify_kafka queue_limit

Specify the maximum limit for undelivered messages. Defaults to 100000.

Comment

Optional

Environment Variable

MINIO_NOTIFY_KAFKA_COMMENT

Configuration Setting

notify_kafka comment

Specify a comment to associate with the Kafka configuration.