From 2ab9e51d34b1fb6bdf57fba2e576b30a0b5ffbbc Mon Sep 17 00:00:00 2001 From: Daryl White <53910321+djwfyi@users.noreply.github.com> Date: Fri, 4 Aug 2023 14:07:02 -0500 Subject: [PATCH] Updates NATS notification docs to point to JetStream service (#937) Streaming configs and envvars are deprecated. Closes #903 --- .../monitoring/publish-events-to-nats.rst | 135 +++++++----------- source/includes/common-mc-admin-config.rst | 9 +- .../minio-mc-admin/mc-admin-config.rst | 27 ++++ .../reference/minio-server/minio-server.rst | 45 ++++-- 4 files changed, 116 insertions(+), 100 deletions(-) diff --git a/source/administration/monitoring/publish-events-to-nats.rst b/source/administration/monitoring/publish-events-to-nats.rst index 5a8347ee..6e641d9c 100644 --- a/source/administration/monitoring/publish-events-to-nats.rst +++ b/source/administration/monitoring/publish-events-to-nats.rst @@ -12,16 +12,21 @@ Publish Events to NATS :local: :depth: 1 -MinIO supports publishing :ref:`bucket notification -` events to a -`NATS `__ service endpoint. +MinIO supports publishing :ref:`bucket notification ` events to a `NATS `__ service endpoint. + +.. admonition:: NATS Streaming Deprecated + :class: important + + NATS Streaming is deprecated. + Migrate to `JetStream `__ instead. + + The related MinIO configuration options and environment variables are deprecated. + Add a NATS Endpoint to a MinIO Deployment ----------------------------------------- -The following procedure adds a new NATS service endpoint for supporting -:ref:`bucket notifications ` in a MinIO -deployment. +The following procedure adds a new NATS service endpoint for supporting :ref:`bucket notifications ` in a MinIO deployment. Prerequisites ~~~~~~~~~~~~~ @@ -35,25 +40,17 @@ See the ``mc`` :ref:`Quickstart ` for installation instructions. 1) Add the NATS Endpoint to MinIO ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can configure a new NATS service endpoint using either environment variables -*or* by setting runtime configuration settings. +You can configure a new NATS service endpoint using either environment variables *or* by setting runtime configuration settings. .. tab-set:: .. tab-item:: Environment Variables - MinIO supports specifying the NATS service endpoint and associated - configuration settings using - :ref:`environment variables - `. The - :mc:`minio server` process applies the specified settings on its - next startup. + MinIO supports specifying the NATS service endpoint and associated configuration settings using :ref:`environment variables `. + The :mc:`minio server` process applies the specified settings on its next startup. - The following example code sets *all* environment variables - related to configuring an NATS service endpoint. The minimum - *required* variables are - :envvar:`MINIO_NOTIFY_NATS_ADDRESS` and - :envvar:`MINIO_NOTIFY_NATS_SUBJECT`: + The following example code sets *all* environment variables related to configuring an NATS service endpoint. + The minimum *required* variables are :envvar:`MINIO_NOTIFY_NATS_ADDRESS` and :envvar:`MINIO_NOTIFY_NATS_SUBJECT`: .. code-block:: shell :class: copyable @@ -67,34 +64,25 @@ You can configure a new NATS service endpoint using either environment variables set MINIO_NOTIFY_NATS_TLS_="" set MINIO_NOTIFY_NATS_TLS_SKIP_VERIFY_="" set MINIO_NOTIFY_NATS_PING_INTERVAL_="" - set MINIO_NOTIFY_NATS_STREAMING_="" - set MINIO_NOTIFY_NATS_STREAMING_ASYNC_="" - set MINIO_NOTIFY_NATS_STREAMING_MAX_PUB_ACKS_IN_FLIGHT_="" - set MINIO_NOTIFY_NATS_STREAMING_CLUSTER_ID_="" + set MINIO_NOTIFY_NATS_QUEUE_DIR_="" + set MINIO_NOTIFY_NATS_QUEUE_LIMIT_="" set MINIO_NOTIFY_NATS_CERT_AUTHORITY_="" set MINIO_NOTIFY_NATS_CLIENT_CERT_="" set MINIO_NOTIFY_NATS_CLIENT_KEY_="" - set MINIO_NOTIFY_NATS_QUEUE_DIR_="" - set MINIO_NOTIFY_NATS_QUEUE_LIMIT_="" set MINIO_NOTIFY_NATS_COMMENT_="" + set MINIO_NOTIFY_NATS_JETSTREAM_="" - - Replace ```` with a unique descriptive string for the - NATS service endpoint. Use the same ```` value for all - environment variables related to the new target service endpoint. + - Replace ```` with a unique descriptive string for the NATS service endpoint. + Use the same ```` value for all environment variables related to the new target service endpoint. The following examples assume an identifier of ``PRIMARY``. - If the specified ```` matches an existing NATS service - endpoint on the MinIO deployment, the new settings *override* - any existing settings for that endpoint. Use - :mc-cmd:`mc admin config get notify_nats ` to - review the currently configured NATS endpoints on the MinIO deployment. + If the specified ```` matches an existing NATS service endpoint on the MinIO deployment, the new settings *override* any existing settings for that endpoint. + Use :mc-cmd:`mc admin config get notify_nats ` to review the currently configured NATS endpoints on the MinIO deployment. - Replace ```` with the URL of the NATS service endpoint. For example: ``htpps://nats-endpoint.example.com:4222`` - See :ref:`NATS Service for Bucket Notifications - ` for complete documentation - on each environment variable. + See :ref:`NATS Service for Bucket Notifications ` for complete documentation on each environment variable. .. tab-item:: Configuration Settings @@ -118,13 +106,10 @@ You can configure a new NATS service endpoint using either environment variables username="" \ password="" \ token="" \ + nats_jetstream="" \ tls="" \ tls_skip_verify="" \ ping_interval="" \ - streaming="" \ - streaming_async="" \ - streaming_max_pub_acks_in_flight="" \ - streaming_cluster_id="" \ cert_authority="" \ client_cert="" \ client_key="" \ @@ -133,24 +118,18 @@ You can configure a new NATS service endpoint using either environment variables comment="" - - Replace ``IDENTIFIER`` with a unique descriptive string for the - NATS service endpoint. The following examples in this procedure - assume an identifier of ``PRIMARY``. + - Replace ``IDENTIFIER`` with a unique descriptive string for the NATS service endpoint. + The following examples in this procedure assume an identifier of ``PRIMARY``. - If the specified ``IDENTIFIER`` matches an existing NATS service - endpoint on the MinIO deployment, the new settings *override* - any existing settings for that endpoint. Use - :mc-cmd:`mc admin config get notify_nats ` to - review the currently configured NATS endpoints on the MinIO deployment. + If the specified ``IDENTIFIER`` matches an existing NATS service endpoint on the MinIO deployment, the new settings *override* any existing settings for that endpoint. + Use :mc-cmd:`mc admin config get notify_nats ` to review the currently configured NATS endpoints on the MinIO deployment. - Replace ``ENDPOINT`` with the URL of the NATS service endpoint. For example: ``htpps://nats-endpoint.example.com:4222``. - See :ref:`NATS Bucket Notification Configuration Settings - ` for complete - documentation on each setting. + See :ref:`NATS Bucket Notification Configuration Settings ` for complete documentation on each setting. -2) Restart the MinIO Deployment +1) Restart the MinIO Deployment ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You must restart the MinIO deployment to apply the configuration changes. @@ -242,8 +221,7 @@ See the ``mc`` :ref:`Quickstart ` for installation instructions. 1) List Configured NATS Endpoints In The Deployment ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Use the :mc-cmd:`mc admin config get` command to list the currently -configured NATS service endpoints in the deployment: +Use the :mc-cmd:`mc admin config get` command to list the currently configured NATS service endpoints in the deployment: .. code-block:: shell :class: copyable @@ -256,23 +234,19 @@ The command output resembles the following: .. code-block:: shell - notify_nats:primary password="yoursecret" streaming_max_pub_acks_in_flight="10" subject="" address="nats-endpoint.example.com:4222" token="" username="yourusername" ping_interval="0" queue_limit="0" tls="off" tls_skip_verify="off" streaming_async="on" queue_dir="" streaming_cluster_id="test-cluster" streaming_enable="on" - notify_nats:secondary password="yoursecret" streaming_max_pub_acks_in_flight="10" subject="" address="nats-endpoint.example.com:4222" token="" username="yourusername" ping_interval="0" queue_limit="0" tls="off" tls_skip_verify="off" streaming_async="on" queue_dir="" streaming_cluster_id="test-cluster" streaming_enable="on" + notify_nats:primary password="yoursecret" subject="" address="nats-endpoint.example.com:4222" token="" username="yourusername" ping_interval="0" queue_limit="0" tls="off" tls_skip_verify="off" queue_dir="" streaming_enable="on" nats_jetstream="on" + notify_nats:secondary password="yoursecret" subject="" address="nats-endpoint.example.com:4222" token="" username="yourusername" ping_interval="0" queue_limit="0" tls="off" tls_skip_verify="off" queue_dir="" streaming_enable="on" nats_jetstream="on" -The :mc-conf:`notify_nats` key is the top-level configuration key for an -:ref:`minio-server-config-bucket-notification-nats`. The -:mc-conf:`address ` key specifies the NATS service endpoint -for the given ``notify_nats`` key. The ``notify_nats:`` suffix -describes the unique identifier for that NATS service endpoint. +The :mc-conf:`notify_nats` key is the top-level configuration key for an :ref:`minio-server-config-bucket-notification-nats`. +The :mc-conf:`address ` key specifies the NATS service endpoint for the given ``notify_nats`` key. +The ``notify_nats:`` suffix describes the unique identifier for that NATS service endpoint. -Note the identifier for the NATS service endpoint you want to update for -the next step. +Note the identifier for the NATS service endpoint you want to update for the next step. 2) Update the NATS Endpoint ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Use the :mc-cmd:`mc admin config set` command to set the new configuration -for the NATS service endpoint: +Use the :mc-cmd:`mc admin config set` command to set the new configuration for the NATS service endpoint: .. code-block:: shell :class: copyable @@ -286,10 +260,7 @@ for the NATS service endpoint: tls="" \ tls_skip_verify="" \ ping_interval="" \ - streaming="" \ - streaming_async="" \ - streaming_max_pub_acks_in_flight="" \ - streaming_cluster_id="" \ + nats_jetstream="" \ cert_authority="" \ client_cert="" \ client_key="" \ @@ -297,10 +268,9 @@ for the NATS service endpoint: queue_limit="" \ comment="" -The :mc-conf:`notify_nats address ` configuration setting -is the *minimum* required for an NATS service endpoint. All other configuration -settings are *optional*. See :ref:`minio-server-config-bucket-notification-nats` -for a complete list of NATS configuration settings. +The :mc-conf:`notify_nats address ` configuration setting is the *minimum* required for an NATS service endpoint. +All other configuration settings are *optional*. +See :ref:`minio-server-config-bucket-notification-nats` for a complete list of NATS configuration settings. 3) Restart the MinIO Deployment ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -313,28 +283,21 @@ Use the :mc-cmd:`mc admin service restart` command to restart the deployment. mc admin service restart ALIAS -Replace ``ALIAS`` with the :ref:`alias ` of the deployment to -restart. +Replace ``ALIAS`` with the :ref:`alias ` of the deployment to restart. -The :mc:`minio server` process prints a line on startup for each configured NATS -target similar to the following: +The :mc:`minio server` process prints a line on startup for each configured NATS target similar to the following: .. code-block:: shell SQS ARNs: arn:minio:sqs::primary:nats -3) Validate the Changes +4) Validate the Changes ~~~~~~~~~~~~~~~~~~~~~~~ -Perform an action on a bucket which has an event configuration using the updated -NATS service endpoint and check the NATS service for the notification data. The -action required depends on which :mc-cmd:`events ` were -specified when configuring the bucket notification. +Perform an action on a bucket which has an event configuration using the updated NATS service endpoint and check the NATS service for the notification data. +The action required depends on which :mc-cmd:`events ` were specified when configuring the bucket notification. -For example, if the bucket notification configuration includes the -``s3:ObjectCreated:Put`` event, you can use the -:mc:`mc cp` command to create a new object in the bucket and trigger -a notification. +For example, if the bucket notification configuration includes the ``s3:ObjectCreated:Put`` event, you can use the :mc:`mc cp` command to create a new object in the bucket and trigger a notification. .. code-block:: shell :class: copyable diff --git a/source/includes/common-mc-admin-config.rst b/source/includes/common-mc-admin-config.rst index 1b33e855..cdaa2b18 100644 --- a/source/includes/common-mc-admin-config.rst +++ b/source/includes/common-mc-admin-config.rst @@ -533,10 +533,17 @@ MinIO supports the following time units: .. start-minio-notify-nats-streaming -Specify ``on`` to enable streaming events to the NATS service endpoint. +Specify ``on`` to enable JetStream support for streaming events to a NATS JetStream service endpoint. .. end-minio-notify-nats-streaming +.. start-minio-notify-nats-jetstream + +Specify ``on`` to enable asynchronous publishing of events to the NATS service endpoint. + +.. end-minio-notify-nats-jetstream + + .. start-minio-notify-nats-streaming-async Specify ``on`` to enable asynchronous publishing of events to the NATS service diff --git a/source/reference/minio-mc-admin/mc-admin-config.rst b/source/reference/minio-mc-admin/mc-admin-config.rst index 619c8e7c..f43e1463 100644 --- a/source/reference/minio-mc-admin/mc-admin-config.rst +++ b/source/reference/minio-mc-admin/mc-admin-config.rst @@ -1342,6 +1342,14 @@ service as a target for :ref:`Bucket Nofitications ` :ref:`minio-bucket-notifications-publish-nats` for a tutorial on using these environment variables. +.. admonition:: NATS Streaming Deprecated + :class: important + + NATS Streaming is deprecated. + Migrate to `JetStream `__ instead. + + The related MinIO configuration options and environment variables are deprecated. + .. mc-conf:: notify_nats The top-level configuration key for defining an NATS service endpoint for use @@ -1480,9 +1488,22 @@ using these environment variables. This configuration setting corresponds with the environment variable :envvar:`MINIO_NOTIFY_NATS_PING_INTERVAL`. + .. mc-conf:: jetstream + :delimiter: " " + + *Optional* + + .. include:: /includes/common-mc-admin-config.rst + :start-after: start-minio-notify-nats-jetstream + :end-before: end-minio-notify-nats-jetstream + + This configuration setting corresponds with the environment variable :envvar:`MINIO_NOTIFY_NATS_JETSTREAM`. + .. mc-conf:: streaming :delimiter: " " + *Deprecated* + *Optional* .. include:: /includes/common-mc-admin-config.rst @@ -1495,6 +1516,8 @@ using these environment variables. .. mc-conf:: streaming_async :delimiter: " " + *Deprecated* + *Optional* .. include:: /includes/common-mc-admin-config.rst @@ -1507,6 +1530,8 @@ using these environment variables. .. mc-conf:: streaming_max_pub_acks_in_flight :delimiter: " " + *Deprecated* + *Optional* .. include:: /includes/common-mc-admin-config.rst @@ -1519,6 +1544,8 @@ using these environment variables. .. mc-conf:: streaming_cluster_id :delimiter: " " + *Deprecated* + *Optional* .. include:: /includes/common-mc-admin-config.rst diff --git a/source/reference/minio-server/minio-server.rst b/source/reference/minio-server/minio-server.rst index dcddb483..8cc72faf 100644 --- a/source/reference/minio-server/minio-server.rst +++ b/source/reference/minio-server/minio-server.rst @@ -998,6 +998,7 @@ These environment variables configure notification targets for use with - :ref:`minio-server-envvar-bucket-notification-elasticsearch` - :ref:`minio-server-envvar-bucket-notification-nsq` - :ref:`minio-server-envvar-bucket-notification-redis` +- :ref:`minio-server-envvar-bucket-notification-nats` - :ref:`minio-server-envvar-bucket-notification-postgresql` - :ref:`minio-server-envvar-bucket-notification-mysql` - :ref:`minio-server-envvar-bucket-notification-kafka` @@ -1713,30 +1714,30 @@ endpoints as ``PRIMARY`` and ``SECONDARY`` respectively: NATS Service for Bucket Notifications +++++++++++++++++++++++++++++++++++++ -The following section documents environment variables for configuring an NATS -service as a target for :ref:`Bucket Nofitications `. See -:ref:`minio-bucket-notifications-publish-nats` for a tutorial on -using these environment variables. +.. admonition:: NATS Streaming Deprecated + :class: important -You can specify multiple NATS service endpoints by appending a unique identifier -``_ID`` for each set of related NATS environment variables: -the top level key. For example, the following commands set two distinct NATS -service endpoints as ``PRIMARY`` and ``SECONDARY`` respectively: + NATS Streaming is deprecated. + Migrate to `JetStream `__ instead. + + The related MinIO configuration options and environment variables are deprecated. + +The following section documents environment variables for configuring an NATS service as a target for :ref:`Bucket Nofitications `. +See :ref:`minio-bucket-notifications-publish-nats` for a tutorial on using these environment variables. + +You can specify multiple NATS service endpoints by appending a unique identifier ``_ID`` for each set of related NATS environment variables no to the top level key. +For example, the following commands set two distinct NATS service endpoints as ``PRIMARY`` and ``SECONDARY`` respectively: .. code-block:: shell :class: copyable set MINIO_NOTIFY_NATS_ENABLE_PRIMARY="on" set MINIO_NOTIFY_NATS_ADDRESS_PRIMARY="https://nats-endpoint.example.net:4222" - set MINIO_NOTIFY_NATS_SUBJECT="minioevents" set MINIO_NOTIFY_NATS_ENABLE_SECONDARY="on" set MINIO_NOTIFY_NATS_ADDRESS_SECONDARY="https://nats-endpoint.example.net:4222" - set MINIO_NOTIFY_NATS_SUBJECT="minioevents" -For example, :envvar:`MINIO_NOTIFY_NATS_ENABLE_PRIMARY -` indicates the environment variable is associated to -an NATS service endpoint with ID of ``PRIMARY``. +For example, :envvar:`MINIO_NOTIFY_NATS_ENABLE_PRIMARY ` indicates the environment variable is associated to an NATS service endpoint with ID of ``PRIMARY``. .. envvar:: MINIO_NOTIFY_NATS_ENABLE @@ -1843,8 +1844,20 @@ an NATS service endpoint with ID of ``PRIMARY``. :mc-conf:`notify_nats ping_interval ` configuration setting. +.. envvar:: MINIO_NOTIFY_NATS_JETSTREAM + + *Optional* + + .. include:: /includes/common-mc-admin-config.rst + :start-after: start-minio-notify-nats-jetstream + :end-before: end-minio-notify-nats-jetstream + + This environment variable corresponds with the :mc-conf:`notify_nats jetstream ` configuration setting. + .. envvar:: MINIO_NOTIFY_NATS_STREAMING + *Deprecated* + *Optional* .. include:: /includes/common-mc-admin-config.rst @@ -1857,6 +1870,8 @@ an NATS service endpoint with ID of ``PRIMARY``. .. envvar:: MINIO_NOTIFY_NATS_STREAMING_ASYNC + *Deprecated* + *Optional* .. include:: /includes/common-mc-admin-config.rst @@ -1869,6 +1884,8 @@ an NATS service endpoint with ID of ``PRIMARY``. .. envvar:: MINIO_NOTIFY_NATS_STREAMING_MAX_PUB_ACKS_IN_FLIGHT + *Deprecated* + *Optional* .. include:: /includes/common-mc-admin-config.rst @@ -1881,6 +1898,8 @@ an NATS service endpoint with ID of ``PRIMARY``. .. envvar:: MINIO_NOTIFY_NATS_STREAMING_CLUSTER_ID + *Deprecated* + *Optional* .. include:: /includes/common-mc-admin-config.rst