From 1aea92b1410e28802983d437d7b9888ac5ee19e9 Mon Sep 17 00:00:00 2001 From: Andrea Longo Date: Fri, 8 Sep 2023 12:45:29 -0600 Subject: [PATCH] SFTP access for Operator Tenants (#993) Document SFTP access for Operator Tenants, from Operator v5.0.7. https://github.com/minio/operator/pull/1685 https://github.com/minio/operator/pull/1692 The new Operator SFTP text and the not-Operator version it is based on were converted to includes rather than have a mess of conditionals in a single file. Staged http://192.241.195.202:9000/staging/DOCS-943-2/k8s/developers/file-transfer-protocol.html http://192.241.195.202:9000/staging/DOCS-943-2/linux/developers/file-transfer-protocol.html Partially addresses https://github.com/minio/docs/issues/943 --- source/developers/file-transfer-protocol.rst | 216 +---------------- .../k8s/file-transfer-protocol-k8s.rst | 167 +++++++++++++ .../linux/file-transfer-protocol-not-k8s.rst | 225 ++++++++++++++++++ source/index.rst | 2 +- 4 files changed, 397 insertions(+), 213 deletions(-) create mode 100644 source/includes/k8s/file-transfer-protocol-k8s.rst create mode 100644 source/includes/linux/file-transfer-protocol-not-k8s.rst diff --git a/source/developers/file-transfer-protocol.rst b/source/developers/file-transfer-protocol.rst index 3e4487e4..3148cc9d 100644 --- a/source/developers/file-transfer-protocol.rst +++ b/source/developers/file-transfer-protocol.rst @@ -14,219 +14,11 @@ File Transfer Protocol (FTP/SFTP) :local: :depth: 1 -.. versionadded:: MinIO RELEASE.2023-04-20T17-56-55Z +.. cond:: not k8s -Overview --------- + .. include:: /includes/linux/file-transfer-protocol-not-k8s.rst -You can use the File Transfer Protocol (``FTP``) to interact with the objects on a MinIO deployment. +.. cond:: k8s and not (openshift or eks or gke or aks) -You must specifically enable FTP or SFTP when starting the server. -Enabling either server type does not affect other MinIO features. + .. include:: /includes/k8s/file-transfer-protocol-k8s.rst -This page uses the abbreviation ``FTP`` throughout, but you can use any of the supported FTP protocols described below. - -Supported Protocols -~~~~~~~~~~~~~~~~~~~ - -When enabled, MinIO supports FTP access over the following protocols: - -- SSH File Transfer Protocol (``SFTP``) - - SFTP is defined by the Internet Engineering Task Force (IETF) as an extension of SSH 2.0. - SFTP allows file transfer over SSH for use with :ref:`Transport Layer Security (TLS) ` and virtual private network (VPN) applications. - - Your FTP client must support SFTP. - -- File Transfer Protocol over SSL/TLS (``FTPS``) - - ``FTPS`` allows for encrypted FTP communication with TLS certificates over the standard FTP communication channel. - ``FTPS`` should not be confused with ``SFTP``, as ``FTPS`` does not communicate over a Secure Shell (``SSH``). - - Your FTP client must support FTPS. - -- File Transfer Protocol (``FTP``) - - Unencrypted file transfer. - - MinIO does **not** recommend using unencrypted FTP for file transfer. - -Supported Commands -~~~~~~~~~~~~~~~~~~ - -When enabled, MinIO supports the following ``ftp`` operations: - -- ``get`` -- ``put`` -- ``ls`` -- ``mkdir`` -- ``rmdir`` -- ``delete`` - -MinIO does not support either ``append`` or ``rename`` operations. - -Considerations --------------- - -Versioning -~~~~~~~~~~ - -You cannot use ``FTP`` to read specific :ref:`object versions ` other than the latest version. - -- For read operations, MinIO only returns the latest version of the requested object(s) to the ftp client. -- For write operations, MinIO applies normal versioning behavior for matching object names. - -Use an S3 API Client, such as the :ref:`MinIO Client `. - -Authentication and Access -~~~~~~~~~~~~~~~~~~~~~~~~~ - -``FTP`` access requires the same authentication as any other S3 client. -MinIO supports the following authentication providers: - -- :ref:`MinIO IDP ` users and their service accounts -- :ref:`Active Directory/LDAP ` users and their service accounts -- :ref:`OpenID/OIDC ` service accounts - -:ref:`STS ` credentials **cannot** access buckets or objects over FTP. -To use STS credentials to authenticate, you must use an S3 API client or port. - -Authenticated users can access buckets and objects based on the :ref:`policies ` assigned to the user or parent user account. - -The FTP protocol does not require any of the ``admin:*`` `permissions `. -The FTP protocols do not support any of the MinIO admin actions. - -Prerequisites -------------- - -- MinIO RELEASE.2023-04-20T17-56-55Z or later. -- Enable an FTP or SFTP port for the server. -- A port to use for the FTP commands and a range of ports to allow the FTP server to request to use for the data transfer. - -Procedure ---------- - -1. Start MinIO with an FTP and/or SFTP port enabled. - - .. code-block:: shell - :class: copyable - - minio server http://server{1...4}/disk{1...4} \ - --ftp="address=:8021" \ - --ftp="passive-port-range=30000-40000" \ - --sftp="address=:8022" \ - --sftp="ssh-private-key=/home/miniouser/.ssh/id_rsa" \ - ... - - See the :mc-cmd:`minio server --ftp` and :mc-cmd:`minio server --sftp` for details on using these flags to start the MinIO service. - To connect to the an ftp port with TLS (``FTPS``), pass the ``tls-private-key`` and ``tls-public-cert`` keys and values, as well, unless using the MinIO default TLS keys. - - The output of the command should return a response that resembles the following: - - .. code-block:: shell - - MinIO FTP Server listening on :8021 - MinIO SFTP Server listening on :8022 - -2. Use your preferred ftp client to connect to the MinIO deployment. - You must connect as a user whose :ref:`policies ` allow access to the desired buckets and objects. - - The specifics of connecting to the MinIO deployment depend on your FTP client. - Refer to the documentation for your client. - - To connect over TLS or through SSH, you must use a client that supports the desired protocol. - -Examples --------- - -The examples here use the ``ftp`` CLI client on a Linux system. - -Connect to MinIO Using FTP -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The following example connects to a server using ``minio`` credentials to list contents in a bucket named ``runner`` - -.. code-block:: shell - - > ftp localhost -P 8021 - Connected to localhost. - 220 Welcome to MinIO FTP Server - Name (localhost:user): minio - 331 User name ok, password required - Password: - 230 Password ok, continue - Remote system type is UNIX. - Using binary mode to transfer files. - ftp> ls runner/ - 229 Entering Extended Passive Mode (|||39155|) - 150 Opening ASCII mode data connection for file list - drwxrwxrwx 1 nobody nobody 0 Jan 1 00:00 chunkdocs/ - drwxrwxrwx 1 nobody nobody 0 Jan 1 00:00 testdir/ - ... - -Start MinIO with FTP over TLS (``FTPS``) Enabled -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The following example starts MinIO with ``FTPS`` enabled. - -.. code-block:: shell - :class: copyable - - minio server http://server{1...4}/disk{1...4} \ - --ftp="address=:8021" \ - --ftp="passive-port-range=30000-40000" \ - --ftp="tls-private-key=path/to/private.key" \ - --ftp="tls-public-cert=path/to/public.crt" \ - ... - -.. note:: - - Omit ``tls-private-key`` and ``tls-public-cert`` to use the MinIO default TLS keys for ``FTPS``. - For more information, see the :ref:`TLS on MinIO documentation `. - -Download an Object over FTP -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -This example lists items in a bucket, then downloads the contents of the bucket. - -.. code-block:: console - - > ftp localhost -P 8021 - Connected to localhost. - 220 Welcome to MinIO FTP Server - Name (localhost:user): minio - 331 User name ok, password required - Password: - 230 Password ok, continue - Remote system type is UNIX. - Using binary mode to transfer files.ftp> ls runner/chunkdocs/metadata - 229 Entering Extended Passive Mode (|||44269|) - 150 Opening ASCII mode data connection for file list - -rwxrwxrwx 1 nobody nobody 45 Apr 1 06:13 chunkdocs/metadata - 226 Closing data connection, sent 75 bytes - ftp> get - (remote-file) runner/chunkdocs/metadata - (local-file) test - local: test remote: runner/chunkdocs/metadata - 229 Entering Extended Passive Mode (|||37785|) - 150 Data transfer starting 45 bytes - 45 3.58 KiB/s - 226 Closing data connection, sent 45 bytes - 45 bytes received in 00:00 (3.55 KiB/s) - ... - -Connect to MinIO Using SFTP -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The following example connects to an SSH FTP server, lists the contents of a bucket named ``runner``, and downloads an object. - -.. code-block:: console - - > sftp -P 8022 minio@localhost - minio@localhost's password: - Connected to localhost. - sftp> ls runner/ - chunkdocs testdir - sftp> get runner/chunkdocs/metadata metadata - Fetching /runner/chunkdocs/metadata to metadata - metadata 100% 226 16.6KB/s 00:00 diff --git a/source/includes/k8s/file-transfer-protocol-k8s.rst b/source/includes/k8s/file-transfer-protocol-k8s.rst new file mode 100644 index 00000000..7840ee63 --- /dev/null +++ b/source/includes/k8s/file-transfer-protocol-k8s.rst @@ -0,0 +1,167 @@ +.. versionadded:: Operator v5.0.7 + +Overview +-------- + +Starting with Operator 5.0.7 and :minio-release:`MinIO Server RELEASE.2023-04-20T17-56-55Z `, you can use the SSH File Transfer Protocol (SFTP) to interact with the objects on a MinIO Operator Tenant deployment. + +SFTP is defined by the Internet Engineering Task Force (IETF) as an extension of SSH 2.0. +It allows file transfer over SSH for use with :ref:`Transport Layer Security (TLS) ` and virtual private network (VPN) applications. + +Enabling SFTP does not affect other MinIO features. + + +Supported Commands +~~~~~~~~~~~~~~~~~~ + +When enabled, MinIO supports the following SFTP operations: + +- ``get`` +- ``put`` +- ``ls`` +- ``mkdir`` +- ``rmdir`` +- ``delete`` + +MinIO does not support either ``append`` or ``rename`` operations. + +MinIO Operator only supports the SFTP file transfer protocol. +Other protocols, such as FTP, are not supported for accessing Tenants. + + +Considerations +-------------- + + +Versioning +~~~~~~~~~~ + +SFTP clients can only operate on the :ref:`latest version ` of an object. +Specifically: + +- For read operations, MinIO only returns the latest version of the requested object(s) to the SFTP client. +- For write operations, MinIO applies normal versioning behavior and creates a new object version at the specified namespace. + ``rm`` and ``rmdir`` operations create ``DeleteMarker`` objects. + + +Authentication and Access +~~~~~~~~~~~~~~~~~~~~~~~~~ + +SFTP access requires the same authentication as any other S3 client. +MinIO supports the following authentication providers: + +- :ref:`MinIO IDP ` users and their service accounts +- :ref:`Active Directory/LDAP ` users and their service accounts +- :ref:`OpenID/OIDC ` service accounts + +:ref:`STS ` credentials **cannot** access buckets or objects over SFTP. + +Authenticated users can access buckets and objects based on the :ref:`policies ` assigned to the user or parent user account. + +The SFTP protocol does not require any of the ``admin:*`` :ref:`permissions `. +You may not perform other MinIO admin actions with SFTP. + + +Prerequisites +------------- + +- MinIO Operator v5.0.7 or later. +- Enable an SFTP port (8022) for the server. +- A port to use for the SFTP commands and a range of ports to allow the SFTP server to request to use for the data transfer. + + +Procedure +--------- + +#. Enable SFTP for the desired Tenant: + + .. tab-set:: + + .. tab-item:: Operator Console + + - In the Operator Console, click on the Tenant for which to enable SFTP. + - In the :guilabel:`Configuration` tab, toggle :guilabel:`SFTP` to :guilabel:`Enabled`. + - Click :guilabel:`Save`. + - Click :guilabel:`Restart` to restart MinIO and apply your changes. + + .. tab-item:: Kubectl + + Use the following Kubectl command to edit the Tenant YAML configuration: + + .. code-block:: yaml + + kubectl edit tenants/my-tenant -n my-tenant-ns + + Replace ``my-tenant`` and ``my-tenant-ns`` with the desired Tenant and namespace. + + In the ``features:`` section, set the value of ``enableSFTP`` to ``true``: + + .. code-block:: yaml + + spec: + configuration: + name: my-tenant-env-configuration + credsSecret: + name: my-tenant-secret + exposeServices: + console: true + minio: true + features: + enableSFTP: true + + Kubectl restarts MinIO to apply the change. + + You may also set ``enableSFTP`` in your `Helm chart `__ or `Kustomize configuration `__ to enable SFTP for newly created Tenants. + + +#. If needed, configure ingress for the SFTP port according to your local policies. + +#. Use your preferred SFTP client to connect to the MinIO deployment. + You must connect as a user whose :ref:`policies ` allow access to the desired buckets and objects. + + The specifics of connecting to the MinIO deployment depend on your SFTP client. + Refer to the documentation for your client. + + +Examples +-------- + +The following examples use the `SFTP CLI client `__ on a Linux system. + +Connect to MinIO Using SFTP +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following example connects to an SFTP server, lists the contents of a bucket named ``test-bucket``, and downloads an object. + +.. code-block:: console + + sftp -P 8022 my-access-key@localhost + my-access-key@localhost's password: + Connected to localhost. + sftp> ls + test-bucket + sftp> ls test-bucket + test-bucket/test-file.txt + sftp> get test-bucket/test-file.txt + Fetching /test-bucket/test-file.txt to test-file.txt + test-file.txt 100% 6 1.3KB/s 00:00 + + +Check if SFTP is Enabled for a Tenant +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following ``kubectl get`` command uses `yq `__ to display the value of ``enableSFTP``, indicating whether SFTP is enabled: + +.. code-block:: console + :class: copyable + + kubectl get tenants/my-tenant -n my-tenant-ns -o yaml | yq '.spec.features' + +Replace ``my-tenant`` and ``my-tenant-ns`` with the desired Tenant and namespace. + +If SFTP is enabled, the output resembles the following: + +.. code-block:: console + + enableSFTP: true + diff --git a/source/includes/linux/file-transfer-protocol-not-k8s.rst b/source/includes/linux/file-transfer-protocol-not-k8s.rst new file mode 100644 index 00000000..7e39abcf --- /dev/null +++ b/source/includes/linux/file-transfer-protocol-not-k8s.rst @@ -0,0 +1,225 @@ +.. versionadded:: MinIO RELEASE.2023-04-20T17-56-55Z + +Overview +-------- + +Starting with :minio-release:`MinIO Server RELEASE.2023-04-20T17-56-55Z `, you can use the File Transfer Protocol (FTP) to interact with the objects on a MinIO deployment. + +You must specifically enable FTP or SFTP when starting the server. +Enabling either server type does not affect other MinIO features. + +This page uses the abbreviation FTP throughout, but you can use any of the supported FTP protocols described below. + +Supported Protocols +~~~~~~~~~~~~~~~~~~~ + +When enabled, MinIO supports FTP access over the following protocols: + +- SSH File Transfer Protocol (SFTP) + + SFTP is defined by the Internet Engineering Task Force (IETF) as an extension of SSH 2.0. + SFTP allows file transfer over SSH for use with :ref:`Transport Layer Security (TLS) ` and virtual private network (VPN) applications. + + Your FTP client must support SFTP. + +- File Transfer Protocol over SSL/TLS (FTPS) + + FTPS allows for encrypted FTP communication with TLS certificates over the standard FTP communication channel. + FTPS should not be confused with SFTP, as FTPS does not communicate over a Secure Shell (SSH). + + Your FTP client must support FTPS. + +- File Transfer Protocol (FTP) + + Unencrypted file transfer. + + MinIO does **not** recommend using unencrypted FTP for file transfer. + +.. admonition:: MinIO Operator Tenants only support SFTP + :class: note + + MinIO Tenants deployed with Operator only support SFTP. + For details, see `File Transfer Protocol for Tenants `__. + + +Supported Commands +~~~~~~~~~~~~~~~~~~ + +When enabled, MinIO supports the following FTP operations: + +- ``get`` +- ``put`` +- ``ls`` +- ``mkdir`` +- ``rmdir`` +- ``delete`` + +MinIO does not support either ``append`` or ``rename`` operations. + +Considerations +-------------- + +Versioning +~~~~~~~~~~ + +SFTP clients can only operate on the :ref:`latest version ` of an object. +Specifically: + +- For read operations, MinIO only returns the latest version of the requested object(s) to the FTP client. +- For write operations, MinIO applies normal versioning behavior and creates a new object version at the specified namespace. + ``rm`` and ``rmdir`` operations create ``DeleteMarker`` objects. + + +Authentication and Access +~~~~~~~~~~~~~~~~~~~~~~~~~ + +FTP access requires the same authentication as any other S3 client. +MinIO supports the following authentication providers: + +- :ref:`MinIO IDP ` users and their service accounts +- :ref:`Active Directory/LDAP ` users and their service accounts +- :ref:`OpenID/OIDC ` service accounts + +:ref:`STS ` credentials **cannot** access buckets or objects over FTP. + +Authenticated users can access buckets and objects based on the :ref:`policies ` assigned to the user or parent user account. + +The FTP protocol does not require any of the ``admin:*`` :ref:`permissions `. +The FTP protocols do not support any of the MinIO admin actions. + +Prerequisites +------------- + +- MinIO RELEASE.2023-04-20T17-56-55Z or later. +- Enable an FTP or SFTP port for the server. +- A port to use for the FTP commands and a range of ports to allow the FTP server to request to use for the data transfer. + +Procedure +--------- + +1. Start MinIO with an FTP and/or SFTP port enabled. + + .. code-block:: shell + :class: copyable + + minio server http://server{1...4}/disk{1...4} \ + --ftp="address=:8021" \ + --ftp="passive-port-range=30000-40000" \ + --sftp="address=:8022" \ + --sftp="ssh-private-key=/home/miniouser/.ssh/id_rsa" \ + ... + + See the :mc-cmd:`minio server --ftp` and :mc-cmd:`minio server --sftp` for details on using these flags to start the MinIO service. + To connect to the an FTP port with TLS (FTPS), pass the ``tls-private-key`` and ``tls-public-cert`` keys and values, as well, unless using the MinIO default TLS keys. + + The output of the command should return a response that resembles the following: + + .. code-block:: shell + + MinIO FTP Server listening on :8021 + MinIO SFTP Server listening on :8022 + +2. Use your preferred FTP client to connect to the MinIO deployment. + You must connect as a user whose :ref:`policies ` allow access to the desired buckets and objects. + + The specifics of connecting to the MinIO deployment depend on your FTP client. + Refer to the documentation for your client. + + To connect over TLS or through SSH, you must use a client that supports the desired protocol. + +Examples +-------- + +The following examples use the `FTP CLI client `__ on a Linux system. + + +Connect to MinIO Using FTP +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following example connects to a server using ``minio`` credentials to list contents in a bucket named ``runner`` + +.. code-block:: shell + + > ftp localhost -P 8021 + Connected to localhost. + 220 Welcome to MinIO FTP Server + Name (localhost:user): minio + 331 User name ok, password required + Password: + 230 Password ok, continue + Remote system type is UNIX. + Using binary mode to transfer files. + ftp> ls runner/ + 229 Entering Extended Passive Mode (|||39155|) + 150 Opening ASCII mode data connection for file list + drwxrwxrwx 1 nobody nobody 0 Jan 1 00:00 chunkdocs/ + drwxrwxrwx 1 nobody nobody 0 Jan 1 00:00 testdir/ + ... + +Start MinIO with FTP over TLS (FTPS) Enabled +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following example starts MinIO with FTPS enabled. + +.. code-block:: shell + :class: copyable + + minio server http://server{1...4}/disk{1...4} \ + --ftp="address=:8021" \ + --ftp="passive-port-range=30000-40000" \ + --ftp="tls-private-key=path/to/private.key" \ + --ftp="tls-public-cert=path/to/public.crt" \ + ... + +.. note:: + + Omit ``tls-private-key`` and ``tls-public-cert`` to use the MinIO default TLS keys for FTPS. + For more information, see the :ref:`TLS on MinIO documentation `. + +Download an Object over FTP +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This example lists items in a bucket, then downloads the contents of the bucket. + +.. code-block:: console + + > ftp localhost -P 8021 + Connected to localhost. + 220 Welcome to MinIO FTP Server + Name (localhost:user): minio + 331 User name ok, password required + Password: + 230 Password ok, continue + Remote system type is UNIX. + Using binary mode to transfer files.ftp> ls runner/chunkdocs/metadata + 229 Entering Extended Passive Mode (|||44269|) + 150 Opening ASCII mode data connection for file list + -rwxrwxrwx 1 nobody nobody 45 Apr 1 06:13 chunkdocs/metadata + 226 Closing data connection, sent 75 bytes + ftp> get + (remote-file) runner/chunkdocs/metadata + (local-file) test + local: test remote: runner/chunkdocs/metadata + 229 Entering Extended Passive Mode (|||37785|) + 150 Data transfer starting 45 bytes + 45 3.58 KiB/s + 226 Closing data connection, sent 45 bytes + 45 bytes received in 00:00 (3.55 KiB/s) + ... + +Connect to MinIO Using SFTP +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following example connects to an SFTP server, lists the contents of a bucket named ``runner``, and downloads an object. + +.. code-block:: console + + > sftp -P 8022 minio@localhost + minio@localhost's password: + Connected to localhost. + sftp> ls runner/ + chunkdocs testdir + sftp> get runner/chunkdocs/metadata metadata + Fetching /runner/chunkdocs/metadata to metadata + metadata 100% 226 16.6KB/s 00:00 + diff --git a/source/index.rst b/source/index.rst index e6da9b4a..ef54f66d 100644 --- a/source/index.rst +++ b/source/index.rst @@ -204,7 +204,7 @@ Any file uploaded to ``play`` should be considered public and non-protected. Software Development Kits (SDK) /developers/sts-for-operator Object Lambda - File Transfer Protocol + /developers/file-transfer-protocol MinIO Client MinIO Admin Client Integrations