diff --git a/source/administration/object-management.rst b/source/administration/object-management.rst index e0b050b8..692e403b 100644 --- a/source/administration/object-management.rst +++ b/source/administration/object-management.rst @@ -10,22 +10,16 @@ Object Management .. _objects: -An :ref:`object ` is binary data, sometimes referred to as a Binary -Large OBject (BLOB). Blobs can be images, audio files, spreadsheets, or even -binary executable code. Object Storage platforms like MinIO provide dedicated -tools and capabilities for storing, retrieving, and searching for blobs. +An :ref:`object ` is binary data, such as images, audio files, spreadsheets, or even binary executable code. +The term "Binary Large Object" or "blob" is sometimes associated to object storage, although blobs can be anywhere from a few bytes to several terabytes in size. +Object Storage platforms like MinIO provide dedicated tools and capabilities for storing, listing, and retrieving objects using a standard S3-compatible API. .. _buckets: MinIO Object Storage uses :ref:`buckets ` to organize objects. -A bucket is similar to a folder or directory in a filesystem, where each -bucket can hold an arbitrary number of objects. MinIO buckets provide the -same functionality as AWS S3 buckets. +A bucket is similar to a top-level drive, folder, or directory in a filesystem (``/mnt/data`` or ``C:\``), where each bucket can hold an arbitrary number of objects. -For example, consider an application that hosts a web blog. The application -needs to store a variety of blobs, including rich multimedia like videos and -images. The structure of objects on the MinIO server might look similar to the -following: +The structure of objects on the MinIO server might look similar to the following: .. code-block:: text @@ -45,10 +39,27 @@ following: 2020-01-02-MinIO-Advanced-Deployment-comments.json 2020-01-04-MinIO-Interview.md -MinIO supports multiple levels of nested directories and objects using a :term:`prefix` structure to support even the most dynamic object storage workloads. +With the example structure, an administrator would create the ``/images``, ``/videos`` and ``/articles`` buckets. +Client applications write objects to those buckets using the full "path" to that object, including all intermediate :term:`prefixes `. -MinIO itself does not limit the number of objects that any specific prefix can contain. -However, hardware and network conditions may show performance impacts with large prefixes. +MinIO supports multiple levels of nested directories and objects using prefixes to support even the most dynamic object storage workloads. +MinIO automatically infers the intermediate prefixes, such as ``/articles/john.doe`` from the full object path using ``/`` as a delimiter. +Clients and administrators should not create these prefixes manually. + +Neither clients nor administrators would manually create the intermediate prefixes, as MinIO automatically infers them from the object name. + +Object Organization and Planning +-------------------------------- + +Administrators typically control the creation and configuration of buckets. +Client applications can then use :ref:`S3-compatible SDKs ` to create, list, retrieve, and delete objects on the MinIO deployment. +Clients therefore drive the overall hierarchy of data within a given bucket or prefix, where Administrators can exercise control using :ref:`policies ` to grant or deny access to an action or resource. + +MinIO has no hard :ref:`thresholds ` on the number of buckets, objects, or prefixes on a given deployment. +The relative performance of the hardware and networking underlying the MinIO deployment may create a practical limit to the number of objects in a given prefix or bucket. +Specifically, hardware using slower drives or network infrastructures tend to exhibit poor performance in buckets or prefixes with a flat hierarchy of objects. + +Consider the following points as general guidance for client applications workload patterns: - Deployments with modest or budget-focused hardware should architect their workloads to target 10,000 objects per prefix as a baseline. Increase this target based on benchmarking and monitoring of real world workloads up to what the hardware can meaningfully handle. @@ -59,16 +70,48 @@ However, hardware and network conditions may show performance impacts with large For a deeper discussion on the benefits of limiting prefix contents, see the article on :s3-docs:`optimizing S3 performance `. Object Versioning ------------------ +----------------- MinIO supports keeping multiple "versions" of an object in a single bucket. -Write operations which would normally overwrite an existing object instead result in the creation of a new versioned object. .. image:: /images/retention/minio-versioning-multiple-versions.svg :alt: Object with Multiple Versions :align: center -MinIO versioning protects from unintended overwrites and deletions while providing support for "undoing" a write operation. +The specific client behavior on write, list, get, or delete operations on a bucket depends on the versioning state of that bucket: + +.. list-table:: + :stub-columns: 1 + :header-rows: 1 + :widths: 20 40 40 + :width: 100% + + * - Operation + - Versioning Enabled + - Versioning Disabled | Suspended + + * - ``PUT`` (Write) + - Create a new full version of the object as the "latest" and assign a unique version ID + - Create the object with overwrite on namespace match. + + * - ``GET`` (Read) + - Retrieve the latest version of the object by default + + Supports retrieving retrieving any object version by version ID. + - Retrieve the object + + * - ``LIST`` (Read) + - Retrieve the latest version of objects at the specified bucket or prefix + + Supports retrieving all objects with their associated version ID. + - Retrieve all objects at the specified bucket or prefix + + * - ``DELETE`` (Write) + - Creates a 0-byte "Delete Marker" for the object as "latest" (soft delete) + + Supports deleting any object version by version ID (hard delete). + You cannot undo hard-delete operations. + - Deletes the object See :ref:`minio-bucket-versioning` for more complete documentation. @@ -83,23 +126,31 @@ MinIO supports both :ref:`duration based object retention `. MinIO Object Locking provides key data retention compliance and meets SEC17a-4(f), FINRA 4511(C), and CFTC 1.31(c)-(d) requirements as per `Cohasset Associates `__. +See :ref:`minio-object-locking` for more complete documentation. + Object Lifecycle Management --------------------------- -MinIO Object Lifecycle Management allows creating rules for time or date -based automatic transition or expiry of objects. For object transition, -MinIO automatically moves the object to a configured remote storage -tier. For object expiry, MinIO automatically deletes the object. +MinIO Object Lifecycle Management allows creating rules for time or date based automatic transition or expiry of objects. +For object transition, MinIO automatically moves the object to a configured remote storage tier. +For object expiry, MinIO automatically deletes the object. -MinIO lifecycle management is built for behavior and syntax compatibility with -:s3-docs:`AWS S3 Lifecycle Management `. For -example, you can export S3 lifecycle management rules and import them into -MinIO or vice-versa. MinIO uses JSON to describe lifecycle management rules, -and conversion to or from XML may be required. +MinIO applies lifecycle management rules on :ref:`versioned and unversioned buckets ` using the same behavior as normal client operations. +You can specify transition or lifecycle rules that handle the latest object versions, non-current object versions, or both. + +MinIO lifecycle management is built for behavior and syntax compatibility with :s3-docs:`AWS S3 Lifecycle Management `. +MinIO uses JSON to describe lifecycle management rules. +Conversion to or from XML may be required for importing rules created on S3 or similar compatible platforms. See :ref:`minio-lifecycle-management` for more complete documentation.