From 330ded5129a8289258700576448686d270eb03b4 Mon Sep 17 00:00:00 2001 From: rocksdb-release-bot Date: Thu, 20 Aug 2026 15:13:31 -0700 Subject: [PATCH] Update version to 11.10.0 for 11.9 release (#15135) Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15135 - Bump version.h from 11.9.0 to 11.10.0 - Add `11.9.fb` to `check_format_compatible.sh` - Sync HISTORY.md `## 11.9.` section from `11.9.fb` - Delete 17 consumed `unreleased_history/` note file(s) from main Part of 11.9 release workflow. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15096 Reviewed By: anand1976 Differential Revision: D116833037 Pulled By: joshkang97 fbshipit-source-id: 04ebda8db5b2c4fdae71a2cf107784b6abdc0f3c --- HISTORY.md | 26 +++++++++++++++++++ include/rocksdb/version.h | 2 +- tools/check_format_compatible.sh | 2 +- .../blob_compaction_wrong_output_path.md | 1 - .../new_features/auto_skip_compression.md | 1 - ...blob_file_writable_file_max_buffer_size.md | 1 - .../new_features/lazy_wide_columns.md | 1 - ...num_running_remote_compactions_property.md | 1 - .../num_unscheduled_compactions_property.md | 1 - .../new_features/per_cf_compaction_abort.md | 1 - ...solete-options-files-background-on-open.md | 1 - .../index_factory_aliases.md | 1 - .../native_coroutine_reads.md | 1 - .../pinnable_multi_get_batch.md | 1 - .../read_io_executor_threads.md | 1 - .../read_options_table_filter_pointer.md | 1 - .../remove_auto_skip_compression_manager.md | 1 - .../stackable_coroutine_reads.md | 1 - ...le_file_creation_reason_sst_file_writer.md | 1 - .../public_api_changes/wal_iterator_rename.md | 1 - 20 files changed, 28 insertions(+), 19 deletions(-) delete mode 100644 unreleased_history/bug_fixes/blob_compaction_wrong_output_path.md delete mode 100644 unreleased_history/new_features/auto_skip_compression.md delete mode 100644 unreleased_history/new_features/blob_file_writable_file_max_buffer_size.md delete mode 100644 unreleased_history/new_features/lazy_wide_columns.md delete mode 100644 unreleased_history/new_features/num_running_remote_compactions_property.md delete mode 100644 unreleased_history/new_features/num_unscheduled_compactions_property.md delete mode 100644 unreleased_history/new_features/per_cf_compaction_abort.md delete mode 100644 unreleased_history/performance_improvements/delete-obsolete-options-files-background-on-open.md delete mode 100644 unreleased_history/public_api_changes/index_factory_aliases.md delete mode 100644 unreleased_history/public_api_changes/native_coroutine_reads.md delete mode 100644 unreleased_history/public_api_changes/pinnable_multi_get_batch.md delete mode 100644 unreleased_history/public_api_changes/read_io_executor_threads.md delete mode 100644 unreleased_history/public_api_changes/read_options_table_filter_pointer.md delete mode 100644 unreleased_history/public_api_changes/remove_auto_skip_compression_manager.md delete mode 100644 unreleased_history/public_api_changes/stackable_coroutine_reads.md delete mode 100644 unreleased_history/public_api_changes/table_file_creation_reason_sst_file_writer.md delete mode 100644 unreleased_history/public_api_changes/wal_iterator_rename.md diff --git a/HISTORY.md b/HISTORY.md index d67cc7b26e..32e7b85e30 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -1,6 +1,32 @@ # Rocksdb Change Log > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt` +## 11.9.0 (08/14/2026) +### New Features +* Added an integrated and improved compression auto-skip feature (new CompressionOptions, still experimental) that saves CPU by skipping compression when and where the overall ratio is inadequate. This improves on the previous auto-skip compression manager by (a) taking achieved compression ratios into account, (b) using a continuously evolving payoff estimate, and (c) combining natively with parallel compression. +* Added the `blob_file_writable_file_max_buffer_size` mutable column family option. When set to a non-zero value, blob-file writers use it as their `WritableFileWriter` max buffer size instead of inheriting `DBOptions::writable_file_max_buffer_size`. +* Added an experimental lazy wide-column read API, `DB::GetEntityLazy()` / `DB::MultiGetEntityLazy()` (see `include/rocksdb/lazy_wide_columns.h`). The returned `LazyWideColumns` leaves blob-backed columns as unresolved references and, like an iterator, stays valid after the call returns, so the caller resolves only the columns -- and only the byte ranges -- it needs; a byte-range read of an uncompressed blob (separate-file or embedded) reads just those bytes from storage. Adds `Env::IOActivity::kLazyResolve` and `rocksdb.blobdb.lazy.*` statistics. Requires `max_open_files == -1`. +* Added DB property `rocksdb.num-running-remote-compactions`, reporting the number of compaction service jobs currently waiting in `CompactionService::Wait()`. +* Added DB property `rocksdb.num-unscheduled-compactions`, reporting compactions waiting in the DB's internal compaction queue but not yet assigned to a background job. +* Added `DB::AbortCompactions()` and `DB::ResumeCompactions()` for per-column-family compaction abort and resume, with the same active-abort semantics as the DB-wide `AbortAllCompactions()`/`ResumeAllCompactions()` but scoped to a single column family. Calls are reference-counted independently per column family and compose with the DB-wide abort. + +### Public API Changes +* Add IndexFactory compatibility names for the existing user-defined index API. +* Added the experimental `CoroDB` capability, exposed by `DB::GetCoroDB()`, with lazy native coroutine variants of `Get` and `MultiGet`. The existing `DB::GetAsync()` and `DB::MultiGetAsync()` APIs dispatch through it. +* Added `rocksdb_batched_multi_get_pinned_cf()` to return batch-owned pinned MultiGet values and per-key errors without allocating one wrapper per found value. +* Add `DBOptions::read_io_executor_threads` to increase the maximum thread count of the shared filesystem read I/O executor used for asynchronous reads when opening a DB. Opening a DB never reduces the shared executor thread count. Change the experimental `FSRandomAccessFile::SubmitReadAsync()` API to return whether the asynchronous path was used, and add the `FILE_SUBMIT_ASYNC_READ_FALLBACK` ticker to count calls that fall back to synchronous reads. +* `ReadOptions::table_filter` is now a *pointer* to a caller-owned `std::function` instead of a `std::function` value, to improve efficiency in copying and destroying `ReadOptions` objects (which RocksDB does internally). This is a breaking change for users of this relatively obscure option, who will now need to declare a `std::function` that outlives the read. As a side effect of `ReadOptions` being trivially copyable, new but legitimate compiler warnings may surface. +* Removed the experimental `CreateAutoSkipCompressionManager()`. Auto-skip compression is now a first-class feature of the block-based table builder, controlled by the new experimental `CompressionOptions::auto_skip`. +* Added `CoroStackableDB` for implementing wrappers over native coroutine reads and used it to add coroutine `Get` and `MultiGet` support to `DBWithTTL` and WritePrepared `TransactionDB`. A stackable wrapper advertises coroutine support only when its wrapped DB supports it; TTL value decoding and transaction visibility semantics are preserved. +* Added a `kSstFileWriter` value to `TableFileCreationReason`, which `SstFileWriter` now reports instead of the catch-all `kMisc`. This, along with new `db_name` and `is_remote_compaction` fields in `FilterBuildingContext`, improves the contextual information available to custom filter and compression strategies. +* Renamed `TransactionLogIterator` to `WalIterator` (and `include/rocksdb/transaction_log.h` to `include/rocksdb/wal_iterator.h`), because it iterates the write-ahead log and is unrelated to `TransactionDB`. The old names still work. `DB::GetUpdatesSince()` is unchanged. In Java, `WalIterator` is the new name and `TransactionLogIterator` remains as a subclass. Also documented existing behavior that was previously unstated: only writes that reached the WAL are visible (so `WriteOptions::disableWAL` and `IngestExternalFile()` leave holes that end iteration), what each iterator state means, and that `GetUpdatesSince()` silently starts later when the requested sequence number is unavailable. + +### Bug Fixes +* Fixed a bug where `BlobFileBuilder` passed an SST path to `OnBlobFileCompleted()` instead of the completed blob path during BlobDB garbage-collection compactions, causing incorrect `SstFileManager` accounting and compaction failures on filesystems where the SST was not yet pathname-visible. + +### Performance Improvements +* When `DBOptions::avoid_unnecessary_blocking_io` is true, obsolete `OPTIONS-*` files found during DB open are deleted by background purge instead of synchronously on the opening thread. + ## 11.8.0 (07/28/2026) ### Public API Changes * Add callback-based asynchronous read APIs, `DB::GetAsync()` and `DB::MultiGetAsync()`. The idea is that when IO is required, RocksDB can suspend its internal coroutine read path, allowing the read executor thread to do other work. When the IO is complete, RocksDB invokes the user callback. This requires filesystem support for full performance benefits. A new filesystem API `FSRandomAccessFile::SubmitReadAsync()` is introduced for this. Unlike `ReadAsync`, the filesystem is responsible for eventually calling the callback. `FileSystem::GetReadExecutor()` returns the IO executor whose EventBases run coroutine read processing. In the Posix filesystem, an IO-uring based event loop is used to complete the IO. diff --git a/include/rocksdb/version.h b/include/rocksdb/version.h index 0fc4c3ef34..eb0b0450c7 100644 --- a/include/rocksdb/version.h +++ b/include/rocksdb/version.h @@ -12,7 +12,7 @@ // NOTE: in 'main' development branch, this should be the *next* // minor or major version number planned for release. #define ROCKSDB_MAJOR 11 -#define ROCKSDB_MINOR 9 +#define ROCKSDB_MINOR 10 #define ROCKSDB_PATCH 0 // Make it easy to do conditional compilation based on version checks, i.e. diff --git a/tools/check_format_compatible.sh b/tools/check_format_compatible.sh index 3df6d5cf9f..ab3f1ab522 100755 --- a/tools/check_format_compatible.sh +++ b/tools/check_format_compatible.sh @@ -161,7 +161,7 @@ EOF # To check for DB forward compatibility with loading options (old version # reading data from new), as well as backward compatibility -declare -a db_forward_with_options_refs=("10.4.fb" "10.5.fb" "10.6.fb" "10.7.fb" "10.8.fb" "10.9.fb" "10.10.fb" "10.11.fb" "11.0.fb" "11.1.fb" "11.2.fb" "11.3.fb" "11.4.fb" "11.5.fb" "11.6.fb" "11.7.fb" "11.8.fb") +declare -a db_forward_with_options_refs=("10.4.fb" "10.5.fb" "10.6.fb" "10.7.fb" "10.8.fb" "10.9.fb" "10.10.fb" "10.11.fb" "11.0.fb" "11.1.fb" "11.2.fb" "11.3.fb" "11.4.fb" "11.5.fb" "11.6.fb" "11.7.fb" "11.8.fb" "11.9.fb") # To check for DB forward compatibility without loading options (in addition # to the "with loading options" set), as well as backward compatibility declare -a db_forward_no_options_refs=() # N/A at the moment diff --git a/unreleased_history/bug_fixes/blob_compaction_wrong_output_path.md b/unreleased_history/bug_fixes/blob_compaction_wrong_output_path.md deleted file mode 100644 index c401949fa6..0000000000 --- a/unreleased_history/bug_fixes/blob_compaction_wrong_output_path.md +++ /dev/null @@ -1 +0,0 @@ -Fixed a bug where `BlobFileBuilder` passed an SST path to `OnBlobFileCompleted()` instead of the completed blob path during BlobDB garbage-collection compactions, causing incorrect `SstFileManager` accounting and compaction failures on filesystems where the SST was not yet pathname-visible. diff --git a/unreleased_history/new_features/auto_skip_compression.md b/unreleased_history/new_features/auto_skip_compression.md deleted file mode 100644 index 723a063913..0000000000 --- a/unreleased_history/new_features/auto_skip_compression.md +++ /dev/null @@ -1 +0,0 @@ -Added an integrated and improved compression auto-skip feature (new CompressionOptions, still experimental) that saves CPU by skipping compression when and where the overall ratio is inadequate. This improves on the previous auto-skip compression manager by (a) taking achieved compression ratios into account, (b) using a continuously evolving payoff estimate, and (c) combining natively with parallel compression. diff --git a/unreleased_history/new_features/blob_file_writable_file_max_buffer_size.md b/unreleased_history/new_features/blob_file_writable_file_max_buffer_size.md deleted file mode 100644 index e30a97e47a..0000000000 --- a/unreleased_history/new_features/blob_file_writable_file_max_buffer_size.md +++ /dev/null @@ -1 +0,0 @@ -Added the `blob_file_writable_file_max_buffer_size` mutable column family option. When set to a non-zero value, blob-file writers use it as their `WritableFileWriter` max buffer size instead of inheriting `DBOptions::writable_file_max_buffer_size`. diff --git a/unreleased_history/new_features/lazy_wide_columns.md b/unreleased_history/new_features/lazy_wide_columns.md deleted file mode 100644 index 448dcebf43..0000000000 --- a/unreleased_history/new_features/lazy_wide_columns.md +++ /dev/null @@ -1 +0,0 @@ -Added an experimental lazy wide-column read API, `DB::GetEntityLazy()` / `DB::MultiGetEntityLazy()` (see `include/rocksdb/lazy_wide_columns.h`). The returned `LazyWideColumns` leaves blob-backed columns as unresolved references and, like an iterator, stays valid after the call returns, so the caller resolves only the columns -- and only the byte ranges -- it needs; a byte-range read of an uncompressed blob (separate-file or embedded) reads just those bytes from storage. Adds `Env::IOActivity::kLazyResolve` and `rocksdb.blobdb.lazy.*` statistics. Requires `max_open_files == -1`. diff --git a/unreleased_history/new_features/num_running_remote_compactions_property.md b/unreleased_history/new_features/num_running_remote_compactions_property.md deleted file mode 100644 index 19bab38ba5..0000000000 --- a/unreleased_history/new_features/num_running_remote_compactions_property.md +++ /dev/null @@ -1 +0,0 @@ -Added DB property `rocksdb.num-running-remote-compactions`, reporting the number of compaction service jobs currently waiting in `CompactionService::Wait()`. diff --git a/unreleased_history/new_features/num_unscheduled_compactions_property.md b/unreleased_history/new_features/num_unscheduled_compactions_property.md deleted file mode 100644 index 882e7de0f5..0000000000 --- a/unreleased_history/new_features/num_unscheduled_compactions_property.md +++ /dev/null @@ -1 +0,0 @@ -Added DB property `rocksdb.num-unscheduled-compactions`, reporting compactions waiting in the DB's internal compaction queue but not yet assigned to a background job. diff --git a/unreleased_history/new_features/per_cf_compaction_abort.md b/unreleased_history/new_features/per_cf_compaction_abort.md deleted file mode 100644 index f269eac7d8..0000000000 --- a/unreleased_history/new_features/per_cf_compaction_abort.md +++ /dev/null @@ -1 +0,0 @@ -Added `DB::AbortCompactions()` and `DB::ResumeCompactions()` for per-column-family compaction abort and resume, with the same active-abort semantics as the DB-wide `AbortAllCompactions()`/`ResumeAllCompactions()` but scoped to a single column family. Calls are reference-counted independently per column family and compose with the DB-wide abort. diff --git a/unreleased_history/performance_improvements/delete-obsolete-options-files-background-on-open.md b/unreleased_history/performance_improvements/delete-obsolete-options-files-background-on-open.md deleted file mode 100644 index 4b59afceb8..0000000000 --- a/unreleased_history/performance_improvements/delete-obsolete-options-files-background-on-open.md +++ /dev/null @@ -1 +0,0 @@ -When `DBOptions::avoid_unnecessary_blocking_io` is true, obsolete `OPTIONS-*` files found during DB open are deleted by background purge instead of synchronously on the opening thread. diff --git a/unreleased_history/public_api_changes/index_factory_aliases.md b/unreleased_history/public_api_changes/index_factory_aliases.md deleted file mode 100644 index a1a585f446..0000000000 --- a/unreleased_history/public_api_changes/index_factory_aliases.md +++ /dev/null @@ -1 +0,0 @@ -Add IndexFactory compatibility names for the existing user-defined index API. diff --git a/unreleased_history/public_api_changes/native_coroutine_reads.md b/unreleased_history/public_api_changes/native_coroutine_reads.md deleted file mode 100644 index 9b2fd09748..0000000000 --- a/unreleased_history/public_api_changes/native_coroutine_reads.md +++ /dev/null @@ -1 +0,0 @@ -Added the experimental `CoroDB` capability, exposed by `DB::GetCoroDB()`, with lazy native coroutine variants of `Get` and `MultiGet`. The existing `DB::GetAsync()` and `DB::MultiGetAsync()` APIs dispatch through it. diff --git a/unreleased_history/public_api_changes/pinnable_multi_get_batch.md b/unreleased_history/public_api_changes/pinnable_multi_get_batch.md deleted file mode 100644 index 57fe5053f9..0000000000 --- a/unreleased_history/public_api_changes/pinnable_multi_get_batch.md +++ /dev/null @@ -1 +0,0 @@ -Added `rocksdb_batched_multi_get_pinned_cf()` to return batch-owned pinned MultiGet values and per-key errors without allocating one wrapper per found value. diff --git a/unreleased_history/public_api_changes/read_io_executor_threads.md b/unreleased_history/public_api_changes/read_io_executor_threads.md deleted file mode 100644 index 64ac4cf10f..0000000000 --- a/unreleased_history/public_api_changes/read_io_executor_threads.md +++ /dev/null @@ -1 +0,0 @@ -Add `DBOptions::read_io_executor_threads` to increase the maximum thread count of the shared filesystem read I/O executor used for asynchronous reads when opening a DB. Opening a DB never reduces the shared executor thread count. Change the experimental `FSRandomAccessFile::SubmitReadAsync()` API to return whether the asynchronous path was used, and add the `FILE_SUBMIT_ASYNC_READ_FALLBACK` ticker to count calls that fall back to synchronous reads. diff --git a/unreleased_history/public_api_changes/read_options_table_filter_pointer.md b/unreleased_history/public_api_changes/read_options_table_filter_pointer.md deleted file mode 100644 index fbae87d62a..0000000000 --- a/unreleased_history/public_api_changes/read_options_table_filter_pointer.md +++ /dev/null @@ -1 +0,0 @@ -`ReadOptions::table_filter` is now a *pointer* to a caller-owned `std::function` instead of a `std::function` value, to improve efficiency in copying and destroying `ReadOptions` objects (which RocksDB does internally). This is a breaking change for users of this relatively obscure option, who will now need to declare a `std::function` that outlives the read. As a side effect of `ReadOptions` being trivially copyable, new but legitimate compiler warnings may surface. diff --git a/unreleased_history/public_api_changes/remove_auto_skip_compression_manager.md b/unreleased_history/public_api_changes/remove_auto_skip_compression_manager.md deleted file mode 100644 index fb4b0342ff..0000000000 --- a/unreleased_history/public_api_changes/remove_auto_skip_compression_manager.md +++ /dev/null @@ -1 +0,0 @@ -Removed the experimental `CreateAutoSkipCompressionManager()`. Auto-skip compression is now a first-class feature of the block-based table builder, controlled by the new experimental `CompressionOptions::auto_skip`. diff --git a/unreleased_history/public_api_changes/stackable_coroutine_reads.md b/unreleased_history/public_api_changes/stackable_coroutine_reads.md deleted file mode 100644 index abc5708eb8..0000000000 --- a/unreleased_history/public_api_changes/stackable_coroutine_reads.md +++ /dev/null @@ -1 +0,0 @@ -Added `CoroStackableDB` for implementing wrappers over native coroutine reads and used it to add coroutine `Get` and `MultiGet` support to `DBWithTTL` and WritePrepared `TransactionDB`. A stackable wrapper advertises coroutine support only when its wrapped DB supports it; TTL value decoding and transaction visibility semantics are preserved. diff --git a/unreleased_history/public_api_changes/table_file_creation_reason_sst_file_writer.md b/unreleased_history/public_api_changes/table_file_creation_reason_sst_file_writer.md deleted file mode 100644 index eae7547d81..0000000000 --- a/unreleased_history/public_api_changes/table_file_creation_reason_sst_file_writer.md +++ /dev/null @@ -1 +0,0 @@ -Added a `kSstFileWriter` value to `TableFileCreationReason`, which `SstFileWriter` now reports instead of the catch-all `kMisc`. This, along with new `db_name` and `is_remote_compaction` fields in `FilterBuildingContext`, improves the contextual information available to custom filter and compression strategies. diff --git a/unreleased_history/public_api_changes/wal_iterator_rename.md b/unreleased_history/public_api_changes/wal_iterator_rename.md deleted file mode 100644 index 43123c4306..0000000000 --- a/unreleased_history/public_api_changes/wal_iterator_rename.md +++ /dev/null @@ -1 +0,0 @@ -Renamed `TransactionLogIterator` to `WalIterator` (and `include/rocksdb/transaction_log.h` to `include/rocksdb/wal_iterator.h`), because it iterates the write-ahead log and is unrelated to `TransactionDB`. The old names still work. `DB::GetUpdatesSince()` is unchanged. In Java, `WalIterator` is the new name and `TransactionLogIterator` remains as a subclass. Also documented existing behavior that was previously unstated: only writes that reached the WAL are visible (so `WriteOptions::disableWAL` and `IngestExternalFile()` leave holes that end iteration), what each iterator state means, and that `GetUpdatesSince()` silently starts later when the requested sequence number is unavailable.