496 Commits

Author	SHA1	Message	Date
Jay Huh	956c7042da	Check async I/O support on the MultiGet thread and fall back to sync (#15337) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15337 ## Why RocksDB crash tests repeatedly fail with `MultiGetEntity error: Not implemented: ReadAsync: failed to init io_uring` when synchronous `MultiGet` requests async I/O on a thread that cannot create an io_uring. The same `NotSupported` status can reach users if capability detection succeeds but an individual asynchronous submission later fails. ## Previous behavior `Version` cached `FSSupportedOps::kAsyncIO` when it was constructed. Posix io_uring instances are thread-local, so that result described the construction thread rather than the thread later executing MultiGet. A cached positive result could route a different thread into `ReadAsync`, where io_uring creation failed and the error reached the caller. A cached negative result could also unnecessarily disable async I/O on a capable read thread. ## New behavior and implementation Synchronous `Version::MultiGet` now checks async-I/O support on the thread executing the read. Both the cross-level `optimize_multiget_for_io` path and the within-level overlap path enter their coroutine-based I/O only when that thread supports it. Posix `SupportedOps` eagerly creates and retains that thread-local io_uring, so subsequent submissions reuse the verified ring. The capability remains advisory: if `ReadAsync` nevertheless returns `NotSupported`, `AsyncFileReader` completes the `FSReadRequest` synchronously, including requests that use filesystem-owned buffers. Poll errors are applied only to requests that were actually submitted asynchronously, so synchronous fallbacks in a mixed batch retain their successful status. Regression coverage forces io_uring capability failure only on a dedicated read thread and exercises both synchronous MultiGet branches. Separate tests cover all-request fallback and a batch mixing submitted and synchronously fallen-back requests under an injected Poll error. A one-line bug-fix release note records the user-visible correction. ## Safety and compatibility This changes no API, ABI, or persisted format. The support check is only performed when synchronous `MultiGet` receives `ReadOptions::async_io`; the Posix steady-state cost is a virtual call and thread-local ring lookup per such MultiGet. Direct I/O continues to use the existing synchronous `MultiRead` route before `AsyncFileReader`, and native coroutine APIs are unchanged. Reviewed By: pdillinger Differential Revision: D117802488 fbshipit-source-id: 0f2991a523607d894436cd53123e7a1b2fe93167	2026-10-12 19:22:21 -07:00
Josh Kang	4b983da055	Spell out special member functions (rule of five) (#15387) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15387 ## Summary RocksDB's `.clang-tidy` enables `cppcoreguidelines-special-member-functions`, `cppcoreguidelines-virtual-class-destructor` and the noexcept-move checks, and clang warns `-Wdeprecated-copy-with-user-provided-dtor`. A full scan reports 476 findings. In most of them, a class declares a destructor (or copy operations) but leaves the rest of copy/move implicit. That has two costs. Readers can't tell whether a class may be copied or moved. And resource-owning classes keep an implicit copy that would double-free if anyone ever used it. This diff resolves 453 of the 476. Approach, picked per class from least to most invasive: 1. An empty user-provided destructor becomes `= default`. This covers most interface classes, such as `virtual ~X() {}` -> `= default`. If the class is still copied, that would only rename the deprecated-copy warning, so the destructor is removed instead when nothing needs it (`Customizable`, `TableFactory` and `InternalKeyComparator` inherit a virtual one; `IteratorWrapperBase` isn't polymorphic), and `MemTableRep::KeyComparator` gets defaulted copy and move. 2. Classes whose copy is already deleted get explicitly deleted move operations too. Move was already unavailable, so this only documents it. 3. Resource-owning classes that are never copied get deleted copy and move operations. Examples: the C API wrapper structs, which run a `destructor_` callback (edited in `tools/c_api_gen/c_base.cc` and regenerated into `db/c.cc`); `ShardedCache`, which frees its shards; iterator wrappers; and RAII handles. 4. Value types that are copied or moved get the missing operations as `= default`, which keeps their current behavior. 5. Exception specifications of existing functions are unchanged, except for `WriteBatch`'s move assignment (below): no other `noexcept` is added or removed. Only the move operations this diff newly declares (`Comparator`, `FileSampledStats` and the thread-local `Entry`) are `noexcept`; they copy only scalars or atomics. `WriteBatch`'s copy and move assignment destroyed `*this` and placement-new'ed a new `WriteBatch` over it. That is undefined behavior when `*this` is a subclass, and if the copy constructor threw, `*this` was left destroyed and was later destroyed again. Move assignment now assigns member-wise and ends in the same state as the move constructor, which leaves `has_key_with_ts_`, the timestamp-size tracking and the other fields it doesn't copy at their defaults. `rep_` is swapped through a temporary, because a plain string move-assignment can keep the old buffer (e.g. a large batch reset with `batch = WriteBatch()` would keep its memory). Nothing in it can throw, so it is now `noexcept`. Copy assignment copies into a temporary and moves it in, so a throwing copy leaves `*this` unchanged. Bugs this exposed and fixed: - `RestoreAfterCopyOrCreateWorkItem` (backup engine): the hand-written move constructor and assignment moved only some fields, so `from_file` and `to_file` were empty after `push_back(std::move(...))`. A restore that hit a checksum mismatch then reported "While restoring -> : ..." without file names. It now uses the compiler-generated moves, which are still `noexcept`. - `ExternalSstFileIngestionJob` lived in a `std::vector`, and its destructor frees raw-owned pointers, so the implicit copy used on vector reallocation would double-free. Its file batches also point into its own inline file storage, so neither a copy nor a move is safe after `Prepare()`. Copy and move are now deleted, and `DBImpl` holds jobs through `std::unique_ptr`. Compatibility: no public class loses an operation users can call. Public interfaces mostly only get `virtual ~X() = default;`. `WriteBatch`'s move assignment becomes `noexcept`, which is not part of the mangled name, so the ABI doesn't change. `SubcompactionJobInfo` and `CompactionJobInfo` intentionally stay C++20 aggregates, so users' aggregate and designated initializers keep compiling. `Configurable`, `CompareInterface`, `WriteBatchBase`, `ObjectLibrary::Entry` and `MemTableRep::KeyComparator` declare defaulted copy and move, and `Customizable` no longer declares a destructor. As a result, `Configurable` and `Customizable` subclasses that declare no destructor or copy operations of their own now move their registered-options list instead of copying it. In-tree, these are the mutable-options adapters, the table factories, the memory allocators and test classes; all are held through smart pointers or used in place, and `Clone()` copies, so none is moved by value. Not changed (23): `IODebugContext` (custom deep copy plus a `std::shared_mutex`), `DBWithColumnFamilies` in db_bench (stored in a `std::vector` with non-movable members), `Snapshot`'s protected virtual destructor (intended), `ManagedSnapshot`, the `SubcompactionJobInfo` and `CompactionJobInfo` aggregates, and the 16 noexcept-move-constructor and noexcept-swap findings on existing functions, including `autovector`'s move assignment (adding `noexcept` would change their exception specifications). Copying `ManagedSnapshot` already double-releases the snapshot. Fixing that needs a real move implementation and is a behavior change, so it is left for a separate diff. Reviewed By: rban1 Differential Revision: D123186309 fbshipit-source-id: 6f8e57e233e12199a007407130bb9a1f6d52e8d1	2026-10-12 16:00:09 -07:00
Xingbo Wang	1334e4dd52	Add offline OPTIONS binding repair (#15310) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15310 Why the change is required: The committed OPTIONS number in MANIFEST gives persisted-options readers a precise source of truth. If that file is missing or corrupt, or an operator must deliberately replace it with a known-good snapshot, persisted-options loading can fail before a normal writable open can publish a replacement. RocksDB therefore needs a supported offline recovery path that obeys the binding and global file-number rules. Previous behavior: ldb update_manifest could rewrite file-temperature metadata but could not repair an OPTIONS binding. Operators could not safely bind a selected snapshot while accounting for the current MANIFEST boundary, every file-number allocation root, recoverable WAL output, and crash durability. Blindly selecting or reusing an existing OPTIONS number could bind the wrong snapshot or collide with another RocksDB file. New behavior: ldb update_manifest --rebind_options_file --options_file=OPTIONS-<number> performs an offline repair from an explicitly selected trusted snapshot whose tracking marker is enabled. Optional --file_number_paths arguments identify external allocation roots that are not serialized in OPTIONS. The command locks the DB and validates that the selected OPTIONS snapshot contains every live MANIFEST column family. Extra OPTIONS CFs are accepted as a safe superset and are filtered by binding-aware persisted-options APIs; a snapshot missing any live CF is rejected. The command raw-copies the selected bytes to a newly allocated canonical OPTIONS number, syncs that file and its directory entry, and commits the new number with one MANIFEST record. It rewrites the current MANIFEST only through the last completely decoded record, appends the commit and next-file boundary to a temporary copy, syncs it, atomically renames it over the original, and syncs the MANIFEST directory. The crash contract has one publication point. Before MANIFEST replacement, the previous commit remains authoritative and the copied tracked OPTIONS file is only an ignorable orphan. After replacement, the new snapshot is authoritative. Implementation: - Parse the selected snapshot for validation but copy it byte-for-byte so unknown or newer option fields are preserved. - Replay MANIFEST to recover live CF membership, the committed number, next file number, and last valid logical-record boundary; discard a torn physical tail during the bounded-memory rewrite. - Scan the DB root, WAL/archive directories, serialized roots, and operator-supplied roots for allocated numbers. - Parse recoverable WALs and reserve conservative headroom for recovery outputs, missing-column-family creation, persistent statistics, asynchronous WAL creation, MANIFEST rollover, and subsequent writable-open allocations. - Reject file-number overflow, missing allocation roots, corrupt or unreadable WALs, invalid input names, missing-live-CF snapshots, and simultaneous temperature rewriting. - Preserve the existing update_temperatures behavior and result text. Safety boundaries and unsupported cases: - This is an explicit offline mutation. It never auto-selects a snapshot and refuses to run while another process holds the DB lock. - The selected input must be a canonical OPTIONS-<number> basename in the DB directory, must parse successfully, must enable MANIFEST tracking, and must contain all live MANIFEST CFs. Extra CFs are allowed but missing live CFs are not. - The command does not repair snapshot contents or alter topology. - OPTIONS does not serialize all DB/CF data roots. Operators must provide every external root through --file_number_paths; the command fails closed when a supplied or inferred root cannot be scanned. - Conservative allocation headroom can reject a repair near kFileNumberMask even when a particular future open would allocate fewer files. - Normal DB open and core publication do not depend on this maintenance command. Validation coverage and known limitations: Coverage includes parsing and incompatible modes, the DB lock, tracking and membership validation, safe-superset acceptance, missing-live-CF rejection, byte-preserving copy, torn MANIFEST tails, one-record commit publication, file-number exhaustion and writable-open headroom, external and missing roots, WAL recovery bounds, asynchronous WAL and persistent-stat allocations, MANIFEST rollover, writable reopen, and later CF creation. The end-to-end test confirms safe-superset rebinding and rejection of a higher tracked snapshot that omits a live CF, verifies the final committed MANIFEST number, and runs the real ldb scan --try_load_options subprocess. Cross-platform CI is not part of local validation. Reviewed By: pdillinger Differential Revision: D123104846 fbshipit-source-id: 8cf7c54c1c7a360411034469614eed8aaffdcf8e	2026-10-11 13:13:46 -07:00
Xingbo Wang	baa4d373a6	Commit authoritative OPTIONS file numbers in MANIFEST (#15305) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15305 Why the change is required: Column-family topology is durable in MANIFEST, while its operational configuration snapshot is stored in a separate OPTIONS file. CreateColumnFamily and DropColumnFamily previously committed MANIFEST first and wrote OPTIONS afterward. An OPTIONS I/O failure could therefore make the API report failure after the topology change was already durable. Persisted-options readers also selected the highest file number and could consume an orphan left by a failed publication. Previous behavior: RocksDB had no durable correspondence between a MANIFEST state and the OPTIONS snapshot describing it. LoadLatestOptions and GetLatestOptionsFileName treated the highest-numbered OPTIONS file as authoritative. Column-family creation and deletion returned the later OPTIONS persistence error even when their MANIFEST edit had already succeeded. Multi-column-family operations wrote a complete OPTIONS snapshot after every successful member, producing quadratic OPTIONS bytes. Ordinary DB::Open consumes caller-supplied DB and column-family options rather than loading OPTIONS. This change preserves that contract. New behavior: Add the default-disabled experimental DB option track_options_file_number_in_manifest. Its value is serialized into each OPTIONS file so the selection rule is self-describing and participates in the normal DB-options compatibility checks. When enabled, RocksDB first durably creates and renames OPTIONS-N and then commits N with one safely ignorable MANIFEST field. A normal OPTIONS rotation uses one MANIFEST commit. Column-family creation and deletion prewrite the proposed snapshot and commit its number together with a topology edit in the same VersionEdit. A failure before the OPTIONS file is durable changes no topology. A failure after the file is durable but before MANIFEST commit leaves an orphan that binding-aware readers ignore, while the previous committed snapshot remains authoritative. Each multi-CF create or drop writes one proposed snapshot, independent of the batch size. Create commits the new number with the first successful add; drop commits it with the last successful drop. Existing partial-success semantics can therefore leave a committed snapshot with extra CFs, but never one missing a live MANIFEST CF. Persisted-options APIs filter this safe superset against recovered MANIFEST membership. A snapshot missing a live CF is corruption. DB::Open with create_missing_column_families uses the same batched path. If the newest OPTIONS file has tracking disabled, it retains the legacy highest-numbered-file behavior, including after an old or disabled writer has run. If it has tracking enabled, readers use the recovered MANIFEST commit and reject newer tracked orphans. A missing usable commit falls back to the highest tracked snapshot for RepairDB and legacy-rewrite recovery. DB open logs that fallback through its configured logger; logger-less utility APIs remain silent. Writable open removes rejected tracked orphans when local obsolete-file deletion is enabled. Read-only open never mutates the DB directory. A DB with no recovered commit and tracking disabled takes the legacy fast path without parsing the newest OPTIONS file. When inspection is required, RocksDB reads only the tracking marker. Marker-read failures used for selection do not fail DB open; a recovered commit remains authoritative and ambiguous newer candidates are conservatively retained. The first topology change after enabling tracking first binds the current topology, so failure of the proposed change cannot make its orphan authoritative. MANIFEST failure during an ordinary tracked OPTIONS rotation also enters the existing manifest-write background-error path. Implementation: - Add the safely ignorable CommittedOptionsFileNumber VersionEdit field and carry it through recovery, MANIFEST snapshots, rollover, and subset checkpoints. - Persist the tracking marker through C++, C, and Java options, generated round-trip coverage, compatibility checking, and db_stress configuration. - Centralize binding-aware selection for DB open and public persisted-options APIs. Reuse RecoveryContext's directory listing, use a narrow marker scan, and recover live CF names while replaying MANIFEST. - Filter manifest-selected safe-superset snapshots to live MANIFEST CFs in LoadLatestOptions and compatibility verification without discarding their option maps. - Prevalidate and sanitize CF batches outside the DB mutex, write one complete snapshot while options_mutex_ serializes topology changes, and hold the DB mutex only for the existing individual MANIFEST commits. Batch missing-CF creation during DB open follows the same locking boundary. - Keep the committed snapshot live in cleanup and live-file APIs, and make subset checkpoints write an OPTIONS snapshot and filtered MANIFEST with identical membership in tracked and legacy modes. - Preserve the first partial multi-CF drop error while consuming legacy OPTIONS persistence Status, and sync the checkpoint MANIFEST directory after replacement. - Pass checkpoint helper callbacks by reference and group VersionEdit storage by alignment so the new paths add neither callback copies nor avoidable object padding. Safety boundaries and unsupported cases: - Tracking defaults to false. Disabled writers and never-tracked databases retain legacy write ordering and highest-numbered selection. - Older binaries safely ignore the new MANIFEST tag and continue choosing the highest OPTIONS number. Downgrade is readable but not crash-atomic: an old binary can consume a tracked orphan. That is the same class of OPTIONS/MANIFEST inconsistency as the previous behavior. - A rollback binary must know the serialized option name before the feature is enabled. Exact compatibility checks intentionally report a mismatch when the caller's tracking setting differs from the persisted snapshot. - A newer untracked OPTIONS file deliberately supersedes an older committed number, allowing an old or disabled writer to become authoritative without rewriting MANIFEST. - Reactive secondary and follower instances do not enforce or repair the binding. Ordinary DB::Open still uses caller-supplied options. - Writable open does not delete rejected tracked or ambiguous files when a compaction service owns OPTIONS-file retention. Read-only open performs no repair writes. - Direct consumers that scan OPTIONS filenames themselves are unchanged; the binding is enforced by RocksDB's persisted-options APIs. - If the committed snapshot is missing and only tracked candidates remain, selection intentionally falls back to the highest candidate so RepairDB output remains usable. Validation coverage and known limitations: Focused coverage exercises encoding compatibility, tracked and legacy selection, the never-tracked open fast path, marker-read failures, exact tracking-marker compatibility, activation and disable/re-enable transitions, batched create/drop and create-missing behavior, partial-batch safe supersets, OPTIONS/MANIFEST failure injection, background-error propagation, orphan cleanup and retention, read-only non-mutation, subset-checkpoint behavior, stale OPTIONS recovery, RepairDB option sanitization, and transient read-corruption reconstruction. The assertion-enabled full executable suite passes. The repository check-format and check-sources wrappers cannot run because they invoke Git inside this Sapling checkout; fbsource lint is used instead. Cross-platform CI is not part of local validation. Reviewed By: pdillinger Differential Revision: D123104845 fbshipit-source-id: 77d94c989243c3f3f9fedfa81fa364c166250303	2026-10-11 10:39:30 -07:00
Xingbo Wang	19b6f85266	Support async file open for SecondaryDB (#15294) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15294 Why the change is required: `open_files_async` reduces startup latency by moving SST validation and table-reader opening to background work, but `DB::OpenAsSecondary()` still performed that work synchronously. Previous behavior: A secondary using `open_files_async=true` replayed its MANIFEST through `ReactiveVersionSet`, synchronously verified and opened every current SST, and only then returned. The option therefore provided no startup benefit for secondary instances. New behavior: `DB::OpenAsSecondary()` can now return after installing its initial read view and start initial SST validation and table-reader pinning through the existing asynchronous opener. With `max_open_files=-1`, the opener processes all current SSTs in the background; a finite table cache can stop the eager pass at capacity and validate remaining files on demand. Later `TryCatchUpWithPrimary()` rounds remain synchronous. Async-open failures notify listeners and are latched according to the existing severity and `paranoid_checks` policy. If an error is not latched, a later read retries the file access and reports a persistent failure. A latched error requires fixing the underlying problem and reopening the secondary. Read-only instances do not start write-oriented recovery. Implementation: Initial reactive MANIFEST replay can defer SST verification and handler loading. The deferral is immutable constructor state on `ReactiveVersionSet`, is cleared before catch-up mode, and is restricted to normal secondary recovery. `VersionBuilder::GetAndClearIntermediateFiles()` now moves out and clears its own historical add-then-delete bookkeeping so callers cannot accidentally retain stale state. Secondary open schedules the background opener after publishing the read view. Its context pins the captured Versions and column families, including retained secondary handles whose column family was subsequently dropped by the primary. Existing table-reader opening performs SST size and unique-ID validation, avoiding a separate serial metadata pass. Closing the DB or canceling background work safely cancels a queued opener or waits for a running opener. Read-only async-open errors use an ErrorHandler path that notifies listeners without automatic recovery. Secondary async open explicitly rejects FIFO compaction. Safety, compatibility, and limitations: The default `open_files_async=false` path and existing primary async-open behavior are unchanged. WAL and blob recovery, as well as later secondary catch-up, remain synchronous. Follower DB and `OpenAndCompact()` do not use this deferral. There are no file-format, ABI, or public API signature changes. Secondary async open requires `skip_stats_update_on_db_open=true` and does not support FIFO compaction. Applications must keep files accessible by coordinating deletion with the primary or by using an Env/FileSystem that delays physical deletion until every instance releases a file. With asynchronous opening, `max_open_files=-1` cannot guarantee every SST is pinned before `OpenAsSecondary()` returns; the table-files-only `max_open_files=-1` workaround applies only when asynchronous opening is disabled. No completion API is added. Validation covers initial deferral and pinning, synchronous catch-up, table-reader detection of physical-size mismatch, FIFO rejection, listener-adjusted errors and synchronous read retry, retained handles after primary column-family deletion, close and explicit cancellation with no HIGH-priority worker, stopping new table opens during shutdown, primary async open, follower isolation, and remote compaction. Linux Buck builds and focused tests passed. Cross-platform CI is pending, and `make check-sources` cannot run successfully in this Sapling checkout because its script requires Git. Reviewed By: pdillinger Differential Revision: D122417547 fbshipit-source-id: 91d408469c35c4ea21165c5e7db75068e65c2389	2026-10-02 12:21:46 -07:00
Peter Dillinger	34d5074bfa	Cross-key coalescing for lazy wide-column blob reads (#15178) ... Summary: Follow-up (Phase 3) to the experimental lazy wide-column read API (DB::GetEntityLazy / DB::MultiGetEntityLazy). A MultiGetEntityLazy batch now resolves blob references across keys with coalesced I/O instead of key-by-key. - MultiGetEntityLazy acquires one SuperVersion and one consistent (implicit) sequence number for the whole batch via MultiCFSnapshot (as multi-CF iterators do) and transfers a single shared per-column-family pin into the LazyWideColumnsBatch, replacing the former per-key SuperVersion + explicit-snapshot loop. Single column family for now; a cross-CF ColumnFamily** overload remains a follow-up (the batch already models a per-CF pin map). - LazyWideColumnsBatch::MultiResolve groups the batch's reads by (Version, blob file) for separate-file references and by SST for embedded references, and issues one coalesced read per group -- for whole-column and byte-range reads alike -- caching whole values in each entity's resolver and pinning partial reads directly. - Adds the coalesced blob-read primitives this needs (none existed before): range-aware BlobFileReader::MultiGetBlobRange / BlobSource::MultiGetBlobRange / Version::MultiGetBlobLazy for separate-file references, and an embedded (SimpleGen2) batch read path (blob_gen2_format batch readers + BlobSource::MultiGetSimpleGen2Blob[Range] + BlockBasedTable::MultiGetSameFileBlob). Whole-value separate-file coalescing reuses the existing MultiGetBlob. The batch readers sort their requests by file offset before issuing the underlying MultiRead (which requires ascending offsets), and gen2 checksum verification is factored into one helper shared by the scalar and batch readers. - The hot non-lazy Get / MultiGet / iterator / compaction paths are untouched: the new multis are lazy-only, and the GetImpl change is gated on a new opt-in GetImplOptions field (an injected shared SuperVersion + sequence number) used only by the batched lazy read. - Blob read failures on these paths (short reads, invalid offsets, out-of-range requests, compression mismatches) now carry file/offset/size and expected-vs-actual context, mirroring the block-checksum diagnostics. - db_bench gains a multireadrandomentitylazy benchmark for measuring the coalesced batch path. No public API change, so this is a performance follow-up to the still-recent lazy API rather than a new feature. EXCEPT that `LazyColumnReadRequest::status == nullptr is now explicitly disallowed`. Left for later follow-up: - When a whole fetch is planned for an (entity, column), bind ordinary range requests for that column to the same fetch and slice them after adoption instead of issuing separate range I/O. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15178 Test Plan: New unit tests in db/wide/db_lazy_entity_test.cc assert cross-key coalescing via a blob-file / SST MultiRead counter added to the test FileSystem wrapper: - N whole-column separate-file reads across keys in one blob file collapse to a single coalesced MultiRead (and the fetched values are cached, so a repeat read does no further blob I/O); - N byte-range separate-file reads collapse to one MultiRead, save the un-read bytes (rocksdb.blobdb.lazy.partial.bytes.saved), and never fill the blob cache; - embedded (same-file) reads across keys in one SST collapse to one MultiRead over that SST; - an out-of-order embedded batch (reads issued in descending record offset) still resolves correctly in one coalesced read, covering the batch readers' internal offset sort; - a mixed batch (inline + whole + byte-range across keys) returns the correct bytes for each read; - a batched MultiGetEntityLazy + one MultiResolve matches resolving each key via a separate GetEntityLazy. The existing db_stress lazy-vs-eager differential (MaybeTestMultiGetEntityLazy) exercises the new batched read and cross-key MultiResolve unchanged (same public API). Reviewed By: xingbowang Differential Revision: D118660682 Pulled By: pdillinger fbshipit-source-id: 6a832175e730917eddeedab8e0b9cb2b8d6ae7b4	2026-09-16 18:10:21 -07:00
Peter Dillinger	e4a7136d3e	Preferred-phase scheduling for periodic (time-based) compaction ... Summary: `periodic_compaction_seconds` softly guarantees every SST is rewritten at least once per interval. Across a fleet of similarly-configured DBs, many events line those intervals up in lockstep -- a shared enable, a coordinated restart (e.g. DC power-outage recovery), or just similar write history -- so the periodic compactions all come due together, a recurring "thundering herd" that is especially painful with remote compaction (SEV S698635). The obvious fixes trade one problem for another. Spreading the herd out means some files get compacted off their natural schedule, which is wasted work; and a clumsy spread just relocates the herd or spawns a new one. So the real goal is narrow: **de-herd while keeping "badness" -- extra and/or late compaction -- small, and without inducing fresh herding at any step.** This change adds an EXPERIMENTAL, opt-in mechanism that gives each `(DB, column family)` a stable, well-spread *preferred phase* within the interval and steers periodic compactions toward it. It is randomized and *uncoordinated* (DBs never talk to each other), which both keeps it simple and, as it turns out, de-herds far more cheaply than adding jitter to the interval. It is controlled by a single mutable DBOption and is **off by default**: - `periodic_compaction_phase_recovery_percent` (default `0` = disabled / kill switch; recommended `33`; clamped to `[0, 100]`) -- how aggressively each trigger converges toward the phase, as a percent of the remaining gap, only ever pulling the trigger *earlier*. `0` reproduces the classic scheduler exactly; `100` jumps to the phase in one cycle; lower values approach gradually with much less extra work. The preferred phase itself needs no tuning: it is derived from the DB ID (stable across re-open and physical cloning/migration, but distinct across DBs) and each column family is offset from the DB's base phase, so the CFs of one DB don't fire together. (An earlier draft exposed a `compaction_schedule_seed` string option for choosing the phase source; it was over-engineered for the one behavior anyone needed, and as a mutable string it also broke OPTIONS-file round-trip verification in crash testing. It has been removed in favor of always using the DB ID.) ## The two kinds of badness "Badness" here is the fraction of compaction work that is ideally unnecessary: over-compaction (rewriting a file earlier than the interval required) plus a penalty for lateness (going overdue past the interval). Steady state is 0; both "never compact" and "compact twice as often as needed" approach 1. De-herding inherently spends a little badness to spread load out; every design choice below exists to bound one of those two costs *without* re-clustering the fleet. The scenario names cited in the design bullets below (`upgrade_dynamic_r33`, `naive_upgrade_r25`, `burst_r*`, `turndown_r33`, `turnup_r33`, ...) all come from the out-of-tree simulator; see its results page (the "Simulator and visualization" section below, with an interactive plot per scenario) for the evidence behind each claim. ## Design - **A stable per-(DB, CF) preferred phase = one fixed attractor per DB.** Each DB gets a base phase from its DB ID; each CF is offset from it. Because the target is fixed (not re-rolled each cycle) and unique per DB, the fleet converges to a uniform spread and then compacts *on time* -- badness decays toward ~0. This is the crucial contrast with naive per-cycle jitter, which de-herds but never stops jittering and so pays a permanent over-compaction cost (see `naive_upgrade_r25`: flat ~12-14%, only diffusive spreading) vs. `upgrade_dynamic_r33`, whose badness decays to ~0 for the same anti-herding. - **Converge gradually, and only ever earlier.** Each trigger closes a `recovery_percent` *fraction* of the gap to the phase, reaching it over a few cycles rather than in one burst, and never later than `periodic_compaction_seconds` in steady state. A lower rate does less extra work, which matters when the phase keeps moving (write-driven / bursty workloads), where a high rate wastefully "chases" it -- the burst sweep shows `burst_r25 < burst_r33 < burst_r50 < burst_r100` in sustained badness. `33` is the default balance: quick spreading, low extra work. - **Fractional recovery (not "snap") is latency-tolerant and doesn't re-herd.** Closing a fraction of the gap gives a single stable convergence point, so real-world scheduling/queueing latency -- the delay between a file becoming due and its compaction actually running -- merely shifts the whole fleet uniformly instead of re-clustering it. A tried "snap onto the phase when in range" variant converged a bit faster with no latency but re-herded around the snap threshold under latency; it was rejected. Graceful degradation beat peak speed. - **Spread a suddenly-due cohort instead of bursting it (bounded lateness).** Enabling the feature, turning `periodic_compaction_seconds` down, or catching up after downtime can leave a whole cohort past due at the same instant. Firing it all at once would just be the herd again, so such a cohort is spread over the first quarter-interval (~N/4) after the phasing anchor on an *absolute*, epoch-aligned per-(DB, CF) grid. This trades a bounded lateness (<= ~N/4 past the deadline, so the interval is a *soft* upper bound during these transients) for anti-herding, and the absolute grid keeps a rapidly restarting DB from being perpetually re-deferred and starved. `turndown_r33` shows the past-due cohort smeared over N/4 rather than bursting; `turnup_r33` confirms a turn-up creates no past-due cohort (deadlines only grow) and so no burst. For immediate enforcement at such a transition, trigger a manual compaction. - **Off by default, with a kill switch.** Inert unless `periodic_compaction_phase_recovery_percent > 0`; at `0` behavior is identical to the classic scheduler. ## Implementation notes - Phase math is integer-only (`Hash64` + `FastRange64`); floating point is avoided for its pitfalls. The per-CF offset is a golden-ratio (Weyl/Kronecker) additive sequence -- low-discrepancy for any number of CFs. - Logic lives in `db/periodic_compaction_phaser.{h,cc}` (`PeriodicCompactionPhaser`), which owns the recovery percent and the phasing anchor and exposes the stateless trigger-time math; `VersionSet` holds one and caches the per-CF result on each `Version`. The anchor is (re)set to now on DB open, on a recovery-percent change, and on any `periodic_compaction_seconds` change -- the three events that can create a past-due cohort. - Plumbed through the C API, `db_bench`, `db_stress`, and `db_crashtest`. ## Simulator and visualization (out-of-tree, not part of this change) A standalone simulator drives 40 real RocksDB instances through a real periodic-task scheduler, records every compaction (periodic vs write-driven), and scores the badness metric above across ~17 scenarios: dynamic/restart enable, interval turn-down/turn-up via `SetOptions`, on-cadence writes, random write bursts with a recovery-rate sweep, and the naive-jitter baseline. It is the basis for the numbers cited above. Interactive HTML (**NOTE** fb-internal URL sanitization might require you to copy-paste the URL yourself): 5759c6a62d/pc_phase_sim.html Reviewed By: jaykorean Differential Revision: D117487278 fbshipit-source-id: a62439548b7dd6fe7dfdbbe21b69531d8b57a1e8	2026-08-26 20:14:26 -07:00
Peter Dillinger	f7b791b213	Fix bottommost-files infinite compaction loop with preserve/preclude (#15146) ... Summary: Leveled compaction combined with preserve_internal_time_seconds or preclude_last_level_data_seconds is a known-broken combination, but it is easy to enable by accident -- for example a fleet-wide config rollout that turns on the preserve time for CFs that happen to be leveled. Rather than degrading gracefully, it triggers an infinite kBottommostFiles compaction loop: a bottommost file gets marked, rewritten with no progress, and immediately re-marked, continuously rewriting the file and growing the MANIFEST so the file set never quiesces. This change makes such a misconfiguration a harmless no-op instead. Before this change, the marking logic (ComputeBottommostFilesMarkedForCompaction) marked a bottommost file whenever its largest seqno was below the oldest snapshot, without checking whether a compaction could actually zero that seqno. Recently written seqnos stay within the seqno->time "preserve" window and cannot be zeroed, so the rewrite never made progress. The same futile marking also occurred with user-defined timestamps when full_history_ts_low is unset. The underlying issue is that the "can this bottommost seqno be zeroed?" question was answered independently by the marking logic and by the compaction output path (CompactionIterator::PrepareOutput), and the two had drifted apart. With this change, both paths answer that question through one shared predicate (BottommostSeqnoCanBeZeroed), and the preserve-window boundary that marking needs is plumbed into VersionStorageInfo from DBImpl, mirroring the existing oldest-snapshot plumbing. Snapshot visibility stays a per-site check. On CFs without preserve/preclude the predicate is a no-op. Confirmed that caller invariants ensure that relocating the `ikey_.sequence <= preserve_seqno_after_` check in CompactionIterator::PrepareOutput() will not trip the ROCKSDB_LOG_FATAL. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15146 Test Plan: - compaction_picker_test: new CompactionPickerTest BottommostFileNotMarkedWithinPreserveWindow / BottommostFileMarkedBelowPreserveWindow cover the preserve-window boundary; new CompactionPickerU64TsTest BottommostNotMarkedWithEmptyFullHistoryTsLowAndUnknownMaxTs covers the UDT edge case. - tiered_compaction_test: new PrecludeBottommostLoopTest LeveledPreserveTimeNoBottommostLoop is an end-to-end regression for the loop and the DBImpl plumbing: a bottommost file with a nonzero largest seqno inside the preserve window, made a marking candidate via a snapshot release, must not be marked. It reproduces the loop (hangs) when the plumbing is neutralized and passes with the fix. Reviewed By: xingbowang Differential Revision: D117299979 Pulled By: pdillinger fbshipit-source-id: 51e8e2d4e2116984d16d3c7ef54a9635d5aded74	2026-08-26 19:44:47 -07:00
Xingbo Wang	e6a2ee0bd2	Fix subset checkpoints with incomplete MANIFEST tails (#15123) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15123 Subset checkpoint post-processing appends column-family drop records to a copied MANIFEST. Recovery can tolerate an incomplete atomic group at the tail while retaining the reader high-water size, so appending at that size can produce a checkpoint that fails recovery. Track the valid logical append boundary together with its MANIFEST file number, refresh it after recovery and successful writes, reset it across reactive MANIFEST switches, and trim only subset-checkpoint copies before appending drop records. Whole-DB checkpoints and the source MANIFEST remain unchanged. Reviewed By: anand1976 Differential Revision: D116449394 fbshipit-source-id: a162dd6a2558f0f9ec40d878ee83956a89df698b	2026-08-22 12:17:33 -07:00
Peter Dillinger	e8554f1b11	Faster and safer MANIFEST recovery for remote compaction (OpenAndCompact) (#15133) ... Summary: Faster and safer MANIFEST recovery for remote compaction (OpenAndCompact) `DB::OpenAndCompact` (remote/distributed compaction) opens the target DB as a secondary and reconstructs the LSM state from the MANIFEST before running the requested compaction. Previously, point-in-time recovery would stat/open the SST (and blob) files referenced by the MANIFEST to classify them found/missing and materialize candidate versions, and eagerly load their table handlers. That is (a) a lot of unnecessary file operations and open latency, especially on remote/distributed storage, and (b) fragile: when a file referenced by the MANIFEST but not an input to the compaction was momentarily unavailable to the worker -- e.g. an obsolete file the primary had already deleted -- recovery could fail the job (surfacing as an incomplete "no valid result") or roll back to an earlier version, even though the compaction only needs its own input files, which the primary keeps referenced for the job's duration. This was seen as a flaky failure of CompactionServiceTest.BasicCompactions on aarch64, and reproduces on x86 under COERCE_CONTEXT_SWITCH. Efficiency + reliability: recovery for the OpenAndCompact path now trusts the MANIFEST. It reconstructs the current version from FileMetaData without stat-ing/opening files to classify them and without eagerly opening candidate-version table handlers, so it does far fewer file operations and never fails or rolls back over a file the compaction does not need. The compaction's input files are opened (and unique-id verified) on demand as it reads them. Correctness hardening: trusting the MANIFEST means the worker's own (possibly eventually-consistent) view of CURRENT/MANIFEST drives the reconstructed LSM shape, which feeds correctness-critical decisions (Compaction::IsBottommostLevel / KeyNotExistsBeyondOutputLevel). A worker that read a stale CURRENT (older MANIFEST), a truncated MANIFEST, or a MANIFEST with a corrupt/torn tail could reconstruct against an older LSM shape and silently drop keys that should be kept. To prevent this, the primary records the MANIFEST position (file number and size) it scheduled the compaction from and passes it in CompactionServiceInput as a floor. After recovery the worker compares the size of the maximal *valid* prefix it reconstructed -- which stops at any corrupt or torn record and at a partial atomic group at EOF, so tolerated tail garbage cannot inflate it -- against that floor, and refuses to compact against an older view (an older file number, or the same file number whose valid prefix is shorter), returning Status::Incomplete so the CompactionService can fall back to a local compaction. Accept-equal-or-later is safe by monotonicity: while the inputs stay locked (being_compacted), a later view can only have moved data further down, never resurrecting a shadowed value, so only an older view is unsafe. Single kill switch: both behaviors are bundled behind one signal and one option. Presence of the floor on the request (min_manifest_file_number != 0) couples on both trust-manifest recovery and the floor check in the worker; absence reverts the worker to the original point-in-time recovery with no floor check. A single mutable DBOption, remote_compaction_manifest_floor (default on), gates whether the primary populates the floor, so it is the one escape hatch that disables the entire new recovery path on a running DB via SetDBOptions if it ever misbehaves. The new CompactionServiceInput fields are wire-compatible in both directions (0 = not provided), so an old worker or an old primary simply behaves as if the switch were off. Follow-up: recovery is not yet strictly "open only the compaction's inputs". A limited number of non-input files are still opened by the bounded file-property sampling in Version::PrepareAppend -> UpdateAccumulatedStats (tolerating any that are missing), which today populates input FileMetaData::num_entries for compaction input-record-count verification. A natural follow-up is to record the needed per-file info (e.g. num_entries) in the MANIFEST so the worker can skip these auxiliary opens entirely; alternatively, source input num_entries from the worker's own read of its input table properties. Relatedly, the floor comparand is the recovered "maximal valid prefix" size, which today equals the MANIFEST file size but is defined to stay correct if a future MANIFEST format writes trailing padding/footer past the last record. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15133 Test Plan: Added tests in compaction_service_test.cc: - RemoteCompactionMissingLowerFileTrustsManifest: a non-input file referenced by the MANIFEST is unavailable during a remote compaction; the job completes with correct results (fails without trust-manifest recovery). The underlying race also reproduces under COERCE_CONTEXT_SWITCH. - RemoteCompactionManifestFloorRejectsStaleView: inflate the floor so the worker's recovered MANIFEST is below it; assert the worker rejects, the job falls back to local, and a below-output deleted key stays deleted (no silent resurrection). - RemoteCompactionManifestFloorSendsPosition: a non-zero floor is attached and the worker accepts its equal-or-later view. - RemoteCompactionManifestFloorKillSwitchOff: with the option off, no floor is sent, the worker uses the original recovery, and a transiently-missing non-input file makes the job fail/roll back (the escape hatch reverts behavior). - RemoteCompactionManifestFloorMutable: SetDBOptions flips the option; the floor stops being attached and the worker skips the check. options_settable_test covers the new mutable DBOption. Crash test: remote compaction is already exercised (db_crashtest randomly enables remote_compaction_worker_threads, driving DB::OpenAndCompact), so with remote_compaction_manifest_floor defaulting on, the new recovery path is covered by the crash test by default -- no new crash-test plumbing is needed. The kill switch is intentionally left un-randomized (always on): turning it off re-enables the original, occasionally-broken recovery, which would only add noisy failures for no benefit. UPDATE: crash test is not well covered because of `"remote_compaction_failure_fall_back_to_local": 1` which is masking other existing failures. Addressing this is a high-priority follow-up item. (Not practical to fix in this diff.) Reviewed By: jaykorean Differential Revision: D116738823 Pulled By: pdillinger fbshipit-source-id: c02e0a437d62fc526dbbfcf70edf43434dfc7c1f	2026-08-20 16:07:48 -07:00
Ranjan Banerjee	2846163d9e	Fix remote compaction input record count mismatch due to uninitialized FileMetaData stats (#15130) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15130 ## Summary Fix a corruption error in remote compaction where `FilterInputsForCompactionIterator` produces different results on the primary host vs. the remote worker. **Root Cause:** When a standalone range tombstone file completely shadows another input file, `FilterInputsForCompactionIterator()` removes the shadowed file from `input_levels_`. This filtering relies on `FileIsStandAloneRangeTombstone()`, which checks `num_entries == num_range_deletions == 1`. On the primary host, these `FileMetaData` fields are always populated (set during flush/compaction when the file was created). On the remote worker, however, these fields are NOT persisted in the MANIFEST and are only lazily loaded from table properties via `UpdateAccumulatedStats()` during DB open, which is capped at 20 files (`kMaxInitCount`). If the standalone range tombstone file is not among those first 20 initialized files, its `num_entries` remains 0 on the remote worker. This causes `FileIsStandAloneRangeTombstone()` to return false, so the remote worker does NOT filter the shadowed file and processes all its keys. Meanwhile the primary's expected record count is computed from its (filtered) view, leading to: ``` Corruption: Compaction number of input keys does not match number of keys processed. Expected 0 but processed 12. ``` **Fix:** Before constructing the `Compaction` object on the remote worker, explicitly initialize `FileMetaData` stats from table properties for all input files. This ensures `FilterInputsForCompactionIterator` produces consistent filtering results on both sides. **Introduced by:** D83375779 (2025-09-29), which enabled `FilterInputsForCompactionIterator` on the remote worker by passing `earliest_snapshot` to `PickCompactionForCompactFiles`. Reviewed By: xingbowang Differential Revision: D116252973 fbshipit-source-id: 2cb2d61dae1f3982edeaf41f9a0f39d5aef1e789	2026-08-20 10:54:16 -07:00
Anand Ananthabhotla	575b6df1b8	Force immediate creation of a new MANIFEST if the current one cannot be reused (#15103) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15103 The crash artifact showed `MANIFEST-000005` was physically well-formed but contained an incomplete atomic group that had been tolerated at EOF. `reuse_manifest_on_open` then appended ordinary records after that tail, turning the tolerated crash tail into mid-file `corrupted atomic group` corruption seen by OpenAndCompact and backup restore. Record whether MANIFEST recovery ended with a pending atomic group. When that marker is set, skip `ReopenManifestForAppend()` so the next manifest write creates a fresh MANIFEST instead of appending after the partial group. The existing physical-size guard remains as a separate torn-tail defense. This replaces the earlier keep-size preallocation hypothesis with the artifact-backed root cause. Reviewed By: xingbowang Differential Revision: D116062618 fbshipit-source-id: fb55ce2891a6fd50fa2c670dc720e91bad27e94d	2026-08-17 16:18:06 -07:00
Andrei Preda	b616b203b8	Drop flushed WAL data from the secondary's active memtable (#15066) ... Summary: Fix https://github.com/facebook/rocksdb/issues/15051: a secondary serves stale reads because entries replayed from a WAL the primary has since flushed are never evicted from the active memtable. During TryCatchUpWithPrimary(), seal the active memtable once the installed Version provably covers everything it holds, so the following RemoveOldMemTables() call drops it. Every memtable-dropping decision takes that same watermark, so a file set the secondary cannot read never turns the stale read into a vanished key. ## Why and How A secondary replays the primary's WALs into its active memtable and stops replaying a WAL once the primary flushes it. Entries already replayed stayed behind, and since reads consult the memtable first, they shadowed the newer flushed values indefinitely. Iterators go stale too once bottommost compaction rewrites the flushed entry's sequence number to 0; until then Get() and an iterator disagree. Seal the active memtable during catch-up, gated on the installed Version: each point-in-time Version now carries the column family's log number it reflects, exposed via the new ReactiveVersionSet::GetInstalledVersionLogNumber(). Sealing requires cf_id_to_current_log_[cfd->GetID()] < that number, which guarantees the primary flushed everything the memtable holds (unflushed WAL data is kept) and that the flushed data is readable through the installed Version. The gate asks the Version directly because proxies fail: the log number alone advances even when no Version was installed, and "no missing files" misses atomic_flush groups parked by a sibling's missing file and corrupt-but-present files with verify_sst_unique_id_in_manifest=false. In those cases the memtable holds the only readable copy, so dropping it would turn a stale read into a vanished key; the gate fails closed and keeps it. Reconcile every initialized column family, not just cfds_changed: a round that failed after advancing the log number leaves a stale memtable that later rounds see no new MANIFEST record for. It can also leave a collectible immutable memtable, so the loop consults the new O(1) MemTableList::HasOldMemTablesToRemove() before skipping a column family. The predicate keeps the skip precise: reconciling any column family with a non-empty immutable list would be correct but wasteful, installing a fresh SuperVersion every catch-up call for memtables legitimately waiting on a primary flush and costing readers a mutex acquisition each time. With it, no extra super versions are installed in the steady state. That predicate and the RemoveOldMemTables() cutoff take the installed Version's log number as well, not cfd->GetLogNumber(): that advances as soon as the flush record is read, so an immutable memtable sealed on a WAL switch could otherwise be freed while the flushed file superseding it is unreadable. A sealed memtable is stamped with cf_id_to_current_log_[cfd->GetID()] + 1, the log number following its contents. cf_id_to_current_log_ is keyed by column family id rather than ColumnFamilyData*, because ids are never reused while a dropped column family's pointer can be recycled, and it is recorded before WriteBatchInternal::InsertInto() rather than after: a batch stops at its first failure with earlier entries already inserted, so recording afterwards could leave the map naming an older WAL than the memtable holds. While the watermark lags and uncollectible memtables accumulate, the secondary warns, since it has no write path to enforce max_write_buffer_number. The seal sequence, shared with RecoverLogFiles() and the follower's TryCatchUpWithLeader(), is extracted into DBImplSecondary::SealActiveMemtable(). The follower is refactored onto it with no behaviour change: it keeps cfd->GetLogNumber() as its cutoff because that catch-up path never replays WALs, so its memtables hold nothing that only a not-yet-installed Version could make readable. ## Testing New tests in db_secondary_test cover: the stale read via both Get() and an iterator, before and after bottommost compaction; unflushed WAL-only data surviving sealing, including two column families sharing a WAL; no data loss with a missing, atomic_flush-sibling-missing, or corrupt flushed file; an immutable memtable sealed on a WAL switch surviving an unreadable flushed file; a batch rejected partway through replay leaving the recorded WAL no older than the memtable's contents; and reconciliation after a failed round (injected via FaultInjectionTestFS at a sync point). Each mechanism was individually neutralized and confirmed to fail its test. Verified with build_tools/rocksptest.sh over db_secondary_test, db_follower_test, memtable_list_test, version_set_test and version_edit_test (all pass, also under ASSERT_STATUS_CHECKED=1), plus db_basic_test, corruption_test, repair_test and db_flush_test; the secondary tests also ran 20x under COERCE_CONTEXT_SWITCH=1 with no flakes. make format-auto and make check-sources are clean. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15066 Reviewed By: jaykorean, anand1976 Differential Revision: D115317523 Pulled By: xingbowang fbshipit-source-id: a1d4615eb951d75e69ed242731336485c0dfa137	2026-08-17 13:47:30 -07:00
Peter Dillinger	8e1569b351	Lazy wide-column byte-range (partial) blob reads (#15067) ... Summary: Builds on the lazy wide-column read API (DB::GetEntityLazy / DB::MultiGetEntityLazy) to serve true byte-range reads: resolving a sub-range of an uncompressed blob column now reads only the requested bytes from storage instead of fetching and slicing the whole column. This covers blobs in separate blob files and embedded (same-file) blobs in the SST, so a caller that knows a large value's layout can pull just the parts it needs. A partial read skips whole-record checksum verification (a strict sub-range cannot cover the checksum) and does not populate the blob cache (the cache entry holds the whole record). A compressed column, a whole-column read, a cache hit, or a read with LazyColumnReadRequest::force_verify still resolves (and verifies) the whole column; force_verify forces verification even when ReadOptions::verify_checksums is off. Mechanics: new range primitives mirror the whole-value read paths -- BlobFileReader::GetBlobRange, BlobSource::GetBlobRange, Version::GetBlobRange, and VersionBlobFetcherBase::FetchBlobRange for separate-file blobs; ReadSimpleGen2BlobRange, BlobSource::GetSimpleGen2BlobRange, and SameFileBlobReader::GetSameFileBlobRange (implemented by BlockBasedTable) for embedded blobs. ReadPathBlobResolver::ResolveColumnRange owns the partial-vs-whole decision and routes same-file references through the current SST's reader; the whole-column path also gained VersionBlobFetcherBase::FetchBlobForceVerify to honor force_verify. Observability: deferred resolve reads are attributed to a new Env::IOActivity::kLazyResolve (distinct from the kGetEntity / kMultiGetEntity of the initial entity read), and lazy resolution's storage reads are counted by new statistics rocksdb.blobdb.lazy.read.count / .read.bytes (all lazy reads) and rocksdb.blobdb.lazy.partial.read.count / .partial.bytes.saved (the byte-range subset and the bytes it avoided). The whole-column read sites gate recording on kLazyResolve so non-lazy reads are excluded. Also fixes a latent bug in LazyWideColumnsBatch::MultiResolve where an empty/default-constructed batch (null rep_) would accept a foreign standalone column (null owning_batch_rep_) through a nullptr == nullptr comparison instead of returning InvalidArgument. Follow-ups (in a local plan): unifying the blob-read API surface -- folding the whole/range method twins and the verify-policy flags (verify_checksums x force_verify) into one internal BlobReadRequest primitive (a range descriptor plus a 3-valued verify policy), which retires FetchBlobForceVerify and the whole/range method pairs; this is deferred for feature velocity and is not public-API affecting (the public force_verify bool stays). Marked in code with TODO(lazy-blob-resolution-cleanup) in db/blob/blob_fetcher.h and db/wide/read_path_blob_resolver.cc. Also cross-key coalescing (Phase 3) and async execution (Phase 4). Pull Request resolved: https://github.com/facebook/rocksdb/pull/15067 Test Plan: Extended db/wide/db_lazy_entity_test.cc: separate-file and embedded partial reads return the exact sub-span and read only the requested bytes; a partial read does not fill the blob cache and a cache hit slices with no I/O; a compressed column resolves whole and slices; force_verify does a full verified read and is detected via a tampered record even with verify_checksums off; the lazy statistics distinguish whole vs partial reads and are not touched by non-lazy reads; deferred reads carry kLazyResolve (verified with a FileSystem wrapper); and a regression test for the empty-batch ownership check. Added an IOActivityToString case in env/env_test.cc and registered the new tickers in the statistics name map. Crash test: extended the db_stress lazy-vs-eager differential (db_stress_tool/db_stress_test_base.cc) to fuzz per-column byte ranges. Each resolved column now randomly picks a whole-column read or a partial (offset, length) sub-range -- occasionally with an offset past the end or a length beyond the remainder to exercise clamping -- for both GetEntityLazy and MultiGetEntityLazy; when an eager reference is available the resolved bytes are compared against the same clamped sub-range of it. No new flags (reuses --lazy_entity_read_one_in); force_verify fuzzing remains a follow-up. Reviewed By: xingbowang Differential Revision: D115130388 Pulled By: pdillinger fbshipit-source-id: 454368dea8fae069d9de649c2c51611469d3c809	2026-08-07 12:08:03 -07:00
Peter Dillinger	fd7399004f	Add lazy wide-column read API (GetEntityLazy / MultiGetEntityLazy) (#15012) ... Summary: Adds an experimental **lazy wide-column read API** that lets callers enumerate a wide-column entity and read back only the columns (and, in a later phase, the byte ranges) they actually need, instead of eagerly materializing every blob-backed column like `GetEntity()` / `MultiGetEntity()`. New public entry points (`include/rocksdb/db.h`): - `DB::GetEntityLazy(read_options, cf, key, LazyWideColumns* result)` - `DB::MultiGetEntityLazy(read_options, cf, num_keys, keys, LazyWideColumnsBatch* result, statuses, sorted_input)` Both leave a wide-column entity's **inline** columns materialized up front but its **blob-backed** columns as *unresolved references*, so blobs for columns that are never pulled are never read. ### Result types (`include/rocksdb/lazy_wide_columns.h`) - **`LazyWideColumn`** — the lazy analogue of `WideColumn`: `name()`, `index()`, `inline_value()` (an `OptSlice`; empty ⇒ unresolved reference), `is_reference()`, and `logical_size()` (`std::optional<uint64_t>`; known for inline and uncompressed references, empty for compressed ones). All no-I/O. - **`LazyWideColumns`** — a move-only result that is **indexable and iterable like a vector** of `LazyWideColumn` (`size()`, `operator[]`, range `begin/end`). Unlike `PinnableWideColumns`, it may be used after the producing call returns: it **pins the SuperVersion** (like an iterator) so referenced blob files / SST readers stay valid for deferred reads. - **`LazyWideColumnsBatch`** — the `MultiGetEntityLazy` result; vector-like over its per-key `LazyWideColumns`. ### Resolution - **`LazyColumnReadRequest { const LazyWideColumn* column; uint64_t offset; size_t length; bool force_verify; PinnableSlice* result; Status* status; }`** — a read names a **typed column pointer**, not an untyped index. Because a column identifies its owning result (and, for a batch, its owning entity), `MultiResolve` routes each read correctly and **rejects a null or foreign column** with `InvalidArgument` on that read's per-read status — cross-result misuse can't be silently mis-indexed. - **`LazyWideColumns::MultiResolve` / `LazyWideColumnsBatch::MultiResolve`** take the same request type (pointer+count and `std::vector` sugar overloads); the batch form resolves reads spanning any entities of the batch in one call. - **`ResolveColumn(column, result)` / `ResolveColumnRange(column, offset, length, result)`** — per-column sugar over a one-entry `MultiResolve`. - A column's blob is fetched **at most once** over the result's lifetime (bytes are cached in the result), so repeated reads of the same column reuse that fetch. Resolution honors the `ReadOptions` captured at `GetEntityLazy()` time (e.g. `read_tier`, `verify_checksums`); the resolve methods intentionally take no per-resolve `ReadOptions`. ### Where it works - Primary, **read-only**, and **secondary** DB instances (each `GetImpl` override forwards the lazy signal and transfers a SuperVersion pin via the shared `DBImpl::TransferSuperVersionPin` helper). - Forwarded through `StackableDB` wrappers (e.g. `TransactionDB`) rather than falling through to the `NotSupported` base default. - Requires `max_open_files == -1` (immortal table readers) so same-file ("embedded") references stay resolvable lazily; otherwise the call returns `InvalidArgument`. ## Implementation notes - Read path: `GetContext` gains a lazy mode that leaves blob references unresolved and captures the SST's `SameFileBlobReader`; `Version::Get` forwards the new (defaulted) `lazy_columns_same_file_reader` out-param. - `ReadPathBlobResolver` performs on-demand, cached blob resolution bound to the pinned `Version` + decoded blob references. - Internal identity is by heap-stable `Rep` pointers, so column/result/batch pointers stay valid across moves with no move-time fix-ups. ## Scope / current limitations - Resolves **whole columns**; byte ranges are served by slicing the whole column for now (so `length`/`offset`/`force_verify` don't yet save I/O or skip checksums). Single column family. Synchronous. `kBlockCacheTier` on the originating call yields `Incomplete` on a blob-cache miss. - User-defined timestamps: entities aren't writable under UDT, so `GetEntityLazy` is at parity with `GetEntity` (same read-timestamp validation; plain values return a single default column). - Planned follow-ups: partial (byte-range) blob reads that skip checksum/cache- fill, cross-key coalescing, and asynchronous execution. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15012 Test Plan: - **`db/wide/db_lazy_entity_test.cc` (15 tests, all passing):** enumeration with no blob I/O; single range and whole-column resolution; `MultiResolve` of many blobs and mixed ranges/columns (incl. the `std::vector` overload); cross-key batch resolution; result-outlives-call / pin survives move; unpulled columns never read; `max_open_files == -1` requirement; `kBlockCacheTier` ⇒ `Incomplete`; foreign/null column ⇒ `InvalidArgument` (single-entity and batch); UDT parity with `GetEntity`; `StackableDB` forwarding; and read-only and secondary instances. - **db_stress:** `GetEntityLazy` / `MultiGetEntityLazy` run as a differential against eager `GetEntity` / `MultiGetEntity` (gated by `--lazy_entity_read_one_in`, which only takes effect under `--open_files=-1`), including coverage on read-only (`--open_read_only_one_in`) and secondary (`--test_secondary`) instances. Smoke-run (`EXIT=0`, "Verification successful") with integrated blob files + `--use_put_entity_one_in`. - `make format-auto` clean; builds `ASSERT_STATUS_CHECKED`-clean. Reviewed By: xingbowang Differential Revision: D113792365 Pulled By: pdillinger fbshipit-source-id: cb73fc845fe0e20ed352d697e552e7b47b2eb9cf	2026-08-05 22:21:40 -07:00
Josh Kang	14ecf7f13c	Route single-blob reads through BlobFetcher (#15046) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15046 `BlobFetcher` is the policy boundary for single-blob reads, but `Version::Get()` and primary direct-write postprocessing still called lower-level backends directly. This duplicated backend selection and could bypass future fetcher policy or decorators. Route those paths through their existing `VersionBlobFetcher` and remove the redundant Slice-based `Version::GetBlob()` overload. This is behavior-preserving: each fetcher delegates to the same Version or direct-write backend with the same `ReadOptions`, cache, and lifetime context. Reviewed By: xingbowang Differential Revision: D114616978 fbshipit-source-id: 0d94cc389138f27791de636e2cca585e51cb142a	2026-08-03 12:59:37 -07:00
Darkhan Imangaliyev	f9df618e4d	Api to checkpoint only selected set of column families (#15019) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15019 RocksDB's Checkpoint API currently copies all column families. For use cases that only need a subset, this is wasteful. This diff adds a new `CreateCheckpoint(dir, column_families, ...)` overload that: - Validates input handles (rejects null/foreign handles, coalesces duplicates, always includes default CF). - Filters GetLiveFilesStorageInfo to capture only the requested CFs' table/blob files. - Post-processes the copied MANIFEST by appending kColumnFamilyDrop records for excluded CFs, so the checkpoint opens without referencing missing files. - An empty column_families vector falls through to the existing whole-DB path. Reviewed By: xingbowang, pdillinger Differential Revision: D113963616 fbshipit-source-id: d1becbbb9528dd13c2dc314165880b76123757f0	2026-07-30 16:16:58 -07:00
Josh Kang	2c77c32ac9	Add Get and MultiGet sync/async plumbing (#14934) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14934 First diff in the async read stack: pure plumbing, no behavior change. Both the point `Get` path and the batched `MultiGet` path gain synchronous/coroutine twins generated by `DEFINE_SYNC_AND_ASYNC`; `DB::GetAsync` and `DB::MultiGetAsync` become the public callback-based async entry points with synchronous default implementations. For `Get`, the existing `DBImpl::Get` / `GetImpl` bodies are relocated into the sync/async wrapper. For `MultiGet`, the same wiring continues through the DBImpl, version, and table-cache layers. The generated coroutine variants still run the existing synchronous logic. This keeps the refactor mechanical while preserving dispatch semantics: the coroutine side remains virtual at the same `GetImpl` boundary as the synchronous path, read-only and secondary DBs use generated sync/coroutine overrides instead of falling back to base `DBImpl`, and `StackableDB` leaves async `Get` / `MultiGet` to the base wrappers so wrapper-specific `Get` / `MultiGet` behavior is preserved. For ABI compatibility and avoiding MACRO blocks in public headers, the asynchronous APIs are callback based instead of coroutine based. Internally it will be developed using coroutines, and applications can also develop their own coroutine wrappers. This is meant to make the later changes to actually enable coroutine-backed async queries much easier to review. Benchmarks: Compared `db_bench` with and without this diff using: `db_bench --benchmarks=fillseq,readrandom --num=1000000 --reads=5000000 --key_size=16 --value_size=100 --seed=12345 --db=<tmp>` | Benchmark | Without diff avg | With diff avg | Delta | | `fillseq` | 4.076 us/op | 4.165 us/op | +2.2% | | `readrandom` | 3.939 us/op | 3.947 us/op | +0.2% | No meaningful local `db_bench` regression was observed. Reviewed By: xingbowang Differential Revision: D109346179 fbshipit-source-id: 0dee1239f49099b67c11c63426204466e963f903	2026-07-23 09:56:30 -07:00
Maciej Szeszko	02941016d0	Fix seqno sync race with WriteUnprepared (#14864) ... Summary: # Summary Pull Request resolved: https://github.com/facebook/rocksdb/pull/14864 With `two_write_queues`=`true`, `WRITE_UNPREPARED` can allocate sequence numbers through `FetchAddLastAllocatedSequence()` before those numbers are published through `SetLastSequence()`. Error recovery can then create new memtable/WAL boundaries from a stale `LastSequence()`, which can later surface as "sequence number going backwards" corruption. Fix this by syncing `LastSequence()` to `LastAllocatedSequence()` at the recovery entry point after draining both write queues, and again at the recovery flush fence after `FlushAllColumnFamilies()` has released/reacquired the DB mutex and before `SwitchMemtable()` consumes `LastSequence()`. Apply the same fence to atomic recovery flushes. # Perf comparative results TLDR; no visible regression. I ran a deterministic local recovery benchmark to check whether the new two-write-queue recovery fence adds measurable recovery latency. ## Workload (P2385173320) - Optimized build. - `TransactionDB` with `WRITE_UNPREPARED`. - `two_write_queues=true`. - Each iteration dirties the memtable with WUP transactions, forces a retryable flush IO error through `FaultInjectionTestFS`, re-enables the filesystem, then measures `DB::Resume()`. - 50 warmup recoveries + 500 measured recoveries per run. ## Why this is relevant This targets the changed recovery path directly rather than measuring generic write throughput. The change only runs during recovery, so the useful signal is recovery latency around `ResumeImpl()` and the recovery flush fence. ## Code path exercised - `DB::Resume()` - `ErrorHandler::RecoverFromBGError(true)` - `DBImpl::ResumeImpl()` - new `two_write_queues_` recovery queue fence - non-atomic recovery `FlushMemTable()` fence before `SwitchMemtable()` This does not cover atomic flush recovery or a writer-backlog stress case. Comparative results: | Revision | Attempt | resume_p50_us | resume_p95_us | Retryable BG errors | | Parent | | 1555.75 | 2092.22 | 550 | | Fix | 1 | 1537.68 | 1905.65 | 550 | | Fix | 2 | 1475.10 | 1883.26 | 550 | | Fix | 3 | 1617.52 | 1993.97 | 550 | Reviewed By: xingbowang, anand1976 Differential Revision: D108946310 fbshipit-source-id: 75fcb4bf0c9a932b5fbed7e934c7f14d981a1bf2	2026-06-20 22:57:11 -07:00
Josh Kang	7785264e5a	Make file opens multi-threaded for commit external sst file ingestion (#14853) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14853 `IngestExternalFileOptions` now exposes `file_opening_threads` so external file ingestion can open table readers for newly committed SSTs in parallel. The requested thread count is carried from each ingestion job into commit and then into manifest `LogAndApply`, where `VersionBuilder::LoadTableHandlers()` uses it while installing the ingested files. This is intended for workloads that ingest many external SSTs in one call, where serial table-reader opens can dominate commit latency. The ingestexternalfile benchmark now has a flag for this option, and `db_stress` now creates multiple data SSTs per ingest operation so stress runs cover multi-file ingestion, file-info ingestion, and the parallel file-opening path together. Benchmarks Ran `db_bench --benchmarks=ingestexternalfile --use_existing_db=0 --num=2200000 --compression_type=none --statistics --use_direct_reads=true --ingest_external_file_batch_size=200 --ingest_external_file_num_batches=1 --ingest_external_file_use_file_info=true --ingest_external_file_fill_cache=false` while varying `--ingest_external_file_file_opening_threads`. The generated SST files were about 256 MiB each. | `file_opening_threads` | `rocksdb.ingest.external.file.prepare.micros` | `rocksdb.ingest.external.file.run.micros` | File opens | | 1 | 16,150 us | 327,136 us | 200 | | 32 | 16,665 us | 38,349 us | 200 | At the same ~256 MiB SST size, 2,000 files is roughly 500 GiB of SST data. A straight 10x extrapolation of the 200-file ingest histograms puts prepare+run at about 3.43s with `file_opening_threads=1` and 0.55s with `file_opening_threads=32`; run alone is about 3.27s vs. 0.38s. Reviewed By: pdillinger Differential Revision: D108347952 fbshipit-source-id: 49efa48524df64cace010b881030f63f771cbb95	2026-06-15 14:45:29 -07:00
Steph Pontikes	e18d41e08c	lazily intialize iterators (#14772) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14772 Updated the iterator creation scheme to happen lazily (on request) as oppsed to eagerly. this allows us to prune the iterator tree structure at the time of requesting iterator preparation as opposed to creation, and allows pruning to become an implementation detail. Version now skips non-overlapping SST levels and files before adding children to the iterator tree, returns direct table iterators when a level has a single matching file, and uses pruned LevelIterator instances when multiple files in one non-L0 level match. The overload no longer prepares iterators during creation; callers that need prepared multiscan execution still call Prepare explicitly after construction, and MultiScan does that itself. Benchmark: ran `db_bench` in opt mode for the base revision and this diff, with `fillseq,compact,levelstats,multiscanrandom`, `--num=1000000`, `--reads=10000000`, single thread, fixed seeds, `--multiscan_use_async_io=false`, and `--use_multiscan=true`. Both A and B had exactly one SST file and no memtable/L0 data (`L0: 0 files`, `L1: 1 file, 61 MB`). `multiscanrandom` creates `MultiScanArgs` and calls `NewMultiScan(...)`, which reaches the new `NewIterator(..., scan_opts)` pruning path in this diff. ``` seed base A pruning B delta 424242 21824.333 17693.333 -18.9% 424243 24042.014 19424.056 -19.2% 424244 22424.974 17636.910 -21.4% 424245 22404.213 18612.840 -16.9% ``` Average: base `22673.9 us/op`, pruning `18341.8 us/op`, about `19.1%` faster. Reviewed By: xingbowang Differential Revision: D104904298 fbshipit-source-id: a742106a1d5813fb795a39eeeb35f8cddc02e886	2026-06-10 17:02:47 -07:00
zaidoon	7affaee1c4	Add use_direct_io_for_compaction_reads option (#14743) ... Summary: Adds a new `DBOption use_direct_io_for_compaction_reads` (default false). When on, compaction-input SST files are opened with `O_DIRECT` so the sequential read-once data from compaction doesn't pollute the OS page cache and evict the hot user-read working set. User reads keep going through the buffered fast path. This protects user-read tail latency on write-heavy workloads without forcing user reads onto the existing global `use_direct_reads` knob (which pays in throughput and P50 — see the bench below). The interesting bit is that just flipping the FileOptions returned by `FileSystem::OptimizeForCompactionTableRead` doesn't actually trigger `O_DIRECT` at the kernel level. The TableCache (and `FileMetaData::pinned_reader`) is already holding buffered handles opened at flush time or at `DB::Open` via `LoadTableHandlers`. When compaction asks for an iterator, it gets back the cached buffered handle and the kernel never sees the `O_DIRECT` flag. So this PR also adds a small bypass path: - `TableCache::FindTable` / `NewIterator` learn a `open_ephemeral_table_reader` mode. When set, the pinned-reader fast path and the shared cache are skipped, `GetTableReader` is called directly with the caller's FileOptions, and ownership of the freshly opened TableReader is handed back via a `unique_ptr`. The iterator takes ownership via `RegisterCleanup` and frees the reader on destruction. - `VersionSet::MakeInputIterator` and `LevelIterator` plumb the flag through both L0 and L1+ compaction-input paths. - `CompactionJob::ProcessKeyValueCompaction` turns the bypass on when `use_direct_io_for_compaction_reads` is set, the global `use_direct_reads` is off, and `OptimizeForCompactionTableRead` produced `use_direct_reads=true` in the compaction-read FileOptions. The option is opt-in: when off, nothing changes for existing users. When on, only the compaction-input opens take the bypass path; user reads keep hitting the TableCache and the buffered fast path normally. There's also a small db_bench helper in the same PR: a new `--bgwriter_num` flag that lets the writer thread in `readwhilewriting` (and the other "while writing" variants) spread its puts across `[0, bgwriter_num)` instead of `[0, num)`. Without this the readers and writer share a key range and you can't have both a hot read subset and meaningful compaction work — this lets you have both. ### Benchmark Setup: Ubuntu 24.04 (kernel 7.0.5, OrbStack Linux VM on Apple Silicon), 14 vCPUs, virtio-blk disk, btrfs. MGLRU disabled (`echo 0 > /sys/kernel/mm/lru_gen/enabled`) so the kernel uses the classic active/inactive LRU. 14 GB DB (3.5M keys × 4 KB values), no compression. Each measurement run is pinned to a 1 GB cgroup via `systemd-run --scope -p MemoryMax=1G -p MemorySwapMax=0`. Page cache is dropped between configs. db_bench is Release build. Workload: `readwhilewriting` for 120s. 4 reader threads doing random reads over a hot key subset, plus 1 writer thread spreading overwrites across the full 3.5M-key keyspace (via `--bgwriter_num=3500000`) throttled at 200 MB/s, so there's continuous compaction running while the readers go. The size of the hot reader subset relative to available page cache controls how visible the optimization is. The Cassandra blog ([Lightfoot 2026](https://lightfoot.dev/direct-i-o-for-cassandra-compaction-cutting-p99-read-latency-by-5x/)) documented the same thing: biggest wins when the hot set is big enough to actually compete for cache, smaller wins when the hot set trivially fits, neutral when the hot set is way bigger than cache. So I ran two hot-set sizes. #### Small hot set: ~30 MB (~3% of the 1 GB cgroup) — N=5 iterations, mean (CV) `--num=7500`. The hot set is small enough that the page cache holds it without much trouble even under compaction, so the wins here are real but on the modest side. | Config | Throughput (ops/s) | Read P50 (µs) | Read P99 (µs) | Read P99.9 (µs) | Read P99.99 (µs) | |---|---|---|---|---|---| | buffered (default) | 233,477 (8.2%) | 16.09 | 82.24 | 721.0 | 2,102.5 | | direct_compaction_writes_only (existing knob alone) | 287,405 (2.8%) — **+23.1%** | 13.00 (−19.2%) | **66.77 (−18.8%)** | 553.9 (−23.2%) | 1,787.6 (−15.0%) | | direct_compaction_read_only (new knob alone) | 250,669 (2.4%) — +7.4% | 14.16 (−12.0%) | 102.99 (+25.2%) | 689.8 (−4.3%) | 1,801.3 (−14.3%) | | direct_compaction_read_write (new + existing, recommended) | 277,920 (3.3%) — **+19.0%** | **12.99 (−19.3%)** | 84.23 (+2.4%) | 613.4 (−14.9%) | **1,738.2 (−17.3%)** | | use_direct_reads=true (existing global) + write-side | 249,014 (2.5%) — +6.7% | 15.95 (−0.9%) | 68.78 (−16.4%) | **450.8 (−37.5%)** | 1,814.5 (−13.7%) | CV is 2.4–3.3% on the optimized configs (8.2% on buffered), so the deltas are real. With a hot set this small, the existing `use_direct_io_for_flush_and_compaction` knob is already doing most of the work — the new flag's main extra contribution here is P99.99 (combined wins it by ~2 points vs writes-only-alone). Worth noting: the new flag *alone* (without the existing write-side flag) improves P99.99 but regresses P99 by 25% on this small-hot-set workload, because direct compaction reads lose kernel readahead and compaction-output writes are still hitting the page cache. That regression goes away once you combine with the existing write-side flag, or once the hot set is bigger (see next table). So if you're using just one knob, use the existing one. If you're using this PR's flag, pair it with `use_direct_io_for_flush_and_compaction=true`. #### Larger hot set: ~400 MB (~40% of cache) — N=5 iterations, mean (CV) `--num=100000`. This is the case the Cassandra blog calls out — hot set big enough to actually fight compaction for cache. Their analogous setup (1M hot partitions, ~33% hot/cache) reported 1.93× p99 improvement. Numbers here are the headline: | Config | Throughput (ops/s) | Read P50 (µs) | Read P99 (µs) | Read P99.9 (µs) | Read P99.99 (µs) | |---|---|---|---|---|---| | buffered (default) | 68,959 (7.7%) | 44.81 | 541.22 | 2,225.2 | 11,334.5 | | direct_compaction_writes_only (existing knob alone) | 73,973 (10.3%) — +7.3% | 42.22 (−5.8%) | 456.27 (−15.7%) | 2,016.9 (−9.4%) | 9,190.0 (−18.9%) | | direct_compaction_read_only (new knob alone) | 84,337 (2.3%) — +22.3% | 38.66 (−13.7%) | 386.97 (−28.5%) | 1,644.8 (−26.1%) | 4,837.9 (−57.3%, 2.34×) | | direct_compaction_read_write (new + existing, recommended) | **104,923 (8.4%) — +52.2%** | **34.26 (−23.5%)** | **290.97 (−46.2%)** | **1,143.4 (−48.6%)** | **3,080.3 (−72.8%, 3.68×)** | | use_direct_reads=true (existing global) + write-side | 71,598 (9.1%) — +3.8% | 51.33 (+14.5%) | 297.91 (−45.0%) | 1,663.6 (−25.2%) | 6,530.0 (−42.4%) | Combined config gets a 3.68× p99.99 win, 1.86× p99, p50 down 23%, throughput up 52%. Same shape as the Cassandra blog's 1.93× p99 result — the improvement just lands at deeper percentiles for us because RocksDB's baseline data path is roughly 40× faster than Cassandra's (their buffered p99 was 35 ms, ours is 0.54 ms), so the cache-miss tail is further out. A few things worth calling out from this table: - The new flag is doing real work on top of the existing write-side flag here, not just shifting things around. Combined throughput is +42% over `direct_compaction_writes_only` alone, and combined p99.99 is 3× better. The existing knob alone gives a fairly modest +7% throughput / -19% p99.99 in this case — there's a clear gap that the new flag fills. - The new flag *alone* (no existing write-side flag) is also a real improvement here: +22% throughput, p99.99 down 57%. The P99 regression we saw in the small-hot-set case is gone, because the cache-protection effect now dominates the lost-readahead cost. - `use_direct_reads=true` (the existing global flag) actually regresses P50 by 14.5% in this workload — taking user reads off the page cache hurts you when the hot data could have been cached. It also gets the worst throughput of any direct config. It's not an equivalent way to get these gains. ### `compaction_readahead_size` matters when this flag is on Direct I/O bypasses kernel readahead, so RocksDB's own `DBOptions::compaction_readahead_size` becomes the only prefetch the iterator has. The default of 2 MB is enough and real users will get it automatically. **But `db_bench`'s `--compaction_readahead_size` CLI default is 0**, which defeats prefetch and makes direct compaction look slower than it actually is. If you're reproducing the numbers above, pass `--compaction_readahead_size=2097152` (or larger). - Recommended production config is `use_direct_io_for_compaction_reads=true` + `use_direct_io_for_flush_and_compaction=true`. Strongest configuration at every percentile and throughput in both benches. - The new flag is the read-side counterpart to `use_direct_io_for_flush_and_compaction`, which handles compaction-write cache pollution. They address different sources of pollution and compose. The gap between "combined" and "writes-only-alone" is 17 percentage points on p99.99 in the small-hot-set bench and 54 points in the larger one, so the new flag is contributing real value, especially as the hot set grows. - The new flag alone is also a real improvement when the hot set is big enough to compete with cache (+22% throughput, 2.34× p99.99 in the larger-hot-set bench). On a very small hot set it improves p99.99 but regresses p99, so pairing with the existing write-side flag is safer. - The benefit is workload-dependent. Small hot sets get modest tail-latency wins. Hot sets sized to actually compete for cache get the big multi-percentile wins shown above. Hot sets bigger than cache (not benched here but covered in the Cassandra blog) see no change either way — every read misses regardless. ### Reproducing Any Linux host (or a Linux VM on macOS via OrbStack / Multipass / lima): ```bash sudo apt-get install -y build-essential clang cmake git pkg-config \ libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev cmake -DCMAKE_BUILD_TYPE=Release -DPORTABLE=1 -DWITH_GFLAGS=1 -DWITH_TESTS=0 .. make -j db_bench echo 0 | sudo tee /sys/kernel/mm/lru_gen/enabled ``` Build the source DB once, unrestricted memory: ```bash ./db_bench --benchmarks=fillrandom,compact,waitforcompaction,stats \ --db=/path/to/source_db --num=3500000 --key_size=16 --value_size=4096 \ --write_buffer_size=16777216 --target_file_size_base=16777216 \ --max_background_jobs=4 --compression_type=none --cache_size=4194304 \ --max_bytes_for_level_base=67108864 --disable_wal=1 --sync=0 ``` For each config, copy `source_db -> scratch_db`, run `sync && echo 3 > /proc/sys/vm/drop_caches`, then: ```bash sudo systemd-run --scope -p MemoryMax=1G -p MemorySwapMax=0 \ ./db_bench --use_existing_db=1 \ --benchmarks=readwhilewriting,stats --db=/path/to/scratch_db \ --threads=5 --duration=120 --statistics=true --histogram=1 \ --num=7500 --bgwriter_num=3500000 \ --key_size=16 --value_size=4096 \ --write_buffer_size=16777216 --target_file_size_base=16777216 \ --max_background_jobs=4 --compression_type=none \ --cache_size=4194304 --open_files=200 \ --skip_stats_update_on_db_open=true \ --max_bytes_for_level_base=67108864 \ --benchmark_write_rate_limit=209715200 \ --compaction_readahead_size=2097152 \ --rate_limiter_bytes_per_sec=0 \ --use_direct_reads={true|false} \ --use_direct_io_for_compaction_reads={true|false} \ --use_direct_io_for_flush_and_compaction={true|false} ``` For the larger hot-set table, change `--num=7500` to `--num=100000`. The five configs in the tables: - `buffered`: all three flags false. - `direct_compaction_writes_only`: `use_direct_io_for_flush_and_compaction=true`, the other two false. This is what users have today without this PR. - `direct_compaction_read_only`: `use_direct_io_for_compaction_reads=true`, the other two false. - `direct_compaction_read_write`: `use_direct_io_for_compaction_reads=true`, `use_direct_io_for_flush_and_compaction=true`, `use_direct_reads=false`. **Recommended.** - `direct_all`: `use_direct_reads=true`, `use_direct_io_for_flush_and_compaction=true`, `use_direct_io_for_compaction_reads=false`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14743 Reviewed By: pdillinger Differential Revision: D108017601 Pulled By: xingbowang fbshipit-source-id: 4039d490d7e77b476db7a477a2f3d24738db6336	2026-06-09 17:02:53 -07:00
Xingbo Wang	4707775ae9	Fix GetContext status propagation and blob-backed wide-column merge operands (#14640) ... Summary: - propagate lower-level read and merge failures through `GetContext` via `read_status`, so `Get` and `GetEntity` preserve the original error instead of synthesizing `Corruption` when blob-backed reads or merge resolution fail - teach `GetMergeOperands` to resolve blob-backed default columns from wide-column entities, covering both the direct base-value path and the merge-plus-base path - add regression coverage for blob-read IO errors during `Get`/`GetEntity` merge resolution and for `GetMergeOperands` on blob-backed wide-column entities - fix the `DBFlushTest.MemPurgeCorrectLogNumberAndSSTFileCreation` test race by waiting for flush callbacks and cleaning up sync points ## Testing - `make db_blob_basic_test -j14` - `/usr/bin/perl -e 'alarm shift; exec ARGV' 60 ./db_blob_basic_test --gtest_filter='DBBlobBasicTest/DBBlobBasicIOErrorTest.GetBlob_IOError/*:DBBlobBasicTest/DBBlobBasicIOErrorTest.GetEntityMergeWithBlobBaseIOError/*'` ## Task T265824017, T265415808 Pull Request resolved: https://github.com/facebook/rocksdb/pull/14640 Reviewed By: anand1976 Differential Revision: D101690700 Pulled By: xingbowang fbshipit-source-id: 2b6fc357b37a01efa72a2d54dcff55be8992f42a	2026-05-12 15:29:27 -07:00
Peter Dillinger	795f3bd61f	Fix check-sources.sh non-ASCII check and remove non-ASCII from sources (#14729) ... Summary: The non-ASCII character check in check-sources.sh used git grep -P (Perl regex), which requires git compiled with PCRE support. On systems without it, the command fails with exit code 128, which is != 1 (no match), so the check always reported a violation -- effectively dead. Even in CI where git has PCRE2 support, the check was silently broken: git grep -P uses PCRE2 in UTF mode by default, which interprets [\x80-\xFF] as a Unicode codepoint range (U+0080 to U+00FF). Characters like em-dash (U+2014), arrows (U+2192), and math symbols (U+2248, etc.) fall outside that range and were not detected. Only Latin-1 Supplement characters (U+0080-U+00FF) would have been caught. Replace with LC_ALL=C git grep using bash $'[\x80-\xff]' literal byte range, which works with basic regex in the C locale, and replace all non-ASCII characters in non-excluded source files: - em-dash to -- - arrow to -> - math symbols to ASCII equivalents (~=, <=, >=) - box-drawing characters to ASCII art Also exclude .github/ from the check, as scripts there can use non-ascii without disrupting RocksDB builds on non-UTF-8 systems. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14729 Test Plan: manual / CI (make check-sources passes clean) Reviewed By: hx235 Differential Revision: D104692574 Pulled By: pdillinger fbshipit-source-id: 1d884c21056dcd83558b825a04b867f1c08e3f45	2026-05-11 17:02:22 -07:00
Anand Ananthabhotla	c734b7cc60	Add reuse_manifest_on_open DBOption (#14704) ... Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14704 Add an immutable DBOption `reuse_manifest_on_open` (default false). When enabled, `DB::Open` can keep using the recovered MANIFEST for the first post-open metadata update instead of rebuilding a fresh MANIFEST, which can reduce warm-open latency for DBs whose MANIFEST is expensive to regenerate. Reuse is still best-effort. If RocksDB cannot safely resume appending to the recovered MANIFEST, it falls back to the existing fresh-MANIFEST path. The option is also disabled under `best_efforts_recovery`. This diff also teaches the reopened MANIFEST writer to adopt the existing file size before appending, documents the small-`max_manifest_file_size` caveat for the reused path, and keeps the full warm-reopen composition working with `optimize_manifest_for_recovery`. Reviewed By: hx235, pdillinger Differential Revision: D103568447 fbshipit-source-id: f4f5c35ea3ef0b80a0d52d94be40c6bd11505999	2026-05-07 13:10:59 -07:00
Xingbo Wang	af4e32945b	Blob direct write v1: write-path blob separation with partitioned files (reduced scope) (#14535) ... Summary: This PR introduces **blob direct write v1**, a reduced-scope write-path optimization where large values (>= `min_blob_size`) are written directly to blob files during `Put()` and replaced in the memtable with compact `BlobIndex` references. This avoids holding full values in memory until flush time. ### Motivation In the existing BlobDB architecture, values are written to the WAL and memtable in their full form and separated into blob files only at flush time. This means: - Large values are held in memory twice (raw in memtable + blob file at flush) - Blob I/O is serialized through a single flush thread per column family Blob direct write addresses both: values leave the write path as small `BlobIndex` references, and multiple **partitions** (configurable via `blob_direct_write_partitions`) allow concurrent blob writes with independent locks. ### Design (v1 — single-writer, WAL-disabled, reduced scope) The v1 design intentionally keeps scope narrow for correctness and reviewability: - **Single writer thread assumption**: no concurrent writes to the same partition file. One logical writer serializes the batch. - **WAL-disabled**: direct-write blob files are only registered in MANIFEST at flush time. WAL replay cannot recover unregistered blob references, so WAL is disabled for this v1. - **Flush-on-write**: each `AddRecord` call flushes to the OS immediately. - **FIFO generation batching**: each memtable switch creates one generation batch. Direct-write files for that memtable are sealed and registered atomically when the batch is flushed to MANIFEST. - **Round-robin partitions**: blob writes are distributed across `blob_direct_write_partitions` files using an atomic counter. ### New components | Component | Description | |---|---| | `BlobFilePartitionManager` | Owns N partition files per CF. Manages open/seal/register lifecycle tied to memtable generations. | | `BlobWriteBatchTransformer` | A `WriteBatch::Handler` that rewrites qualifying `Put` values as `BlobIndex` entries before the batch enters the write group. | ### Write path integration 1. `DBImpl::WriteImpl` calls `BlobWriteBatchTransformer::TransformBatch` before entering the writer group (for default write path), or before joining the batch group (for pipelined/unordered write). 2. Values >= `min_blob_size` are written to a partition file; the key is stored with a `BlobIndex` in the transformed batch. A rollback guard marks blob bytes as initial garbage if the write fails. 3. On `SwitchMemtable`, `RotateCurrentGeneration` moves active partitions into the next immutable batch. 4. `FlushMemTableToOutputFile` / `AtomicFlushMemTablesToOutputFiles` call `PrepareFlushAdditions` to seal partition files and collect `BlobFileAddition` + `BlobFileGarbage` entries registered to MANIFEST alongside the flush. 5. Shutdown paths (`CancelAllBackgroundWork`, `WaitForCompact` with `close_db=true`) force-flush all CFs with active direct-write managers to ensure blob files are registered before close. ### Read path - **Get/MultiGet**: `MaybeResolveBlobForWritePath` resolves `BlobIndex` references found in memtable or immutable memtable via `BlobFilePartitionManager::ResolveBlobDirectWriteIndex`, which first checks manifest-visible state and falls back to direct blob-file reads via `BlobFileCache`. - **Iterator**: `DBIter::BlobReader` is extended with a `BlobFilePartitionManager*` to resolve direct-write blob indexes during iteration. The unified `ResolveBlobDirectWriteIndex` path handles both manifest-visible and not-yet-flushed files. ### New options | Option | Default | Description | |---|---|---| | `enable_blob_direct_write` | `false` | Enable write-path blob separation for this CF. Requires `enable_blob_files = true`. Not dynamically changeable. | | `blob_direct_write_partitions` | `1` | Number of parallel partition files per CF. Not dynamically changeable. | ### Feature incompatibilities (reduced v1 scope) The following features are *not supported* when `enable_blob_direct_write = true`, and are enforced both in `db_stress_tool` validation and `db_crashtest.py` sanitization: **Write model constraints:** - `threads` must be 1 (single writer assumption) - `allow_concurrent_memtable_write` = 0 - `enable_pipelined_write` = 0 (transformation done before batch group, but pipelined path supported with pre-transform) - `two_write_queues` = 0 - `unordered_write` = 0 (transformation done before batch group, but unordered path supported with pre-transform) **WAL and recovery:** - `disable_wal` = 1 (required — WAL replay of unregistered blob files is out of v1 scope) - `best_efforts_recovery` = 0 - `reopen` = 0 (no crash-restart with WAL replay) - All WAL-related stress features disabled: `manual_wal_flush_one_in`, `sync_wal_one_in`, `lock_wal_one_in`, `get_sorted_wal_files_one_in`, `get_current_wal_file_one_in`, `track_and_verify_wals`, `rate_limit_auto_wal_flush`, `recycle_log_file_num` **Blob GC and dynamic options:** - `use_blob_db` = 0 (stacked BlobDB not supported) - `allow_setting_blob_options_dynamically` = 0 - `enable_blob_garbage_collection` = 0 - `blob_compaction_readahead_size` = 0 - `blob_file_starting_level` = 0 **Unsupported value types and APIs:** - Merge (`use_merge`, `use_full_merge_v1`) — merge values pass through untransformed - Entity APIs (`use_put_entity_one_in`, `use_get_entity`, `use_multi_get_entity`, `use_attribute_group`) - `use_timed_put_one_in` - User-defined timestamps (`user_timestamp_size`, `persist_user_defined_timestamps`, `create_timestamped_snapshot_one_in`) - Transactions (`use_txn`, `use_optimistic_txn`, `test_multi_ops_txns`, `commit_bypass_memtable_one_in`) — though `WriteCommittedTxn::CommitInternal` falls back from bypass-memtable to normal path when BDW is active - `IngestWriteBatchWithIndex` returns `NotSupported` - `inplace_update_support` = 0 **Fault injection:** - All write/read/metadata fault injection disabled (`sync_fault_injection`, `write_fault_one_in`, `metadata_write_fault_one_in`, `read_fault_one_in`, `metadata_read_fault_one_in`, `open_*_fault_one_in`) **Infrastructure/snapshot APIs:** - `remote_compaction_worker_threads` = 0 - `test_secondary` = 0 - `backup_one_in` = 0 - `checkpoint_one_in` = 0 - `get_live_files_apis_one_in` = 0 - `ingest_external_file_one_in` = 0 - `ingest_wbwi_one_in` = 0 ### Tests - `db/blob/db_blob_basic_test.cc`: ~660 lines of new direct-write unit tests covering basic put/get, multi-partition, flush/compaction, recovery, and error injection. - `db/blob/blob_file_cache_test.cc`: ~96 lines of new tests for direct-write blob file cache behavior. - `db/write_batch_test.cc`: ~96 lines of tests for WriteBatch with blob index entries. - `utilities/transactions/transaction_test.cc`: verifies transaction commit path falls back correctly with direct write enabled. - `db_stress_tool/`: full stress test support with `--enable_blob_direct_write` and `--blob_direct_write_partitions` flags, integrated into `db_crashtest.py` with 10% random selection alongside regular blob params. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14535 Test Plan: ``` make -j128 db_blob_basic_test && ./db_blob_basic_test make -j128 blob_file_cache_test && ./blob_file_cache_test make -j128 write_batch_test && ./write_batch_test make -j128 transaction_test && ./transaction_test make -j128 check ``` Stress test: ``` python3 tools/db_crashtest.py blackbox --enable_blob_direct_write=1 \ --enable_blob_files=1 --blob_direct_write_partitions=4 \ --disable_wal=1 --threads=1 ``` Reviewed By: pdillinger Differential Revision: D98766843 Pulled By: xingbowang fbshipit-source-id: 1577653826913a59d05680a87bce5534ac5a5e69	2026-04-02 07:31:56 -07:00
Josh Kang	5db0603613	Read-triggered compactions (#14426) ... Summary: Add read-triggered compaction, a new feature that reduces read amplification by compacting SST files that receive high read traffic. When an SST file's read frequency (`num_reads_sampled / file_size`) exceeds a configurable threshold, it is marked for compaction to a lower level. The feature introduces two new options: a CF option `read_triggered_compaction_threshold` (default 0, disabled) and a DB option `max_periodic_compaction_trigger_seconds` (default 43200s) that controls how often the background thread re-evaluates compaction scores on quiet databases. Both options are dynamically changeable. Lowering `max_periodic_compaction_trigger_seconds` does add some overhead, but generally is minimal, so running this every couple of minutes in a production environment seems fairly reasonable. ## Key changes - **New CF option `read_triggered_compaction_threshold`** (`advanced_options.h`): When positive, files with `reads_per_byte > threshold` are marked for compaction. Files at the last non-empty level are skipped (bottommost compaction handles those separately). Marked files are sorted by hotness (reads_per_byte descending). - **New DB option `max_periodic_compaction_trigger_seconds`** (`options.h`): Replaces the hardcoded 12-hour ceiling in `ComputeTriggerCompactionPeriod()`. Essential for read-triggered compaction on quiet DBs since there are no writes to trigger score re-evaluation. - **Leveled compaction picker** (`compaction_picker_level.cc`): Adds read-triggered as the lowest-priority compaction reason in `SetupInitialFiles()`, using the existing `PickFileToCompact` helper. - **Universal compaction picker** (`compaction_picker_universal.cc`): Adds `PickReadTriggeredCompaction` as lowest priority. Refactors shared "find output level + compute overlapping inputs + create Compaction" logic from both `PickDeleteTriggeredCompaction` and `PickReadTriggeredCompaction` into `BuildCompactionToNextLevel`, handling both single-level and multi-level universal cases. - **Periodic trigger integration** (`db_impl.cc`): `TriggerPeriodicCompaction` now also fires for CFs with `read_triggered_compaction_threshold > 0`, even without time-based compaction configured. - **Stress test & db_bench support**: Both `db_stress` and `db_bench` support the new options. `db_crashtest.py` randomly enables read-triggered compaction and sets a short periodic trigger interval when enabled. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14426 Test Plan: **Unit tests**: - `compaction_picker_test` — 7 new tests: `ReadTriggeredCompactionDisabled`, `ReadTriggeredCompactionBelowThreshold`, `ReadTriggeredCompactionAboveThreshold`, `NeedsCompactionReadTriggered`, `ReadTriggeredPicksFile`, `UniversalReadTriggeredCompaction`, `ReadTriggeredSkipsLastLevel`, `UniversalReadTriggeredNoPickWhenNotMarked` - `db_compaction_test` — `ReadTriggeredCompaction` integration test verifying end-to-end behavior with sync points - Stress test coverage **Stress test**: ``` make V=1 -j "CRASH_TEST_EXT_ARGS=--duration=600 --max_key=2500000 --max_compaction_trigger_wakeup_seconds=10 --read_triggered_compaction_threshold=0.0001 --interval=600" blackbox_crash_test ``` - confirmed read triggered compactions from LOGS **Benchmark** (`db_bench`): Setup: 5M keys (100B values, 16B keys), leveled compaction, 5 levels, 4MB target file size. DB fully compacted, then 2M overlapping keys written without compaction to create L0/L1 overlap (82 files, ~294MB). LSM shape change during readrandom with read-triggered compaction: ``` BEFORE: L0=9 files (15MB), L1=4 (16MB), L2=20 (69MB), L3=49 (194MB) — 82 files, 294MB AFTER: L3=66 files (223MB) ``` | Benchmark | Config | avg ops/s | % change | |-----------|--------|-----------|----------| | readrandom (8 threads, 5M reads) | baseline (threshold=0) | 1,086,965 | — | | readrandom (8 threads, 5M reads) | threshold=0.000001, trigger=5s | 1,453,697 | **+33.7%** | Reviewed By: xingbowang Differential Revision: D97838716 Pulled By: joshkang97 fbshipit-source-id: a21fcb270c7fadd4f78d98b9c821982f220dd3f0	2026-03-27 14:43:52 -07:00
Danny Chen	b6f498b2c9	Add verify_manifest_content_on_close option (#14451) ... Summary: Add a new mutable DB option `verify_manifest_content_on_close` (default: false). When enabled, on DB close the MANIFEST file is read back and all records are validated (CRC checksums via log::Reader and logical content via VersionEdit::DecodeFrom). If corruption is detected, a fresh MANIFEST is written from in-memory state using the existing LogAndApply recovery path. This complements the existing size validation in VersionSet::Close() with content validation, reusing the same manifest reading pattern as VersionSet::Recover(). Implementation plan: ## Part 1: New DB Option — verify_manifest_content_on_close - A new mutable bool DB option (default: false) that can be dynamically toggled via SetDBOptions() at runtime, following the pattern of other mutable manifest options like max_manifest_file_size. - Propagation: SetDBOptions() -> DBImpl::mutable_db_options_ -> versions_->UpdatedMutableDbOptions() -> VersionSet::verify_manifest_content_on_close_ ## Part 2: Core Implementation — Content Validation in VersionSet::Close() - Inserted after existing size check, before closed_ = true - Opens manifest as SequentialFileReader, creates log::Reader with checksum=true - Loops ReadRecord with WALRecoveryMode::kAbsoluteConsistency, decodes each record as VersionEdit - On corruption: fires OnIOError listeners, logs error, calls LogAndApply with empty edit to trigger manifest rewrite from in-memory state - If manifest can't be opened for reading: logs warning, doesn't fail close ## Part 3: Unit Tests (in version_set_test.cc) - ManifestContentValidationOnClose_Clean: enable option, normal close, verify no manifest rotation - ManifestContentValidationOnClose_CorruptRecord: enable option, corrupt manifest via SyncPoint, verify rotation occurs and DB reopens cleanly - ManifestContentValidationOnClose_Disabled: default off, verify content validation does not run - ManifestContentValidationOnClose_SizeCheckFails: truncate manifest so size check fails first, verify recovery via size-check path ## What Happens If a Corruption is Detected If corruption was detected, four things happen: 1. **Notify listeners** — Fires `OnIOError` on all registered event listeners (from db_options_->listeners) so monitoring/alerting systems can observe the corruption event. Uses `FileOperationType::kVerify` to categorize it. 2. **Permit unchecked errors** — `PermitUncheckedError()` silences RocksDB's debug-mode assertion that every `IOStatus` must be inspected. These statuses are informational-only here; the real recovery is via `LogAndApply`. 3. **Log the error** — Writes a `ROCKS_LOG_ERROR` message with the filename for operational visibility (grep-able in production logs). 4. **Rewrite the manifest via `LogAndApply`** — This is the actual recovery. `LogAndApply` is called with an empty `VersionEdit` (no changes). Internally, `LogAndApply` detects that the current `descriptor_log_` is null (it was reset at line 5551, or by the previous `LogAndApply` in the size-check path) and creates a brand-new MANIFEST file. It serializes the entire current in-memory LSM state — all column families, all levels, all file metadata, sequence numbers, etc. — into this new file. It then atomically updates the `CURRENT` file pointer to reference the new MANIFEST. This works because the in-memory state was built from the original manifest during `DB::Open()` and has been kept fully up to date through all subsequent operations (flushes, compactions, etc.) during the DB's lifetime. The on-disk manifest is essentially a journal of changes; `LogAndApply` with an empty edit produces a fresh, compacted snapshot of that state. ## Flow Diagram of Manifest Content Validation VersionSet::Close() │ ├─ Close descriptor_log_ and check size │ └─ Size mismatch? → LogAndApply (rewrite manifest) │ ├─ Content validation (if s.ok() && option enabled) │ ├─ Open manifest for sequential reading │ │ └─ Can't open? → WARN log, continue │ │ │ ├─ For each record: │ │ ├─ ReadRecord (CRC32 check, kAbsoluteConsistency) │ │ └─ DecodeFrom (VersionEdit logical check) │ │ │ └─ Corruption detected? │ ├─ Notify OnIOError listeners │ ├─ LOG_ERROR │ └─ LogAndApply (rewrite manifest from in-memory state) │ └─ closed_ = true; return s; ## How This Relates to the Existing Size Check The existing size check (lines 5556-5582) and the new content validation are complementary: | Check | What it catches | How it checks | |----------------|-----------------------------------------|----------------------------| | Size check | Truncation, partial writes, extra bytes | Compare expected vs actual file size | | Content check | Bit-rot, silent corruption, bad records | CRC32 + VersionEdit decode | The size check catches gross corruption (file too short or too long). The content check catches subtle corruption where the file is the right size but individual bytes have been flipped (e.g., storage media bit-rot, buggy filesystem, incomplete block write). Both recovery paths use the same mechanism: `LogAndApply` with an empty `VersionEdit` to rewrite the manifest from in-memory state. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14451 Reviewed By: xingbowang Differential Revision: D96004906 Pulled By: dannyhchen fbshipit-source-id: 0b0ecdada3a74e97d2cadbba2091b8b577f1d684	2026-03-19 12:01:23 -07:00
anand76	653fd9c65b	Bug fix for bg error recovery in TransactionDB (#14313) ... Summary: This PR fixes a bug in the interaction between WritePrepared/WriteUnprepared TransactionDB (with two_write_queues=true) and background error recovery. This bug caused crash tests to fail with a "sequence number going backwards" error during DB open. Root Cause ------------ When two_write_queues=true, sequence numbers are allocated via FetchAddLastAllocatedSequence() before a write completes, but are only published via SetLastSequence() after the write succeeds. If a background error occurs (e.g., a MANIFEST write failure during flush), the error recovery path in DBImpl::ResumeImpl creates new memtables and WAL files. The new WAL's starting sequence number is based on LastSequence() (the published value), which can be lower than already-allocated sequence numbers that were written to the old WAL. On subsequent recovery, RocksDB detects that sequence numbers in the new WAL are lower than those in the old WAL and reports a "sequence number going backwards" corruption error, causing the DB to fail to open. Fix --- The fix adds a call to a new VersionSet::SyncLastSequenceWithAllocated() method at the beginning of DBImpl::ResumeImpl, before any new memtables or WALs are created. This method advances last_sequence_ to match last_allocated_sequence_ if the latter is higher, ensuring the new WAL starts with a sequence number that is at least as high as any previously allocated one. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14313 Test Plan: --------- Add new unit tests in write_prepared_transaction_test_seqno Reviewed By: pdillinger Differential Revision: D92746944 Pulled By: anand1976 fbshipit-source-id: 34385fc13fd74435dd1c3283637eb118f45d887e	2026-02-17 14:52:00 -08:00
Xingbo Wang	ea5e649225	Fix an infinite compaction loop bug with udt (#14228) ... Summary: Problem The TEST_WaitForCompact in TimestampCompatibleCompactionTest.UdtTombstoneCollapsingTest would sometimes run forever, indicating an infinite compaction loop. Issue https://github.com/facebook/rocksdb/issues/14223 Root Cause In ComputeBottommostFilesMarkedForCompaction(), files were marked for bottommost compaction based only on the condition largest_seqno < oldest_snapshot_seqnum. However, for User-Defined Timestamps (UDT) columns, compaction can only zero sequence numbers when the file's maximum timestamp is below full_history_ts_low. When timestamps were above this threshold: 1. File gets marked for compaction (seqno condition met) 2. Compaction runs but cannot zero seqno (timestamp condition not met) 3. Output file immediately gets re-marked for compaction 4. Infinite loop Solution Added timestamp range tracking to FileMetaData and updated the marking logic to check timestamps before marking files. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14228 Test Plan: Unit test Reviewed By: pdillinger Differential Revision: D90586045 Pulled By: xingbowang fbshipit-source-id: addfa4f988db8c87fb513a1bf58ee54623a6c210	2026-01-20 14:10:41 -08:00
Ranjan Banerjee	9fbb68be17	Api to get SST file with key ranges for a particular level and key range (startKey, EndKey)rocksdb [Internal version] (#14009) ... Summary: There are instances where an application might be interested in knowing the distribution in SST files for a key range in a particular level. This implementation creates an overloaded GetColumnFamilyMetaData api where (startKey, EndKey) can be passed along with level information to filter the necessary sst files along with the keyranges for each sst file Pull Request resolved: https://github.com/facebook/rocksdb/pull/14009 Reviewed By: anand1976 Differential Revision: D83389707 fbshipit-source-id: 6df1dc1f9233efe9000b03cc1831b3c618cbcef3	2025-11-10 17:13:34 -08:00
Peter Dillinger	37176a4a44	Auto-tune manifest file size (#14076) ... Summary: Adds auto-tuning of manifest file size to avoid the need to scale `max_manifest_file_size` in proportion to things like number of SST files to properly balance (a) manifest file write amp and new file creation, vs. (b) manifest file space amp and replay time, including non-incremental space usage in backups. (Manifest file write amp comes from re-writing a "live" record when the manifest file is re-created, or "compacted"; space amp is usage beyond what would be used by a compacted manifest file.) In more detail, * Add new option `max_manifest_space_amp_pct` with default value of 500, which defaults to 0.2 write amp and up to roughly 5.0 space amp, except `max_manifest_file_size` is treated as the "minimum" size before re-creating ("compacting") the manifest file. * `max_manifest_file_size` in a way means the same thing, with the same default of 1GB, but in a way has taken on a new role. What is the same is that we do not re-create the manifest file before reaching this size (except for DB re-open), and so users are very unlikely to see a change in default behavior (auto-tuning only kicking in if auto-tuning would exceed 1GB for effective max size for the current manifest file). The new role is as a file size lower bound before auto-tuning kicks in, to minimize churn in files considered "negligibly small." We recommend a new setting of around 1MB or even smaller like 64KB, and expect something like this to become the default soon. * These two options along with `manifest_preallocation_size` are now mutable with SetDBOptions. The effect is nearly immediate, affecting the next write to the current manifest file. Also in this PR: * Refactoring of VersionSet to allow it to get (more) settings from MutableDBOptions. This touches a number of files in not very interesting ways, but notably we have to be careful about thread-safe access to MutableDBOptions fields, and even fields within VersionSet. I have decided to save copies of relevant fields from MutableDBOptions to simplify testing, etc. by not saving a reference to MutableDBOptions but getting notified of updates. * Updated some logging in VersionSet to provide some basic data about final and compacted manifest sizes (effects of auto-tuning), making sure to avoid I/O while holding DB mutex. * Added db_etc3_test.cc which is intended as a successor to db_test and db_test2, but having "test.cc" in its name for easier exclusion of test files when using `git grep`. Intended follow-up: rename db_test2 to db_etc2_test * Moved+updated `ManifestRollOver` test to the new file to be closer to other manifest file rollover testing. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14076 Test Plan: As for correctness, new unit test AutoTuneManifestSize is pretty thorough. Some other unit tests updated appropriately. Manual tests in the performance section were also audited for expected behavior based on the new logging in the DB LOG. Example LOG data with -max_manifest_file_size=2048 -max_manifest_space_amp_pct=500: ``` 2025/10/24-11:12:48.979472 2150678 [/version_set.cc:5927] Created manifest 5, compacted+appended from 52 to 116 2025/10/24-11:12:49.626441 2150682 [/version_set.cc:5927] Created manifest 24, compacted+appended from 2169 to 1801 2025/10/24-11:12:52.194592 2150682 [/version_set.cc:5927] Created manifest 91, compacted+appended from 10913 to 8707 2025/10/24-11:13:02.969944 2150682 [/version_set.cc:5927] Created manifest 362, compacted+appended from 52259 to 13321 2025/10/24-11:13:18.815120 2150681 [/version_set.cc:5927] Created manifest 765, compacted+appended from 80064 to 13304 2025/10/24-11:13:35.590905 2150681 [/version_set.cc:5927] Created manifest 1167, compacted+appended from 79863 to 13304 ``` As you can see, it only took a few iterations of ramp-up to settle on the auto-tuned max manifest size for tracking ~122 live SST files, around 80KB and compacting down to about 13KB. (13KB * (500 + 100) / 100 = 78KB). With the default large setting for max_manifest_file_size, we end up with a 232KB manifest, which is more than 90% wasted space. (A long-running DB would be much worse.) As for performance, we don't expect a difference, even with TransactionDB because actual writing of the manifest is done without holding the DB mutex. I was not able to see a performance regression using db_bench with FIFO compaction and >1000 ~10MB SST files, including settings of -max_manifest_file_size=2048 -max_manifest_space_amp_pct={500,10,0}. No "hiccups" visible with -histogram either. I also tried seeding a 1 second delay in writing new manifest files (other than the first). This had no significant effect at -max_manifest_space_amp_pct=500 but at 100 started causing write stalls in my test. In many ways this is kind of a worst case scenario and out-of-proportion test, but gives me more confidence that a higher number like 500 is probably the best balance in general. Reviewed By: xingbowang Differential Revision: D85445178 Pulled By: pdillinger fbshipit-source-id: 1e6e07e89c586762dd65c65bb7cb2b8b719513f9	2025-11-07 09:04:52 -08:00
Changyu Bi	144e9f1e42	Fix compaction picking with L0 standalone range deletion file (#14061) ... Summary: When a standalone range deletion file is ingested in L0, currently it is compacted with any overlapping L0 files. This is not desirable when we ingest new data on top of the range deletion file. This PR fixes the compaction picking logic to only consider L0 files older than the standalone range deletion file. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14061 Test Plan: added a new unit test and updated an existing one. Reviewed By: xingbowang Differential Revision: D84930780 Pulled By: cbi42 fbshipit-source-id: 65f4403ccb40ba964b9e65b09e2f7f7efebe81df	2025-10-23 13:34:07 -07:00
Changyu Bi	82b5a2d3fc	Allow ingestion of any DB generated SST file (#13878) ... Summary: `IngestExternalFileOptions::allow_db_generated_files` requires SST files to have zero sequence number. This PR opens it up for any DB generated SST files. Currently we don't do global sequence number assignment when `allow_db_generated_files` is true, so we require that files do not overlap with any key in the CF. One behavior difference is that now we allow ingesting overlapping files when `allow_db_generated_files` is true. Users need to ensure that files are ordered such that later files have more recent updates. Intended follow ups: - Record smallest seqno in table property, so that we don't need to scan the file for it. - Cover allow_db_generated_files in crash test. We may create a new DB and ingest all files from a CF for verification. - Add APIs that uses allow_db_generated_files. For example, an API for ingesting SST files from a source CF, so that we take care of ingestion file ordering for user. If we are already getting metadata from the source CF, we may be use it as a hint for level placement instead of dividing input files into batches again (`ExternalSstFileIngestionJob::DivideInputFilesIntoBatches`). Pull Request resolved: https://github.com/facebook/rocksdb/pull/13878 Test Plan: two new unit tests. Reviewed By: hx235, xingbowang Differential Revision: D80233727 Pulled By: cbi42 fbshipit-source-id: 74209386d8426c434bff2d9a734f06db537eb50c	2025-08-22 16:05:56 -07:00
Yu Zhang	934cf2d40d	Implement the DB::GetPropertiesOfTablesForLevels API (#13469) ... Summary: As titled. This API returns the table properties of files per level. It can be handy for use cases that needed file's leveling info while retrieving TableProperties. We will use this API to later aggregate per level data write time info. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13469 Test Plan: Added unit tests Reviewed By: pdillinger Differential Revision: D71353096 Pulled By: jowlyzhang fbshipit-source-id: dc1fbb2c97e4365fc8d7241f9a59c65fbf4fb766	2025-03-21 17:23:01 -07:00
Maciej Szeszko	d0374a0a72	Control SST write lifetime hints based on compaction style (#13472) ... Summary: This PR is a followup to https://github.com/facebook/rocksdb/pull/13461. We're introducing an experimental option / killswitch to control SST write lifetime hint calculation based on the selected compaction style. By default (and mostly for backwards compatibility reasons), we'll calculate the SST hints only for level compactions. With this change users have an option to configure SST lifetime hint policy in their environments to enable the calculations in the universal compaction mode as well. It's important to underline that as currently implemented, SST write lifetime hints are calculated in a static way and solely based on the level, which might not be suitable for non-uniform workloads with dynamic / high-variance lifespan of data within the same level. In those cases (or when the performance is not satisfactory), it's recommended to disable the hints by setting the set to empty. Please see the comment in `options.h` for more. **NOTE:** We deliberately decided to introduce a new option to ensure no impact to external users running their RocksDB instances on local flash with the default `PosixWritableFile` file implementation. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13472 Reviewed By: pdillinger, anand1976 Differential Revision: D71445488 Pulled By: mszeszko-meta fbshipit-source-id: 57dc5e56662fa0b0fd686e183c0ec7090ff12d66	2025-03-21 13:10:43 -07:00
Peter Dillinger	b9c7481fc2	Fix some secondary/read-only DB logic (#13441) ... Summary: Primarily, fix an issue from https://github.com/facebook/rocksdb/issues/13316 with opening secondary DB with preserve/preclude option (crash test disable in https://github.com/facebook/rocksdb/issues/13439). The issue comes down to mixed-up interpretations of "read_only" which should now be resolved. I've introduced the stronger notion of "unchanging" which means the VersionSet never sees any changes to the LSM tree, and the weaker notion of "read_only" which means LSM tree changes are not written through this VersionSet/etc. but can pick up externally written changes. In particular, ManifestTailer should use read_only=true (along with unchanging=false) for proper handling of preserve/preclude options. A new assertion in VersionSet::CreateColumnFamily to help ensure sane usage of the two boolean flags is incompatible with the known wart of allowing CreateColumnFamily on a read-only DB. So to keep that assertion, I have fixed that issue by disallowing it. And this in turn required downstream clean-up in ldb, where I cleaned up some call sites as well. Also, rename SanitizeOptions for ColumnFamilyOptions to SanitizeCfOptions, for ease of search etc. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13441 Test Plan: * Added preserve option to a test in db_secondary_test, which reproduced the failure seen in the crash test. * Revert https://github.com/facebook/rocksdb/issues/13439 to re-enable crash test functionality * Update some tests to deal with disallowing CF creation on read-only DB * Add some testing around read-only DBs and CreateColumnFamily(ies) * Resurrect a nearby test for read-only DB to be sure it doesn't write to the DB dir. New EnforcedReadOnlyReopen should probably be used in more places but didn't want to attempt a big migration here and now. (Suggested follow-up.) Reviewed By: jowlyzhang Differential Revision: D70808033 Pulled By: pdillinger fbshipit-source-id: 486b4e9f9c9045150a0ebb9cb302753d03932a3f	2025-03-07 14:56:45 -08:00
Peter Dillinger	da8eba8b49	Improve consistency of SeqnoToTime tracking in SuperVersion (#13316) ... Summary: This is an unexpectedly complex follow-up to https://github.com/facebook/rocksdb/issues/13269. This change solves (and detects regressed) inconsistencies between whether a CF's SuperVersion is configured with a preserve/preclude option and whether it gets a usable SeqnoToTimeMapping. Operating with preserve/preclude and no usable mapping is degraded functionality we need to avoid. And no mapping is useful for actually disabling the feature (except with respect to existing SST files, but that's less of a concern for now). The challenge is that how we maintain the DB's SeqnoToTimeMapping can depend on all the column families, and we don't want to iterate over all column families *for each column family* (e.g. on initially creating each). The existing code was a bit relaxed: * On initially creating or re-configuring a CF, we might install an empty mapping, but soon thereafter (after releasing and re-acquiring the DB mutex) re-install another SuperVersion with a useful mapping. The solution here is to refactor the logic so that there's a distinct but related workflow for (a) ensuring a quality set of mappings when we might only be considering a single CF (`EnsureSeqnoToTimeMapping()`), and (b) massaging that set of mappings to account for all CFs (`RegisterRecordSeqnoTimeWorker`) which doesn't need to re-install new SuperVersions because each CF already has good mappings and will get updated SuperVersions when the periodic task adds new mappings. This should eliminate the extra SuperVersion installs associated with preserve/preclude on CF creation or re-configure, making it the same as any other CF. Some more details: * Some refactorings such as removing new_seqno_to_time_mapping from SuperVersionContext. (Now use parameter instead of being stateful.) * Propagate `read_only` aspect of DB to more places so that we can pro-actively disable preserve/preclude on read-only DBs, so that we don't run afoul of the assertion expecting SeqnoToTime entries. * Introduce a utility struct `MinAndMaxPreserveSeconds` for aggregating preserve/preclude settings in a useful way, sometimes on one CF and sometimes across multiple CFs. Much cleaner! (IMHO) * Introduce a function `InstallSuperVersionForConfigChange` that is a superset of `InstallSuperVersionAndScheduleWork` for when a CF is new or might have had a change to its mutable options. * Eliminate redundant re-install SuperVersions of created "missing" CFs in DBImpl::Open. Intended follow-up: * Ensure each flush has an "upper bound" SeqnoToTime entry, which would resolve a FIXME in tiered_compaction_test, but causes enough test churn to deserve its own PR + investigation. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13316 Test Plan: This change is primarily validated by a new assertion in SuperVersion::Init to ensure consistency between (a) presence of any SeqnoToTime mappings in the SuperVersion and (b) preserve/preclude option being currently set. One unit test update was needed because we now ensure at least one SeqnoToTime entry is created on any DB::Open with preserve/preclude, so that there is a lower bound time on all the future data writes. This required a small hack in associating the time with Seqno 1 instead of 0, which is reserved for "unspecified old." Reviewed By: cbi42 Differential Revision: D70540638 Pulled By: pdillinger fbshipit-source-id: bb419fdbeb5a1f115fc429c211f9b8efaf2f56d7	2025-03-04 17:44:01 -08:00
Hui Xiao	affcad0cc9	Fix corrupted wal number when predecessor wal corrupts + minor cleanup (#13359) ... Summary: **Context/Summary:** 02b4197544 recently added the ability to detect WAL hole presents in the predecessor WAL. It forgot to update the corrupted wal number to point to the predecessor WAL in that corruption case. This PR fixed it. As a bonus, this PR also (1) fixed the `FragmentBufferedReader()` constructor API to expose less parameters as they are never explicitly passed in in the codebase (2) a INFO log wording (3) a parameter naming typo (4) the reporter naming Pull Request resolved: https://github.com/facebook/rocksdb/pull/13359 Test Plan: 1. Manual printing to ensure the corrupted wal number is set to the right number 2. Existing UTs Reviewed By: jowlyzhang Differential Revision: D69068089 Pulled By: hx235 fbshipit-source-id: f7f8a887cded2d3a26cf9982f5d1d1ab6a78e9e1	2025-02-13 21:49:51 -08:00
Peter Dillinger	c9ce4a3d6b	Improve atomicity of SetOptions, skip manifest write (#13384) ... Summary: Motivated by code review issue in https://github.com/facebook/rocksdb/issues/13316, we don't want to release the DB mutex in SetOptions between updating the cfd latest options and installing the new Version and SuperVersion. SetOptions uses LogAndApply to install a new Version but this currently incurs an unnecessary manifest write. (This is not a big performance concern because SetOptions dumps a new OPTIONS file, which is much larger than the redundant manifest update.) Since we don't want IO while holding the DB mutex, we need to get rid of the manifest write, and that's what this change does. We introduce a kind of dummy VersionEdit that allows the existing code paths of LogAndApply to install a new Version (with the updated mutable options), recompute resulting compaction scores etc., but without the manifest write. Part of the validation for this is new assertions in SetOptions verifying the consistency of the various copies of MutableCFOptions. (I'm not convinced we need it in SuperVersion in addition to Version, but that's not for here and now.) These checks depend on defaulted `operator==` so depend on C++20. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13384 Test Plan: New unit test in addition to new assertions. SetOptions already tested heavily in crash test. Used `ROCKSDB_CXX_STANDARD=c++20 make -j100 check` to ensure the new assertions are verified Reviewed By: cbi42 Differential Revision: D69408829 Pulled By: pdillinger fbshipit-source-id: 4cf026010c6bb381e0ea27567cce2708d4678e7d	2025-02-10 16:46:13 -08:00
Peter Dillinger	780280b52e	Reduce unnecessary MutableCFOptions copies and parameters (#13301) ... Summary: As follow-up to https://github.com/facebook/rocksdb/issues/13239, this change is primarily motivated by simplifying the calling conventions of LogAndApply. Since it must be called while holding the DB mutex, it can read safely read cfd->GetLatestMutableCFOptions(), until it releases the mutex within ProcessManifestWrites. Before it releases the mutex, it makes a copy of the mutable options in a new, unpublished Version object, which can be used when not holding the DB mutex. This eliminates the need for callers of LogAndApply to copy mutable options for its sake, or even specify mutable options at all. And it eliminates the need for *another* copy to be saved in ManifestWriter. Other functions that don't need the mutable options parameter: * ColumnFamilyData::CreateNewMemtable() * CompactionJob::Install() / InstallCompactionResults() * MemTableList::*InstallMemtable*() * Version::PrepareAppend() Pull Request resolved: https://github.com/facebook/rocksdb/pull/13301 Test Plan: existing tests, CI with sanitizers Reviewed By: mszeszko-meta Differential Revision: D68234865 Pulled By: pdillinger fbshipit-source-id: 6ce95f9cc479834e09ffc8ce93cbae7b664329e5	2025-01-16 14:10:11 -08:00
Peter Dillinger	b3333587eb	Clean up some CFOptions code hygiene, fix SetOptions() bug (#13294) ... Summary: To start, I wanted to remove the unnecessary new_options parameter of `InstallSuperVersionAndScheduleWork()`. Passing it something other than the latest mutable options would be inconsistent/outdated. There was even a comment "Use latest MutableCFOptions" on a place that was using the saved options in effect for the compaction. On investigation, this fixes an undiagnosed but longstanding serious bug in SetOptions() where the new settings can be reverted if a flush or compaction started before the SetOptions() finishes after. Fix confirmed with new unit test in db_test.cc. I also got tired of seeing the cumbersome usage of pointer rather than const reference for related options accesses, so there's kind of a large (but trivial) refactoring tied in here as well. (Sorry for combining them; wasn't planning a major bug fix) Intended follow-up: Clarify/simplify the crazy calling conventions of LogAndApply, and remove some unnecessary copying of MutableCFOptions (see new FIXMEs) Pull Request resolved: https://github.com/facebook/rocksdb/pull/13294 Test Plan: test for bug fix, confirmed fails on main and at least as far back as version 8.10. Plus existing tests and CI Reviewed By: mszeszko-meta Differential Revision: D68141563 Pulled By: pdillinger fbshipit-source-id: f6c3290145afa06cc2fe8b485a5de17560a5deea	2025-01-15 13:11:40 -08:00
jonahgao	00a3eb6990	Remove unused variable: number_of_files_to_sort_ (#13259) ... Summary: This variable is not used, the one actually being used is `kNumberFilesToSort`. 02b4197544/db/version_set.h (L544-L549) Pull Request resolved: https://github.com/facebook/rocksdb/pull/13259 Reviewed By: archang19 Differential Revision: D67765254 Pulled By: cbi42 fbshipit-source-id: 3c32b0396d0ba9e08e77b96b098e6c26d0bf8aac	2025-01-02 11:34:58 -08:00
Maciej Szeszko	be7703b27d	Offline file checksum manifest retriever (#13178) ... Summary: This change introduces a new, lightweight _experimental_ API that reconstructs the [file # -> file checksum -> file checksum function] 1-1-1 mapping directly from the `MANIFEST` file considered `CURRENT` in scope of specific DB instance at the time. The goal is to provide a cheap alternative to `DB::GetLiveFilesMetaData` that doesn't require opening the database, reconstructing version sets and/or accessing files that are _potentially_ in disaggregated storage. ### Housekeeping: 1. Moved the `GetCurrentManifestPath` out of `version_set` to a new `manifest_ops` file(s) dedicated to manifest related operations. 2. Introduced new `Env::IOActivity::kReadManifest` to better reflect the IO intent in offline file checksum retrieving function. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13178 Test Plan: Added a unit test comparing the outcome of newly introduced API against the established `GetLiveFilesMetaData`: ```hcl ./db_test2 --gtest_filter="*GetFileChecksumsFromCurrentManifest_CRC32*" ``` Reviewed By: pdillinger Differential Revision: D66711910 Pulled By: mszeszko-meta fbshipit-source-id: 57091c550a14ac2e832bf7eea136dab5450e71bc	2024-12-06 13:29:52 -08:00
anand76	ee258619be	Fix missing cases of corruption retries (#13122) ... Summary: This PR fixes a few cases where RocksDB was not retrying checksum failure/corruption of file reads with the `verify_and_reconstruct_read` IO option. After fixing these cases, we can almost always successfully open the DB and execute reads even if we see transient corruptions, provided the `FileSystem` supports the `verify_and_reconstruct_read` option. The specific cases fixed in this PR are - 1. CURRENT file 2. IDENTITY file 3. OPTIONS file 4. SST footer Pull Request resolved: https://github.com/facebook/rocksdb/pull/13122 Test Plan: Unit test in `db_io_failure_test.cc` that injects corruption at various stages of DB open and reads Reviewed By: jaykorean Differential Revision: D65617982 Pulled By: anand1976 fbshipit-source-id: 4324b88cc7eee5501ab5df20ef7a95bb12ed3ea7	2024-11-08 12:43:21 -08:00
Peter Dillinger	a28cc4a38c	Fix a leak of open Blob files (#13106) ... Summary: An earlier change (b34cef57b7) removed apparently unused functionality where an obsolete blob file number is passed for removal from TableCache, which manages SST files. This was actually relying on broken/fragile abstractions wherein TableCache and BlobFileCache share the same Cache and using the TableCache interface to manipulate blob file caching. No unit test was actually checking for removal of obsolete blob files from the cache (which is somewhat tricky to check and a second order correctness requirement). Here we fix the leak and add a DEBUG+ASAN-only check in DB::Close() that no obsolete files are lingering in the table/blob file cache. Fixes https://github.com/facebook/rocksdb/issues/13066 Important follow-up (FIXME): The added check discovered some apparent cases of leaked (into table_cache) SST file readers that would stick around until DB::Close(). Need to enable that check, diagnose, and fix. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13106 Test Plan: added a check that is called during DB::Close in ASAN builds (to minimize paying the cost in all unit tests). Without the fix, the check failed in at least these tests: ``` db_blob_basic_test DBBlobBasicTest.DynamicallyWarmCacheDuringFlush db_blob_compaction_test DBBlobCompactionTest.CompactionReadaheadMerge db_blob_compaction_test DBBlobCompactionTest.MergeBlobWithBase db_blob_compaction_test DBBlobCompactionTest.CompactionDoNotFillCache db_blob_compaction_test DBBlobCompactionTest.SkipUntilFilter db_blob_compaction_test DBBlobCompactionTest.CompactionFilter db_blob_compaction_test DBBlobCompactionTest.CompactionReadaheadFilter db_blob_compaction_test DBBlobCompactionTest.CompactionReadaheadGarbageCollection ``` Reviewed By: ltamasi Differential Revision: D65296123 Pulled By: pdillinger fbshipit-source-id: 2276d76482beb2c75c9010bc1bec070bb23a24c0	2024-10-31 15:29:30 -07:00
Yu Zhang	9c94559de7	Optimize compaction for standalone range deletion files (#13078) ... Summary: This PR adds some optimization for compacting standalone range deletion files. A standalone range deletion file is one with just a single range deletion. Currently, such a file is used in bulk loading to achieve something like atomically delete old version of all data with one big range deletion and adding new version of data. These are the changes included in the PR: 1) When a standalone range deletion file is ingested via bulk loading, it's marked for compaction. 2) When picking input files during compaction picking, we attempt to only pick a standalone range deletion file when oldest snapshot is at or above the file's seqno. To do this, `PickCompaction` API is updated to take existing snapshots as an input. This is only done for the universal compaction + UDT disabled combination, we save querying for existing snapshots and not pass it for all other cases. 3) At `Compaction` construction time, the input files will be filtered to examine if any of them can be skipped for compaction iterator. For example, if all the data of the file is deleted by a standalone range tombstone, and the oldest snapshot is at or above such range tombstone, this file will be filtered out. 4) Every time a snapshot is released, we examine if any column family has standalone range deletion files that becomes eligible to be scheduled for compaction. And schedule one for it. Potential future improvements: - Add some dedicated statistics for the filtered files. - Extend this input filtering to L0 files' compactions cases when a newer L0 file could shadow an older L0 file Pull Request resolved: https://github.com/facebook/rocksdb/pull/13078 Test Plan: Added unit tests and stress tested a few rounds Reviewed By: cbi42 Differential Revision: D64879415 Pulled By: jowlyzhang fbshipit-source-id: 02b8683fddbe11f093bcaa0a38406deb39f44d9e	2024-10-25 09:32:14 -07:00
Nicholas Ormrod	0e04ef1a96	Deshim coro in fbcode/internal_repo_rocksdb ... Summary: The following rules were deshimmed: ``` //folly/experimental/coro:accumulate -> //folly/coro:accumulate //folly/experimental/coro:async_generator -> //folly/coro:async_generator //folly/experimental/coro:async_pipe -> //folly/coro:async_pipe //folly/experimental/coro:async_scope -> //folly/coro:async_scope //folly/experimental/coro:async_stack -> //folly/coro:async_stack //folly/experimental/coro:baton -> //folly/coro:baton //folly/experimental/coro:blocking_wait -> //folly/coro:blocking_wait //folly/experimental/coro:collect -> //folly/coro:collect //folly/experimental/coro:concat -> //folly/coro:concat //folly/experimental/coro:coroutine -> //folly/coro:coroutine //folly/experimental/coro:current_executor -> //folly/coro:current_executor //folly/experimental/coro:detach_on_cancel -> //folly/coro:detach_on_cancel //folly/experimental/coro:detail_barrier -> //folly/coro:detail_barrier //folly/experimental/coro:detail_barrier_task -> //folly/coro:detail_barrier_task //folly/experimental/coro:detail_current_async_frame -> //folly/coro:detail_current_async_frame //folly/experimental/coro:detail_helpers -> //folly/coro:detail_helpers //folly/experimental/coro:detail_malloc -> //folly/coro:detail_malloc //folly/experimental/coro:detail_manual_lifetime -> //folly/coro:detail_manual_lifetime //folly/experimental/coro:detail_traits -> //folly/coro:detail_traits //folly/experimental/coro:filter -> //folly/coro:filter //folly/experimental/coro:future_util -> //folly/coro:future_util //folly/experimental/coro:generator -> //folly/coro:generator //folly/experimental/coro:gmock_helpers -> //folly/coro:gmock_helpers //folly/experimental/coro:gtest_helpers -> //folly/coro:gtest_helpers //folly/experimental/coro:inline_task -> //folly/coro:inline_task //folly/experimental/coro:invoke -> //folly/coro:invoke //folly/experimental/coro:merge -> //folly/coro:merge //folly/experimental/coro:mutex -> //folly/coro:mutex //folly/experimental/coro:promise -> //folly/coro:promise //folly/experimental/coro:result -> //folly/coro:result //folly/experimental/coro:retry -> //folly/coro:retry //folly/experimental/coro:rust_adaptors -> //folly/coro:rust_adaptors //folly/experimental/coro:scope_exit -> //folly/coro:scope_exit //folly/experimental/coro:shared_lock -> //folly/coro:shared_lock //folly/experimental/coro:shared_mutex -> //folly/coro:shared_mutex //folly/experimental/coro:sleep -> //folly/coro:sleep //folly/experimental/coro:small_unbounded_queue -> //folly/coro:small_unbounded_queue //folly/experimental/coro:task -> //folly/coro:task //folly/experimental/coro:timed_wait -> //folly/coro:timed_wait //folly/experimental/coro:timeout -> //folly/coro:timeout //folly/experimental/coro:traits -> //folly/coro:traits //folly/experimental/coro:transform -> //folly/coro:transform //folly/experimental/coro:unbounded_queue -> //folly/coro:unbounded_queue //folly/experimental/coro:via_if_async -> //folly/coro:via_if_async //folly/experimental/coro:with_async_stack -> //folly/coro:with_async_stack //folly/experimental/coro:with_cancellation -> //folly/coro:with_cancellation //folly/experimental/coro:bounded_queue -> //folly/coro:bounded_queue //folly/experimental/coro:shared_promise -> //folly/coro:shared_promise //folly/experimental/coro:cleanup -> //folly/coro:cleanup //folly/experimental/coro:auto_cleanup_fwd -> //folly/coro:auto_cleanup_fwd //folly/experimental/coro:auto_cleanup -> //folly/coro:auto_cleanup ``` The following headers were deshimmed: ``` folly/experimental/coro/Accumulate.h -> folly/coro/Accumulate.h folly/experimental/coro/Accumulate-inl.h -> folly/coro/Accumulate-inl.h folly/experimental/coro/AsyncGenerator.h -> folly/coro/AsyncGenerator.h folly/experimental/coro/AsyncPipe.h -> folly/coro/AsyncPipe.h folly/experimental/coro/AsyncScope.h -> folly/coro/AsyncScope.h folly/experimental/coro/AsyncStack.h -> folly/coro/AsyncStack.h folly/experimental/coro/Baton.h -> folly/coro/Baton.h folly/experimental/coro/BlockingWait.h -> folly/coro/BlockingWait.h folly/experimental/coro/Collect.h -> folly/coro/Collect.h folly/experimental/coro/Collect-inl.h -> folly/coro/Collect-inl.h folly/experimental/coro/Concat.h -> folly/coro/Concat.h folly/experimental/coro/Concat-inl.h -> folly/coro/Concat-inl.h folly/experimental/coro/Coroutine.h -> folly/coro/Coroutine.h folly/experimental/coro/CurrentExecutor.h -> folly/coro/CurrentExecutor.h folly/experimental/coro/DetachOnCancel.h -> folly/coro/DetachOnCancel.h folly/experimental/coro/detail/Barrier.h -> folly/coro/detail/Barrier.h folly/experimental/coro/detail/BarrierTask.h -> folly/coro/detail/BarrierTask.h folly/experimental/coro/detail/CurrentAsyncFrame.h -> folly/coro/detail/CurrentAsyncFrame.h folly/experimental/coro/detail/Helpers.h -> folly/coro/detail/Helpers.h folly/experimental/coro/detail/Malloc.h -> folly/coro/detail/Malloc.h folly/experimental/coro/detail/ManualLifetime.h -> folly/coro/detail/ManualLifetime.h folly/experimental/coro/detail/Traits.h -> folly/coro/detail/Traits.h folly/experimental/coro/Filter.h -> folly/coro/Filter.h folly/experimental/coro/Filter-inl.h -> folly/coro/Filter-inl.h folly/experimental/coro/FutureUtil.h -> folly/coro/FutureUtil.h folly/experimental/coro/Generator.h -> folly/coro/Generator.h folly/experimental/coro/GmockHelpers.h -> folly/coro/GmockHelpers.h folly/experimental/coro/GtestHelpers.h -> folly/coro/GtestHelpers.h folly/experimental/coro/detail/InlineTask.h -> folly/coro/detail/InlineTask.h folly/experimental/coro/Invoke.h -> folly/coro/Invoke.h folly/experimental/coro/Merge.h -> folly/coro/Merge.h folly/experimental/coro/Merge-inl.h -> folly/coro/Merge-inl.h folly/experimental/coro/Mutex.h -> folly/coro/Mutex.h folly/experimental/coro/Promise.h -> folly/coro/Promise.h folly/experimental/coro/Result.h -> folly/coro/Result.h folly/experimental/coro/Retry.h -> folly/coro/Retry.h folly/experimental/coro/RustAdaptors.h -> folly/coro/RustAdaptors.h folly/experimental/coro/ScopeExit.h -> folly/coro/ScopeExit.h folly/experimental/coro/SharedLock.h -> folly/coro/SharedLock.h folly/experimental/coro/SharedMutex.h -> folly/coro/SharedMutex.h folly/experimental/coro/Sleep.h -> folly/coro/Sleep.h folly/experimental/coro/Sleep-inl.h -> folly/coro/Sleep-inl.h folly/experimental/coro/SmallUnboundedQueue.h -> folly/coro/SmallUnboundedQueue.h folly/experimental/coro/Task.h -> folly/coro/Task.h folly/experimental/coro/TimedWait.h -> folly/coro/TimedWait.h folly/experimental/coro/Timeout.h -> folly/coro/Timeout.h folly/experimental/coro/Timeout-inl.h -> folly/coro/Timeout-inl.h folly/experimental/coro/Traits.h -> folly/coro/Traits.h folly/experimental/coro/Transform.h -> folly/coro/Transform.h folly/experimental/coro/Transform-inl.h -> folly/coro/Transform-inl.h folly/experimental/coro/UnboundedQueue.h -> folly/coro/UnboundedQueue.h folly/experimental/coro/ViaIfAsync.h -> folly/coro/ViaIfAsync.h folly/experimental/coro/WithAsyncStack.h -> folly/coro/WithAsyncStack.h folly/experimental/coro/WithCancellation.h -> folly/coro/WithCancellation.h folly/experimental/coro/BoundedQueue.h -> folly/coro/BoundedQueue.h folly/experimental/coro/SharedPromise.h -> folly/coro/SharedPromise.h folly/experimental/coro/Cleanup.h -> folly/coro/Cleanup.h folly/experimental/coro/AutoCleanup-fwd.h -> folly/coro/AutoCleanup-fwd.h folly/experimental/coro/AutoCleanup.h -> folly/coro/AutoCleanup.h ``` This is a codemod. It was automatically generated and will be landed once it is approved and tests are passing in sandcastle. You have been added as a reviewer by Sentinel or Butterfly. Autodiff project: dcoro Autodiff partition: fbcode.internal_repo_rocksdb Autodiff bookmark: ad.dcoro.fbcode.internal_repo_rocksdb Reviewed By: dtolnay Differential Revision: D62684411 fbshipit-source-id: 8dbd31ab64fcdd99435d322035b9668e3200e0a3	2024-09-14 09:48:21 -07:00
Yu Zhang	0c6e9c036a	Make compaction always use the input version with extra ref protection (#12992) ... Summary: `Compaction` is already creating its own ref for the input Version: 4b1d595306/db/compaction/compaction.cc (L73) And properly Unref it during destruction: 4b1d595306/db/compaction/compaction.cc (L450) This PR redirects compaction's access of `cfd->current()` to this input `Version`, to prepare for when a column family's data can be replaced all together, and `cfd->current()` is not safe to access for a compaction job. Because a new `Version` with just some other external files could be installed as `cfd->current()`. The compaction job's expectation of the current `Version` and the corresponding storage info to always have its input files will no longer be guaranteed. My next follow up is to do a similar thing for flush, also to prepare it for when a column family's data can be replaced. I will make it create its own reference of the current `MemTableListVersion` and use it as input, all flush job's access of memtables will be wired to that input `MemTableListVersion`. Similarly this reference will be unreffed during a flush job's destruction. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12992 Test Plan: Existing tests Reviewed By: pdillinger Differential Revision: D62212625 Pulled By: jowlyzhang fbshipit-source-id: 9a781213469cf366857a128d50a702af683a046a	2024-09-06 14:07:33 -07:00
Yu Zhang	945f60b157	Add some documentation for version edit handlers (#12948) ... Summary: As titled. No functional change. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12948 Reviewed By: hx235 Differential Revision: D61551254 Pulled By: jowlyzhang fbshipit-source-id: ccf53d78bd2f18f174d7e61972e5de467c96ce76	2024-08-21 10:09:10 -07:00