mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-23 08:09:02 +03:00
ded3cb72a5
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15296 **Why:** trace replay (`DB::NewDefaultReplayer()`, db_bench `replay`) cannot replay a trace of a DB with only the default column family, and multi-threaded replay reports success when records fail. Together these make db_bench replay look successful while replaying no reads. **Previous behavior:** - `TraceExecutionHandler` resolves a record's column family ID only against the `handles` passed to `NewDefaultReplayer()`. Records on a column family missing from `handles`, including the default one, fail with `Corruption: Invalid Column Family ID.`. An empty `handles` also trips `assert(!handles.empty())` in debug builds. - db_bench fills its handle list (`db_.cfh`) only when `--num_column_families` > 1, so with the default of 1 it passes an empty list. Every Get, MultiGet and iterator record fails. Write records are not affected, because `DB::Write()` resolves the column family from the batch itself. - In multi-threaded `Replay()` (`num_threads` > 1), `BackgroundWork()` sends only decode errors to the error callback. Execution errors are dropped, so `Replay()` returns OK. Single-threaded replay stops and returns the error, including a NotSupported from execution (for example `DB::Write()` of a DeleteRange with `row_cache` set). It skips only records whose trace type it cannot decode. - Result in db_bench on a single column family DB: `--trace_replay_threads=1` prints `Replay failed. Error: Corruption: Invalid Column Family ID.`. `--trace_replay_threads=4` prints `Replay completed` with `rocksdb.number.keys.read` = 0. **New behavior:** - The replay handler always includes the DB's default column family, so `handles` may be empty or leave it out. A handle the caller passes for a column family is still the one used. - Multi-threaded `Replay()` reports execution errors through the same callback as decode errors. It stops scheduling new records and returns the error of the failed record with the earliest trace timestamp, as the existing code already does for decode errors. Get records that find no key still count as success, since the handler treats NotFound as OK. - As in single-threaded replay, NotSupported is skipped only when decoding returns it (a trace type the replayer does not support). NotSupported from executing a record stops the replay. - The scheduling loop no longer reads the shared status `bg_s` while worker threads write it. It already did so for decode errors; with execution errors reported, TSAN flags it in the new test (see Test Plan). **Implementation:** - `TraceExecutionHandler` constructor: drop the non-empty assert; after the caller's handles, insert `db->DefaultColumnFamily()`. `unordered_map::insert` does not overwrite, so a caller's handle wins. - `TraceExecutionHandler` constructor: skip the default column family if `DefaultColumnFamily()` returns nullptr (some test `DB` mocks do), so such DBs behave as before. - `ReplayerImpl::BackgroundWork()`: call `error_cb` when `Accept()` returns an error. On a decode error, skip `error_cb` for NotSupported, matching single-threaded replay. - `ReplayerImpl::Replay()`: `error_cb` no longer filters out NotSupported (the decode path now does that); it still keeps the error of the earliest record. - `ReplayerImpl::Replay()`: `error_cb` also sets an `std::atomic<bool> bg_failed` while holding `mtx`, and the scheduling loop checks that flag instead of `bg_s.ok()`. `bg_s` is read only after `WaitForJobsAndJoinAllThreads()`. - Document both contracts in `db.h` (`NewDefaultReplayer`), `trace_record.h` (`NewExecutionHandler`) and `replayer.h` (`Replay`). - Release notes in `unreleased_history/bug_fixes/`. **Safety, compatibility, limitations:** - No API or ABI change. - Callers that already pass the default column family handle see no change from the first fix. - Behavior change: multi-threaded replay callers that got OK while records failed now get the error, including NotSupported from execution. That matches single-threaded replay and the new `Replay()` comment. - A record read by the scheduling loop while a failing worker is setting `bg_failed` may still be scheduled. This is allowed by the `Replay()` comment (records already scheduled still run) and does not change the returned status, which is that of the earliest failed record. - Cost: one extra map entry per replayer, and one branch per record in threaded replay. The loop's `bg_s.ok()` read becomes a relaxed atomic load. - Read records on column families that are neither passed nor the default still fail with Corruption, as before. Write records never used `handles` and are unchanged. Reviewed By: xingbowang Differential Revision: D122626982 fbshipit-source-id: 9a0973448710bac25233feedbaf264696ac85035