mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-23 08:09:02 +03:00
bb77d17072
Summary: DB::Flush() with wait=true only waits until the flush result is committed (memtables removed from the immutable list). The EventListener::OnFlushCompleted callbacks run afterward, on the background flush thread, with the DB mutex released, and the waiter is only woken by a shared condition variable that many unrelated events also signal. As a result a Flush(wait=true) caller can return before -- or in the middle of -- its OnFlushCompleted callbacks. This surfaced as a flaky failure in the c_test table_filter_listener_metadata case, which asserts listener_state.flush_count == 3 immediately after Flush() returns (https://github.com/facebook/rocksdb/actions/runs/29976431449). It is also long-standing behavior that other tests have worked around with TEST_WaitForBackgroundWork(). This adds an opt-in FlushOptions::listener_wait (default false). When set together with wait=true, Flush() does not return until the OnFlushCompleted callbacks for the flushed memtables have finished. Default behavior is unchanged. Implementation: MemTableList gains a per-column-family num_pending_flush_notifications_ counter (guarded by the DB mutex). The background flush thread reserves a pending notification at flush-commit time -- before the mutex is released to run the callbacks -- and releases it after they return, in both the single-CF (FlushMemTableToOutputFile) and atomic (AtomicFlushMemTablesToOutputFiles) paths. WaitForFlushMemTables treats a CF as finished only when its normal predicate holds and the counter is zero. Because the memtables are removed and the counter incremented under one continuous mutex hold (and, for atomic flush, all CFs are reserved before the per-CF notify loop starts releasing the mutex), a listener_wait waiter can never observe the flush as finished before the callbacks complete. The counter is per-CF rather than per-flush, so a listener_wait waiter can wait somewhat longer than strictly necessary: under rapid flushing it may block until an unrelated, later flush's OnFlushCompleted on the same CF also drains (e.g. a waiter satisfied by GetEarliestMemTableID() advancing still waits for the whole CF's pending notifications). This can only over-wait, never return early, and is tolerated in favor of implementation simplicity; a per-memtable-id watermark could tighten it if it ever matters. To keep the counter balanced against future edits, the release uses an RAII Defer (runs on every exit path) and ~MemTableList asserts the counter has drained to zero; SubPendingFlushNotifications asserts against underflow. A reservation is taken only when a callback will actually fire (committed flush info non-empty, not a mempurge, CF not dropped, listeners registered). Also adds the C API rocksdb_flushoptions_set/get_listener_wait (regenerated via tools/c_api_gen) and a DBImpl::TEST_FlushMemTableWithListenerWait() helper. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14992 Test Plan: New db_flush_test cases ListenerWaitWaitsForOnFlushCompleted and ListenerWaitAtomicFlushWaitsForAllOnFlushCompleted use a listener that blocks inside OnFlushCompleted plus spurious bg_cv_ wakeups to deterministically verify that Flush(wait=true, listener_wait=true) does not return until the callback (all CFs, for atomic flush) has finished; the atomic case guards the per-CF mutex-release gap in the notify loop. Converted existing racy/worked-around cases to the new option: c_test table_filter_listener_metadata, listener_test OnSingleDBFlushTest and MultiDBMultiListeners, and db_flush_test FireOnFlushCompletedAfterCommittedResult (the last previously relied on TEST_WaitForBackgroundWork()). The auto-generated C API roundtrip test covers the new flushoptions field. db_stress StressTest::TestFlush now sets listener_wait on a hard-coded 1 in 4 of its manual flushes (both the atomic and non-atomic branches), so the crash test exercises the new path with the DbStressListener registered. Reviewed By: jaykorean Differential Revision: D113425407 Pulled By: pdillinger fbshipit-source-id: 6fd0f5bc93fabaccd8010c2fead1d93c4c44ccfb