mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-23 08:09:02 +03:00
a38cd3a439
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15331 ## Motivation The parent diff already implements cross-DB flushable-memory accounting and a background sorter that ranks DBs outside the write path. This change does not alter how the best candidate is selected. The remaining liveness problem occurs after the sorter selects a DB and hands it off for execution. Previously, the coordinator tracked one remote handoff as pending and waited for that DB to report completion before advancing. A selected DB could therefore monopolize coordinator progress if its job waited in Env::LOW, blocked on the DB mutex, waited behind an existing write leader or write stall, or otherwise failed to complete promptly. Other ranked DBs could be ready to reclaim memory but would not be attempted while the first handoff remained pending. ## Solution Make cross-DB flush handoff execution bounded and bypassable. Cross-DB attempts now run on a small four-worker executor owned by the WriteBufferManager instead of consuming Env::LOW compaction capacity. The executor is created only for kFlushLargestAcrossDBs and its workers are retired when that policy is disabled. The coordinator distinguishes queued and executing handoffs. A candidate lease begins when an executor worker starts the callback, rather than while the callback is merely waiting in the executor queue. If an executing handoff exceeds its lease, the coordinator releases ownership of that attempt and advances to the next ranked candidate. The original callback is not cancelled and may still complete, so the executor and outstanding-handoff count provide a strict bound on concurrent stuck work. Each registration has a per-DB outstanding-job reservation, preventing the sorter from dispatching the same DB again while an earlier callback remains alive. The coordinator also records the identity of the active candidate, so late completion from an expired attempt cannot clear or corrupt a newer candidate lease. DB mutex acquisition, write-thread joining, memtable switching, and flush dispatch continue to happen outside registry and coordinator locks. The executor invokes the existing BackgroundCallWBMFlush path directly; actual immutable-memtable flushing still uses RocksDB normal flush infrastructure. ## Compatibility This behavior is limited to kFlushLargestAcrossDBs. Existing policies retain their previous local flush behavior, and no WBM executor threads are started for them. ## Validation Unit coverage exercises lease expiry, advancement to the next candidate, late completion, duplicate suppression, bounded worker capacity, policy changes with blocked callbacks, shutdown, and reranking after progress. The end-to-end stack was also compared against the original local policy in one matched memory-pressure run with 1,000 DBs and 16 writers. The complete largest-across-DBs policy delivered 61.0% higher throughput, 55.5% fewer WBM flushes, 37.9% lower mean latency, 47.9% lower P99 latency, 5.17x larger average flushes, and 62.8% lower peak RSS. These measurements validate the full policy stack rather than isolating the performance contribution of this liveness follow-up. Reviewed By: xingbowang Differential Revision: D123461329 fbshipit-source-id: 4ad86102c2a43beb469f7672d722674dfbda9871