mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-24 19:59:35 +03:00
58d835d951
Summary: Add a db_crashtest liveness mode that runs db_stress with a watchdog for foreground progress. The profile disables fault injection and uses longer runs so failures are more likely to represent real forward-progress bugs instead of expected injected I/O errors. The watchdog tracks completed operations, active operation type/start time, and per-operation completion counts to distinguish stuck writes from long admin operations. Bug: The new mode exposed a latent flush scheduling bug. A flush request can be queued with an effective max memtable id, then a newer immutable memtable can be added before the background flush picks work. PickMemtablesToFlush() could pick only the older memtables and still clear flush_requested_. A later queued flush for the newer memtable would then be skipped because the column family no longer looked flush-pending, leaving the immutable memtable permanently unflushed. In stress this surfaced as IngestExternalFile() waiting in WaitForFlushMemTables(), with writes eventually blocked behind the same unflushed memtable. Fix: Keep flush_requested_ set until all unstarted immutable memtables have been picked. Add a direct MemTableList regression for the partial-pick case and a DB-level regression covering the ingest/get-live-files ordering that reproduced the lost request. ## Testing - CI - Local liveness mode 32 full 20-minute runs passed, 0 failures Pull Request resolved: https://github.com/facebook/rocksdb/pull/14830 Reviewed By: joshkang97 Differential Revision: D107895052 Pulled By: xingbowang fbshipit-source-id: ca67c5f9376899eefdc9af7bad4af06200b52446