From fbd446cebbba55e4fc8bbde60fd25e9d46b91fb2 Mon Sep 17 00:00:00 2001 From: Xingbo Wang Date: Mon, 27 Jul 2026 11:00:39 -0700 Subject: [PATCH] Disable open_read_only_one_in when mmap_read is enabled (#15011) Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15011 Root cause: `open_read_only_one_in` can create a read-only DB instance on the same live directory as the primary DB. When the stress run also enables `mmap_read=1`, that read-only instance can hold mmap-backed slices into SST files while the primary continues compacting and deleting obsolete files. If `SstFileManager` uses chunked truncation (`sst_file_manager_bytes_per_truncate > 0`), Linux can raise SIGBUS when the read-only path later touches a page whose backing SST was truncated or removed. The range tombstone fragmenter/comparator is where the invalid mmap-backed key slice was touched, not necessarily the root cause. Evidence from T281873950 / Sandcastle job `63050395314610170`: the crashing command had `--mmap_read=1`, `--open_read_only_one_in=16`, and `--sst_file_manager_bytes_per_truncate=1048576`. The crash stack was `BlockBasedTable::ReadRangeDelBlock()` -> `FragmentedRangeTombstoneList::FragmentTombstones()` -> `std::set::emplace()` -> comparator -> `memcmp`, ending in signal 7/SIGBUS. The crash DB LOG shows `allow_mmap_reads: 1`; table `104662.sst` was created cleanly at size `6257069` with range tombstones and was later deleted by the primary. The artifact does not show persistent SST corruption or checksum failure, and injected read faults would normally surface as RocksDB statuses rather than an OS SIGBUS. Why this started surfacing: D111995911 / `rFBS0f7a2ef93e63cfa6677b6fa32c1bd34eb818c978` added randomized read-only-on-primary coverage via `open_read_only_one_in` on 2026-07-20. Before that change, blackbox crash test did not randomly keep a separate read-only DB open on the primary live directory, so the existing `mmap_read + SstFileManager truncation` hazard lacked the second reader needed to trigger SIGBUS. The failure is therefore a probabilistic exposure of the new stress mode when it combines with mmap reads and chunked SST truncation. Fix: disable `open_read_only_one_in` in the existing `mmap_read=1` sanitizer path. This preserves read-only-on-primary stress coverage for non-mmap runs while avoiding an OS-level mmap/truncation hazard that RocksDB cannot convert into a normal `Status`. Reviewed By: jaykorean Differential Revision: D113774892 fbshipit-source-id: f82ec2924c12f4512aeeb7a913659dc9a72a4c2f --- tools/db_crashtest.py | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/db_crashtest.py b/tools/db_crashtest.py index db581432c9..37ec28da95 100644 --- a/tools/db_crashtest.py +++ b/tools/db_crashtest.py @@ -944,6 +944,7 @@ def finalize_and_sanitize(src_params): dest_params["use_direct_reads"] = 0 dest_params["use_direct_io_for_compaction_reads"] = 0 dest_params["multiscan_use_async_io"] = 0 + dest_params["open_read_only_one_in"] = 0 if dest_params.get("min_tombstones_for_range_conversion", 0) > 0: # SQFC range-query filtering installs ReadOptions::table_filter on # iterators. Read-write iterators reject table_filter when read-path