144deb248a
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14979 `MaybeOpenReadOnlyOnPrimary` (added in D111995911) opens a read-only DB on the primary's live directory while the primary keeps mutating it. Two separate stress-test false positives were observed there; this diff fixes both and consolidates the earlier one-off fixes into a single diff. 1) TSAN deadlock-detector crash in the CF-name snapshot (`fbcode_tsan_blackbox_crash_test`): ThreadSanitizer: CHECK failed: sanitizer_deadlock_detector.h:67 "((n_all_locks_)) < (...)" (0x80, 0x80) `n_all_locks_` is the number of mutexes a thread holds *simultaneously*; TSAN caps it at 128 (0x80). The snapshot loop called `LockColumnFamily(cf)` to read `column_family_names_[cf]`, and that helper `.Lock()`s ALL per-key mutexes for the CF at once (`ceil(max_key / 2^log2_keys_per_lock)` -- e.g. 25,000 in blackbox with max_key=100000, log2_keys_per_lock=2) without releasing any, so a single call blows past 128. The writer (`MaybeClearOneColumnFamily`) holds all key locks while renaming a CF, so a single key lock is enough to synchronize the read. Fix: take one `MutexLock` on key-lock index 0 per CF instead of `LockColumnFamily`, keeping the simultaneously-held count at 1. (`LockColumnFamily` is a latent hazard for any caller; the pre-existing caller `MaybeClearOneColumnFamily` never runs in the crash-test config -- `clear_column_family_one_in=0` -- so only this new, ungated caller exposed it.) 2) False-positive read-only Get (folded in from D112987468). After flushing the primary, the test performed a `Get(Key(0))` on each CF of the read-only instance. That Get races against concurrent primary compaction, which can delete SST files the read-only's frozen MANIFEST still references, surfacing as IOError/PathNotFound/Corruption ("truncated block read"). The Get adds no unique coverage -- the scenario under test (the read-only close wrongly deleting primary live files) is detected on the primary's own verification paths -- so remove the Get block entirely instead of widening a tolerated-status list. Note on D112936376: that diff tried to fix crash (1) by setting `table_cache_numshardbits=0` on the read-only instance, on the theory that the primary's and reader's table caches contribute "128 unique lock objects." That is a misdiagnosis: the failing CHECK counts locks held *simultaneously*, and sharded-cache shard mutexes are held transiently (one at a time; read-path nesting is a constant depth of 2), so shard count does not affect `n_all_locks_`. The real cause is `LockColumnFamily`, fixed in (1), so `table_cache_numshardbits=0` is unnecessary and is not included here. This diff subsumes and replaces D112936376 and D112987468. Reviewed By: joshkang97 Differential Revision: D112980148 fbshipit-source-id: 012ff852109223f3d19d8dbbc065fde9340f9c66
RocksDB: A Persistent Key-Value Store for Flash and RAM Storage
RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)
This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.
Start with example usage here: https://github.com/facebook/rocksdb/tree/main/examples
See the github wiki for more explanation.
The public interface is in include/. Callers should not include or
rely on the details of any other header files in this package. Those
internal APIs may be changed without warning.
Questions and discussions are welcome on the RocksDB Developers Public Facebook group and email list on Google Groups.
License
RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.