Summary:
To be compatible with some upcoming compression change/refactoring where we supply a fixed size buffer to CompressBlock, we need to support CompressedSecondaryCache storing uncompressed values when the compression ratio is not suitable. It seems crazy that CompressedSecondaryCache currently stores compressed values that are *larger* than the uncompressed value, and even explicitly exercises that case (almost exclusively) in the existing unit tests. But it's true.
This change fixes that with some other nearby refactoring/improvement:
* Update the in-memory representation of these cache entries to support uncompressed entries even when compression is enabled. AFAIK this also allows us to safely get rid of "don't support custom split/merge for the tiered case".
* Use more efficient in-memory representation for non-split entries
* For CompressionType and CacheTier, which are defined as single-byte data types, use a single byte instead of varint32. (I don't know if varint32 was an attempt at future-proofing for a memory-only schema or what.) Now using lossless_cast will raise a compiler error if either of these types is made too large for a single byte.
* Don't wrap entries in a CacheAllocationPtr object; it's not necessary. We can rely on the same allocator being provided at delete time.
* Restructure serialization/deserialization logic, hopefully simpler or easier to read/understand.
* Use a RelaxedAtomic for disable_cache_ to avoid race.
Suggested follow-up on CompressedSecondaryCache:
* Refine the exact strategy for rejecting compressions
* Still have a lot of buffer copies; try to reduce
* Revisit the split-merge logic and try to make it more efficient overall, more unified with non-split case
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13797
Test Plan:
Unit tests updated to use actually compressible strings in many places and more testing around non-compressible string.
## Performance Test
There was some pre-existing issue causing decompression failures in compressed secondary cache with cache_bench that is somehow fixed in this change. This decompression failures were present before the new compression API, but since then cause assertion failures rather than being quietly ignored. For the "before" test here, they are back to quietly ignored. And the cache_bench changes here were back-ported to the "before" configuration.
### No compressed secondary (setting expectations)
```
./cache_bench --cache_type=auto_hyper_clock_cache -cache_size=8000000000 -populate_cache
```
Max key : 3906250
Before:
Complete in 12.784 s; Rough parallel ops/sec = 2503123
Thread ops/sec = 160329; Lookup hit ratio: 0.686771
After:
Complete in 12.745 s; Rough parallel ops/sec = 2510717 (in the noise)
Thread ops/sec = 159498; Lookup hit ratio: 0.68686
### Compressed secondary, no split/merge
Same max key and approximate total memory size
```
/usr/bin/time ./cache_bench --cache_type=auto_hyper_clock_cache -cache_size=4000000000 -populate_cache -resident_ratio=0.125 -compressible_to_ratio=0.4 --secondary_cache_uri=compressed_secondary_cache://capacity=4000000000
```
Before:
Complete in 18.690 s; Rough parallel ops/sec = 1712144
Thread ops/sec = 108683; Lookup hit ratio: 0.776683
Latency: P50: 4205.19 P75: 15281.76 P99: 43810.98 P99.9: 71487.41 P99.99: 165453.32
max RSS (according to /usr/bin/time): 9341856
After:
Complete in 17.878 s; Rough parallel ops/sec = 1789951 (+4.5%)
Thread ops/sec = 114957; Lookup hit ratio: 0.792998 (+0.016)
Latency: P50: 4012.70 P75: 14477.63 P99: 40039.70 P99.9: 62521.04 P99.99: 167049.18
max RSS (according to /usr/bin/time): 9235688
The improved hit ratio is probably from fixing the failed decompressions (somehow). And my modifications could have improved CPU efficiency, or it could be the small penalty the benchmark naturally imposes on most misses (generate another value and insert it).
### Compressed secondary, with split/merge
```
/usr/bin/time ./cache_bench --cache_type=auto_hyper_clock_cache -cache_size=4000000000 -populate_cache -resident_ratio=0.125 -compressible_to_ratio=0.4 --secondary_cache_uri='compressed_secondary_cache://capacity=4000000000;enable_custom_split_merge=true'
```
Before:
Complete in 20.062 s; Rough parallel ops/sec = 1595075
Thread ops/sec = 101759; Lookup hit ratio: 0.787129
Latency: P50: 5338.53 P75: 16073.46 P99: 46752.65 P99.9: 73459.11 P99.99: 201318.75
max RSS (according to /usr/bin/time): 9049852
After:
Complete in 18.564 s; Rough parallel ops/sec = 1723771 (+8.1%)
Thread ops/sec = 110724; Lookup hit ratio: 0.813414 (+0.026)
Latency: P50: 5234.75 P75: 14590.43 P99: 41401.03 P99.9: 65606.50 P99.99: 157248.04
max RSS (according to /usr/bin/time): 8917592
Looks like an improvement
Reviewed By: anand1976
Differential Revision: D78842120
Pulled By: pdillinger
fbshipit-source-id: 5f754b160c37ebee789279178ebb5e862071bdb2
Summary:
* Mostly, remove `sample_for_compression` from CompressionInfo because it's not used by the core function it serves, `CompressData()`. Confusing (and inefficient), especially in db_bench where it appears to use `FLAGS_sample_for_compression` in places where it is actually ignored.
* Various clarifying comments, clean-ups, and tiny optimizations
* Prepare some structures like `CompressionDict` for more usage
* Some TODOs and FIXMEs about some things I've noticed are amiss, confusing, or excessive
* A notable optimization opportunity that might become a "pay as you go" improvement for the potential indirection costs of customizable compression: use C++23's resize_and_overwrite() in compress functions to avoid zeroing the string buffer contents before populating it.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13539
Test Plan: existing tests / CI
Reviewed By: hx235
Differential Revision: D73451273
Pulled By: pdillinger
fbshipit-source-id: 0373627466d695043d21146ce34d52f189ae9432
Summary:
The following are risks associated with pointer-to-pointer reinterpret_cast:
* Can produce the "wrong result" (crash or memory corruption). IIRC, in theory this can happen for any up-cast or down-cast for a non-standard-layout type, though in practice would only happen for multiple inheritance cases (where the base class pointer might be "inside" the derived object). We don't use multiple inheritance a lot, but we do.
* Can mask useful compiler errors upon code change, including converting between unrelated pointer types that you are expecting to be related, and converting between pointer and scalar types unintentionally.
I can only think of some obscure cases where static_cast could be troublesome when it compiles as a replacement:
* Going through `void*` could plausibly cause unnecessary or broken pointer arithmetic. Suppose we have
`struct Derived: public Base1, public Base2`. If we have `Derived*` -> `void*` -> `Base2*` -> `Derived*` through reinterpret casts, this could plausibly work (though technical UB) assuming the `Base2*` is not dereferenced. Changing to static cast could introduce breaking pointer arithmetic.
* Unnecessary (but safe) pointer arithmetic could arise in a case like `Derived*` -> `Base2*` -> `Derived*` where before the Base2 pointer might not have been dereferenced. This could potentially affect performance.
With some light scripting, I tried replacing pointer-to-pointer reinterpret_casts with static_cast and kept the cases that still compile. Most occurrences of reinterpret_cast have successfully been changed (except for java/ and third-party/). 294 changed, 257 remain.
A couple of related interventions included here:
* Previously Cache::Handle was not actually derived from in the implementations and just used as a `void*` stand-in with reinterpret_cast. Now there is a relationship to allow static_cast. In theory, this could introduce pointer arithmetic (as described above) but is unlikely without multiple inheritance AND non-empty Cache::Handle.
* Remove some unnecessary casts to void* as this is allowed to be implicit (for better or worse).
Most of the remaining reinterpret_casts are for converting to/from raw bytes of objects. We could consider better idioms for these patterns in follow-up work.
I wish there were a way to implement a template variant of static_cast that would only compile if no pointer arithmetic is generated, but best I can tell, this is not possible. AFAIK the best you could do is a dynamic check that the void* conversion after the static cast is unchanged.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12308
Test Plan: existing tests, CI
Reviewed By: ltamasi
Differential Revision: D53204947
Pulled By: pdillinger
fbshipit-source-id: 9de23e618263b0d5b9820f4e15966876888a16e2
Summary:
In IDE navigation I find it annoying that there are two statistics.h files (etc.) and often land on the wrong one. Here I migrate several headers to use the blah.h <- blah_impl.h <- blah.cc idiom. Although clang-format wants "blah.h" to be the top include for "blah.cc", I think overall this is an improvement.
No public API changes.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/11408
Test Plan: existing tests
Reviewed By: ltamasi
Differential Revision: D45456696
Pulled By: pdillinger
fbshipit-source-id: 809d931253f3272c908cf5facf7e1d32fc507373
