mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-23 08:09:02 +03:00
8f492b2f47
Summary:
Building on D116105558, this change finishes and publishes `format_version = 8` as
production-ready.
The headline feature is a tweaked block encoding that stores the common user-key
prefix shared by all keys in a block **once**, in the block's leading bytes
`[0, restarts[0])`, instead of repeating it at every restart point. This shrinks
prefix-heavy blocks (and their block-cache footprint) and, for bytewise-ordered
keys, speeds up `Seek` by comparing prefix-stripped suffixes. It is applied to
**data blocks** and **every index tier** (leaf/partition index, the
partitioned-index top-level, and the partitioned-filter top-level).
Motivation: MyRocks/ZippyDB/timeseries-style keys carry a long shared prefix
(table/index id, partition, etc.). Repeating it at each restart wastes space and
cache, and every Seek re-compares those bytes. Profiles show index-block seek CPU
is comparable to data-block seek CPU (partitioned index does two index seeks per
data seek), so index blocks are a first-class target, not just data blocks.
### Format (self-describing, no new footer bit)
- The common prefix occupies the block's leading bytes; a non-zero `restarts[0]`
(the offset of the first entry, i.e. the prefix length) self-signals its
presence. `restarts[0]` is 0 in every block that does not use the feature (all
versions, all block types), so a single fv8 reader transparently handles any
mix of stripped and unstripped blocks -- no footer bit is consumed.
- Previous versions would misinterpret this encoding by quietly ignoring the
common prefix, which (along with the lack of extensibility in data-block
metadata) is why it is gated behind format_version 8.
- Only restart-point entries change on disk; non-restart entries are byte
identical to fv7, so the writer produces the exact same bytes regardless of
implementation.
### Public option
New `BlockBasedTableOptions::optimize_key_common_prefix`
(`enum class OptimizeKeyCommonPrefix`), default `kIfFastSeek`:
- `kDisabled` -- never use it.
- `kIfFastSeek` (default) -- use it only where it also enables the Seek speedup:
format_version >= 8 with a built-in (reverse-)bytewise comparator.
- `kEnabled` -- use it for all comparators at fv8; (reverse-)bytewise get space +
fast seek, other comparators get the space savings only (the reader
reconstructs full keys and compares with the real comparator).
Expressed as a criteria so it need not be toggled per CF based on the comparator
(which can vary by CF). It additionally requires `use_delta_encoding` (the
feature builds on delta encoding) and `ts_sz == 0`; when `use_delta_encoding` is
false -- the setting that also preserves zero-copy key pinning -- it is a no-op.
Default format_version is unchanged (still 7); the feature is opt-in via fv8.
### Scope: every index tier
The reader handles all index-like blocks uniformly (a non-zero `restarts[0]`
self-signals a stripped block regardless of block type), so the writer strips the
common prefix from every tier:
- **Data blocks** and **leaf / partition index blocks**: enabled.
- **Partitioned-index top-level block**: enabled. Under the default
`index_shortening = kShortenSeparatorsAndSuccessor` the file's last separator
would otherwise be a short successor sharing no leading bytes, collapsing the
single-block top-level's common prefix to 0. When common-prefix is on the
builder suppresses that one successor (keeps the full last key -- i.e.
`kShortenSeparators` behavior for that one entry), so the top-level strips
uniformly under every shortening mode; the full last key is also a tighter file
upper bound, so it is not worse.
- **Partitioned-filter top-level index**: enabled (same uniform treatment). Its
space/CPU effect is small at default partitioning (the top-level filter index
is tiny) but grows with many small partitions -- a filter miss short-circuits
at, and is dominated by, the top-level filter-index seek (see Performance).
Included for a uniform on-disk surface across tiers.
Bloom filter *contents* still hash full user keys, so prefix/whole-key filtering
is unaffected.
### Implementation
- Writer (`block_builder.{h,cc}`): keys are encoded incrementally in the fv7
layout while the running common prefix is tracked cheaply (reusing the delta
`shared`); at `Finish()` only the restart-point entries are rewritten in place
with the prefix stripped, and the prefix is written once at the block start.
The rewrite is generalized to the value-delta (fv4) index layout used by index
blocks (no `value_size` header; bulk-copy the inline BlockHandle and
non-restart entries verbatim).
- Reader (`block.{h,cc}`, `dbformat.h`): `Block` decodes the prefix from
`restarts[0]`; `BlockIter::ParseNextKey` prepends it on a restart point
(`shared == 0`). The suffix-seek helpers (strip target once, compare shortened
suffixes) live in the templated `BlockIter` base and are shared by data and
index iterators (binary search, templated on `DecodeKey`/`DecodeKeyV4`); for
non-bytewise comparators (data) and the hash `PrefixSeek` refine step, the
reader reconstructs the probed restart key and compares full keys.
- Gating (`block_based_table_builder.cc`, `index_builder.{h,cc}`,
`partitioned_filter_block.{h,cc}`): the criteria is threaded to the data-block
builder and to every index tier -- leaf/partition index, the partitioned-index
top-level (`use_common_prefix_top`), and the partitioned-filter top-level --
together with the last-separator successor suppression
(`ShortenedIndexBuilder::GetSeparatorWithSeq`) that lets the top-level strip
under the default shortening mode. The index base gate also requires
`super_block_alignment_size == 0` (block alignment can trigger
`skip_delta_encoding`, producing non-restart `shared == 0` entries the reader
could not distinguish from stripped restart keys).
- Memory: the per-block/per-iterator prefix is stored as a `uint32_t` length
(the prefix is always at offset 0) rather than a full `Slice`, and `Block` /
`BlockBuilder` members were repacked so the feature adds little or no struct
size.
- Java: a new `OptimizeKeyCommonPrefix` enum and a `BlockBasedTableConfig`
getter/setter, plumbed through JNI in both directions (the
`newTableFactoryHandle` writer and the options-reconstruction constructor).
### Whole-file (table-level) prefix: investigated and rejected
A natural extension is to store the whole-file common prefix once per *file*
(when an `SstPartitioner` guarantees every key in the file shares a fixed prefix)
instead of once per *block* -- e.g. the MyRocks index/table id, or a future
128-bit global id. It was prototyped end-to-end (partitioner-provided prefix
captured at file open, stored in a `user_key_common_prefix` table property, and
stripped from every data/index block; the reader re-prepends it) and then
dropped. The `user_key_common_prefix` table property reserved in the
previous diff is dropped in this one. The rationale, in case it is revisited:
- Marginal benefit over per-block. Per-block already strips the full shared
prefix from every key; the only thing whole-file removes on top is the single
per-block copy of that prefix, i.e. `~= prefix_len / block_size` of the
uncompressed (block-cache) footprint, with no meaningful read-CPU change
(per-block already delivers strip-once + suffix compares). Measured at
prefix_len=12, 4 KB blocks this was ~-0.3% incremental space; at the 8-16 KB
data blocks typical of the space-constrained target (MyRocks/UDB) it is
~0.1-0.2% even with a 16-byte global id -- a fraction of a percent, seen only
by partitioner users. Small block-size workloads are becoming more
niche, and even where they exist, they are also CPU-sensitive (see next
paragraph).
- Real, if small, CPU tax on the *per-block* path. Reconstructing a key from two
non-contiguous parts (file prefix + block residual) cost ~0.5-1.0% additional
retired instructions on the per-block read path (deterministic across gcc and
clang), wall-clock-neutral in cache-resident tests but present even when no
whole-file prefix is configured. Attempts to remove it did not pan out: a
compile-time specialization of the reconstruction *increased* instructions
(+2.9%) by defeating inlining; a per-block combined-prefix buffer trades the
CPU cost for a memory cost on every (default) block; and a per-iterator buffer
was the original larger regression. Only invasive iterator-level
monomorphization would truly zero it, which is not justified on this hot path.
- Decision. A fraction-of-a-percent, partitioner-only space win does not justify
a permanent per-block-read-path CPU tax plus additional on-disk format surface
(a table property that changes whole-file interpretation). Dropped. The generic
`SstPartitioner::ShouldPartitionByPrefix` hook is kept (cheap, and useful
independently), and the reserved `user_key_common_prefix` table property is
removed since no writer will emit it.
### Performance (db_bench, release/native, cache-warm, best-of-3, pinned)
Three arms: (A) fv7 on the base/main branch, (B) fv7 with this change (should be
neutral / byte-identical writing), (C) fv8 with this change and
`optimize_key_common_prefix=enabled` (the optimization is active). Partitioned
index + filters; `compression_type=none`, so SST bytes are the uncompressed
block-cache footprint. `int` = db_bench default keys (variation-early, near worst
case); `pfxN` = an N-byte shared prefix + varying suffix (MyRocks-like).
Setup: `num=3,000,000`, `reads=1,500,000`, single thread,
`metadata_block_size=4096`, cache holds the working set, pinned to one core.
| shape (key/val) | arm | SST bytes | readrandom us/op | seekrandom us/op |
|---|---|---:|---:|---:|
| int (32/8) | A fv7 main | 138,983,658 | 3.484 | 4.995 |
| | B fv7 ours | 138,983,146 | 3.518 | 5.024 |
| | C fv8 ours | 137,681,420 | 3.490 | 4.986 |
| pfx24 (32/8) | A fv7 main | 71,697,531 | 3.417 | 5.561 |
| | B fv7 ours | 71,698,043 | 3.448 | 5.579 |
| | C fv8 ours | 65,578,236 | 3.284 | 5.386 |
| pfx56 (64/16) | A fv7 main | 102,942,266 | 3.624 | 5.830 |
| | B fv7 ours | 102,942,266 | 3.678 | 5.835 |
| | C fv8 ours | 90,554,642 | 3.444 | 5.626 |
fv8 vs fv7-main (C vs A) -- space / readrandom / seekrandom:
- int (32/8): **-0.9%** / ~neutral / ~neutral
- pfx24 (32/8): **-8.5%** / **-3.9%** / **-3.1%**
- pfx56 (64/16): **-12.0%** / **-5.0%** / **-3.5%**
fv7-ours vs fv7-main (B vs A): SST bytes byte-identical (within 512 B; the fv8
paths are gated off), read/seek us/op within run-to-run noise (~1%, best-of-3 on
a shared host). Net: fv8 is a space **and** read/seek-CPU win on prefix-heavy
data and neutral on variation-early (`int`) data, so `kIfFastSeek` is safe on by
default at fv8.
### Space by tier (compression=none, num=3M, mdb=4096; fv7 -> fv8)
Where the win comes from -- the index tiers strip the most:
| shape | data | leaf/partition index | top-level index | filter |
|---|---:|---:|---:|---:|
| int | -0.8% | -29% | -45% | -0.2% |
| pfx24 | -8.3% | -69% | -89% | -0.8% |
| pfx56 | -11.1% | -83% | -96% | -1.6% |
Data bytes dominate the absolute total, but every index tier is stripped:
leaf/partition index -29% to -83% and the partitioned-index top-level -45% to
-96%. Filter *blocks* barely move (-0.2% to -1.6%) because filter bits hash full
user keys; only the small top-level filter index is stripped.
### Point-miss lookups (readmissing)
A partitioned-filter miss short-circuits at the filter (no data-block read), so
it is dominated by the top-level filter-index seek plus the partition bloom. At a
typical `metadata_block_size=4096` (few partitions, tiny top-level filter index)
fv8 is neutral on misses (int +0.9%, pfx24 -1.0%, pfx56 -0.7% us/op -- all
noise). The top-level-filter stripping pays only with many small partitions: at
`metadata_block_size=256` it is pfx24 1.166->1.138 (**-2.4%**) and pfx56
1.273->1.188 (**-6.7%**) us/op.
### Backward / forward compatibility
- Default `format_version` remains 7; the feature is inert unless a CF opts into
fv8. Older RocksDB rejects fv8 files at footer decode (fail-safe).
- Within fv8, the stripped-block set a writer can emit is fixed and every fv8
reader can read it (the `restarts[0]` signal plus uniform reader handling), so
the on-disk surface is self-consistent across builds/downgrade.
### Follow-ups (not in this change)
- Suffix-aware interpolation / hash index seek (currently correct via the
reconstruction fallback, not perf-tuned).
Pull Request resolved: https://github.com/facebook/rocksdb/pull/15117
Test Plan:
- New focused unit tests (`table_test`), run at the full `format_version` matrix
including the now-supported fv8:
`GeneralTableTest.OptimizeKeyCommonPrefix` (data-block space+read+Seek
correctness: existing and missing keys straddling the prefix, before/after-all,
SeekForPrev), `OptimizeKeyCommonPrefixNonBytewise` (reconstruction path), and
`IndexBlockCommonPrefixEquivalence` across `{kBinarySearch,
kBinarySearchWithFirstKey, kHashSearch, kTwoLevelIndexSearch}` x
`{partitioned filters}` x `{bytewise, reverse-bytewise}` (so both top-level
tiers -- partitioned-index and partitioned-filter -- are exercised), asserting
identical reads and a shrunken fv8 index.
- The randomized `ParameterizedHarnessTest` now runs at fv8 (reverse-bytewise,
restart intervals `{1,16,1024}`, special/empty keys fuzzed against the feature).
- `db_crashtest.py` randomizes `format_version` including 8 and
`optimize_key_common_prefix` in {0,1,2}.
- Removed the reserved-property carve-out: dropped the `ReservedUserKeyCommonPrefix`
reader test and the C API getter for the property; C API regenerated via
`tools/c_api_gen/` (`make regen-c-api`), and the intentional public-C-API
removal is recorded in `tools/c_api_gen/api_compatibility_allowlist.json`.
- Java: `BlockBasedTableConfigTest.optimizeKeyCommonPrefix` (default
`kIfFastSeek`, plus set/get of every value).
- `table_test` `FooterTests` updated for the now-published fv8 footer, whose
metaindex-gap encoding stores the metaindex block size in 16 bits (the test
caps that size for fv >= 8).
- Manual verification: the db_bench runs in Performance above (three-arm
space + read/seek CPU, per-tier byte breakdown via `sst_dump`, and
`readmissing`); and `readrandom`/`seekrandom` with
`--block_protection_bytes_per_key=8` find 100% of keys at both fv7 and fv8
(reconstructed keys pass per-key checksum).
Reviewed By: xingbowang
Differential Revision: D116365016
Pulled By: pdillinger
fbshipit-source-id: 66f4b5c60ea4fe0ff48d34c98d191ba8ebdf0665
RocksDB: A Persistent Key-Value Store for Flash and RAM Storage
RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)
This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.
Start with example usage here: https://github.com/facebook/rocksdb/tree/main/examples
See the github wiki for more explanation.
The public interface is in include/. Callers should not include or
rely on the details of any other header files in this package. Those
internal APIs may be changed without warning.
Questions and discussions are welcome on the RocksDB Developers Public Facebook group and email list on Google Groups.
License
RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.