507206e46b
Summary: `BlockBasedTableIterator::SeekImpl()` sets `seek_key_prefix_for_readahead_trimming_` only when given a seek key, and `ResetBlockCacheLookupVar()` does not clear it. After a `Seek()`, a `SeekToFirst()` on the same iterator trims readahead against the previous seek key's prefix. `DBIter::SeekToFirst()` under `prefix_same_as_start` takes its prefix from the first key found, so the old prefix does not describe the keys being scanned. With the stale prefix, `IsNextBlockOutOfReadaheadBound()` is true at every block instead of only at the prefix boundary, and each prefetch is clamped to the current block. The same predicate sets `is_index_out_of_bound_`, which `FindBlockForward()` converts to `is_out_of_bound_` and ends the scan. Under `ReadOptions::async_io` this drops keys. The bug needs `auto_readahead_size` and a preceding `Seek()` that reads no data block. A `Seek()` that lands on a key leaves the prefetch buffer anchored above the scan, and the trimming callback is not called. The bug does not occur under `allow_mmap_reads`, where `FilePrefetchBuffer` is constructed disabled. The fix clears the prefix in `SeekToFirst()`, so prefix-based trimming is disabled for that positioning. The `auto_readahead_size` documentation in `include/rocksdb/options.h` already says prefix-based trimming requires `Seek(key)`. On buffered reads with `prefix_same_as_start`, `auto_readahead_size` and `readahead_size=8192`, a cold-cache `SeekToFirst()` scan of 2M keys in a single 222 MB file, preceded on the same iterator by a `Seek()` past the last key, took 391 ms before and 308 ms after. The same scan with no preceding `Seek()` takes 307 ms. Under direct IO on a local disk it went from 33.1 s to 11.8 s. Data blocks read are unchanged; file reads drop by 3x. Also fixes a pre-existing test bug: `SeekParallelizationTestWithPosix` asserts on `number_async_seek` without resetting that cumulative thread-local, unlike its three siblings. The new tests move `prefetch_test` from 11 shards to 13, so different tests share a process, and the assertion fails at the parent commit under that grouping. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15216 Test Plan: New regression test in `PrefetchTrimReadaheadTestParam`, in sync and `async_io` variants over the fixture's three `index_shortening` modes. It runs two scans of the same keys from an equally cold block cache, differing only in a preceding `Seek()`. Reverting the `clear()` fails 3 of 6 sync parameterizations on `READAHEAD_TRIMMED` (6 vs 40) and 3 of 6 `async_io` parameterizations on key count (40 vs 1). `make check` and `ASSERT_STATUS_CHECKED=1 make check` show no new failures. `prefetch_test` 122/122, `db_iterator_test` 249/249, `table_test` 7629/7629, `block_based_table_reader_test` 5347/5347, `db_basic_test` 300/300. New cases ran 1200 times under `COERCE_CONTEXT_SWITCH=1` without failure. Added db_stress coverage for `SeekToFirst()` after `Seek()` under `prefix_same_as_start`. With the fix reverted and io_uring available, the configuration in the second commit fails 8 of 8 seeds with "Iterator diverged from control iterator"; with the fix, the same seeds pass. Reviewed By: rban1 Differential Revision: D120228466 Pulled By: xingbowang fbshipit-source-id: a4935f03a0080cc8dd4c7a8c1438404e9b8f5c3a
User Documentation for rocksdb.org
This directory will contain the user and feature documentation for RocksDB. The documentation will be hosted on GitHub pages.
Contributing
See CONTRIBUTING.md for details on how to add or modify content.
Run the Site Locally
The requirements for running a GitHub pages site locally is described in GitHub help. The steps below summarize these steps.
If you have run the site before, you can start with step 1 and then move on to step 5.
-
Ensure that you are in the
/docsdirectory in your local RocksDB clone (i.e., the same directory where thisREADME.mdexists). The below RubyGems commands, etc. must be run from there. -
Make sure you have Ruby and RubyGems installed.
Ruby >= 2.2 is required for the gems. On the latest versions of Mac OS X, Ruby 2.0 is the default. Use
brew install ruby(or your preferred upgrade mechanism) to install a newer version of Ruby for your Mac OS X system. -
Make sure you have Bundler installed.
# may require sudo gem install bundler -
Install the project's dependencies
# run this in the 'docs' directory bundle installIf you get an error when installing
nokogiri, you may be running into the problem described in this nokogiri issue. You can eitherbrew uninstall xz(and thenbrew install xzafter the bundle is installed) orxcode-select --install(although this may not work if you have already installed command line tools). -
Run Jekyll's server.
- On first runs or for structural changes to the documentation (e.g., new sidebar menu item), do a full build.
bundle exec jekyll serve- For content changes only, you can use
--incrementalfor faster builds.
bundle exec jekyll serve --incrementalWe use
bundle execinstead of running straightjekyllbecausebundle execwill always use the version of Jekyll from ourGemfile. Just runningjekyllwill use the system version and may not necessarily be compatible.- To run using an actual IP address, you can use
--host=0.0.0.0
bundle exec jekyll serve --host=0.0.0.0This will allow you to use the IP address associated with your machine in the URL. That way you could share it with other people.
e.g., on a Mac, you can your IP address with something like
ifconfig | grep "inet " | grep -v 127.0.0.1. -
Either of commands in the previous step will serve up the site on your local device at http://127.0.0.1:4000/ or http://localhost:4000.
Updating the Bundle
The site depends on Github Pages and the installed bundle is based on the github-pages gem.
Occasionally that gem might get updated with new or changed functionality. If that is the case,
you can run:
bundle update
to get the latest packages for the installation.