mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-24 19:59:35 +03:00
2850ccb96b
Summary: initial support for Prepare() to optimize the performance of MultiScan when using block-based tables. In Prepare(), we do the following: 1. Load all data blocks that will be read in multiscan to block cache 2. Pin the data blocks during the scan 3. if I/O is needed, coalesce I/Os when they are adjacent. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13778 Test Plan: Added a new unit test. Benchmark: 1. Set up the DB, I use FIFO here so that files will be in L0 and iterator will use BlockBasedTableIterator directly instead of LevelIterator, where Prepare() call is not implemented yet. ``` ./db_bench --benchmarks="fillseq,compact" --disable_wal=1 --threads=1 --num_levels=1 --compaction_style=2 --fifo_compaction_max_table_files_size_mb=1000 --write_buffer_size=268435456 ``` 2. Multi-scan: based on https://github.com/facebook/rocksdb/issues/13765 ``` ./db_bench --db="/tmp/rocksdbtest-543376/dbbench" --use_existing_db=1 --benchmarks=multiscan --disable_auto_compactions=1 --seek_nexts=100 --threads=32 --duration=10 --statistics=1 multiscan_stride = 100 multiscan_size = 10 seek_nexts = 100 Main: multiscan : 449.386 micros/op 70562 ops/sec 10.359 seconds 730968 operations; (multscans:22999) multiscan : 453.606 micros/op 69433 ops/sec 10.369 seconds 719968 operations; (multscans:22999) rocksdb.non.last.level.read.bytes COUNT : 47763519421 rocksdb.non.last.level.read.count COUNT : 21573878 Branch: multiscan : 332.670 micros/op 94698 ops/sec 10.285 seconds 973968 operations; (multscans:29999) rocksdb.non.last.level.read.bytes COUNT : 111791308336 rocksdb.non.last.level.read.count COUNT : 1062942 With direct-IO: ./db_bench --db="/tmp/rocksdbtest-543376/dbbench" --use_existing_db=1 --benchmarks=multiscan --disable_auto_compactions=1 --seek_nexts=100 --threads=32 --duration=10 --statistics=1 --use_direct_reads=1 Main: multiscan : 586.045 micros/op 53825 ops/sec 10.366 seconds 557968 operations; (multscans:14999) rocksdb.non.last.level.read.bytes COUNT : 69107458693 rocksdb.non.last.level.read.count COUNT : 6724651 Branch: multiscan : 386.679 micros/op 81282 ops/sec 10.359 seconds 841968 operations; (multscans:25999) rocksdb.non.last.level.read.bytes COUNT : 96605800558 rocksdb.non.last.level.read.count COUNT : 918973 ``` Throughput is 36% higher with non-direct IO and 50% higher with direct IO. The improvement is likely from doing less number of I/Os due to I/O coalescing during Prepare(), as shown in `rocksdb.non.last.level.read.count`. The total bytes read is more with this PR for the same reason. 3. Regular iterator: ``` ./db_bench --use_existing_db=1 --db="/tmp/rocksdbtest-543376/dbbench" --benchmarks=seekrandom --disable_auto_compactions=1 --seek_nexts=10 --threads=32 --duration=10 Main: seekrandom : 13.014 micros/op 2456735 ops/sec 10.014 seconds 24602968 operations; 2717.8 MB/s (773999 of 773999 found) Branch: seekrandom : 13.048 micros/op 2450554 ops/sec 10.013 seconds 24537968 operations; 2710.9 MB/s (772999 of 772999 found) ``` The result fluctuates but without noticeable regression. Reviewed By: anand1976 Differential Revision: D78440807 Pulled By: cbi42 fbshipit-source-id: 80ac6fd222696fa65ac0b4b5441748be5ee0b979