mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-23 08:09:02 +03:00
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15017 ## Summary RocksDB async `GetAsync` and `MultiGetAsync` callbacks now default to a completion-only path. Callbacks can override `EnableStats()` when they need request-specific `PerfContext` or `IOStatsContext`; otherwise RocksDB avoids creating a coroutine stats context and returns null stats pointers. Callbacks must not initiate another async read from `OnComplete()`. RocksDB stats normally use thread-local storage, which does not compose with interleaved coroutine reads. A stats-enabled request captures only the caller's configuration: perf level, per-level perf enablement, and whether io stats are disabled. It then starts with empty request contexts. `onSet()` loads that request's contexts into TLS, while `onUnset()` saves the accumulated contexts back into request data. On resume, the same contexts are loaded again. This isolates request counters without preserving or restoring pre-existing executor TLS state. The synchronous fallback similarly resets TLS before executing a stats-enabled read. `PerfContext` and `IOStatsContext` gain `Merge()` support for caller-side aggregation. `PerfContext` move construction and assignment now transfer ownership of per-level state rather than deep-copying it, avoiding allocations during coroutine context handoffs. A fbcode-wide audit found no caller that reads a moved-from `PerfContext`. `db_bench` opts in only for coroutine jobs that print per-job perf context and merges every completed request's callback stats into the job aggregate. ## Example: Two Async Gets on One Thread Both callbacks enable stats. Each request starts with empty counters and carries its contexts across suspensions. ```text Shared executor thread | | onSet(A): load A's empty contexts into TLS | A records 3 reads | onUnset(A): save TLS contexts back into A | | onSet(B): load B's empty contexts into TLS | B records 4 reads | onUnset(B): save TLS contexts back into B | | onSet(A): load A's contexts containing 3 reads | A records 2 more reads | callback A receives block_read_count = 5 | | onSet(B): load B's contexts containing 4 reads | callback B receives block_read_count = 4 ``` Isolation comes from replacing TLS with the active request's saved contexts on every `onSet()`. ### Timers Existing wall-clock timers remain in coroutine frames, so timers spanning `co_await` include suspended time and measure end-to-end latency. Request-level `get_cpu_nanos` is suspension-aware: it starts in `onSet()` and stops in `onUnset()` or `Finalize()`, charging only time while the request is active. Nested CPU timers remain disabled because normal RAII CPU timers would also charge executor work performed while their coroutine is suspended. `MultiGetAsync` fanout additionally requires per-child timer state to avoid double counting. ## Benchmarks Benchmarked to confirm no performance regressions when perf is disabled, and also to see the overhead of enabling perf. ```bash db_bench --benchmarks=<readrandomcoroutine|multireadrandomcoroutine> --db=<db_path> --coro_jobs_per_thread=10 --use_existing_db=true --num=2000000 --value_size=1024 --compression_type=none --threads=8 --batch_size=16 --multiread_batched=true --use_direct_reads=true --cache_size=1428076626 --reads=320000 --perf_level=<1|3> --seed=<seed> ``` | Workload | Before change | Perf level 1 | Before -> 1 | Perf level 3 | 1 -> 3 | | -------- | ------------- | ------------ | ----------- | ------------ | ------ | | Async Get | 475.6K ops/s | 476.7K ops/s | +0.2% | 470.5K ops/s | -1.3% | | Async MultiGet | 567.3K ops/s | 567.4K ops/s | +0.0% | 552.2K ops/s | -2.7% | Reviewed By: xingbowang Differential Revision: D112634874 fbshipit-source-id: 94e240aeb1496fa12dac995b5aa5a9c3543cc5fc
138 lines
4.9 KiB
C++
138 lines
4.9 KiB
C++
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
#pragma once
|
|
|
|
#include <atomic>
|
|
#include <limits>
|
|
|
|
#include "port/port.h"
|
|
#include "rocksdb/system_clock.h"
|
|
#include "test_util/mock_time_env.h"
|
|
#include "test_util/sync_point.h"
|
|
#include "util/random.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
// NOTE: SpecialEnv offers most of this functionality, along with hooks
|
|
// for safe DB behavior under a mock time environment, so should be used
|
|
// instead of MockSystemClock for DB tests.
|
|
class MockSystemClock : public SystemClockWrapper {
|
|
public:
|
|
explicit MockSystemClock(const std::shared_ptr<SystemClock>& base)
|
|
: SystemClockWrapper(base) {}
|
|
|
|
static const char* kClassName() { return "MockSystemClock"; }
|
|
const char* Name() const override { return kClassName(); }
|
|
Status GetCurrentTime(int64_t* time_sec) override {
|
|
assert(time_sec != nullptr);
|
|
*time_sec = static_cast<int64_t>(current_time_us_ / kMicrosInSecond);
|
|
return Status::OK();
|
|
}
|
|
|
|
virtual uint64_t NowSeconds() { return current_time_us_ / kMicrosInSecond; }
|
|
|
|
uint64_t NowMicros() override { return current_time_us_; }
|
|
|
|
uint64_t NowNanos() override {
|
|
assert(current_time_us_ <= std::numeric_limits<uint64_t>::max() / 1000);
|
|
return current_time_us_ * 1000;
|
|
}
|
|
|
|
uint64_t CPUMicros() override {
|
|
if (!mock_cpu_time_.load(std::memory_order_relaxed)) {
|
|
return SystemClockWrapper::CPUMicros();
|
|
}
|
|
return current_cpu_time_ns_.load(std::memory_order_relaxed) / 1000;
|
|
}
|
|
|
|
uint64_t CPUNanos() override {
|
|
if (!mock_cpu_time_.load(std::memory_order_relaxed)) {
|
|
return SystemClockWrapper::CPUNanos();
|
|
}
|
|
return current_cpu_time_ns_.load(std::memory_order_relaxed);
|
|
}
|
|
|
|
uint64_t RealNowMicros() { return target_->NowMicros(); }
|
|
|
|
void SetCurrentTime(uint64_t time_sec) {
|
|
assert(time_sec < std::numeric_limits<uint64_t>::max() / kMicrosInSecond);
|
|
assert(time_sec * kMicrosInSecond >= current_time_us_);
|
|
current_time_us_ = time_sec * kMicrosInSecond;
|
|
}
|
|
|
|
// It's a fake sleep that just updates the Env current time, which is similar
|
|
// to `NoSleepEnv.SleepForMicroseconds()` and
|
|
// `SpecialEnv.MockSleepForMicroseconds()`.
|
|
// It's also similar to `set_current_time()`, which takes an absolute time in
|
|
// seconds, vs. this one takes the sleep in microseconds.
|
|
// Note: Not thread safe.
|
|
void SleepForMicroseconds(int micros) override {
|
|
assert(micros >= 0);
|
|
assert(current_time_us_ + static_cast<uint64_t>(micros) >=
|
|
current_time_us_);
|
|
current_time_us_.fetch_add(micros);
|
|
}
|
|
|
|
void MockSleepForSeconds(int seconds) {
|
|
assert(seconds >= 0);
|
|
uint64_t micros = static_cast<uint64_t>(seconds) * kMicrosInSecond;
|
|
assert(current_time_us_ + micros >= current_time_us_);
|
|
current_time_us_.fetch_add(micros);
|
|
}
|
|
|
|
void SetCurrentCPUTimeNanos(uint64_t nanos) {
|
|
current_cpu_time_ns_.store(nanos, std::memory_order_relaxed);
|
|
mock_cpu_time_.store(true, std::memory_order_relaxed);
|
|
}
|
|
|
|
void MockSleepForCPUNanos(uint64_t nanos) {
|
|
assert(mock_cpu_time_.load(std::memory_order_relaxed));
|
|
assert(current_cpu_time_ns_.load(std::memory_order_relaxed) <=
|
|
std::numeric_limits<uint64_t>::max() - nanos);
|
|
current_cpu_time_ns_.fetch_add(nanos, std::memory_order_relaxed);
|
|
}
|
|
|
|
bool TimedWait(port::CondVar* cv,
|
|
std::chrono::microseconds deadline) override {
|
|
uint64_t now_micros = NowMicros();
|
|
uint64_t deadline_micros = static_cast<uint64_t>(deadline.count());
|
|
uint64_t delay_micros;
|
|
if (deadline_micros > now_micros) {
|
|
delay_micros = deadline_micros - now_micros;
|
|
} else {
|
|
delay_micros = 0;
|
|
}
|
|
// To prevent slowdown, this `TimedWait()` is completely synthetic. First,
|
|
// it yields to coerce other threads to run while the lock is released.
|
|
// Second, it randomly selects between mocking an immediate wakeup and a
|
|
// timeout.
|
|
cv->GetMutex()->Unlock();
|
|
std::this_thread::yield();
|
|
bool mock_timeout = Random::GetTLSInstance()->OneIn(2);
|
|
if (mock_timeout) {
|
|
TEST_SYNC_POINT("MockSystemClock::TimedWait:UnlockedPreSleep");
|
|
current_time_us_.fetch_add(delay_micros);
|
|
TEST_SYNC_POINT("MockSystemClock::TimedWait:UnlockedPostSleep1");
|
|
TEST_SYNC_POINT("MockSystemClock::TimedWait:UnlockedPostSleep2");
|
|
}
|
|
cv->GetMutex()->Lock();
|
|
return mock_timeout;
|
|
}
|
|
|
|
// TODO: this is a workaround for the different behavior on different platform
|
|
// for timedwait timeout. Ideally timedwait API should be moved to env.
|
|
// details: PR #7101.
|
|
void InstallTimedWaitFixCallback();
|
|
|
|
private:
|
|
std::atomic<uint64_t> current_time_us_{0};
|
|
std::atomic<uint64_t> current_cpu_time_ns_{0};
|
|
std::atomic<bool> mock_cpu_time_{false};
|
|
static constexpr uint64_t kMicrosInSecond = 1000U * 1000U;
|
|
};
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|