Summary:
### Motivation
Claude's auto review workflow classifies the PR as complex and sets `MAX_THINKING_TOKENS`=**32000** in `.github/workflows/ai-review-analysis.yml:534`. The Claude action then sends a request where `thinking.budget_tokens` is **32000**, but the request `max_tokens` is not greater than that, so Anthropic rejects it before the review starts ([example](https://github.com/facebook/rocksdb/actions/runs/25691112276/job/75427435133)).
### Fix
Reduce the "complex" thinking budget from **32000** to **24000** tokens and clamps manual overrides to the same ceiling, preventing the thinking budget from exceeding or equalling the action's effective `max_tokens`. 24k is still generous — enough for deep reasoning on complex PRs while guaranteeing the model can emit the formatted review.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14731
Reviewed By: xingbowang
Differential Revision: D104749426
Pulled By: mszeszko-meta
fbshipit-source-id: bf94d0f50e7c2c9bc9e4d4cdffd61087d739018a
Summary:
Previously, `Classify PR complexity (Codex)` ran under `bash -e`, so any `codex exec` failure aborted the entire Codex review before the real review step could run. The classifier only selects the review budget, so on failure we now log the classifier output tail, default to the `complex` review budget, and continue. This keeps actual Codex review failures visible through the existing review exit-code/log handling while preventing the auxiliary classifier from blocking review generation.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14721
Reviewed By: xingbowang
Differential Revision: D104326081
Pulled By: mszeszko-meta
fbshipit-source-id: c17388bbf576f71ff85ded320e0740f89072f8c1
Summary:
This commit fixes several potential issues introduced by cc8d9ea04 (CI: AI review workflow improvements) that broke the agent review jobs in GitHub CI.
1. Codex workflows fail when OPENAI_API_KEY is not configured:
- Added Codex review workflows fail with exit code 1 if the API key secret is missing
- Added script-level checks to skip Codex steps gracefully when OPENAI_API_KEY is unset, avoiding CI noise
2. Missing pull-requests: read permission in manual-review job:
- The manual-review job calls github.rest.pulls.get() but lacked the required permission
- Added 'pull-requests: read' to the manual-review job permissions
3. Potential crash when headSha is undefined:
- parse-claude-review.js and parse-codex-review.js accessed meta.headSha.substring() without null check
- Added defensive checks to handle undefined headSha gracefully
4. Claude model claude-opus-4-7 incompatible with thinking API:
- The commit upgraded default model to claude-opus-4-7, but this model doesn't support the 'thinking.type.enabled' API used by the Claude Code action
- Error: 'thinking.type.enabled' is not supported for this model. Use 'thinking.type.adaptive' and 'output_config.effort' to control thinking behavior
- Reverted default model to claude-opus-4-6 and removed claude-opus-4-7 from options
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14696
Test Plan: Created a draft PR with this change and draft production code changes, including a branch on facebook/rocksdb. Go to https://github.com/facebook/rocksdb/actions/workflows/claude-review.yml and choose "Run workflow" with that branch and PR. (Changes in the PR do are not picked up for the agent code review workflows; the only way to test what is outside of main is with a branch on facebook/rocksdb)
Reviewed By: joshkang97
Differential Revision: D103453631
Pulled By: pdillinger
fbshipit-source-id: 15a82257b5762e9e0b1b393dc45c4b343c866f7a
Summary:
- Upgrade the Claude review workflow models and add complexity-based thinking budget selection for the main review run.
- Keep AI review comments tied to the reviewed commit and restructure long reviews so high-severity findings stay visible while details live in a collapsible section.
- Add early auto-trigger gating plus Codex review/comment workflows, including shared comment-building and parsing helpers.
## Testing
- Not run. The branch refresh was a no-op rebase onto current `upstream/main`, and no new code changes were made during this task.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14659
Reviewed By: pdillinger
Differential Revision: D102224743
Pulled By: xingbowang
fbshipit-source-id: 39c25494062cbcd52d3e85f8a859b5a3907c758e
Summary:
Fix nightly build regressions introduced by recent CI/toolchain changes.
This PR includes two parts:
- CMake/link fixes for the Folly and Folly Lite nightly jobs
- a targeted nightly workflow fix for the clang-21 ASAN/UBSAN + Folly job
Changes:
- link gflags explicitly for USE_FOLLY CMake tool and benchmark targets
- resolve USE_FOLLY_LITE glog via its installed library path instead of bare -lglog
- in the clang-21 nightly job, build Folly/getdeps with gcc/g++ instead of inheriting clang-21 for third-party dependency builds
- disable ccache only for that standalone Folly build step so getdeps/CMake does not inject the broken ccache compiler launcher for ASM
Root cause:
- recent CI/container changes exposed missing explicit link dependencies in the CMake Folly paths
- the clang-21 nightly job exported CC/CXX at job scope, so Folly getdeps inherited clang-21 into libiberty/binutils build logic that expects GCC-style driver behavior such as -print-multi-os-directory
- the same standalone Folly build path also misbehaved when ccache was auto-detected and used as a compiler launcher for assembler-with-cpp inputs
Verification:
- make format-auto
- git diff --check
- local CMake configure sanity check for default config
- upstream nightly reruns used to confirm failure signatures before and after the first-round fixes
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14609
Reviewed By: anand1976
Differential Revision: D100695693
Pulled By: xingbowang
fbshipit-source-id: 703546ad3ddc518ab80c936442709d65fe2d22af
Summary:
This fixes the Claude review workflow YAML parse error and adds a CI validation step for GitHub Actions workflow YAML via `make check-
workflow-yaml`.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14624
Reviewed By: archang19
Differential Revision: D101225058
Pulled By: xingbowang
fbshipit-source-id: 0f371dcbd70ece2c25220c96e43305d9665554b1
Summary:
### Summary
Fix flaky Claude Code Review workflow failures caused by stale workflow_run SHAs.
Problem:
- The auto-review workflow is triggered from pr-jobs via workflow_run.
- In some cases, by the time the review job starts, the PR branch head has already advanced.
- Then Get PR info cannot find an open PR whose head SHA matches the older workflow_run.head_sha, so it fails with:
- Could not find PR for SHA ... after retries
- This creates unnecessary CI noise even though a newer review job will be triggered for the updated commit.
Solution:
- Treat “PR not found for this head SHA” as a stale run, not an error.
- In Get PR info, replace workflow failure with:
- a warning
- skip=true
- skip_reason=stale_sha:<sha>
- Gate all later auto-review steps on steps.pr_info.outputs.skip != 'true'
- Add a small logging step to report the skip reason
Result:
- stale auto-review runs exit successfully instead of failing
- CI becomes less noisy
- the newest commit still gets reviewed by the later workflow run
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14591
Reviewed By: mszeszko-meta
Differential Revision: D100190375
Pulled By: xingbowang
fbshipit-source-id: fa4a0479a10e953e52b3d99c2b483dae01d3e7fb
Summary:
Upgrade the Ubuntu 24 CI Docker image to include clang-21 (from clang-18), and bump all workflow references to the new image tags. Also adds ccache to both images and renames all clang-18 job references to clang-21.
This is mainly for upgrading the clang version later used for generating C API automatically. See PR https://github.com/facebook/rocksdb/issues/14572
### Changes
**`build_tools/ubuntu24_image/Dockerfile`**
- Add clang-21 installation from LLVM snapshot repo (`apt.llvm.org/noble/llvm-toolchain-noble-21`)
- Add ccache
- Add comment pointing Meta employees to internal devvm build guide
**`build_tools/ubuntu22_image/Dockerfile`**
- Add ccache
- Add comment pointing Meta employees to internal devvm build guide
**`.github/workflows/pr-jobs.yml`**
- Rename `build-linux-clang-18-no_test_run` → `build-linux-clang-21-no_test_run`
- Rename `build-linux-clang18-asan-ubsan` → `build-linux-clang21-asan-ubsan`
- Rename `build-linux-clang18-mini-tsan` → `build-linux-clang21-mini-tsan`
- Update all `clang-18`/`clang++-18` references to `clang-21`/`clang++-21`
- Update ccache key prefixes: `clang18-asan-ubsan` → `clang21-asan-ubsan`, `clang18-tsan` → `clang21-tsan`
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`
**`.github/workflows/nightly.yml`**
- Rename `build-linux-clang-18-asan-ubsan-with-folly` → `build-linux-clang-21-asan-ubsan-with-folly`
- Update clang-18 → clang-21 compiler references
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`
**`.github/workflows/clang-tidy.yml`**
- Update clang-18 → clang-21
- Bump `rocksdb_ubuntu:24.0` → `rocksdb_ubuntu:24.1`
### New Docker Images
Both images have been built and tested locally. They will be pushed to `ghcr.io/facebook/rocksdb_ubuntu` before this PR is merged.
- `ghcr.io/facebook/rocksdb_ubuntu:22.2` — adds ccache, adds devvm build note
- `ghcr.io/facebook/rocksdb_ubuntu:24.1` — adds clang-21 from LLVM snapshot repo, adds ccache
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14576
Test Plan: Built both images locally and verified CI job names/compiler flags are consistent.
Reviewed By: joshkang97
Differential Revision: D99694293
Pulled By: xingbowang
fbshipit-source-id: c23c27f5cf870fb2c8b4e3d1cba281d0ce63f9d6
Summary:
`teardown-ccache` runs with if: always(), but in folly jobs `setup-folly` precedes `setup-ccache`. When `setup-folly` fails, `setup-ccache` is skipped, so `CCACHE_DIR` is unset and ccache is not on `PATH`. This is exactly what happened [here](https://github.com/facebook/rocksdb/actions/runs/23660261947/job/68928337162). Fix guards `teardown-ccache` to exit gracefully when `CCACHE_DIR` is unset, and tolerate missing ccache binary for stats.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14542
Reviewed By: joshkang97
Differential Revision: D98964757
Pulled By: mszeszko-meta
fbshipit-source-id: 8e83c1b62ef66130ba479142f002897d0a97f6c0
Summary:
## Problems
**(1) Claude review times out on large PRs**
The `claude-code-base-action` defaults to `timeout_minutes=10`. Large PRs (e.g. https://github.com/facebook/rocksdb/issues/14499) get killed with exit code 124 after 600 seconds mid-review.
**(2) Exported PRs never get reviewed**
PRs exported from Meta's internal pipeline (e.g. https://github.com/facebook/rocksdb/issues/14515) trigger CI within milliseconds of PR creation. The `workflow_run` payload has `pull_requests=[]`, and the SHA-based fallback also misses because GitHub hasn't registered the PR yet — so Claude review never fires.
## Fixes
- Set `timeout_minutes: "60"` on both auto-review and manual-review `Run Claude` steps
- Retry the SHA→PR lookup up to 5 times with a 10s delay, giving GitHub up to ~50s to register the PR. Also bumped `per_page` from 30 to 100.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14526
Reviewed By: pdillinger
Differential Revision: D98636217
Pulled By: xingbowang
fbshipit-source-id: bba19095ab2ddf468c5c19cf5c77d19536f498b0
Summary:
Fix build failures, flaky tests, and Windows ccache issues exposed by PR https://github.com/facebook/rocksdb/issues/14478.
### 1. Release build (NDEBUG) compile error
**Job**: `build-linux-release-with-folly`
The SyncPoint cleanup listener in testharness.cc referenced `SyncPoint::GetInstance()` unconditionally, but SyncPoint is only declared in debug builds. Wrapped with `#ifndef NDEBUG`.
### 2. Folly-lite link error
**Job**: `build-linux-cmake-with-folly-lite`
The `USE_FOLLY_LITE` cmake path unconditionally linked `-lglog`, which fails with lld (added in https://github.com/facebook/rocksdb/issues/14478) when glog isn't installed. Use `find_library()` to link only when available.
### 3. Flaky tests — leaked `Env::Default()` thread pool state
Sharded execution runs multiple tests per process. Several tests in `db_compaction_test.cc` modified the global `Env::Default()` BOTTOM thread pool without resetting. With a leaked BOTTOM pool, subsequent tests' last-level compactions get forwarded to the bottom pool, freeing the LOW thread to pick additional compactions unexpectedly.
Fix: add `TearDown()` override to `DBCompactionTest` that captures default thread pool sizes in the constructor and restores them after every test. This is more robust than per-test cleanup because:
- It runs even when a test fails (gtest calls TearDown after assertion failures)
- It catches all current and future leakers without per-test maintenance
### 4. Windows ccache — 0.26% hit rate → should be ~99%
The Windows nightly build takes 57 minutes because ccache has near-zero hit rate. Two issues:
- **Cache key**: The `hendrikmuhs/ccache-action` used a timestamp-based key, so each nightly run created a unique key and never found the previous run's cache (`No cache found.`). Fixed by using a stable key `ccache-windows-<workflow>` with prefix-based restore.
- **Compiler check**: Missing `compiler_check=content` setting, so MSVC path/version changes between runners invalidated all cache entries. Added `compiler_check=content` (same fix as macOS in https://github.com/facebook/rocksdb/issues/14478).
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14508
Test Plan:
- Release and debug builds compile cleanly
- 70 consecutive shuffled runs of db_compaction_test (367 tests each) pass — 50 on full cores + 20 on 4 cores
- Format check passes
- Windows ccache fix requires CI run to verify (first run populates cache, second run should see high hit rate)
Reviewed By: anand1976
Differential Revision: D98380757
Pulled By: xingbowang
fbshipit-source-id: d74079b75786ba3299e335b145a6c5fdc81fd5c1
Summary:
Improves the Claude Code review CI workflow to produce deeper, more reliable reviews.
**Motivation:** The previous 30-turn limit caused reviews to silently produce "no output" on complex PRs (e.g. https://github.com/facebook/rocksdb/issues/14477). The review prompt was also too shallow — single-pass with no codebase context phase.
**Changes:**
**1. Comprehensive multi-agent review prompt** (`claude_md/ci_review_prompt.md`)
- 9 specialized review agents: design, correctness, cross-component, invariant-adversary, caller-audit, performance, API, serialization, test coverage
- Deep codebase context phase before agents spawn: caller-chain analysis (3-5 levels up), callee side-effect tracing, cross-component data consumer analysis, execution context verification, assumption stress-testing
- Inter-agent debate with round-robin critique assignments
- Final report quality rules: disproven findings removed, no stream-of-consciousness
**2. Incremental findings + recovery flow**
- `Write` tool added so Claude saves findings to `review-findings.md` after each phase
- If the review hits the turn limit, a recovery step launches a Sonnet session to format partial findings into the standard output
- Recovery file existence check prevents crash if recovery step fails
- `getLastAssistantText` fallback truncated to 50KB to avoid enormous PR comments
**3. Prompts extracted to files** (`claude_md/ci_*.md`)
- `ci_review_prompt.md` — full review methodology
- `ci_query_prompt.md` — `/claude-query` system prompt
- `ci_recovery_prompt.md` — recovery formatting prompt
- Secure: checkout is from base branch (main), not PR head
**4. `max_turns` increased from 30 to 300**
- Orchestrator budget for multi-agent workflow; sub-agents get their own turns
- Recovery flow ensures partial results if limit is still hit
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14507
Reviewed By: archang19
Differential Revision: D98170111
Pulled By: xingbowang
fbshipit-source-id: 390626a53e7a7f91c2d3e91ed4403494532425ed
Summary:
Reduce `make check` time from ~9.4 minutes to ~3.7 minutes (2.6x) on a 192-core machine, speed up CI by 1.8x across all jobs, and fix ccache hit rates (macOS cmake: 0.51% → 99.80%).
## Changes
### 1. Auto-detect lld linker (`build_detect_platform`)
- Probe for `lld` at configure time; fall back to default `ld.bfd` if unavailable
- Linux-only guard (`TARGET_OS = Linux`)
- **Measured: 12x faster linking** (7.4s → 0.6s per test binary)
- Add `-L/usr/local/lib` for lld library resolution on CI
- Install lld in CI pre-steps action
- Opt out with `ROCKSDB_NO_FAST_LINKER=1`
### 2. Sharded test execution (`Makefile`)
**Before:** `make check` enumerated every individual gtest case via `--gtest_list_tests`, then created a separate shell script for each case with `--gtest_filter=TestName`. For example, `block_based_table_reader_test` has 9,788 parameterized test cases — each spawned its own process, loading a ~50MB binary, initializing gtest, registering all ~10K tests, running exactly ONE, then tearing down. The per-process overhead was ~1.5s, so 9,788 × 1.5s ≈ 15,000s wasted on overhead alone. Across all 302 test binaries this produced 39,467 individual processes.
**After:** Use gtest's built-in sharding (`GTEST_TOTAL_SHARDS`/`GTEST_SHARD_INDEX`) to group ~10 test cases per process. Each shard loads the binary once and runs multiple tests sequentially — eliminating the per-process overhead. The 9,788 cases in `block_based_table_reader_test` become ~980 shards. Total process count drops from 39,467 to ~4,036. Same tests, same coverage, just fewer process spawns.
The shard count adapts to machine size: `min(ceil(test_count / GTEST_SHARD_SIZE), NCORES * 8)`. This ensures large machines (192 cores) get many small shards for parallelism, while small CI runners (4 cores) get fewer larger shards to avoid excessive overhead. Both `GTEST_SHARD_SIZE` (default 10) and `NCORES` can be overridden.
CI sharding uses round-robin distribution across 3 shards to balance heavy tests (db_test, db_compaction_test, etc.) instead of contiguous alphabetical ranges.
Note: gtest continues running all tests after a failure (`GTEST_THROW_ON_FAILURE=0`), so grouping tests does not mask failures — all failures within a shard are reported.
**Measured: 3.6x CPU reduction** (76,121s → 21,033s)
### 3. Fix SyncPoint leaks for test isolation (5 files)
Sharded execution exposed 43 test files that set SyncPoint callbacks but never clean them up. Stale callbacks with captured local variables cause segfaults or data corruption when subsequent tests in the same process trigger them.
**Systematic fix:** Global gtest `TestEventListener` in `testharness.cc` that calls `SyncPoint::DisableProcessing()` + `ClearAllCallBacks()` + `ClearTrace()` + `LoadDependency({})` after every test case. This cleans up all SyncPoint state: callbacks, dependency maps, cleared points, and the point filter. Registered via static initialization — no changes needed to individual test `main()` functions.
**SyncPoint infrastructure fixes:**
- `DisableProcessing()` now calls `cv_.notify_all()` to wake threads blocked in `Process()`. Previously, threads waiting for predecessor sync points that would never fire (because the test ended) would hang forever.
- `Process()` now rechecks `enabled_` after waking from `cv_.wait()`, so threads exit promptly when processing is disabled instead of looping forever.
**Specific fixes (defense-in-depth):**
- `SstFileReaderTest::VerifyNumEntriesCorruption` — leaked `PropertyBlockBuilder::AddTableProperty:Start` callback that corrupted SST entry counts
- `WritePreparedTransactionTest::CommitAndSnapshotDuringCompaction` — leaked `CompactionIterator:AfterInit` callback causing segfault via dangling pointers
- `TransactionTestBase` destructor — cleanup for 26 SyncPoint uses across transaction tests
- `RetriableLogTest` destructor — cleanup for log reader SyncPoint callbacks
### 4. Fix ccache for reliable cross-run caching (`setup-ccache`, `CMakeLists.txt`)
- **`CCACHE_COMPILERCHECK=content`**: Default `mtime` compared compiler binary modification time, which differs on every fresh CI runner → 0% hit rate. `content` hashes compiler output instead — stable across runner instances
- **`CCACHE_SLOPPINESS`**: CMake generates varying `-MF` paths and timestamps. Without sloppiness settings, ccache treated these as different compilations
- **`CMAKE_C/CXX_COMPILER_LAUNCHER`** instead of `RULE_LAUNCH_COMPILE`: Avoids double-wrapping ccache when also injected via PATH. Removes `RULE_LAUNCH_LINK` since ccache cannot cache link operations
- **macOS cmake hit rate: 0.51% → 99.80%**
### 5. Align GTEST_THROW_ON_FAILURE (`Makefile`)
- Set `GTEST_THROW_ON_FAILURE=0` in Makefile default to match CI's `pre-steps/action.yml`
- `=1` caused `std::terminate` in multi-threaded stress tests (e.g., `point_lock_manager_stress_test`, `rate_limiter_test`) when assertions fail — the gtest exception propagates across thread boundaries, triggering undefined behavior and heap-use-after-free under ASAN
### 6. Portability fixes
- `[[maybe_unused]]` instead of `__attribute__((unused))` for MSVC compatibility
- Remove blanket `-fPIC` from `COMMON_FLAGS` — only apply via `PLATFORM_SHARED_CFLAGS` for shared builds, preserving optimal codegen for release/static builds
- `make -s list_all_tests` to suppress make noise in test enumeration
## Results
### Local (192-core devvm, clean build)
| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Build time (clean) | 2m43s | ~2m | lld linker |
| Test wall clock | 400s | 200s | 2.0x |
| Test CPU total | 76,121s | 21,033s | 3.6x |
| Test failures | 0 | 0 | — |
| **Total make check** | **~9.4min** | **3m41s** | **2.6x** |
### CI (GitHub Actions)
| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Aggregate CI time | 15,220s | 8,425s | 1.8x |
| build-linux-arm | 763s | 221s | 3.5x |
| build-linux-release | 616s | 244s | 2.5x |
| build-windows-vs2022 | 2,868s | 1,205s | 2.4x |
| build-linux-java-static | 858s | 335s | 2.6x |
### ccache hit rates (all jobs)
| Job | Before | After |
|-----|--------|-------|
| macOS cmake (0-3) | 0.51% | **99.80%** |
| build-linux | 99.68% | **99.84%** |
| build-linux-clang18-asan-ubsan | 98.58% | **99.84%** |
| build-linux-clang18-mini-tsan | 98.74% | **99.84%** |
| build-windows-vs2022 | 99.22% | **99.83%** |
| All 27 ccache jobs | — | **>94%** |
## Remaining bottleneck
The critical path is now dominated by genuinely slow integration tests:
- `external_sst_file_test` shards: 97–113s each
- `db_test` shards: 109–124s each
- `db_bloom_filter_test`: 103s (non-parallel)
Further improvement would require test optimization or a `make check-fast` target.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14478
Test Plan:
- `make check -j192 SKIP_FORMAT_BUCK_CHECKS=1` — all 4,045 shards pass (ASAN+UBSAN), zero failures
- Verified SyncPoint fixes: each crash/corruption reproduces before fix, passes after
- Verified SyncPoint::DisableProcessing wakes blocked threads (previously caused indefinite hangs)
- Verified gtest does NOT stop after failure with `GTEST_THROW_ON_FAILURE=0` — all tests in a shard run and all failures are reported
- CI: all 34 checks pass across two consecutive runs
- ccache: validated hit rates >94% on second run after cache seeding
Reviewed By: hx235
Differential Revision: D97623305
Pulled By: xingbowang
fbshipit-source-id: b299e6d43c8713a9ef2b5659e8bbd74958afe155
Summary:
Add GitHub Actions workflows for AI-powered code review on PRs using Claude (Anthropic). Follows the clang-tidy security pattern with two separate workflows for privilege separation.
## Trigger Modes
**1. Auto** — Runs after `pr-jobs` workflow completes successfully via `workflow_run`. Safe for fork PRs (runs from default branch, never executes PR code).
**2. Manual** — Maintainers comment `/claude-review [focus area]` or `/claude-query <question>` on any PR. Restricted to 15 authorized team members.
**3. workflow_dispatch** — For manual testing.
## Security Model (Two-Workflow Separation)
Same pattern as clang-tidy:
**`claude-review.yml`** (analysis):
- Runs Claude with `ANTHROPIC_API_KEY`
- Has ONLY `contents: read` — no PR write, no issue write
- Saves review markdown + metadata as artifact
**`claude-review-comment.yml`** (posting):
- Triggers on `workflow_run` completion
- Downloads artifact and posts/updates PR comment
- Has `pull-requests: write` but never runs AI
This separation prevents a crafted PR from tricking Claude into exfiltrating write tokens.
## Review Methodology
Review prompt in `claude_md/code_review.md` (shared with local Claude Code reviews). Five perspectives:
- Call-chain analysis (3-5 levels up/down)
- Correctness & edge cases
- Cross-component & adversarial (10 execution contexts)
- Performance
- API compatibility & test coverage
## Shared Scripts
- `.github/scripts/post-pr-comment.js` — Create-or-update PR comment with marker-based dedup. Now used by both clang-tidy and Claude review.
- `.github/scripts/parse-claude-review.js` — Parses `claude-code-base-action` execution log into markdown.
## Files Changed
| File | Description |
|------|-------------|
| `.github/workflows/claude-review.yml` | Analysis workflow (476 lines) |
| `.github/workflows/claude-review-comment.yml` | Comment posting workflow (146 lines) |
| `.github/scripts/post-pr-comment.js` | Shared PR comment utility (57 lines) |
| `.github/scripts/parse-claude-review.js` | Execution log parser (78 lines) |
| `.github/workflows/clang-tidy-comment.yml` | Updated to use shared script |
| `claude_md/code_review.md` | Review methodology (104 lines) |
## Setup Required
Add `ANTHROPIC_API_KEY` secret to the repo settings.
## Testing
Tested end-to-end on `xingbowang/rocksdb` fork — both auto and manual triggers, artifact upload/download, comment posting, and duplicate detection all verified working.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14480
Reviewed By: omkarhgawde
Differential Revision: D97832666
Pulled By: xingbowang
fbshipit-source-id: f80c7d8683ac980614dc4ca66c1e545deb3be504
Summary:
The nightly build had a flag for portable build on folly, but rocksdb does not. This causes link error. Fix this by removing portable build flag in folly build in nightly run. Nightly run will always build without cache using native flag. Only PR jobs uses cache.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14391
Test Plan: nightly run
Reviewed By: joshkang97
Differential Revision: D94520182
Pulled By: xingbowang
fbshipit-source-id: 7c7d3b089744c7e2e8ea073158f1b5b80db420d4
Summary:
-march=native in cmake builds causes SIGILL after cache hits
Issue: PORTABLE=1 env var only works for Makefile builds. cmake ignores it and injects -march=native via its own CPU detection. Since ccache hashes the flag string literally, a cache compiled on an AVX-512 runner and restored on a non-AVX-512 runner produces a SIGILL crash.
Fix: Added -DPORTABLE=ON to build-linux-cmake-with-benchmark-no-thread-status and build-linux-cmake-with-folly-coroutines cmake commands.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14388
Test Plan: Github CI
Reviewed By: joshkang97
Differential Revision: D94367557
Pulled By: xingbowang
fbshipit-source-id: 5aa39cc36fc004d0ee636cd3fa197880d6cb773a
Summary:
Accelerate CI: ccache integration, replace scan-build, upgrade runners
1. ccache integration via reusable composite action (setup-ccache)
- Created .github/actions/setup-ccache/action.yml
- Applied to 13+ Linux compilation jobs in pr-jobs.yml
- PORTABLE=1 by default to avoid illegal instruction errors on
heterogeneous runners; disabled for jobs linking pre-built Folly
- On cache hit, reducing compilation from 20+ min to ~2-5 min per job
- ccache placed before Folly build so Folly compilation also
benefits from cache on Folly cache misses
2. Replace scan-build with clang-tidy (30+ min -> seconds)
- Removed build-linux-clang18-clang-analyze job from pr-jobs.yml
- Expanded .clang-tidy to enable all clang-analyzer-* checks,
matching scan-build's full coverage
- Existing clang-tidy-comment.yml workflow now handles both
clang-tidy and clang-analyzer checks on changed files only
3. Upgrade Folly job runners for faster builds and tests
- build-linux-make-with-folly: 16-core -> 32-core, -j32 -> -j64
- build-linux-cmake-with-folly-coroutines: 16-core -> 32-core,
-j20 -> -j64 (both make and ctest)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14368
Reviewed By: joshkang97
Differential Revision: D94166095
Pulled By: xingbowang
fbshipit-source-id: 3dfbd71aace3ba9cb2e790873bdbedba20215657
Summary:
Introduce a new V2 serialization format for wide column entities that supports storing individual column values in blob files. The V2 format adds a column type section that marks each column as either inline or blob-index, enabling per-column blob storage for large values.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14314
Reviewed By: pdillinger
Differential Revision: D92832066
Pulled By: xingbowang
fbshipit-source-id: 13c24347e1f481a059d67eef987d2d2b184b4a51
Summary:
RocksDB has been using clang-tidy for a long time inside Meta. However, it is not efficient for external contributor, as the result from clang-tidy has to be ferried back through internal contributor. This PR added support to run clang-tidy on external github CI. It added .clang-tidy file based on internal version. It run clang-tidy in a separate pr job and a workflow step would post the pr job result to the PR itself. See example below.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14347
Test Plan: Github CI
Reviewed By: archang19
Differential Revision: D93862467
Pulled By: xingbowang
fbshipit-source-id: bb4330241036894deb619470efd73a7041a8b62f
Summary:
Add clang-tidy-comment workflow. This workflow allows pr clang tidy pr job to post the clang-tidy finding directly on the PR page.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14348
Test Plan: Will be tested with next clang-tidy PR
Reviewed By: joshkang97
Differential Revision: D93670150
Pulled By: xingbowang
fbshipit-source-id: 8245f9d5bde8cf800d88034c4339de9f387c5692
Summary:
To make CI consistent with internal meta clang version.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14311
Test Plan:
CI shows correct version
```
Successfully installed clang-format-21.1.2
clang-format version 21.1.2
```
Reviewed By: xingbowang
Differential Revision: D92535441
Pulled By: joshkang97
fbshipit-source-id: ea21ea97b13a35b286f0c2ce18b3f01ffbf49afd
Summary:
Update HISTORY, version number, format compatible test, and folly version
folly build now depends on libaio
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14259
Test Plan: CI
Reviewed By: anand1976
Differential Revision: D91356493
Pulled By: pdillinger
fbshipit-source-id: 9d85960c647758d5cb33e3910e714e2f7785fd06
Summary:
Folly download dependencies directly from external source. Sometimes, this could fail due to external website instability. To solve this, we added github cache to cache the dependencies. We also added a python script to try different sources during download to reduce the chance of failure.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14226
Test Plan: github CI
Reviewed By: krhancoc, archang19
Differential Revision: D90343051
Pulled By: xingbowang
fbshipit-source-id: 3faad6aaa6c1bfd361b9e405c298856cd64bf457
Summary:
To help find potential issues not showing up in ARM unit tests. I'm running it with and without TransactionDB (write-committed) for better coverage. The job expands the size of /dev/shm for adequate space on maximum performance storage, and adds swap space to reduce risk of OOM in case we fill that up.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14172
Test Plan: earlier drafts of this PR added the job to PR jobs, and the last before putting in "nightly" can be seen here: https://github.com/facebook/rocksdb/actions/runs/19945493840/job/57193797390?pr=14172
Reviewed By: archang19
Differential Revision: D88429479
Pulled By: pdillinger
fbshipit-source-id: bd4d9cda9256950c3c6c126c299a44dbbbc30c7e
Summary:
This test is now taking > 6 hours, timing out, and has low signal, so creating a weekly job for it, with an explicit timeout of 12 hours.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14110
Test Plan: watch CI
Reviewed By: virajthakur
Differential Revision: D86428262
Pulled By: pdillinger
fbshipit-source-id: 44103518064ca378f3fd2ff8d21967ede698c8ea
Summary:
* Fix nightly build-linux-cmake-with-folly-lite-no-test for real this time
with correct include directory. (CMakeLists.txt)
* Add test runs to that build (and rename)
* Improve folly build caching with a folly.mk file with most of the relevant
parts of Makefile that contribute to the checkout_folly and
build_folly builds. This reduces the risk of false passing of CI job with
cache folly build. This caching is still only for folly debug builds, (which
is probably OK with just a single nightly build relying on release folly
build, which also serves as a rough canary against false passing
because of caching).
* Use `make VERBOSE=1` after cmake calls for detailed output
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14099
Test Plan:
temporary CI change to put the relevant parts in pr-jobs,
then back to homes including in nightly
Reviewed By: mszeszko-meta
Differential Revision: D86243363
Pulled By: pdillinger
fbshipit-source-id: f7975fa190ef45195c6d0b74417f7886e551516a
Summary:
... caused by public headers depending on build parameters (macro definitions). This change also adds a check under 'make check-headers' (already in CI) looking for potential future violations.
I've audited the uses of '#if' in public headers and either
* Eliminated them
* Systematically excluded them because they are intentional or similar (details in comments in check-public-header.sh
* Manually excluded them as being ODR-SAFE
In the case of ROCKSDB_USING_THREAD_STATUS, there was no good reason for this to appear in public headers so I've replaced it with a static bool ThreadStatus::kEnabled. I considered getting rid of the ability to disable this code but some relatively recent PRs have been submitted for fixing that case. I've added a release note and updated one of the CI jobs to use this build configuration. (I didn't want to combine with some jobs like no_compression and status_checked because the interaction might limit what is checked.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14096
Test Plan: manual 'make check-headers' + manual cmake as in new CI config + CI
Reviewed By: jaykorean
Differential Revision: D86241864
Pulled By: pdillinger
fbshipit-source-id: d16addc9e3480706b174a006720a4def0740bf2e
Summary:
Following up on https://github.com/facebook/rocksdb/pull/14071, updating folly to
8a9fc1e80a or beyond was failing an F14Table assertion for a very subtle reason: ODR violation between the folly build and RocksDB build because folly build was release mode and RocksDB build was debug mode. What was happening was that folly change introduced a dependence on kDebug (whether build is debug) in a hashing implementation in a .h file, and the inconsistency between the inlined implementation during RocksDB build and the linked-to implementation from the folly build was leading to inconsistencies in the data structure.
The primary fix is to ensure we build folly in debug mode for debug mode RocksDB builds. Also,
* Needed to use the `patchelf` tool in `build_folly` to ensure the glog dependency shared library can always find its own gflags dependency. I explored many options for working around this, and this is what would work without reworking folly's own build.
* Updated folly to latest commit.
* Thrown in an ad hoc folly patch to use ftp.gnu.org mirrors (the canonical is super slow)
* Moved the placement of GETDEPS_USE_WGET=1 to apply to local builds also, to avoid the issue of a large download almost reaching completion and then stalling indefinitely.
* Fix failing nightly build-linux-cmake-with-folly-lite-no-test with fmt includes in cmake build (as was done with make build)
* Add a release mode folly+RocksDB to nightly CI, including both cmake and make. This also serves as a non-cached folly build to detect potential problems with PR jobs working from cached folly build.
* Move build-linux-cmake-with-folly to nightly because it's mostly covered by build-linux-cmake-with-folly-coroutines
Intended follow-up:
* folly-lite build with tests
* Make the folly build caching more friendly+accurate by hashing the relevant Makefile parts and tagging whether debug or release. Not in this PR because then you wouldn't be able to see what changed in the folly build steps themselves.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14094
Test Plan: manual + CI
Reviewed By: mszeszko-meta
Differential Revision: D85864871
Pulled By: pdillinger
fbshipit-source-id: 50009b33422d5781074fcbbdf18089be9e36800d
Summary:
With cache hit and compiler option optimization, the compilation time build time is reduced from 40 min to 2 min. Overall build time is reduced from 60 min to less 20 minutes on cache hit on majority of the source file. On 100% cache miss, it would be around 40 minutes.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14064
Test Plan: Github CI
Reviewed By: mszeszko-meta
Differential Revision: D85023882
Pulled By: xingbowang
fbshipit-source-id: 98551880c98f14d36133ff43e6af8c3be94ab465
Summary:
* Reduce build time of folly from 45m~1hr down to 25m. This is achieved by caching folly build artifact from previous build.
* Reduce windows build time of folly from 1hr 15m down to 50m. This is done by increase windows build machine size.
* Fix build on macos on other macos target.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14057
Test Plan: github CI
Reviewed By: archang19, nmk70
Differential Revision: D84848041
Pulled By: xingbowang
fbshipit-source-id: 00306750737070e7e446ee436d607ed6ecae79ae
Summary:
We are adding more and more tests, so we need to increase the number of shards in macos build to reduce overall CI time.
macos-15-xlarge image is ARM, which has 5 vCPU cores, but is still 50% faster than the intel x86 12 vCPU.
Test time reduced from 1h 37m to 14m.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14048
Reviewed By: archang19
Differential Revision: D84741917
Pulled By: xingbowang
fbshipit-source-id: 9ba9bd696d3b2152f11dec2fb4280572b98233d5
Summary:
Fix the binutils truncated download issue by switching to wget in the folly build scripts for downloading dependencies.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14030
Test Plan: make build_folly
Reviewed By: jaykorean
Differential Revision: D84033126
Pulled By: anand1976
fbshipit-source-id: bc6706d7e57c97d6edff149a965aa12c7959825f
Summary:
Complete redo of parallel compression in block_based_table_builder.cc to greatly reduce cross-thread hand-off and blocking. A ring buffer of blocks-in-progress is used to essentially bound working memory while enabling high throughput. Unlike before, all threads can participate in compression work, for a kind of work-stealing algorithm that reduces the need for threads to block. This builds on improvements in https://github.com/facebook/rocksdb/pull/13850
Previously, there was either
* parallel_threads==1, the *emit thread* (caller from flush/compaction) doing all the work
* parallel_threads > 1, the emit thread generates uncompressed blocks, `parallel_threads` worker threads compress blocks, and a writer thread writes to the SST file. Total of `parallel_threads + 2` threads participating. (Other bookkeeping in emit and write steps omitted from description for simplicity.)
Now we have either
* parallel_threads==1 (same), the emit thread doing all the work
* parallel_threads > 1, the emit thread generates uncompressed blocks and can take up compression work when the ring buffer is full; `parallel_threads` worker threads have as their top priority to write compressed blocks to the SST file but also take up compression work in priority order of next-to-write. Total of `parallel_threads + 1` threads participating. In some cases, this could result in less throughput than before, but arguably the previous implementation was using more threads than explicitly allowed.
## Future/alternate considerations
Although we could likely have used some framework for micro-work sharing across threads, that could be difficult with the asymmetry of work loads and thread affinity. Specifically, (a) it would be quite challenging to allow emit work in other threads, because it happens in the caller of BlockBasedTableBuilder, (b) async programming is unlikely to pay off until we have an async interface for writing SST files, and (c) this implementation will nevertheless serve as a benchmark for what we lose or gain in such a framework vs. a hand-tuned system.
This implementation still creates and destroys threads for each SST file created. We hope in the future to have more governance and/or pooling of worker threads across various flushes and compactions, but that is not available currently and would require significant design and implementation work.
## More details
* This implementation makes use of semaphores for idling and re-waking threads. `std::counting_semaphore` and `binary_semaphore` offer the best performance (see benchmark results below) but some implementations are known to have correctness bugs. Also, my attempt at upgrading CI for C++20 support (required for these) in https://github.com/facebook/rocksdb/pull/13904 is actually incomplete. Therefore, using these structures is opt-in with `-DROCKSDB_USE_STD_SEMAPHORES` at compile time, and a naive semaphore implementation based on mutex and condvar is used by default. A folly alternative (folly::fibers::Semaphore) was dropped in during development and found to be less efficient than the naive implementation. One CI job is upgraded to test with the new opt-in.
* One of the biggest concerns about correctness/reliability for this implementation is the possibility of hitting a deadlock, in part because that is not well checked in the DB crash test (a challenging problem!). Note also that with the parallel compression improvements in this release, I am calling the feature production-ready, so there is an extra level of confidence needed in the reliability of the feature. Thus, for DEBUG builds including crash test, I have added a watchdog thread to each parallel SST construction that heuristically checks for the most likely kinds of deadlock that could happen, including for the case of buggy semaphore implementations. It periodically verifies that some thread is outside of its "idle" state, and if the watchdog wakes up repeatedly to see all live threads stuck in their idle state (even if wake-up was attempted) then it declares a deadlock. This feature was manually verified for several seeded deadlock bugs. (More details in code comments.)
* For CPU efficiency, this implementation greatly simplifies the logic to estimate the outstanding or "inflight" size not yet written to the SST file. I expect this size to generally be insignificant relative to the full SST file size so is not worth careful engineering. And based on Meta's current needs, landing under-size for an SST file is better than over-size. See comments on `estimated_inflight_size` for details.
* Some other existing atomics in block_based_table_builder.cc modified to use safe atomic wrappers.
* Status handling in BlockBasedTableBuilder was streamlined to get rid of essentially redundant `status`+`io_status` fields and associated code. Made small optimizations to reduce unnecessary IOStatus copies (with StatusOk()) and mark status conditional branches as LIKELY or UNLIKELY.
* Prefer inline field initialization to initialization in constructor.
* Minimize references to the `parallel_threads` configuration parameter for better separation of concerns / sanitization / etc. For example, use non-nullity of `pc_rep` to indicate that parallel compression is enabled (and active).
* Some other refactoring to aid the new implementation.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13910
Test Plan:
## Correctness
Already integrated into unit tests and crash test. CI updated for opt-in semaphore implementation. Basic semaphore unit tests added/updated.
As for the tremendous simplification of logic relating to hitting target SST file size, as expected, the new behavior could under-shoot the single-threaded behavior by a small number of blocks, which will typically affect the file size by ~1/1000th or less. I think that's a good trade-off for cutting out unnecessarily complex code with non-trivial CPU cost (FileSizeEstimator).
```
./db_bench -db=/dev/shm/dbbench_filesize_after8 -benchmarks=fillseq,compact -num=10000000 -compression_type=zstd -compression_level=8 -compression_parallel_threads=8
```
Before, PT=8 & PT=1, and After PT=1 the same or very similar
```
-rw-r--r-- 1 peterd users 67474097 Sep 12 15:32 000052.sst
-rw-r--r-- 1 peterd users 67474214 Sep 12 15:32 000053.sst
-rw-r--r-- 1 peterd users 67473834 Sep 12 15:32 000054.sst
-rw-r--r-- 1 peterd users 67473437 Sep 12 15:32 000055.sst
-rw-r--r-- 1 peterd users 67473835 Sep 12 15:32 000056.sst
-rw-r--r-- 1 peterd users 67473204 Sep 12 15:33 000057.sst
-rw-r--r-- 1 peterd users 67473294 Sep 12 15:33 000058.sst
-rw-r--r-- 1 peterd users 67473839 Sep 12 15:33 000059.sst
```
After, PT=8 (worst case here ~0.05% smaller)
```
-rw-r--r-- 1 peterd users 67463189 Sep 12 14:55 000052.sst
-rw-r--r-- 1 peterd users 67465233 Sep 12 14:55 000053.sst
-rw-r--r-- 1 peterd users 67466822 Sep 12 14:55 000054.sst
-rw-r--r-- 1 peterd users 67466221 Sep 12 14:55 000055.sst
-rw-r--r-- 1 peterd users 67441675 Sep 12 14:55 000056.sst
-rw-r--r-- 1 peterd users 67467855 Sep 12 14:55 000057.sst
-rw-r--r-- 1 peterd users 67455132 Sep 12 14:55 000058.sst
-rw-r--r-- 1 peterd users 67458334 Sep 12 14:55 000059.sst
```
## Performance, modest load
We are primarily interested in balancing throughput in building SST files and CPU usage in doing so. (For example, we could maximize throughput by having worker threads only spin waiting for work, but that would likely be extra CPU usage we want to avoid to allow other productive CPU work to be scheduled.) No read path code has been touched.
A benchmark script running "before" and "after" configurations at the same time to minimize random machine load effects:
```
$ SUFFIX=`tty | sed 's|/|_|g'`; for CT in none lz4 zstd; do for PT in 1 2 3 4 6 8; do echo -n "$CT pt=$PT -> "; (for I in `seq 1 10`; do BIN=/tmp/dbbench${SUFFIX}.bin; rm -f $BIN; cp db_bench $BIN; /usr/bin/time $BIN -db=/dev/shm/dbbench$SUFFIX --benchmarks=fillseq -num=10000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=1000 -fifo_compaction_allow_compaction=0 -disable_wal -write_buffer_size=12000000 -format_version=7 -compression_type=$CT -compression_parallel_threads=$PT 2>&1; done) | awk '/micros.op/ {n++; sum += $5;} /system / { cpu += $1 + $2; } END { print "ops/s: " int(sum/n) " cpu*s: " cpu; }'; done; done
```
Before this change:
```
none pt=1 -> ops/s: 1999603 cpu*s: 72.08
none pt=2 -> ops/s: 1871094 cpu*s: 148.3
none pt=3 -> ops/s: 1882907 cpu*s: 147.7
lz4 pt=1 -> ops/s: 1987858 cpu*s: 94.74
lz4 pt=2 -> ops/s: 1590192 cpu*s: 182.65
lz4 pt=3 -> ops/s: 1896294 cpu*s: 174.7
lz4 pt=4 -> ops/s: 1949174 cpu*s: 172.26
lz4 pt=6 -> ops/s: 1912517 cpu*s: 175.91
lz4 pt=8 -> ops/s: 1930585 cpu*s: 176.71
zstd pt=1 -> ops/s: 1239379 cpu*s: 129.85
zstd pt=2 -> ops/s: 1171742 cpu*s: 226.12
zstd pt=3 -> ops/s: 1832574 cpu*s: 214.21
zstd pt=4 -> ops/s: 1887124 cpu*s: 212.51
zstd pt=6 -> ops/s: 1920936 cpu*s: 211.7
zstd pt=8 -> ops/s: 1885544 cpu*s: 214.87
```
After this change:
```
none pt=1 -> ops/s: 1964361 cpu*s: 72.66
none pt=2 -> ops/s: 1914033 cpu*s: 104.95
none pt=3 -> ops/s: 1978567 cpu*s: 100.24
lz4 pt=1 -> ops/s: 2041703 cpu*s: 92.88
lz4 pt=2 -> ops/s: 1903210 cpu*s: 121.64
lz4 pt=3 -> ops/s: 1973906 cpu*s: 122.22
lz4 pt=4 -> ops/s: 1952605 cpu*s: 123.05
lz4 pt=6 -> ops/s: 1957524 cpu*s: 124.31
lz4 pt=8 -> ops/s: 1986274 cpu*s: 129.06
zstd pt=1 -> ops/s: 1233748 cpu*s: 130.43
zstd pt=2 -> ops/s: 1675226 cpu*s: 158.41
zstd pt=3 -> ops/s: 1929878 cpu*s: 159.77
zstd pt=4 -> ops/s: 1916403 cpu*s: 160.99
zstd pt=6 -> ops/s: 1942526 cpu*s: 166.21
zstd pt=8 -> ops/s: 1966704 cpu*s: 171.56
```
For parallel_threads=1, results are very similar, as expected.
For parallel_threads>1, throughput is usually improved a bit, but cpu consumption is dramatically reduced. For zstd, maximum throughput is essentially achieved with pt=3 rather than the previous roughly pt=4 to 6. And the old used about 30% more CPU.
We can also compare with more expensive compression by raising the compression level.
```
SUFFIX=`tty | sed 's|/|_|g'`; CT=zstd; for CL in 4 6 8; do for PT in 1 4 8; do echo -n "$CT@$CL pt=$PT -> "; (for I in `seq 1 10`; do BIN=/tmp/dbbench${SUFFIX}.bin; rm -f $BIN; cp db_bench $BIN; /usr/bin/time $BIN -db=/dev/shm/dbbench$SUFFIX --benchmarks=fillseq -num=10000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=1000 -fifo_compaction_allow_compaction=0 -disable_wal -write_buffer_size=12000000 -format_version=7 -compression_type=$CT -compression_parallel_threads=$PT -compression_level=$CL 2>&1; done) | awk '/micros.op/ {n++; sum += $5;} /system / { cpu += $1 + $2; } END { print "ops/s: " int(sum/n) " cpu*s: " cpu; }'; done; done
```
Before:
```
zstd@4 pt=1 -> ops/s: 883630 cpu*s: 161.12
zstd@4 pt=4 -> ops/s: 1878206 cpu*s: 243.25
zstd@4 pt=8 -> ops/s: 1885002 cpu*s: 245.89
zstd@6 pt=1 -> ops/s: 710767 cpu*s: 189.44
zstd@6 pt=4 -> ops/s: 1706377 cpu*s: 277.29
zstd@6 pt=8 -> ops/s: 1866736 cpu*s: 275.07
zstd@8 pt=1 -> ops/s: 529047 cpu*s: 237.87
zstd@8 pt=4 -> ops/s: 1401379 cpu*s: 330.61
zstd@8 pt=8 -> ops/s: 1895601 cpu*s: 321.59
```
After:
```
zstd@4 pt=1 -> ops/s: 889905 cpu*s: 161.03
zstd@4 pt=4 -> ops/s: 1942240 cpu*s: 193.18
zstd@4 pt=8 -> ops/s: 1922367 cpu*s: 205.21
zstd@6 pt=1 -> ops/s: 713870 cpu*s: 188.91
zstd@6 pt=4 -> ops/s: 1832314 cpu*s: 219.66
zstd@6 pt=8 -> ops/s: 1949631 cpu*s: 229.34
zstd@8 pt=1 -> ops/s: 530324 cpu*s: 238.02
zstd@8 pt=4 -> ops/s: 1479767 cpu*s: 271.65
zstd@8 pt=8 -> ops/s: 1949631 cpu*s: 275.6
```
And we can also look at the cumulative effect of this change and https://github.com/facebook/rocksdb/pull/13850 that will combine for the parallel compression improvements in the upcoming 10.7 release:
Before both:
```
lz4 pt=1 -> ops/s: 1954445 cpu*s: 95.14
lz4 pt=3 -> ops/s: 1687043 cpu*s: 186.62
lz4 pt=5 -> ops/s: 1708196 cpu*s: 188.33
zstd pt=1 -> ops/s: 1220649 cpu*s: 131.2
zstd pt=3 -> ops/s: 1658100 cpu*s: 227.08
zstd pt=5 -> ops/s: 1685074 cpu*s: 226.08
```
After:
```
lz4 pt=1 -> ops/s: 2048214 cpu*s: 93.24
lz4 pt=3 -> ops/s: 1922049 cpu*s: 122.9
lz4 pt=5 -> ops/s: 1980165 cpu*s: 122.49
zstd pt=1 -> ops/s: 1245165 cpu*s: 128.84
zstd pt=3 -> ops/s: 1956961 cpu*s: 158.73
zstd pt=5 -> ops/s: 1970458 cpu*s: 161.02
```
In summary, before with zstd default level, you could see only
* about 38% increase in throughput for about 73% increase in CPU usage
Now you can get
* about 58% increase in throughput for about 25% increase in CPU usage
## Performance, high load
To validate this for usage on remote compaction workers, we also need to test whether it falls over at high load or anything concerning like that. For this I did a lot of testing with concurrent db_bench and zstd compression_level=8 and parallel_thread (PT) in {1,8} trying to observe "bad" behaviors such as stalls due to preempted threads and such. On a 166 core machine where a "job" is a db_bench process running a fillseq benchmark similar to above in parallel with others, I could summarize the results like this:
10 jobs PT=8 vs. PT=1 -> 12% more CPU usage, 75% reduction in wall time, 1.9 jobs/sec (vs. 0.5)
50 jobs PT=8 vs. PT=1 -> 89% more CPU usage, 27% reduction in wall time, 3.1 jobs/sec (vs. 2.3)
100 jobs PT=8 vs. PT=1 -> 24% more CPU usage, 5% reduction in wall time, 3.25 jobs/sec (vs. 3.1)
150 jobs PT=8 vs. PT=1 -> 4% more CPU usage, 2% increase in wall time, 3.3 jobs/sec (vs. 3.4)
500 jobs PT=8 vs. PT=1 -> 1% more CPU usage, insignificant difference in wall time, 3.3 jobs/sec
Even when there are 4000 threads potentially competing for 166 cores, the throughput (3.3 jobs / sec) is still very close to maximum (3.4). Enabling parallel compression didn't result in notably less throughput (based on wall clock time for all jobs to complete) in any case tested above, and much higher throughput for many cases. If parallel compression causes us to tip from comfortably under-saturating to over-saturating the cores (as in the 50 jobs case), the overall CPU usage can be much higher, presumably due to lower CPU cache hit rates and maybe clock throttling, but parallel compression still has the throughput advantage in those cases.
In other words, what would we stand to gain from being able to intelligently share worker threads between compaction jobs? It doesn't seem that much.
Reviewed By: xingbowang
Differential Revision: D81365623
Pulled By: pdillinger
fbshipit-source-id: 5db5151a959b5d25b84dbe185bc208bd188f2d1c
Summary:
A follow-up to https://github.com/facebook/rocksdb/issues/13904 which was incomplete in updating CI jobs to support C++20 because the C++20 usage was only in tests. Here we add subtle C++20 usage in the public API ("using enum" feature in db.h) to force the issue.
A lot of the work for this PR was in updating the Ubuntu22 docker image, for earlier compiler/runtime versions supporting C++20, and generating a new Ubuntu24 docker image, for later compiler/runtime versions. The Ubuntu22 image needed to be updated because there are incompatibilities with clang-13 + c++20 + libstdc++ for gcc 11, seen on these examples
```
#include <chrono>
int main(int argc, char *argv[]) {
std::chrono::microseconds d = {}; return 0;
}
```
and
```
#include <coroutine>
int main() { return 0; }
```
The second was causing recurring failures in build-linux-clang-13-asan-ubsan-with-folly, now fixed.
So we have to install clang's libc++ to compile with clang-13. I haven't been able to get this to work with some of the libraries like benchmark, glog, and/or gflags, but I'm able to compile core RocksDB with clang-13. On this docker image, an extra compiler parameter is needed to compile with gcc and glog because it's built from source perhaps not perfectly, because the ubuntu package transitively conflicts with libc++.
The Ubuntu24 image seems to be low-drama and generally work for testing out newer compiler versions. The mingw build uses Ubuntu24 because the mingw package on Ubuntu22 uses a gcc version that is too old.
And the mass of other code changes are trying to work around new warnings, mostly from clang-analyze, which I upgraded to clang-18 in CI.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13915
Test Plan: CI, including temporarily including the nightly jobs in the PR jobs in earlier revisions to test and stabilize
Reviewed By: archang19
Differential Revision: D81933067
Pulled By: pdillinger
fbshipit-source-id: 7e33823006a79d5f3cf5bc1d625f0a3c08a7d74c
Summary:
I am wanting to use std::counting_semaphore for something and the timing seems good to require C++20 support. The internets suggest:
* GCC >= 10 is adequate, >= 11 preferred
* Clang >= 10 is needed
* Visual Studio >= 2019 is adquate
And popular linux distributions look like this:
* CentOS Stream 9 -> GCC 11.2 (CentOS 8 is EOL)
* Ubuntu 22.04 LTS -> GCC 11.x (Ubuntu 20 just ended standard support)
* Debian 12 (oldstable) -> GCC 12.2
* (Debian 11 has ended security updates, uses GCC 10.2)
This required generating a new docker image based on Ubuntu 22 for CI using gcc. The existing Ubuntu 20 image works for covering appropriate clang versions (though we should maybe add a much later version as well, in the next increment of our Ubuntu 22 image; however the minimum available clang build from apt.llvm.org for Ubuntu 22 is clang 13).
Update to SetDumpFilter is to quiet a mysterious gcc-13 warning-as-error.
Removed --compile-no-warning-as-error from a cmake command line because cmake in the new docker image is too old for this option.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13904
Test Plan: CI, one minor unit test added to verify std::counting_semaphor works
Reviewed By: xingbowang
Differential Revision: D81266435
Pulled By: pdillinger
fbshipit-source-id: 26040eeccca7004416e29a6ff4f6ea93f2052684
Summary:
Building db_bench with clang and DEBUG_LEVEL=0 was failing with unused variable. This was not caught by CI so I have added this to the build-linux-clang-13-no_test_run job.
Also, while I was touching CI:
* Fold build-linux-release-rtti into build-linux-release by reducing the number of combinations tested between static/dynamic lib and rtti/not. I don't expect these to interact meaningfully with an extremely mature compiler.
* Combine build-linux-clang10-asan and build-linux-clang10-ubsan because clang is extremely reliable running both together
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13813
Test Plan: manual builds, CI
Reviewed By: krhancoc
Differential Revision: D79112643
Pulled By: pdillinger
fbshipit-source-id: 4ffc672718c05fa4597d637aacbc5a179ad8a0cf
Summary:
So that we can use --command=recompress with a custom CompressionManager. (It's not required for reading files using a custom CompressionManager because those can already use ObjectLibrary for dependency injection.)
Suggested follow-up:
* These tests should not be using C arrays, snprintf, manual delete, etc. except for thin compatibility with argc/argv.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13783
Test Plan: unit test added, some manual testing
Reviewed By: archang19
Differential Revision: D78574434
Pulled By: pdillinger
fbshipit-source-id: 609e6c6439090e6b7e9b63fbd4c2d3f04b104fcf
Summary:
See `Makefile` for actual changes:
* ZLIB remains the same
* BZIP2 remains the same
* SNAPPY is a minor update
* LZ4 is a significant update with multithreaded/multicore compression https://github.com/lz4/lz4/releases/tag/v1.10.0
* ZSTD is a significant update RocksDB is called out as benefiting in particular from the performance improvements herein https://github.com/facebook/zstd/releases/tag/v1.5.7
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13609
Reviewed By: archang19
Differential Revision: D77877295
Pulled By: mszeszko-meta
fbshipit-source-id: bf9a257e8f68dec3d02743b339aa2df65df4ab2c
Summary:
Somehow this was previously not being tested in our Windows CI jobs so was accidentally broken in https://github.com/facebook/rocksdb/pull/13540 This fix will need to be backported to 10.3.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13649
Test Plan: CI
Reviewed By: hx235
Differential Revision: D75655418
Pulled By: pdillinger
fbshipit-source-id: a56bb213270904a1b7a13b905c2cc1919116df1c
Summary:
Usual release steps
* Release notes from 10.3 branch
* Update version.h
* Add 10.3.fb to check_format_compatible.sh
* Update folly commit hash. Added a few hacks to fix build errors.
Bonus:
* Add a check_format_compatible.sh sanity check to the per-PR GitHub actions jobs. It should be quick enough and catch typos in release diffs as we've seen in the past.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13622
Test Plan: CI
Reviewed By: jowlyzhang
Differential Revision: D74943843
Pulled By: pdillinger
fbshipit-source-id: 4ff1db9a635e111f8830cadff2d3ee51cf2de512
Summary:
**Context/Summary:**
- This is an attempt to fix our [build-window-vs2022 failure](https://github.com/facebook/rocksdb/actions/runs/14215681026/job/39831770554?fbclid=IwZXh0bgNhZW0CMTAAAR2BQLjp8kC1u1yyvN1_S5qwmrHEZOfzxJdcbj2vq7mvwwq83n1cbkmiBCA_aem_ygYxQA5EUmxh2y4EjMlTfg) below. snappy-1.1.8's cmake_minimum_required being less than 3.5 seems to trigger the complaint. Hopefully downloading the 1.2.2 which is the [first version starting to use higher cmake_minimum_required version](https://github.com/google/snappy/releases/tag/1.2.2) solves the failure.
```
Directory: D:\a\rocksdb\rocksdb\thirdparty\snappy-1.1.8
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 4/2/2025 9:02 AM build
CMake Error at CMakeLists.txt:29 (cmake_minimum_required):
Compatibility with CMake < 3.5 has been removed from CMake.
Update the VERSION argument <min> value. Or, use the <min>...<max> syntax
to tell CMake that the project requires at least <min> but has been updated
to work with policies introduced by <max> or earlier.
Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.
```
- The downloaded snappy do not include the content under nested repos Google Test and Google Benchmark. But snappy cmake by default will attempt to build them. Since we don't change snappy, we don't need building such development suit. This PR also disabled snappy cmake's attempt to build them.
- By running above changes, the same build [complained](https://github.com/facebook/rocksdb/actions/runs/14228883966/job/39874927730?pr=13514) about java cmakelists requiring too low cmake_minimum_required as well. So this PR also upgraded its cmake_minimum_required to be 3.11 aligning with its warning message
```
if(${CMAKE_VERSION} VERSION_LESS "3.11.4")
message("Please consider switching to CMake 3.11.4 or newer")
endif()
```
**Test plan**
Monitor build-window-vs2022 for this PR
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13514
Reviewed By: pdillinger
Differential Revision: D72333581
Pulled By: hx235
fbshipit-source-id: 1a9096738d39c8b1d270fe17fbd78c1ea4c4c45e
Summary:
Originally I was trying to update `build-linux-clang10-mini-tsan` to actually use `clang10` (as the name implied). https://github.com/facebook/rocksdb/issues/13220 was supposed to also update this configuration, but I did not see that we had a definition for `build-linux-clang10-mini-tsan` in both `config.yml` and `pr-jobs.yml`. I was wondering why I could not see my changes reflected in the CI checks after merging. After I updated `pr-jobs.yml` for this PR, I found that the CI check started failing https://github.com/facebook/rocksdb/actions/runs/12417441052/job/34668411263?pr=13232. I don't think it makes sense for me to tackle looking into all the TSAN warnings being reported in `clang10` (at least in this PR), so for now I have updated the name of the PR job to accurately reflect the command that is being run.
This PR also gets rid of the entire `.circleci` folder, which I think is the more significant change.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13232
Test Plan: Existing CI check is unchanged
Reviewed By: pdillinger
Differential Revision: D67462454
Pulled By: archang19
fbshipit-source-id: f1aabfe4c8793616d6cbaae36fdf007319bf7ab2
