Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/15366
## Summary
RocksDB's `.clang-tidy` enables `modernize-use-override`, and clang
reports `-Winconsistent-missing-destructor-override` when a class marks
its other overrides `override` but not its destructor. A full scan finds
190 `modernize-use-override`, 100
`inconsistent-missing-destructor-override` and 2 `suggest-override`
findings in existing code. Each one comes back whenever a nearby line is
edited. They also leave classes where it isn't obvious which methods are
overrides and which introduce new virtuals.
This diff marks every overriding member function and destructor
`override` and drops the redundant `virtual` on them. It also changes
`final override` to plain `final`, following the check's default. Methods
that introduce a new virtual keep `virtual`. Most edits are clang-tidy's
own fix-its, applied mechanically and then clang-formatted.
Compatibility: no signatures change. In public headers the only effect is
`override` on existing overrides (mostly destructors, plus
`Cache::Name()` and `Cache::GetPrintableOptions()`, whose base
declarations are unconditional), so user code that subclasses these types
is unaffected. Every touched class now marks all of its overrides
consistently. This matters because a partially-marked class would make
clang emit `-Winconsistent-missing-override` in user builds.
Reviewed By: rban1
Differential Revision: D123186308
fbshipit-source-id: 1f489ea32d938e3ec0faf9abc4f7ff49e042d37a
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/15325
## Summary
RocksDB's `.clang-tidy` enables the `clang-analyzer-*` and `bugprone-*`
checks. A full scan of the repo still reports 78 findings from these
checks in existing code. Each time a nearby line is edited, they come
back as noise, and a few of them are real defects. This diff resolves 73
of the 78; upstream has since fixed one more, and the four left are
listed under "Intentionally not changed". Some fixes repair real bugs.
The rest make the code's intent explicit to the analyzer without
changing behavior.
Real bugs fixed:
- `port/mmap.cc`: `MemMapping::operator=(MemMapping&&)` ran the
destructor on `*this`, `memcpy`'d over a non-trivially-copyable object,
then placement-new'd the source. That is undefined behavior, and `*this`
can be the base subobject of a `TypedMemMapping`. It now moves the
members one by one, and the previous mapping is released through a
local `MemMapping`.
- `port/mmap.h`: `TypedMemMapping::operator=(MemMapping&&)` had no
`return *this;`, so calling it was undefined behavior.
- `cache/typed_cache.h`: `if constexpr (sizeof(TCreateContext) > 0)` is
always true, so the branch meant for empty create contexts (for example
`BlobContentsCreator`) could never run. The check is now
`!std::is_empty_v<TCreateContext>`. Existing instantiations still call
the same `Create` function.
- `cassandra_row_merge_test.cc`: the test reused a vector after
`std::move`-ing it into `RowValue::Merge(std::vector<RowValue>&&)`. That
only worked because `Merge` happens not to consume its input. The test
now rebuilds the input.
Behavior-preserving changes:
- Virtual calls in constructors and destructors are now explicitly
qualified (for example `DBImpl::CloseImpl()` in `~DBImpl`). During
construction and destruction, dispatch already resolved to these
functions.
- Stale `/*name=*/` argument comments now match the parameter names.
- `unique_ptr::release()` after a successful cache `Insert` (where the
cache takes ownership) uses `[[maybe_unused]] T* released = x.release();`,
the pattern already used in `table_cache.cc`.
- Removed `class Mutex;` and `class StaticMeta;` forward declarations
that sat in the wrong namespace.
- Added default member initializers for the
`TablePropertiesCollectorFactory::Context` fields that its default
constructor left uninitialized.
- Smaller cleanups: a redundant `cond ? 3 : 3` in a test,
`sizeof(std::string)` in a memory estimate, and an explicit
`Cleanable` base move via `up_cast`. Tests that intentionally use
moved-from objects now carry `NOLINT(bugprone-use-after-move)`,
matching the existing convention.
Intentionally not changed: enum casts in tests that check invalid-value
handling; three `clang-analyzer-core.NullDereference` reports the
analyzer can't prove safe (the C API timestamp out-parameters in
`db/c.cc`, which are only written when the caller asked for a
timestamp, and the waiter-list walk in `util/async_file_reader.cc`);
bit-flag enums in public headers (`FileOpenContract`,
`OptionTypeFlags`), where OR-combined values are valid; ownership
transfers the analyzer can't see across translation units
(`partitioned_index_reader.cc`, `fs_posix.cc`); and the intentional
two-separator check in `file/filename.cc`.
Reviewed By: rban1
Differential Revision: D123186307
fbshipit-source-id: 623e871286b1c73263b1309cb194692194199e0f
Summary:
... which is the default for CentOS 9 and Ubuntu 24, the latter of which is now available in GitHub Actions. Relevant CI job updated.
Re-formatted all cc|c|h files except in third-party/, using
```
clang-format -i `git ls-files | grep -E '[.](cc|c|h)$' | grep -v third-party/`
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13233
Test Plan: CI
Reviewed By: jaykorean, archang19
Differential Revision: D67461638
Pulled By: pdillinger
fbshipit-source-id: 0c9ac21a3f5eea6f5ade68bb6af7b6ba16c8b301
Summary:
We haven't been actively mantaining RocksDB LITE recently and the size must have been gone up significantly. We are removing the support.
Most of changes were done through following comments:
unifdef -m -UROCKSDB_LITE `git grep -l ROCKSDB_LITE | egrep '[.](cc|h)'`
by Peter Dillinger. Others changes were manually applied to build scripts, CircleCI manifests, ROCKSDB_LITE is used in an expression and file db_stress_test_base.cc.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/11147
Test Plan: See CI
Reviewed By: pdillinger
Differential Revision: D42796341
fbshipit-source-id: 4920e15fc2060c2cd2221330a6d0e5e65d4b7fe2
Summary:
Cleans up some of the dependencies on test code in the Makefile while building tools:
- Moves the test::RandomString, DBBaseTest::RandomString into Random
- Moves the test::RandomHumanReadableString into Random
- Moves the DestroyDir method into file_utils
- Moves the SetupSyncPointsToMockDirectIO into sync_point.
- Moves the FaultInjection Env and FS classes under env
These changes allow all of the tools to build without dependencies on test_util, thereby simplifying the build dependencies. By moving the FaultInjection code, the dependency in db_stress on different libraries for debug vs release was eliminated.
Tested both release and debug builds via Make and CMake for both static and shared libraries.
More work remains to clean up how the tools are built and remove some unnecessary dependencies. There is also more work that should be done to get the Makefile and CMake to align in their builds -- what is in the libraries and the sizes of the executables are different.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/7097
Reviewed By: riversand963
Differential Revision: D22463160
Pulled By: pdillinger
fbshipit-source-id: e19462b53324ab3f0b7c72459dbc73165cc382b2
Summary:
When dynamically linking two binaries together, different builds of RocksDB from two sources might cause errors. To provide a tool for user to solve the problem, the RocksDB namespace is changed to a flag which can be overridden in build time.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6433
Test Plan: Build release, all and jtest. Try to build with ROCKSDB_NAMESPACE with another flag.
Differential Revision: D19977691
fbshipit-source-id: aa7f2d0972e1c31d75339ac48478f34f6cfcfb3e
Summary:
There are too many types of files under util/. Some test related files don't belong to there or just are just loosely related. Mo
ve them to a new directory test_util/, so that util/ is cleaner.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5377
Differential Revision: D15551366
Pulled By: siying
fbshipit-source-id: 0f5c8653832354ef8caa31749c0143815d719e2c
Summary:
Ran the following commands to recursively change all the files under RocksDB:
```
find . -type f -name "*.cc" -exec sed -i 's/ unique_ptr/ std::unique_ptr/g' {} +
find . -type f -name "*.cc" -exec sed -i 's/<unique_ptr/<std::unique_ptr/g' {} +
find . -type f -name "*.cc" -exec sed -i 's/ shared_ptr/ std::shared_ptr/g' {} +
find . -type f -name "*.cc" -exec sed -i 's/<shared_ptr/<std::shared_ptr/g' {} +
```
Running `make format` updated some formatting on the files touched.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4638
Differential Revision: D12934992
Pulled By: sagar0
fbshipit-source-id: 45a15d23c230cdd64c08f9c0243e5183934338a8
Summary:
This reverts the previous commit 1d7048c598, which broke the build.
Did a `git revert 1d7048c`.
Closes https://github.com/facebook/rocksdb/pull/2627
Differential Revision: D5476473
Pulled By: sagar0
fbshipit-source-id: 4756ff5c0dfc88c17eceb00e02c36176de728d06
Summary: This uses `clang-tidy` to comment out unused parameters (in functions, methods and lambdas) in fbcode. Cases that the tool failed to handle are fixed manually.
Reviewed By: igorsugak
Differential Revision: D5454343
fbshipit-source-id: 5dee339b4334e25e963891b519a5aa81fbf627b2
Summary:
introduce new methods into a public threadpool interface,
- allow submission of std::functions as they allow greater flexibility.
- add Joining methods to the implementation to join scheduled and submitted jobs with
an option to cancel jobs that did not start executing.
- Remove ugly `#ifdefs` between pthread and std implementation, make it uniform.
- introduce pimpl for a drop in replacement of the implementation
- Introduce rocksdb::port::Thread typedef which is a replacement for std::thread. On Posix Thread defaults as before std::thread.
- Implement WindowsThread that allocates memory in a more controllable manner than windows std::thread with a replaceable implementation.
- should be no functionality changes.
Closes https://github.com/facebook/rocksdb/pull/1823
Differential Revision: D4492902
Pulled By: siying
fbshipit-source-id: c74cb11
Summary:
The persistent cache is designed to hop over errors and return key not found. So far, it has shown resilience to write errors, encoding errors, data corruption etc. It is not resilient against disappearing files/directories. This was exposed during testing when multiple instances of persistence cache was started sharing the same directory simulating an unpredictable filesystem environment.
This patch
- makes the write code path more resilient to errors while creating files
- makes the read code path more resilient to handle situation where files are not found
- added a test that does negative write/read testing by removing the directory while writes are in progress
Closes https://github.com/facebook/rocksdb/pull/1472
Differential Revision: D4143413
Pulled By: kradhakrishnan
fbshipit-source-id: fd25e9b
Summary:
Travis times out with persistent_cache_test, there is no way to increase the
timeout, so lower the test case for Travis
Test Plan: Run on travis and sandcastle
Subscribers: leveldb, dhruba, andrewkr
Differential Revision: https://reviews.facebook.net/D61863
Summary:
The patch is a continuation of part 5. It glues the abstraction for
file layout and metadata, and flush out the implementation of the API. It
adds unit tests for the implementation.
Test Plan: Run unit tests
Subscribers: andrewkr, dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D57549
Summary: Enabled build in Windows and corresponding fixes
Test Plan:
Compile and run persistent_cache_test in Windows and make check in
Linux
Reviewers: sdong
Subscribers: andrewkr, dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D59307
Summary:
This provides provides an implementation of PersistentCacheTier that is
specialized for RAM. This tier does not persist data though.
Why do we need this tier ?
This is ideal as tier 0. This tier can host data that is too hot.
Why can't we use Cache variants ?
Yes you can use them instead. This tier can potentially outperform BlockCache
in RAW mode by virtue of compression and compressed cache in block cache doesn't
seem very popular. Potentially this tier can be modified to under stand the
disadvantage of the tier below and retain data that the tier below is bad at
handling (for example index and bloom data that is huge in size)
Test Plan: Run unit tests added
Subscribers: andrewkr, dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D57069
