database/rocksdb
1
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2026-10-23 08:09:02 +03:00
Code Activity
Files
main
rocksdb/utilities/counted_fs.cc
Josh Kang 4b983da055 Spell out special member functions (rule of five) (#15387) 
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/15387

## Summary

RocksDB's `.clang-tidy` enables `cppcoreguidelines-special-member-functions`,
`cppcoreguidelines-virtual-class-destructor` and the noexcept-move checks,
and clang warns `-Wdeprecated-copy-with-user-provided-dtor`. A full scan
reports 476 findings. In most of them, a class declares a destructor (or
copy operations) but leaves the rest of copy/move implicit. That has two
costs. Readers can't tell whether a class may be copied or moved. And
resource-owning classes keep an implicit copy that would double-free if
anyone ever used it. This diff resolves 453 of the 476.

Approach, picked per class from least to most invasive:
1. An empty user-provided destructor becomes `= default`. This covers most
   interface classes, such as `virtual ~X() {}` -> `= default`. If the
   class is still copied, that would only rename the deprecated-copy
   warning, so the destructor is removed instead when nothing needs it
   (`Customizable`, `TableFactory` and `InternalKeyComparator` inherit a
   virtual one; `IteratorWrapperBase` isn't polymorphic), and
   `MemTableRep::KeyComparator` gets defaulted copy and move.
2. Classes whose copy is already deleted get explicitly deleted move
   operations too. Move was already unavailable, so this only documents
   it.
3. Resource-owning classes that are never copied get deleted copy and move
   operations. Examples: the C API wrapper structs, which run a
   `destructor_` callback (edited in `tools/c_api_gen/c_base.cc` and
   regenerated into `db/c.cc`); `ShardedCache`, which frees its shards;
   iterator wrappers; and RAII handles.
4. Value types that are copied or moved get the missing operations as
   `= default`, which keeps their current behavior.
5. Exception specifications of existing functions are unchanged, except
   for `WriteBatch`'s move assignment (below): no other `noexcept` is
   added or removed. Only the move operations this diff
   newly declares (`Comparator`, `FileSampledStats` and the thread-local
   `Entry`) are `noexcept`; they copy only scalars or atomics.

`WriteBatch`'s copy and move assignment destroyed `*this` and
placement-new'ed a new `WriteBatch` over it. That is undefined behavior
when `*this` is a subclass, and if the copy constructor threw, `*this`
was left destroyed and was later destroyed again. Move assignment now
assigns member-wise and ends in the same state as the move constructor,
which leaves `has_key_with_ts_`, the timestamp-size tracking and the
other fields it doesn't copy at their defaults. `rep_` is swapped through
a temporary, because a plain string move-assignment can keep the old
buffer (e.g. a large batch reset with `batch = WriteBatch()` would keep
its memory). Nothing in it can throw, so it is now `noexcept`. Copy
assignment copies into a temporary and moves it in, so a throwing copy
leaves `*this` unchanged.

Bugs this exposed and fixed:
- `RestoreAfterCopyOrCreateWorkItem` (backup engine): the hand-written
  move constructor and assignment moved only some fields, so `from_file`
  and `to_file` were empty after `push_back(std::move(...))`. A restore
  that hit a checksum mismatch then reported "While restoring  -> : ..."
  without file names. It now uses the compiler-generated moves, which
  are still `noexcept`.
- `ExternalSstFileIngestionJob` lived in a `std::vector`, and its
  destructor frees raw-owned pointers, so the implicit copy used on
  vector reallocation would double-free. Its file batches also point into
  its own inline file storage, so neither a copy nor a move is safe after
  `Prepare()`. Copy and move are now deleted, and `DBImpl` holds jobs
  through `std::unique_ptr`.

Compatibility: no public class loses an operation users can call. Public
interfaces mostly only get `virtual ~X() = default;`.
`WriteBatch`'s move assignment becomes `noexcept`, which is not part of
the mangled name, so the ABI doesn't change.
`SubcompactionJobInfo` and `CompactionJobInfo` intentionally stay C++20
aggregates, so users' aggregate and designated initializers keep
compiling. `Configurable`, `CompareInterface`, `WriteBatchBase`,
`ObjectLibrary::Entry` and `MemTableRep::KeyComparator` declare defaulted
copy and move, and `Customizable` no longer declares a destructor. As a
result, `Configurable` and `Customizable` subclasses that declare no
destructor or copy operations of their own now move their
registered-options list instead of copying it. In-tree, these are the
mutable-options adapters, the table factories, the memory allocators and
test classes; all are held through smart pointers or used in place, and
`Clone()` copies, so none is moved by value.

Not changed (23): `IODebugContext` (custom deep copy plus a
`std::shared_mutex`), `DBWithColumnFamilies` in db_bench (stored in a
`std::vector` with non-movable members), `Snapshot`'s protected virtual
destructor (intended), `ManagedSnapshot`, the `SubcompactionJobInfo` and
`CompactionJobInfo` aggregates, and the 16 noexcept-move-constructor and
noexcept-swap findings on existing functions, including `autovector`'s
move assignment (adding `noexcept` would change their exception
specifications). Copying `ManagedSnapshot` already double-releases the
snapshot. Fixing that needs a real move implementation and is a behavior
change, so it is left for a separate diff.

Reviewed By: rban1

Differential Revision: D123186309

fbshipit-source-id: 6f8e57e233e12199a007407130bb9a1f6d52e8d1
2026-10-12 16:00:09 -07:00

398 lines
13 KiB
C++
Raw Permalink Blame History

// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#include "utilities/counted_fs.h"
#include <sstream>
#include "rocksdb/file_system.h"
#include "rocksdb/utilities/options_type.h"
namespace ROCKSDB_NAMESPACE {
namespace {
class CountedSequentialFile : public FSSequentialFileOwnerWrapper {
private:
CountedFileSystem* fs_;
public:
CountedSequentialFile(std::unique_ptr<FSSequentialFile>&& f,
CountedFileSystem* fs)
: FSSequentialFileOwnerWrapper(std::move(f)), fs_(fs) {}
// No copying allowed
CountedSequentialFile(const CountedSequentialFile&) = delete;
CountedSequentialFile& operator=(const CountedSequentialFile&) = delete;
CountedSequentialFile(CountedSequentialFile&&) = delete;
CountedSequentialFile& operator=(CountedSequentialFile&&) = delete;
~CountedSequentialFile() override { fs_->counters()->closes++; }
IOStatus Read(size_t n, const IOOptions& options, Slice* result,
char* scratch, IODebugContext* dbg) override {
IOStatus rv = target()->Read(n, options, result, scratch, dbg);
fs_->counters()->reads.RecordOp(rv, result->size());
return rv;
}
IOStatus PositionedRead(uint64_t offset, size_t n, const IOOptions& options,
Slice* result, char* scratch,
IODebugContext* dbg) override {
IOStatus rv =
target()->PositionedRead(offset, n, options, result, scratch, dbg);
fs_->counters()->reads.RecordOp(rv, result->size());
return rv;
}
};
class CountedRandomAccessFile : public FSRandomAccessFileOwnerWrapper {
private:
CountedFileSystem* fs_;
public:
CountedRandomAccessFile(std::unique_ptr<FSRandomAccessFile>&& f,
CountedFileSystem* fs)
: FSRandomAccessFileOwnerWrapper(std::move(f)), fs_(fs) {}
// No copying allowed
CountedRandomAccessFile(const CountedRandomAccessFile&) = delete;
CountedRandomAccessFile& operator=(const CountedRandomAccessFile&) = delete;
CountedRandomAccessFile(CountedRandomAccessFile&&) = delete;
CountedRandomAccessFile& operator=(CountedRandomAccessFile&&) = delete;
~CountedRandomAccessFile() override { fs_->counters()->closes++; }
IOStatus Read(uint64_t offset, size_t n, const IOOptions& options,
Slice* result, char* scratch,
IODebugContext* dbg) const override {
IOStatus rv = target()->Read(offset, n, options, result, scratch, dbg);
fs_->counters()->reads.RecordOp(rv, result->size());
return rv;
}
IOStatus MultiRead(FSReadRequest* reqs, size_t num_reqs,
const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->MultiRead(reqs, num_reqs, options, dbg);
for (size_t r = 0; r < num_reqs; r++) {
fs_->counters()->reads.RecordOp(reqs[r].status, reqs[r].result.size());
}
return rv;
}
};
class CountedWritableFile : public FSWritableFileOwnerWrapper {
private:
CountedFileSystem* fs_;
public:
CountedWritableFile(std::unique_ptr<FSWritableFile>&& f,
CountedFileSystem* fs)
: FSWritableFileOwnerWrapper(std::move(f)), fs_(fs) {}
IOStatus Append(const Slice& data, const IOOptions& options,
IODebugContext* dbg) override {
IOStatus rv = target()->Append(data, options, dbg);
fs_->counters()->writes.RecordOp(rv, data.size());
return rv;
}
IOStatus Append(const Slice& data, const IOOptions& options,
const DataVerificationInfo& info,
IODebugContext* dbg) override {
IOStatus rv = target()->Append(data, options, info, dbg);
fs_->counters()->writes.RecordOp(rv, data.size());
return rv;
}
IOStatus PositionedAppend(const Slice& data, uint64_t offset,
const IOOptions& options,
IODebugContext* dbg) override {
IOStatus rv = target()->PositionedAppend(data, offset, options, dbg);
fs_->counters()->writes.RecordOp(rv, data.size());
return rv;
}
IOStatus PositionedAppend(const Slice& data, uint64_t offset,
const IOOptions& options,
const DataVerificationInfo& info,
IODebugContext* dbg) override {
IOStatus rv = target()->PositionedAppend(data, offset, options, info, dbg);
fs_->counters()->writes.RecordOp(rv, data.size());
return rv;
}
IOStatus Close(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->Close(options, dbg);
if (rv.ok()) {
fs_->counters()->closes++;
}
return rv;
}
IOStatus Flush(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->Flush(options, dbg);
if (rv.ok()) {
fs_->counters()->flushes++;
}
return rv;
}
IOStatus Sync(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->Sync(options, dbg);
if (rv.ok()) {
fs_->counters()->syncs++;
}
return rv;
}
IOStatus Fsync(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->Fsync(options, dbg);
if (rv.ok()) {
fs_->counters()->fsyncs++;
}
return rv;
}
IOStatus RangeSync(uint64_t offset, uint64_t nbytes, const IOOptions& options,
IODebugContext* dbg) override {
IOStatus rv = target()->RangeSync(offset, nbytes, options, dbg);
if (rv.ok()) {
fs_->counters()->syncs++;
}
return rv;
}
};
class CountedRandomRWFile : public FSRandomRWFileOwnerWrapper {
private:
mutable CountedFileSystem* fs_;
public:
CountedRandomRWFile(std::unique_ptr<FSRandomRWFile>&& f,
CountedFileSystem* fs)
: FSRandomRWFileOwnerWrapper(std::move(f)), fs_(fs) {}
IOStatus Write(uint64_t offset, const Slice& data, const IOOptions& options,
IODebugContext* dbg) override {
IOStatus rv = target()->Write(offset, data, options, dbg);
fs_->counters()->writes.RecordOp(rv, data.size());
return rv;
}
IOStatus Read(uint64_t offset, size_t n, const IOOptions& options,
Slice* result, char* scratch,
IODebugContext* dbg) const override {
IOStatus rv = target()->Read(offset, n, options, result, scratch, dbg);
fs_->counters()->reads.RecordOp(rv, result->size());
return rv;
}
IOStatus Flush(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->Flush(options, dbg);
if (rv.ok()) {
fs_->counters()->flushes++;
}
return rv;
}
IOStatus Sync(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->Sync(options, dbg);
if (rv.ok()) {
fs_->counters()->syncs++;
}
return rv;
}
IOStatus Fsync(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->Fsync(options, dbg);
if (rv.ok()) {
fs_->counters()->fsyncs++;
}
return rv;
}
IOStatus Close(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = target()->Close(options, dbg);
if (rv.ok()) {
fs_->counters()->closes++;
}
return rv;
}
};
class CountedDirectory : public FSDirectoryWrapper {
private:
mutable CountedFileSystem* fs_;
bool closed_ = false;
public:
CountedDirectory(std::unique_ptr<FSDirectory>&& f, CountedFileSystem* fs)
: FSDirectoryWrapper(std::move(f)), fs_(fs) {}
// No copying allowed
CountedDirectory(const CountedDirectory&) = delete;
CountedDirectory& operator=(const CountedDirectory&) = delete;
CountedDirectory(CountedDirectory&&) = delete;
CountedDirectory& operator=(CountedDirectory&&) = delete;
IOStatus Fsync(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = FSDirectoryWrapper::Fsync(options, dbg);
if (rv.ok()) {
fs_->counters()->dsyncs++;
}
return rv;
}
IOStatus Close(const IOOptions& options, IODebugContext* dbg) override {
IOStatus rv = FSDirectoryWrapper::Close(options, dbg);
if (rv.ok()) {
fs_->counters()->closes++;
fs_->counters()->dir_closes++;
closed_ = true;
}
return rv;
}
IOStatus FsyncWithDirOptions(const IOOptions& options, IODebugContext* dbg,
const DirFsyncOptions& dir_options) override {
IOStatus rv =
FSDirectoryWrapper::FsyncWithDirOptions(options, dbg, dir_options);
if (rv.ok()) {
fs_->counters()->dsyncs++;
}
return rv;
}
~CountedDirectory() override {
if (!closed_) {
// TODO: fix DB+CF code to use explicit Close, not rely on destructor
fs_->counters()->closes++;
fs_->counters()->dir_closes++;
}
}
};
} // anonymous namespace
std::string FileOpCounters::PrintCounters() const {
std::stringstream ss;
ss << "Num files opened: " << opens.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num files deleted: " << deletes.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num files renamed: " << renames.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num Flush(): " << flushes.load(std::memory_order_relaxed) << std::endl;
ss << "Num Sync(): " << syncs.load(std::memory_order_relaxed) << std::endl;
ss << "Num Fsync(): " << fsyncs.load(std::memory_order_relaxed) << std::endl;
ss << "Num Dir Fsync(): " << dsyncs.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num Close(): " << closes.load(std::memory_order_relaxed) << std::endl;
ss << "Num Dir Open(): " << dir_opens.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num Dir Close(): " << dir_closes.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num Read(): " << reads.ops.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num Append(): " << writes.ops.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num bytes read: " << reads.bytes.load(std::memory_order_relaxed)
<< std::endl;
ss << "Num bytes written: " << writes.bytes.load(std::memory_order_relaxed)
<< std::endl;
return ss.str();
}
CountedFileSystem::CountedFileSystem(const std::shared_ptr<FileSystem>& base)
: FileSystemWrapper(base) {}
IOStatus CountedFileSystem::NewSequentialFile(
const std::string& f, const FileOptions& options,
std::unique_ptr<FSSequentialFile>* r, IODebugContext* dbg) {
std::unique_ptr<FSSequentialFile> base;
IOStatus s = target()->NewSequentialFile(f, options, &base, dbg);
if (s.ok()) {
counters_.opens++;
r->reset(new CountedSequentialFile(std::move(base), this));
}
return s;
}
IOStatus CountedFileSystem::NewRandomAccessFile(
const std::string& f, const FileOptions& options,
std::unique_ptr<FSRandomAccessFile>* r, IODebugContext* dbg) {
std::unique_ptr<FSRandomAccessFile> base;
IOStatus s = target()->NewRandomAccessFile(f, options, &base, dbg);
if (s.ok()) {
counters_.opens++;
r->reset(new CountedRandomAccessFile(std::move(base), this));
}
return s;
}
IOStatus CountedFileSystem::NewWritableFile(const std::string& f,
const FileOptions& options,
std::unique_ptr<FSWritableFile>* r,
IODebugContext* dbg) {
std::unique_ptr<FSWritableFile> base;
IOStatus s = target()->NewWritableFile(f, options, &base, dbg);
if (s.ok()) {
counters_.opens++;
r->reset(new CountedWritableFile(std::move(base), this));
}
return s;
}
IOStatus CountedFileSystem::ReopenWritableFile(
const std::string& fname, const FileOptions& options,
std::unique_ptr<FSWritableFile>* result, IODebugContext* dbg) {
std::unique_ptr<FSWritableFile> base;
IOStatus s = target()->ReopenWritableFile(fname, options, &base, dbg);
if (s.ok()) {
counters_.opens++;
result->reset(new CountedWritableFile(std::move(base), this));
}
return s;
}
IOStatus CountedFileSystem::ReuseWritableFile(
const std::string& fname, const std::string& old_fname,
const FileOptions& options, std::unique_ptr<FSWritableFile>* result,
IODebugContext* dbg) {
std::unique_ptr<FSWritableFile> base;
IOStatus s =
target()->ReuseWritableFile(fname, old_fname, options, &base, dbg);
if (s.ok()) {
counters_.opens++;
result->reset(new CountedWritableFile(std::move(base), this));
}
return s;
}
IOStatus CountedFileSystem::NewRandomRWFile(
const std::string& name, const FileOptions& options,
std::unique_ptr<FSRandomRWFile>* result, IODebugContext* dbg) {
std::unique_ptr<FSRandomRWFile> base;
IOStatus s = target()->NewRandomRWFile(name, options, &base, dbg);
if (s.ok()) {
counters_.opens++;
result->reset(new CountedRandomRWFile(std::move(base), this));
}
return s;
}
IOStatus CountedFileSystem::NewDirectory(const std::string& name,
const IOOptions& options,
std::unique_ptr<FSDirectory>* result,
IODebugContext* dbg) {
std::unique_ptr<FSDirectory> base;
IOStatus s = target()->NewDirectory(name, options, &base, dbg);
if (s.ok()) {
counters_.opens++;
counters_.dir_opens++;
result->reset(new CountedDirectory(std::move(base), this));
}
return s;
}
} // namespace ROCKSDB_NAMESPACE
View Git Blame Copy Permalink