mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-23 08:09:02 +03:00
af0eae4de3
Summary:
`TransactionLogIterator` iterates the write-ahead log. It does not return transactions, does not require `TransactionDB`, and in fact returns `Status::NotSupported()` for two of the three `TransactionDB` write policies (WritePrepared and WriteUnprepared). The name predates `TransactionDB` and comes from "transaction log" being an old synonym for WAL. Everything else in this header was already renamed away from that terminology -- `LogFile` -> `WalFile`, `VectorLogPtr` -> `VectorWalPtr` -- and the C API has long used `rocksdb_wal_iterator_t` and `rocksdb_wal_iter_*`. This finishes the job.
The public class is now `WalIterator` in `include/rocksdb/wal_iterator.h`. `include/rocksdb/transaction_log.h` remains as a forwarding header and `TransactionLogIterator` remains as an alias, both marked deprecated by comment only (not `[[deprecated]]`) so existing users compiling with -Werror are unaffected, matching how `LogFile` and `VectorLogPtr` were retired. `DB::GetUpdatesSince()` keeps its name.
More importantly, the documentation was incomplete in ways that could cost a user data, so this states plainly what the API has always done:
- It only sees the WAL. Writes made with `WriteOptions::disableWAL`, and sequence numbers consumed by `IngestExternalFile()`, advance the DB sequence number without producing a WAL record. They leave permanent holes that end iteration. This was not documented anywhere, so it was unreasonable to expect a user to know those features silently break WAL tailing.
- The three iterator states, in particular that `!Valid()` with an OK `status()` means caught up rather than finished, and that calling `Next()` again is the intended way to tail a DB. The old comment on `Next()` said "REQUIRES: Valid() to be true", which forbids the primary use case.
- `Status::TryAgain`, which a caller must handle but which appeared nowhere in the public headers.
- That `GetUpdatesSince(seq)` silently starts later when `seq` is unavailable. Combined with the above this is the sharp edge: recovering from a spent iterator means calling `GetUpdatesSince()` again, so the recovery step is exactly where data can be skipped unnoticed. A consumer applying these batches to a copy of the DB diverges permanently and undetectably if it misses one, so the header now says such consumers must verify continuity themselves using `BatchResult::sequence` and `WriteBatch::Count()`.
Almost all of this diff is mechanical: renaming the type and its files, updating `#include` paths, rewriting comments, and updating build lists (`src.mk`, `CMakeLists.txt`, `BUCK`, `java/Makefile`, `java/CMakeLists.txt`). `db/c.cc` is regenerated from `tools/c_api_gen/c_base.cc`. Reviewers should focus on the four places that are not:
1. `WalIteratorImpl::UpdateCurrentWriteBatch()` in `db/wal_iterator_impl.cc` (renamed from `db/transaction_log_impl.cc`) drops dead code. On a sequence discontinuity it set `current_status_` to `NotFound` and then called `SeekToStartSequence()`, which returns immediately when `current_status_` is not OK. That guard and the early return in `Next()` were both added by fe63899d1 ("Add checks to GetUpdatesSince", https://github.com/facebook/rocksdb/issues/9459), which made the reseek unreachable without mentioning it. The surviving behavior -- a discontinuity ends the run with `NotFound` -- is what the class has always documented, so this deletes the unreachable reseek and the comment claiming it can restore an OK status rather than resurrecting it. There is no observable change: `Valid()` is `started_ && is_valid_`, `is_valid_` is false either way, and a non-OK `current_status_` already makes `Next()` a no-op.
2. `WalIteratorImpl::IsBatchExpected()` in the same file drops "Log iterator will reseek the correct batch." from its info log message, since it no longer does.
3. `org.rocksdb.WalIterator` now holds the implementation and the nested `BatchResult`, and `org.rocksdb.TransactionLogIterator` becomes a subclass of it. Existing Java code keeps compiling, `RocksDB.getUpdatesSince()` keeps its return type, and `TransactionLogIterator.BatchResult` still resolves through inheritance. Because the native methods are now declared on `WalIterator`, the JNI entry points in `java/rocksjni/wal_iterator.cc` (renamed from `transaction_log.cc`) become `Java_org_rocksdb_WalIterator_*`.
4. `BatchResultJni::getJClass()` in `java/rocksjni/portal.h` looks up `org/rocksdb/WalIterator$BatchResult` to match.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/15056
Test Plan:
Adds three tests to `db_log_iter_test.cc` that pin the semantics now being documented, none of which had coverage:
- `CaughtUpThenResumes` -- an iterator that has run out of records is `!Valid()` with an OK `status()`, and a later write to the same WAL is picked up by calling `Next()` again. This is the tailing pattern the old `Next()` comment prohibited.
- `GapFromDisableWAL` -- a `disableWAL` write between two normal writes ends the run with `NotFound` instead of skipping to the later sequence number, and the iterator stays spent across further `Next()` calls.
- `SilentlySkipsUnavailableStart` -- `GetUpdatesSince(1)` where sequence 1 never reached the WAL returns OK and positions at sequence 2, documenting the silent skip.
`IngestExternalFile()` is documented as the other source of holes but is not covered by a test; it needs `SstFileWriter` setup and is worth a follow-up.
The Java rename is verified by `WalIteratorTest`, which declares `WalIterator` while `RocksDB.getUpdatesSince()` still returns `TransactionLogIterator`, so the compatibility subclass is exercised on every use. It is otherwise an identifier-only rename of the previous test.
Renames the C++ test fixture to `DBWalIteratorTest`, drops the now-redundant `TransactionLogIterator` prefix from its test names, and updates the comments in `wal_manager.cc`, `sync_point.h` and `fs_posix.cc` that refer to those tests by name. Those comments already pointed at the wrong fixture and file (`DBTest` in `db_test.cc`) before this change.
Reviewed By: xingbowang
Differential Revision: D114809792
Pulled By: pdillinger
fbshipit-source-id: d3cfd23867b4738b62a096db1dbc895bada069b7