Summary:
Point-in-time MANIFEST replay -- best-efforts recovery (VersionSet::TryRecoverFromOneManifest) and secondary/follower catch-up (ManifestTailer) -- was quadratic in the number of MANIFEST records.
d458331ee (D61171320, "Move file tracking in VersionEditHandlerPointInTime to VersionBuilder") took a clean approach: it consolidated applying a VersionEdit and checking Version completeness into a single VersionBuilder implementation, so those checks were no longer duplicated between the handler and the builder. To let VersionEditHandlerPointInTime still build a Version reflecting the state before a valid->invalid edit, it added a save point -- but that save point proved inefficient: it was implemented by deep-copying the entire VersionBuilder::Rep before every edit (all of levels_[].added_files with a ref bump per file, table_file_levels_, found_files_, mutable_blob_file_metas_, ...). With N edits accumulating O(F) live files that is O(N*F) time and memory, and in the common healthy case the copy is pure waste since no Version is built during replay.
D110842287 (66bccb467, "Store BySmallestKey by value to avoid shared_ptr atomic copy in Rep") addressed a CPU-overhead symptom that had been misattributed to the shared_ptr atomic copies in that per-edit Rep copy. The real expense was copying Rep's containers -- notably table_file_levels_ -- on every edit. This change keeps d458331ee's clean single-implementation design but makes the save point cheap, removing the copy entirely:
- Replace the whole-Rep save point with a single-edit undo. Apply() records a compact, self-inverse undo of just the current edit (the pre-edit value of each touched key), and the PIT handler builds the pre-edit Version by rolling that one edit back, then redoing it to continue replay. This is O(edit) per edit, single-copy memory, and adds no extra file-verification I/O except on the rare valid->invalid negative edge. Also removes the Rep copy constructor.
- Store the undo registries in UnorderedMap (hash map) where ordering is not needed, which reduces the remaining per-edit CPU further. added_files and deleted_files keep an ordered map keyed by (level, file number), which trivial moves (same file number at two levels in one edit) require.
- Group the scalar/flag state mutated during Apply into a MutableScalars base so the undo can snapshot/restore it with a single copy/swap without duplicating the field list. Add util/cast_util.h up_cast<Base> for the explicit, compile-time-checked upcasts this uses instead of raw static_cast.
- Generalize D110842287: also store the stateless L0 comparators level_zero_cmp_by_epochno_ and level_zero_cmp_by_seqno_ by value rather than shared_ptr, dropping their make_shared allocation and atomic refcounting.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14929
Test Plan: Debug build (asserts + force_consistency_checks on). The single-edit rollback path is exercised by the existing recovery coverage, all passing: version_builder_test, version_set_test, db_secondary_test (ManifestTailer), db_follower_test (PartialVersionRecovery = the negative-edge rebuild), and the db_basic_test best-efforts suite -- including BestEffortsRecoverIncompleteVersion with blob-file variants (blob addition rolled back/redone) and BestEffortRecoveryFailureWithTableCacheUseAfterFree (FileMetaData/table-cache lifecycle) -- plus corruption_test.
Reviewed By: joshkang97
Differential Revision: D111753669
Pulled By: pdillinger
fbshipit-source-id: 7e6b5238f55cd906b1d4ff0044aeb0a02ec1cdcd
c9536aaef7