mirror of
https://github.com/facebook/rocksdb.git
synced 2026-10-23 08:09:02 +03:00
8585299552
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15285 Why the change is required: MyRocks bulk-load v2 creates a temporary RocksDB and calls DestroyDB when the load is committed, rolled back, abandoned, or recovered. Successfully ingested SSTs are moved into the main DB, so leftover SST cleanup is primarily an error, abort, or recovery case; WALs, obsolete files, and compaction-created trash can still remain. All physical deletion performed by DestroyDB should honor the configured SstFileManager rate, including rocksdb_sst_mgr_rate_bytes_per_sec in MyRocks. Previous behavior: DestroyDB sent recognized table, blob, and WAL files through SstFileManager, but ignored filenames ending in .trash because ParseFileName rejects that suffix. The final best-effort DeleteDir then silently left the directory behind. DB::Open handled the same files through CleanupDirectory, forcing MyRocks cleanup code to discover and register trash itself. A global WaitForEmptyTrash workaround would also drain unrelated compaction, DROP TABLE, and DROP INDEX deletion work from every DB sharing the manager. New behavior: DestroyDB waits only for pending deletion work associated with each DB, data, WAL, and archive directory. It then schedules any remaining .trash file through the existing rate-limited scheduler and waits only for the private bucket created for this DestroyDB call. Directory keys are normalized for bookkeeping, so equivalent path spellings identify the same pending work while filesystem operations retain the original path. Accounting and failure handling: A failed accounted trash deletion retains its original file path and remaining trash-byte charge. If DB open or DestroyDB later discovers the file, the retry preserves the accounted classification, avoids charging the trash bytes twice, and clears SstFileManager tracked-size and trash-size accounting after successful deletion. Scope and safety: There is no new sysvar, public RocksDB API, or persisted-format change. Existing rate settings, hard-link handling, partial deletion, and separate-WAL foreground deletion remain unchanged. The bookkeeping is confined to file enqueue/completion and DestroyDB cleanup. A shared FIFO can still delay this DB behind unrelated work already ahead of its files, but unrelated work queued behind this DB no longer gates DestroyDB. Reviewed By: xingbowang Differential Revision: D122221503 fbshipit-source-id: ec106f57efa70e24827234b4173366e5dbcbda5c