Jay Huh
3aa706c2bf
Enforce WriteBufferManager during WAL recovery ( #14305 )
...
Summary:
Add a new immutable DB option `enforce_write_buffer_manager_during_recovery` to control whether WriteBufferManager buffer_size is enforced during WAL recovery. When multiple RocksDB instances share a WriteBufferManager, a recovering instance could exceed the global memory limit by replaying large amounts of WAL data into memtables. This can cause OOM, especially when other instances are actively using the shared memory budget. When this option is enabled (and a WriteBufferManager is configured), RocksDB will check `WriteBufferManager::ShouldFlush()` after each batch insertion during WAL recovery and schedule flushes when needed to keep memory bounded.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14305
Test Plan:
Two unit tests added - `WriteBufferManagerLimitDuringWALRecoverySingleDB` and `WriteBufferManagerLimitDuringWALRecoveryMultipleDBs`
```
./db_write_buffer_manager_test --gtest_filter="*WriteBufferManagerLimitDuringWALRecovery*"
```
Basic stress test added with options toggled
## To follow up
Multiple DB scenario in stress test
Reviewed By: hx235
Differential Revision: D92533792
Pulled By: jaykorean
fbshipit-source-id: 35ca70e2300a8bfd6b81a6646a65ef284fb90a9a
2026-03-05 15:56:46 -08:00
..
2026-02-21 06:19:45 -08:00
2026-03-03 21:40:36 -08:00
2026-03-05 15:56:46 -08:00
2026-02-21 06:19:45 -08:00
2025-05-05 17:42:57 -07:00
2025-08-08 10:33:36 -07:00
2024-04-16 08:45:38 -07:00
2024-11-04 18:06:07 -08:00
2026-03-05 14:18:08 -08:00
2026-01-20 14:10:41 -08:00
2026-02-15 10:04:58 -08:00
2026-03-02 16:18:14 -08:00
2024-04-16 08:45:38 -07:00
2024-11-18 11:23:17 -08:00
2026-02-17 23:33:39 -08:00
2026-03-03 21:40:36 -08:00
2025-08-05 23:19:09 -07:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2026-02-17 13:05:44 -08:00
2026-02-17 13:05:44 -08:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2026-02-19 16:45:51 -08:00
2023-05-18 13:25:01 -07:00
2026-01-30 05:53:04 -08:00
2024-03-22 13:40:42 -07:00
2026-03-02 16:18:14 -08:00
2024-12-19 10:58:40 -08:00
2025-07-09 10:40:28 -07:00
2025-11-07 09:04:52 -08:00
2025-10-24 13:11:26 -07:00
2026-03-05 14:18:08 -08:00
2025-10-03 14:28:37 -07:00
2024-02-05 10:26:41 -08:00
2022-06-27 16:37:09 -07:00
2025-06-04 10:44:17 -07:00
2025-05-05 17:42:57 -07:00
2025-05-05 17:42:57 -07:00
2026-02-02 13:18:02 -08:00
2025-10-13 12:48:04 -07:00
2026-02-19 16:45:51 -08:00
2025-04-11 10:08:29 -07:00
2026-02-19 20:26:38 -08:00
2026-02-17 23:33:39 -08:00
2026-02-19 16:45:51 -08:00
2025-02-18 11:01:19 -08:00
2026-02-17 23:33:39 -08:00
2026-01-08 15:15:31 -08:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2025-10-09 14:31:47 -07:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2026-03-02 16:18:14 -08:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2024-12-19 10:58:40 -08:00
2026-03-04 13:26:19 -08:00
2026-03-02 16:18:14 -08:00
2026-03-02 16:18:14 -08:00
2026-03-04 13:48:48 -08:00
2025-08-05 23:19:09 -07:00
2026-03-02 16:18:14 -08:00
2026-02-17 23:33:39 -08:00
2026-01-20 14:10:41 -08:00
2022-05-19 18:39:41 -07:00
2022-05-19 18:39:41 -07:00
2026-03-05 15:56:46 -08:00
2026-01-05 20:47:46 -08:00
2025-09-25 17:35:23 -07:00
2024-11-22 17:53:35 -08:00
2026-02-19 16:45:51 -08:00
2025-01-24 22:28:41 -08:00
2026-02-17 23:33:39 -08:00
2026-03-02 16:18:14 -08:00
2024-12-19 16:57:51 -08:00
2025-09-17 20:20:33 -07:00
2023-01-27 13:14:19 -08:00
2026-02-13 11:56:22 -08:00
2026-02-17 23:33:39 -08:00
2026-02-17 13:05:44 -08:00
2025-08-22 16:05:56 -07:00
2026-02-19 22:10:47 -08:00
2026-02-17 23:33:39 -08:00
2022-11-02 14:34:24 -07:00
2023-07-03 09:41:48 -07:00
2022-11-02 14:34:24 -07:00
2022-11-02 14:34:24 -07:00
2025-11-07 09:04:52 -08:00
2026-01-20 14:10:41 -08:00
2025-05-22 09:42:15 -07:00
2026-02-17 23:33:39 -08:00
2025-01-15 13:11:40 -08:00
2025-05-05 17:42:57 -07:00
2023-02-22 12:28:18 -08:00
2026-02-17 13:05:44 -08:00
2023-06-15 12:25:04 -07:00
2026-02-17 23:33:39 -08:00
2026-01-30 05:53:04 -08:00
2026-01-30 05:53:04 -08:00
2025-05-22 09:42:15 -07:00
2023-04-25 12:08:23 -07:00
2026-02-19 20:26:38 -08:00
2024-12-26 13:20:35 -08:00
2025-08-13 12:02:12 -07:00
2025-10-03 14:28:37 -07:00
2025-02-13 21:49:51 -08:00
2024-12-26 13:20:35 -08:00
2025-10-03 14:28:37 -07:00
2022-11-02 14:34:24 -07:00
2022-11-02 14:34:24 -07:00
2024-02-07 10:44:11 -08:00
2024-12-19 10:58:40 -08:00
2024-12-06 13:29:52 -08:00
2024-12-06 13:29:52 -08:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2025-04-17 13:19:52 -07:00
2025-04-17 13:19:52 -07:00
2025-09-18 16:15:50 -07:00
2026-03-05 15:56:46 -08:00
2025-02-10 17:15:47 -08:00
2022-10-31 22:28:58 -07:00
2025-07-30 12:00:54 -07:00
2025-10-15 13:43:53 -07:00
2025-09-08 13:11:28 -07:00
2026-02-17 23:33:39 -08:00
2024-11-04 18:06:07 -08:00
2024-10-20 20:53:08 -07:00
2025-10-13 12:48:04 -07:00
2026-02-17 23:33:39 -08:00
2026-02-17 23:33:39 -08:00
2024-01-31 16:30:26 -08:00
2024-01-31 16:30:26 -08:00
2026-02-17 23:33:39 -08:00
2026-02-28 13:43:55 -08:00
2026-02-28 13:43:55 -08:00
2025-07-01 11:07:51 -07:00
2024-02-07 10:44:11 -08:00
2026-02-17 23:33:39 -08:00
2022-06-10 16:07:03 -07:00
2026-02-19 16:45:51 -08:00
2022-10-25 11:33:04 -07:00
2025-12-04 12:28:01 -08:00
2025-12-04 12:28:01 -08:00
2024-03-22 13:40:42 -07:00
2022-12-19 15:06:22 -08:00
2024-01-29 11:37:34 -08:00
2024-10-14 12:28:35 -07:00
2024-12-19 10:58:40 -08:00
2026-01-20 14:10:41 -08:00
2026-02-17 23:33:39 -08:00
2025-03-04 17:44:01 -08:00
2025-03-04 17:44:01 -08:00
2024-10-25 09:32:14 -07:00
2022-11-02 14:34:24 -07:00
2022-11-02 14:34:24 -07:00
2026-03-02 16:18:14 -08:00
2026-03-02 16:18:14 -08:00
2026-03-02 16:18:14 -08:00
2024-11-01 10:08:35 -07:00
2022-11-02 14:34:24 -07:00
2024-06-18 10:51:29 -07:00
2024-12-19 10:58:40 -08:00
2025-02-13 21:49:51 -08:00
2022-11-02 14:34:24 -07:00
2026-01-20 14:10:41 -08:00
2026-03-02 16:18:14 -08:00
2026-03-02 16:18:14 -08:00
2025-08-05 10:48:49 -07:00
2026-03-02 16:18:14 -08:00
2025-10-01 14:21:55 -07:00
2026-03-02 16:18:14 -08:00
2026-03-02 16:18:14 -08:00
2024-10-17 14:13:20 -07:00
2026-01-20 14:10:41 -08:00
2026-03-02 16:18:14 -08:00
2026-02-17 14:52:00 -08:00
2026-03-02 16:18:14 -08:00
2026-03-02 16:18:14 -08:00
2022-07-21 13:35:36 -07:00
2022-07-21 13:35:36 -07:00
2022-07-21 13:35:36 -07:00
2025-11-07 09:04:52 -08:00
2025-10-24 17:10:48 -07:00
2024-12-19 10:58:40 -08:00
2025-06-18 17:32:59 -07:00
2025-02-10 17:15:47 -08:00
2025-06-18 17:32:59 -07:00
2026-02-17 23:33:39 -08:00
2022-11-02 14:34:24 -07:00
2022-11-02 14:34:24 -07:00
2023-04-05 14:42:31 -07:00
2024-01-29 10:38:08 -08:00
2024-12-05 15:00:17 -08:00
2025-10-03 14:28:37 -07:00