continuwuation/rocksdb
12
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity
rocksdb/tools
History
Exact
Anand Ananthabhotla b21eaa91c2 Fix memory accounting leak in IODispatcher ReadIndex() (#14569) 
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14569

ReadSet::ReadIndex() moves block values out of pinned_blocks_ via std::move,
but never releases the associated prefetch memory accounting. This causes
ReleaseBlock() and the destructor to skip ReleaseMemory() since they check
pinned_blocks_.GetValue() which returns null after the move. Over time, the
memory budget is exhausted and no further prefetches can be dispatched when
max_prefetch_memory_bytes is set. The bug was introduced in https://github.com/facebook/rocksdb/pull/14401.

The fix releases memory accounting in ReadIndex() when moving values out
(both for Case 1: block already available, and Case 2: after async IO
polling), and zeros block_sizes_ to prevent double-release.

Also adds multiscan_max_prefetch_memory_bytes option to db_stress/crashtest
for stress testing this code path.

Reviewed By: hx235

Differential Revision: D99488961

fbshipit-source-id: 5ddd1f50e2f6ebb357f86e013d781a790e7e558a
2026-04-09 22:43:39 -07:00
..
advisor Deprecate random access max buffer size references - take #2 (#13288) 2025-01-10 15:32:38 -08:00
block_cache_analyzer Remove obsolete compression code and some .h->.cc movement (#14325) 2026-02-13 11:18:05 -08:00
dump Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
analyze_txn_stress_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
auto_sanity_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
backup_db.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
benchmark.sh optimize file size statistics in benchmark script (#12363) 2024-02-21 15:45:18 -08:00
benchmark_ci.py internal_repo_rocksdb 2024-10-14 03:01:20 -07:00
benchmark_compare.sh Fix file modes (#10815) 2022-10-13 09:00:37 -07:00
benchmark_leveldb.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
blob_dump.cc Restore blob_dump_tool compression support (partially revert PR #14266) (#14382) 2026-02-24 14:10:13 -08:00
check_all_python.py internal_repo_rocksdb 2024-10-14 03:01:20 -07:00
check_format_compatible.sh Support automated interpolation search (#14383) 2026-03-06 10:13:51 -08:00
CMakeLists.txt Mark dependencies as PRIVATE and fix missing dependencies in tools. (#6790) 2020-05-12 21:07:55 -07:00
compact_db.sh Fix format compatibility issues, extend test (#14323) 2026-02-13 09:18:40 -08:00
db_bench.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
db_bench_tool.cc Add verify_manifest_content_on_close option (#14451) 2026-03-19 12:01:23 -07:00
db_bench_tool_test.cc Delete max_write_buffer_number_to_maintain (#13491) 2025-04-07 21:44:36 -07:00
db_crashtest.py Fix memory accounting leak in IODispatcher ReadIndex() (#14569) 2026-04-09 22:43:39 -07:00
db_repl_stress.cc Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
db_sanity_test.cc Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
dbench_monitor Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
Dockerfile adding docker build script and dockerfile 2015-05-22 16:03:39 -07:00
generate_random_db.sh Fix format compatibility issues, extend test (#14323) 2026-02-13 09:18:40 -08:00
ingest_external_sst.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
io_tracer_parser.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
io_tracer_parser_test.cc Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
io_tracer_parser_tool.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
io_tracer_parser_tool.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
ldb.cc Use/endorse (Auto)HyperClockCache by default over LRUCache (#13964) 2025-09-18 13:27:51 -07:00
ldb_cmd.cc Support automated interpolation search (#14383) 2026-03-06 10:13:51 -08:00
ldb_cmd_impl.h Restore blob_dump_tool compression support (partially revert PR #14266) (#14382) 2026-02-24 14:10:13 -08:00
ldb_cmd_test.cc Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
ldb_test.py Restore blob_dump_tool compression support (partially revert PR #14266) (#14382) 2026-02-24 14:10:13 -08:00
ldb_tool.cc Support automated interpolation search (#14383) 2026-03-06 10:13:51 -08:00
pflag Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
reduce_levels_test.cc Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
regression_test.sh Fix regression test script for internal use (#14079) 2025-10-27 14:22:47 -07:00
restore_db.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
rocksdb_dump_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
run_blob_bench.sh add exe and script path check (#11621) 2023-07-19 12:05:24 -07:00
run_clang_tidy.py Run clang-tidy in github CI (#14347) 2026-02-20 14:58:21 -08:00
run_flash_bench.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
run_leveldb.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
sample-dump.dmp First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
simulated_hybrid_file_system.cc Standardize on clang-format version 18 (#13233) 2024-12-19 10:58:40 -08:00
simulated_hybrid_file_system.h Standardize on clang-format version 18 (#13233) 2024-12-19 10:58:40 -08:00
sst_dump.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
sst_dump_test.cc Allow standalone file and directory arguments to sst_dump (#13978) 2025-09-19 16:01:43 -07:00
sst_dump_tool.cc DROP support for block-based SST format_version < 2 (#14315) 2026-02-11 14:43:41 -08:00
tool_hooks.cc Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
trace_analyzer.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
trace_analyzer_test.cc Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
trace_analyzer_tool.cc Standardize on clang-format version 18 (#13233) 2024-12-19 10:58:40 -08:00
trace_analyzer_tool.h Standardize on clang-format version 18 (#13233) 2024-12-19 10:58:40 -08:00
verify_random_db.sh Fix ldb dump swallowing errors, but ldb scan for compatibility test (#14422) 2026-03-05 09:41:58 -08:00
write_external_sst.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
write_stress.cc Remove deprecated DB::Open raw pointer variants (and more) (#14335) 2026-02-17 23:33:39 -08:00
write_stress_runner.py internal_repo_rocksdb 2024-10-14 03:01:20 -07:00