forked from continuwuation/rocksdb
94e65a2e0b
Summary: Add a new CF immutable option `paranoid_memory_check_key_checksum_on_seek` that allows additional data integrity validations during seek on SkipList Memtable. When this option is enabled and memtable_protection_bytes_per_key is non zero, skiplist-based memtable will validate the checksum of each key visited during seek operation. The option is opt-in due to performance overhead. This is an enhancement on top of paranoid_memory_checks option. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13902 Test Plan: * new unit test added for paranoid_memory_check_key_checksum_on_seek=true. * existing unit test for paranoid_memory_check_key_checksum_on_seek=false. * enable in stress test. Performance Benchmark: we check for performance regression in read path where data is in memtable only. For each benchmark, the script was run at the same time for main and this PR: ### Memtable-only randomread ops/sec: * Value size = 100 Bytes ``` for B in 0 1 2 4 8; do (for I in $(seq 1 50);do ./db_bench --benchmarks=fillseq,readrandom --write_buffer_size=268435456 --writes=250000 --value_size=100 --num=250000 --reads=500000 --seed=1723056275 --paranoid_memory_check_key_checksum_on_seek=true --memtable_protection_bytes_per_key=$B 2>&1 | grep "readrandom"; done;) | awk '{ t += $5; c++; print } END { print 1.0 * t / c }'; done; ``` 1. Main: 928999 2. PR with paranoid_memory_check_key_checksum_on_seek=false: 930993 (+0.2%) 3. PR with paranoid_memory_check_key_checksum_on_seek=true: 3.1 memtable_protection_bytes_per_key=1: 464577 (-50%) 3.2 memtable_protection_bytes_per_key=2: 470319 (-49%) 3.3 memtable_protection_bytes_per_key=4: 468457 (-50%) 3.4 memtable_protection_bytes_per_key=8: 465061 (-50%) * Value size = 1000 Bytes ``` for B in 0 1 2 4 8; do (for I in $(seq 1 50);do ./db_bench --benchmarks=fillseq,readrandom --write_buffer_size=268435456 --writes=250000 --value_size=1000 --num=250000 --reads=500000 --seed=1723056275 --paranoid_memory_check_key_checksum_on_seek=true --memtable_protection_bytes_per_key=$B 2>&1 | grep "readrandom"; done;) | awk '{ t += $5; c++; print } END { print 1.0 * t / c }'; done; ``` 1. Main: 601321 2. PR with paranoid_memory_check_key_checksum_on_seek=false: 607885 (+1.1%) 3. PR with paranoid_memory_check_key_checksum_on_seek=true: 3.1 memtable_protection_bytes_per_key=1: 185742 (-69%) 3.2 memtable_protection_bytes_per_key=2: 177167 (-71%) 3.3 memtable_protection_bytes_per_key=4: 185908 (-69%) 3.4 memtable_protection_bytes_per_key=8: 183639 (-69%) Reviewed By: pdillinger Differential Revision: D81199245 Pulled By: xingbowang fbshipit-source-id: e3c29552ab92f2c5f360361366a293fa26934913 |
||
|---|---|---|
| .. | ||
| alloc_tracker.cc | ||
| hash_linklist_rep.cc | ||
| hash_skiplist_rep.cc | ||
| inlineskiplist.h | ||
| inlineskiplist_test.cc | ||
| memtablerep_bench.cc | ||
| skiplist.h | ||
| skiplist_test.cc | ||
| skiplistrep.cc | ||
| stl_wrappers.h | ||
| vectorrep.cc | ||
| wbwi_memtable.cc | ||
| wbwi_memtable.h | ||
| write_buffer_manager.cc | ||
| write_buffer_manager_test.cc | ||