7ecc12110c
Summary: See https://github.com/facebook/rocksdb/issues/14240 which brought this to my attention. Here I've added range deletions and compactions to the format compatible test, and fixed or worked-around compatibility issues (likely longstanding). The first fix was in Version::MaybeInitializeFileMetaData for an assertion failure simply from adding range deletions from some 5.x version. The second fix is a broader work-around for older SST files with unreliable num_entries/num_range_deletions/num_deletions statistics in their table properties. We depend on them only for some paranoid checks for compaction, so in my assessment the best way to deal with those files is to exclude the paranoid checks when dealing with the files with unrelaible data. (Details in code comments.) The important part is that compacting old files is exceptionally rare, so we aren't really interefering with the paranoid checks doing thier job on an ongoing basis. This depends on https://github.com/facebook/rocksdb/issues/14315 (just landed) because there is a remaining undiagnosed problem with some very early releases, but I'm not fixing that because its support is being dropped. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14323 Test Plan: test extended (ran locally excluding some releases) Reviewed By: xingbowang Differential Revision: D93032653 Pulled By: pdillinger fbshipit-source-id: f90b32f30ba4764692e68d23705f42c778e0dc1d
31 lines
859 B
Bash
Executable file
31 lines
859 B
Bash
Executable file
#!/usr/bin/env bash
|
|
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
|
|
#
|
|
# A shell script to compact DB generated by generate_random_db.sh.
|
|
# ./ldb needs to be available to be executed.
|
|
#
|
|
# Usage: <SCRIPT> <DB Path> [if_try_load_options] [if_ignore_unknown_options]
|
|
|
|
if [ "$#" -lt 1 ]; then
|
|
echo "usage: $BASH_SOURCE <db_directory> [if_try_load_options] [if_ignore_unknown_options]"
|
|
exit 1
|
|
fi
|
|
|
|
db_dir=$1
|
|
try_load_options=${2:-"1"}
|
|
ignore_unknown_options=${3:-"0"}
|
|
extra_params=
|
|
|
|
if [ "$try_load_options" = "0" ]; then
|
|
extra_params=" --try_load_options=false"
|
|
elif [ "$try_load_options" = "1" ]; then
|
|
extra_params=" --try_load_options=true"
|
|
fi
|
|
|
|
if [ "$ignore_unknown_options" = "1" ]; then
|
|
extra_params="$extra_params --ignore_unknown_options"
|
|
fi
|
|
|
|
set -e
|
|
echo == Compacting DB at $db_dir
|
|
./ldb compact --db=$db_dir $extra_params
|