rocksdb/table/block_based/block_based_table_factory.h
Peter Dillinger d8b1893c9d DROP support for block-based SST format_version < 2 (#14315)
Summary:
... and remove some old code and tech debt in the process.

This is arguably a great milestone and precendent in RocksDB history as for the first time we are explicitly dropping support for the ability to read source-of-truth data in old formats. (We previously dropped support for reading some old bloom filters, but those are performance optimizers not source-of-truth. https://github.com/facebook/rocksdb/issues/10184) However, DBs written with default settings since release 4.6.0, which is very nearly 10 years ago, can still be read. And by using compaction with intermediate versions, there's an upgrade path going back to (AFAIK) early releases of LevelDB (from which RocksDB was forked).

Some detail:
* The magic number for LevelDB SST files (0xdb4775248b80fb57, most recently called kLegacyBlockBasedTableMagicNumber) now only exists in the code to provide a good error message and to test that good error message.
* There is some notable refactoring and renaming around format_version handling. This is a bit of a messy area of code because the footer code being shared between different table formats (block-based, plain, cuckoo) means format_version in the footer is in ways tied to all of them, but in other ways is just tied to block-based table where we have been making updates. Hopefully code comments keep this clear.
* Now that there are old format_versions we can't read (and can't write authoritatively in tests), I've needed to split out kMinSupportedFormatVersion into a constant for reads and for writes, currently the same at format_version=2. Comments describe how to update these in the future.
* The idea of versioning the compression format is basically going away, though we're keeping BuiltinV2 in places just because it's already there. There's lots of room in the BuiltinV2 schema to expand to new built-in compression types, or new ways of handling existing compression algorithms. CompressionManager with CompatibilityName gives users the power to customize compression without the need for versions tied to format_version.

Immediate follow-up:
* Clean up compression loose ends like OLD_Compress, OLD_Uncompress

Suggested follow-up:
* Update plain table builder to migrate to new footer version so that we can drop support for legacy footer. We have to be careful that the (likely untested) forward compatibility path I put in place a while back works (or fix it and wait a while) before dropping support for plain table with legacy footer.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/14315

Test Plan:
* Some tests updated / added
* A couple tests are obsolete: removed
* Also updated format compatible test, which now doesn't need to dig as far back into history building RocksDB.

Reviewed By: hx235

Differential Revision: D92577766

Pulled By: pdillinger

fbshipit-source-id: a23be846189d901ce087af4ca9a99cef18445cb7
2026-02-11 14:43:41 -08:00

112 lines
3.6 KiB
C++

// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file. See the AUTHORS file for names of contributors.
#pragma once
#include <stdint.h>
#include <memory>
#include <string>
#include "cache/cache_reservation_manager.h"
#include "port/port.h"
#include "rocksdb/flush_block_policy.h"
#include "rocksdb/table.h"
namespace ROCKSDB_NAMESPACE {
struct ColumnFamilyOptions;
struct ConfigOptions;
struct DBOptions;
struct EnvOptions;
class BlockBasedTableBuilder;
class RandomAccessFileReader;
class WritableFileWriter;
// TODO: deprecate this class as it can be replaced with
// `FileMetaData::tail_size`
//
// A class used to track actual bytes written from the tail in the recent SST
// file opens, and provide a suggestion for following open.
class TailPrefetchStats {
public:
void RecordEffectiveSize(size_t len);
// 0 indicates no information to determine.
size_t GetSuggestedPrefetchSize();
private:
const static size_t kNumTracked = 32;
size_t records_[kNumTracked];
port::Mutex mutex_;
size_t next_ = 0;
size_t num_records_ = 0;
};
class BlockBasedTableFactory : public TableFactory {
public:
explicit BlockBasedTableFactory(
const BlockBasedTableOptions& table_options = BlockBasedTableOptions());
~BlockBasedTableFactory() {}
// Method to allow CheckedCast to work for this class
static const char* kClassName() { return kBlockBasedTableName(); }
const char* Name() const override { return kBlockBasedTableName(); }
using TableFactory::NewTableReader;
Status NewTableReader(
const ReadOptions& ro, const TableReaderOptions& table_reader_options,
std::unique_ptr<RandomAccessFileReader>&& file, uint64_t file_size,
std::unique_ptr<TableReader>* table_reader,
bool prefetch_index_and_filter_in_cache = true) const override;
TableBuilder* NewTableBuilder(
const TableBuilderOptions& table_builder_options,
WritableFileWriter* file) const override;
// Valdates the specified DB Options.
Status ValidateOptions(const DBOptions& db_opts,
const ColumnFamilyOptions& cf_opts) const override;
Status PrepareOptions(const ConfigOptions& opts) override;
std::string GetPrintableOptions() const override;
bool IsDeleteRangeSupported() const override { return true; }
std::unique_ptr<TableFactory> Clone() const override {
return std::make_unique<BlockBasedTableFactory>(*this);
}
TailPrefetchStats* tail_prefetch_stats() {
return &shared_state_->tail_prefetch_stats;
}
protected:
const void* GetOptionsPtr(const std::string& name) const override;
Status ParseOption(const ConfigOptions& config_options,
const OptionTypeInfo& opt_info,
const std::string& opt_name, const std::string& opt_value,
void* opt_ptr) override;
void InitializeOptions();
private:
BlockBasedTableOptions table_options_;
// Share some state among cloned instances
struct SharedState {
std::shared_ptr<CacheReservationManager> table_reader_cache_res_mgr;
TailPrefetchStats tail_prefetch_stats;
};
std::shared_ptr<SharedState> shared_state_;
};
extern const std::string kHashIndexPrefixesBlock;
extern const std::string kHashIndexPrefixesMetadataBlock;
extern const std::string kPropTrue;
extern const std::string kPropFalse;
} // namespace ROCKSDB_NAMESPACE