rust-rocksdb

A high-performance Rust wrapper for Facebook's RocksDB embeddable database.

RocksDB is a fast key-value storage engine based on LSM-trees, optimized for SSDs with excellent performance for both reads and writes. This crate provides safe, idiomatic Rust bindings with support for all major RocksDB features including transactions, column families, backups, and advanced compression.

📋 Table of Contents

• 🚀 Quick Start
• Usage Examples
• ⚙️ Features & Configuration
• 🔧 Building from Source
• 🤝 Contributing
• ❓ Why This Fork

🚀 Quick Start

Requirements:

• Clang and LLVM - Required for building RocksDB C++ components
• Rust 1.81.0+ - Current MSRV (rolling 6-month policy)

Add this to your Cargo.toml:

[dependencies]
rust-rocksdb = "0.43"

Basic Example

use rust_rocksdb::{DB, Options};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Open database
    let path = "./my_db";
    let mut opts = Options::default();
    opts.create_if_missing(true);
    let db = DB::open(&opts, path)?;

    // Write data
    db.put(b"key1", b"value1")?;
    
    // Read data
    match db.get(b"key1")? {
        Some(value) => println!("Retrieved: {}", String::from_utf8_lossy(&value)),
        None => println!("Key not found"),
    }

    // Delete data
    db.delete(b"key1")?;
    
    Ok(())
}

Usage Examples

Working with Iterators

use rust_rocksdb::{DB, Options, IteratorMode};

let db = DB::open(&Options::default(), path)?;

// Insert some data
db.put(b"key1", b"value1")?;
db.put(b"key2", b"value2")?;
db.put(b"key3", b"value3")?;

// Iterate over all keys
let iter = db.iterator(IteratorMode::Start);
for (key, value) in iter {
    println!("{}: {}", 
        String::from_utf8_lossy(&key), 
        String::from_utf8_lossy(&value)
    );
}

// Iterate from a specific key
let iter = db.iterator(IteratorMode::From(b"key2", rust_rocksdb::Direction::Forward));
for (key, value) in iter {
    println!("{}: {}", 
        String::from_utf8_lossy(&key), 
        String::from_utf8_lossy(&value)
    );
}

Using Column Families

use rust_rocksdb::{DB, Options, ColumnFamilyDescriptor};

let mut opts = Options::default();
opts.create_if_missing(true);

// Define column families
let cf_opts = Options::default();
let cf_descriptors = vec![
    ColumnFamilyDescriptor::new("users", cf_opts.clone()),
    ColumnFamilyDescriptor::new("posts", cf_opts),
];

let db = DB::open_cf_descriptors(&opts, path, cf_descriptors)?;

// Get column family handles
let users_cf = db.cf_handle("users").unwrap();
let posts_cf = db.cf_handle("posts").unwrap();

// Write to specific column families
db.put_cf(&users_cf, b"user:1", b"alice")?;
db.put_cf(&posts_cf, b"post:1", b"Hello World!")?;

// Read from specific column families
let user = db.get_cf(&users_cf, b"user:1")?;

Using Transactions

use rust_rocksdb::{TransactionDB, TransactionDBOptions, TransactionOptions, Options, WriteOptions};

let mut opts = Options::default();
opts.create_if_missing(true);

let txn_db_opts = TransactionDBOptions::default();
let db = TransactionDB::open(&opts, &txn_db_opts, path)?;

// Start a transaction
let txn_opts = TransactionOptions::default();
let txn = db.transaction_opt(&WriteOptions::default(), &txn_opts);

// Perform operations within transaction
txn.put(b"key1", b"value1")?;
txn.put(b"key2", b"value2")?;

// Commit the transaction
txn.commit()?;

⚙️ Features & Configuration

Compression Support

By default, support for Snappy, LZ4, Zstd, Zlib, and Bzip2 compression is enabled. To enable only specific algorithms:

[dependencies.rust-rocksdb]
default-features = false
features = ["lz4"]  # Enable only LZ4 compression

Available compression features:

• snappy - Google's Snappy compression (fast, moderate compression)
• lz4 - LZ4 compression (very fast, light compression)
• zstd - Zstandard compression (configurable speed/ratio tradeoff)
• zlib - Zlib compression (widely compatible)
• bzip2 - Bzip2 compression (high compression ratio, slower)

Performance Features

Jemalloc Memory Allocator

⚠️ Highly Recommended for Production

[dependencies.rust-rocksdb]
features = ["jemalloc"]

Enables jemalloc memory allocator which significantly reduces memory fragmentation compared to libc malloc, especially for RocksDB workloads.

Platform Support:

• Supported platforms (Linux, macOS): RocksDB will be properly informed that Jemalloc is enabled, allowing internal optimizations
• Unsupported platforms: See build.rs - You still get Jemalloc benefits but some RocksDB internal optimizations are skipped

See GitHub issue for more details on memory fragmentation with RocksDB.

Malloc Usable Size

[dependencies.rust-rocksdb]
features = ["malloc-usable-size"]

Required if you want to use RocksDB's optimize_filters_for_memory feature. See RocksDB documentation for details.

Platform-Specific Features

Multi-threaded Column Family Operations

[dependencies.rust-rocksdb]
features = ["multi-threaded-cf"]

Enables concurrent column family creation/deletion from multiple threads using RwLock. Alternatively, use DBWithThreadMode<MultiThreaded> directly.

Windows Runtime Library

[dependencies.rust-rocksdb]
features = ["mt_static"]

Windows Only: The mt_static feature requests the library to be built with the /MT flag, which results in the library using the static version of the run-time library.

Use case: This can be useful when there's a conflict in the dependency tree between different run-time versions.

Bindgen Linking

Dynamic Linking (Default):

[dependencies.rust-rocksdb]
features = ["bindgen-runtime"]  # Enabled by default

The bindgen-runtime feature enables the runtime feature of bindgen, which dynamically links to libclang. This is suitable for most platforms and is enabled by default.

Static Linking (Alpine Linux/musl):

[dependencies.rust-rocksdb]
default-features = false
features = ["bindgen-static", "snappy", "lz4", "zstd", "zlib", "bzip2"]

The bindgen-static feature enables the static feature of bindgen, which statically links to libclang. This is suitable for musllinux platforms, such as Alpine Linux.

⚠️ Important: The runtime and static features are mutually exclusive and won't compile if both are enabled.

Advanced Features

ZSTD Dictionary Optimization

[dependencies.rust-rocksdb]
features = ["zstd", "zstd-static-linking-only"]

Holds digested dictionaries in block cache for read-heavy workloads. Uses experimental APIs but is production-tested at Facebook. See Dictionary Compression Blog.

Link-Time Optimization (LTO)

[dependencies.rust-rocksdb]
features = ["lto"]

⚠️ CRITICAL REQUIREMENTS

• Must use clang: CC=/usr/bin/clang CXX=/usr/bin/clang++
• Clang LLVM version must match Rust compiler
• Rust flags: RUSTFLAGS="-Clinker-plugin-lto -Clinker=clang -Clink-arg=-fuse-ld=lld"

CC=/usr/bin/clang CXX=/usr/bin/clang++ \
RUSTFLAGS="-Clinker-plugin-lto -Clinker=clang -Clink-arg=-fuse-ld=lld" \
cargo build --release --features lto

See Rust LTO documentation for details.

🔧 Building from Source

Clone with submodules for RocksDB and compression libraries:

git clone --recursive https://github.com/zaidoon1/rust-rocksdb.git
cd rust-rocksdb

# Or if already cloned:
git submodule update --init --recursive

🤝 Contributing

Feedback and pull requests welcome! Open an issue for feature requests or submit PRs. This fork maintains regular updates with latest RocksDB releases and Rust versions.

Current MSRV: 1.81.0 (rolling 6-month policy)

❓ Why This Fork

This fork of the original rust-rocksdb focuses on:

• Regular updates with latest RocksDB releases
• Modern Rust support with up-to-date MSRV policy
• Active maintenance and quick issue resolution
• Performance optimizations and new feature integration

---

📚 Resources

• API Documentation - Complete API reference
• RocksDB Wiki - Upstream documentation
• RocksDB Tuning Guide - Performance tuning