perf(build): Option to use precompiled RocksDB. Isolate spurious debug crates from release #1509

Closed
gamesguru wants to merge 55 commits from gamesguru/continuwuity:guru/perf/build-optimizations-dep-tree into main

2026-03-15

run xtask
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-15 04:25:28 -04:00
Apply suggestions from code review
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-15 00:41:04 -04:00
Update justfile
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-03-15 00:38:34 -04:00

2026-03-14

tidy/arrange justfile fix rockdb glitch
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-14 14:59:06 -04:00

2026-03-11

prune makefile; fix my stupid slowdowns/regressions, but not quite as good as 9e76153a nor even a6d377f8 yet again.
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 21:10:39 -04:00
git: 3-way patch from upstream, see:
from this branch (origin is Jade's, github is mine), I ran:

```shell
FILES=$(gds origin/main --name-only)
BASE=$(git merge-base HEAD github/main)
git diff $BASE..github/main  -- $FILES | git apply --3way
```
2026-03-11 19:22:20 -04:00
prune makefile from this branch
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 14:34:45 -04:00
update justfile targets (fix). format files/whitespace.
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 13:21:12 -04:00
add some build/compile profiling targets to justfile
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 08:31:02 -04:00
gitignore the .tmp/ in case not everyone echoed a gitignore in it with '**'
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 02:41:48 -04:00
TODO: investigate dep. inversion w/ ruma, since it rebuilds on trivial/unrelated changes it shouldn't. [empty-commit]
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 02:26:09 -04:00
update "debugging over SSH" docs to use rust-gdb
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 01:59:45 -04:00
cargo configs so RustRover uses prebuilt libraries
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 01:53:47 -04:00
justfile targets to prebuild jemalloc, lz4, rocksdb, and zstd. down to 553 crates.
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 01:30:59 -04:00
add a proof of concept justfile for profiling cpu/mem usage and heat maps
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 00:48:49 -04:00
add docs on debugging over SSH with gdb or RustRover
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 00:36:04 -04:00
help RustRover see. You may need below commands. And i may nest things eventually in /usr/loca/uwu for isolation.
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
This will tell your linker to look in `/usr/local`, but it may also cause conflicts or unexpected behavior:

```bash
echo "/usr/local/lib" | sudo tee /etc/ld.so.conf.d/usr-local.conf
sudo ldconfig
```
2026-03-11 00:17:07 -04:00
RustRover default to prebuilt RocksDB if found in/usr/local
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-11 00:02:08 -04:00

2026-03-10

add redundant config since RustRover has the IQ of a carrot. TODO: prevent this breaking if sccache or mold aren't found
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-10 23:47:51 -04:00
direnv auto-detect sccache, faster builds from CLI
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-10 23:37:30 -04:00
add cargo config to use mold, faster linker than ld or cc
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
this should also apply to RustRover by default, and you will notice less redundant rebuilding of built static imports.
2026-03-10 04:56:15 -04:00
leaner runtime 560 crates. take tracers out of default features. give the people choice
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
42 megabytes. 8 minutes. release-max-perf.

we need to improve our documentation, so admins understand how to get non-default features.

i suspect nothing additional is needed to run the full Complement Test suite and serve all basic homeserver functions.

not even URL preview zero overhead: extreme cases may leak IPs or allow forgery attacks.
there's no harm in recompiling once you find you want it.

testing: add WIP engage target for complement tests (not everyone has buildkit or buildx)
2026-03-10 03:03:17 -04:00
document tech debt: wasteful presence DB i/o and a need for LRU caching
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-03-10 01:42:32 -04:00
build: prelim support for bindgen, bundling the RocksDB assembly into standalone Rust binaries
revised commands to build rocksdb static assembly:

```shell
DISABLE_JEMALLOC=1 EXTRA_CXXFLAGS="-Wno-error=unused-parameter" make shared_lib static_lib -j$(nproc)
sudo make install-shared INSTALL_PATH=/usr/local
sudo make install-static INSTALL_PATH=/usr/local
sudo ldconfig
```
2026-03-10 00:33:15 -04:00

2026-03-09

update env vars defs for building. see below note on DISABLE_JEMALLOC=1,
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
```shell
git clone https://forgejo.ellis.link/continuwuation/rust-rocksdb-zaidoon1.git
cd rust-rocksdb-zaidoon1
git checkout 61d9d23872197e9ace4a477f2617d5c9f50ecb23
git submodule update --init --recursive
cd librocksdb-sys/rocksdb
PORTABLE=0 DISABLE_JEMALLOC=1 CXXFLAGS="-Wno-error=unused-parameter" make shared_lib -j$(nproc)
sudo make install-shared INSTALL_PATH=/usr/local
sudo ldconfig
```
2026-03-09 05:50:13 -04:00

2026-03-08

build(docs): old compilers may need: CXXFLAGS="-Wno-error=unused-parameter"
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
2026-03-08 21:11:44 -04:00
build(perf): document benefits of pre-compiling rocksdb:
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
git clone https://forgejo.ellis.link/continuwuation/rust-rocksdb-zaidoon1.git
    cd rust-rocksdb-zaidoon1
    git checkout 61d9d23872197e9ace4a477f2617d5c9f50ecb23
    git submodule update --init --recursive
    cd librocksdb-sys/rocksdb
    make shared_lib -j$(nproc)
    sudo make install-shared INSTALL_PATH=/usr/local
    sudo ldconfig
2026-03-08 06:26:00 -04:00