perf(build): Option to use precompiled RocksDB. Isolate spurious debug crates from release #1509

Open
gamesguru wants to merge 48 commits from gamesguru/continuwuity:guru/perf/build-optimizations-dep-tree into main
Contributor

This PR places heavy debug libraries like tracing-flame BEHIND debug feature flags.

I have also documented the benefits of pre-compiling RocksDB (see below), which requires exposing the ROCKSDB_LIB_DIR env var.

Net result is a beefed up build process, or perhaps leaned down?
It won't cure the deadlocks in our runtime, but at least now we can compile in under 3 minutes and or on 512 MB of ram (realistically probably not both at once!).

This doesn't really help docker-in-docker, since you'd need to compile RustDB every CI run. But I think we may be able to find a docker image which supports the same major and minor version of the RocksDB headers, which would replace building a million lines of code with downloading 150 MB of binaries over enterprise-grade Ethernet.

git clone https://forgejo.ellis.link/continuwuation/rust-rocksdb-zaidoon1.git
cd rust-rocksdb-zaidoon1
git checkout 61d9d23872197e9ace4a477f2617d5c9f50ecb23
git submodule update --init --recursive
cd librocksdb-sys/rocksdb
make shared_lib -j$(nproc)
sudo make install-shared INSTALL_PATH=/usr/local
sudo ldconfig
# you must do this where your tools (i.e., make, engage, or just) will see it
export ROCKSDB_LIB_DIR=/usr/local/lib

Pull request checklist:

  • This pull request targets the main branch, and the branch is named something other than
    main.
  • I have written an appropriate pull request title and my description is clear.
  • I understand I am responsible for the contents of this pull request.
  • I have followed the contributing guidelines:
This PR places heavy debug libraries like `tracing-flame` BEHIND debug feature flags. I have also documented the benefits of **pre-compiling RocksDB** (see below), which requires exposing the `ROCKSDB_LIB_DIR` env var. Net result is a beefed up build process, or perhaps leaned down? It won't cure the deadlocks in our runtime, but at least now we can compile in under 3 minutes ~~and~~ or on 512 MB of ram (realistically probably not both at once!). This doesn't really help docker-in-docker, since you'd need to compile RustDB every CI run. But I think we may be able to find a docker image which supports the same major and minor version of the RocksDB headers, which would replace building a million lines of code with downloading 150 MB of binaries over enterprise-grade Ethernet. ```shell git clone https://forgejo.ellis.link/continuwuation/rust-rocksdb-zaidoon1.git cd rust-rocksdb-zaidoon1 git checkout 61d9d23872197e9ace4a477f2617d5c9f50ecb23 git submodule update --init --recursive cd librocksdb-sys/rocksdb make shared_lib -j$(nproc) sudo make install-shared INSTALL_PATH=/usr/local sudo ldconfig # you must do this where your tools (i.e., make, engage, or just) will see it export ROCKSDB_LIB_DIR=/usr/local/lib ``` <!-- Example: This pull request allows us to warp through time and space ten times faster than before by double-inverting the warp drive with hyperheated jump fluid, both making the drive faster and more efficient. This resolves the common issue where we have to wait more than 10 milliseconds to engage, use, and disengage the warp drive when travelling between galaxies. --> <!-- Closes: #... --> <!-- Fixes: #... --> <!-- Uncomment the above line(s) if your pull request fixes an issue or closes another pull request by superseding it. Replace `#...` with the issue/pr number, such as `#123`. --> **Pull request checklist:** <!-- You need to complete these before your PR can be considered. If you aren't sure about some, feel free to ask for clarification in #dev:continuwuity.org. --> - [x] This pull request targets the `main` branch, and the branch is named something other than `main`. - [x] I have written an appropriate pull request title and my description is clear. - [x] I understand I am responsible for the contents of this pull request. - I have followed the [contributing guidelines][c1]: - [x] My contribution follows the [code style][c2], if applicable. - [x] I ran [pre-commit checks][c1pc] before opening/drafting this pull request. - [ ] I have [tested my contribution][c1t] (or proof-read it for documentation-only changes) myself, if applicable. This includes ensuring code compiles. - [x] My commit messages follow the [commit message format][c1cm] and are descriptive. - [ ] I have written a [news fragment][n1] for this PR, if applicable<!--(can be done after hitting open!)-->. <!-- Notes on these requirements: - While not required, we encourage you to sign your commits with GPG or SSH to attest the authenticity of your changes. - While we allow LLM-assisted contributions, we do not appreciate contributions that are low quality, which is typical of machine-generated contributions that have not had a lot of love and care from a human. Please do not open a PR if all you have done is asked ChatGPT to tidy up the codebase with a +-100,000 diff. - In the case of code style violations, reviewers may leave review comments/change requests indicating what the ideal change would look like. For example, a reviewer may suggest you lower a log level, or use `match` instead of `if/else` etc. - In the case of code style violations, pre-commit check failures, minor things like typos/spelling errors, and in some cases commit format violations, reviewers may modify your branch directly, typically by making changes and adding a commit. Particularly in the latter case, a reviewer may rebase your commits to squash "spammy" ones (like "fix", "fix", "actually fix"), and reword commit messages that don't satisfy the format. - Pull requests MUST pass the `Checks` CI workflows to be capable of being merged. This can only be bypassed in exceptional circumstances. If your CI flakes, let us know in matrix:r/dev:continuwuity.org. - Pull requests have to be based on the latest `main` commit before being merged. If the main branch changes while you're making your changes, you should make sure you rebase on main before opening a PR. Your branch will be rebased on main before it is merged if it has fallen behind. - We typically only do fast-forward merges, so your entire commit log will be included. Once in main, it's difficult to get out cleanly, so put on your best dress, smile for the cameras! --> [c1]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md [c2]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/docs/development/code_style.mdx [c1pc]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#pre-commit-checks [c1t]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#running-tests-locally [c1cm]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#commit-messages [n1]: https://towncrier.readthedocs.io/en/stable/tutorial.html#creating-news-fragments
build(perf): document benefits of pre-compiling rocksdb:
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
a241fde347
git clone https://forgejo.ellis.link/continuwuation/rust-rocksdb-zaidoon1.git
    cd rust-rocksdb-zaidoon1
    git checkout 61d9d23872197e9ace4a477f2617d5c9f50ecb23
    git submodule update --init --recursive
    cd librocksdb-sys/rocksdb
    make shared_lib -j$(nproc)
    sudo make install-shared INSTALL_PATH=/usr/local
    sudo ldconfig
@ -59,16 +59,18 @@ standard = [
"systemd",
"url_preview",
"zstd_compression",
"sentry_telemetry",
Owner

Why disable sentry and opentelemetry?

Why disable sentry and opentelemetry?
Owner

(I use both of these in prod, as do others)

(I use both of these in prod, as do others)
Author
Contributor

typically telemetry is opt-in

you guys will have to figure out what this means for your release process

typically telemetry is opt-in you guys will have to figure out what this means for your release process
Owner

This is a misunderstanding of what these options do.

This is a misunderstanding of what these options do.
Contributor

@gamesguru

The telemetry here is opt-in via configuration. Asking end users to recompile continuwuity themselves for observability features seems a little odd, when these features are documented. I'd expect to have to compile a version without them, if I particularly wanted them gone.

Unless you're suggesting they add another build to the release pipeline with the telemetry features compiled in, which would double the current builds, this seems odd.

@gamesguru The telemetry here is opt-in via configuration. Asking end users to recompile continuwuity themselves for observability features seems a little odd, when these features are documented. I'd expect to have to compile a version without them, if I particularly wanted them gone. Unless you're suggesting they add another build to the release pipeline with the telemetry features compiled in, which would double the current builds, this seems odd.
Author
Contributor

What i mean is that should be configured as part of the release. I am perfectly okay with you passing the telemetry flag in with every action, workflow, runner, or prod build agent you think needs it.

But i don't think it's appropriate that this should be hard-coded in the rust configurations. I don't think someone compiling from source should be forced or automatically opted-into telemetry unless they have the expertise or time to reverse engineer and modify our rust code's build process. We should allow people who wish to compile their own source the option to disable it.

I'm not even sure if rust supports enablement by default for certain flags? I'm even even okay with enabling the flag by default.

What I'm resisting is the idea that the flag shouldn't exist at all, or there shouldn't be an option to disable the feature.

What i mean is that should be configured as part of the release. I am perfectly okay with you passing the telemetry flag in with every action, workflow, runner, or prod build agent you think needs it. But i don't think it's appropriate that this should be hard-coded in the rust configurations. I don't think someone compiling from source should be *forced* or automatically opted-into telemetry unless they have the expertise or time to reverse engineer and modify our rust code's build process. We should allow people who wish to compile their own source the option to disable it. I'm not even sure if rust supports enablement by default for certain flags? I'm even even okay with enabling the flag by default. What I'm resisting is the idea that the flag shouldn't exist at all, or there shouldn't be an option to disable the feature.
Contributor

I'm not sure what you mean by there's no option to disable the feature. You can select features when you compile. --no-default-features --features "x y z" will only include features x, y, and z. I feel like that's not what you meant, but I can't tell what else you could be talking about, so do correct me if I'm wrong.

Also, "enablement by default for certain flags"? It's done by default right now. That's what the default feature set is. None of this is forced upon people compiling from source or set in stone.

To clarify, the situation is currently that observability features are compiled in by default, but you always have the option to compile continuwuity without them.

I apologise, but I'm very confused.

I'm not sure what you mean by there's no option to disable the feature. You can select features when you compile. `--no-default-features --features "x y z"` will only include features `x`, `y`, and `z`. I feel like that's not what you meant, but I can't tell what else you could be talking about, so do correct me if I'm wrong. Also, "enablement by default for certain flags"? It's done by default right now. That's what the default feature set is. None of this is forced upon people compiling from source or set in stone. To clarify, the situation is currently that observability features are compiled in by default, but you always have the option to compile continuwuity without them. I apologise, but I'm very confused.
Author
Contributor

for example cargo build --features full,otlp_telemetry --profile release will after this PR accomplish exactly the same as the old default behavior that you're used to.

i admittedly did not have time to document things in all the relevant places that may be affected. I am going to lean on the team a bit to help fill in those gaps and make sure this gets laid out as effectively and clearly as we can

for example `cargo build --features full,otlp_telemetry --profile release` will after this PR accomplish exactly the same as the old default behavior that you're used to. i admittedly did not have time to document things in all the relevant places that may be affected. I am going to lean on the team a bit to help fill in those gaps and make sure this gets laid out as effectively and clearly as we can
Author
Contributor

ok so, even if you use the --no-default-features (today on main), it looks like it skips the runtime but still was compiling the crates.

and that's really what this PR is about, cutting build times drastically and making caching of our largest dependency, which has stayed at the same version for a while, trivially easy.

if it's too confusing what to do now about telemetry, we need to figure out some ideas there

ok so, even if you use the `--no-default-features` (today on `main`), it looks like it skips the runtime but still was compiling the crates. and that's really what this PR is about, cutting build times drastically and making caching of our largest dependency, which has stayed at the same version for a while, trivially easy. if it's too confusing what to do now about telemetry, we need to figure out some ideas there
Author
Contributor

plus, even if it no-ops the runtime... a open-source project like ours should probably exclude the telemetry SDK from the build by default

plus, even if it no-ops the runtime... a open-source project like ours should probably exclude the telemetry SDK from the build by default
Contributor

I think the confusion here comes from the term "telemetry" when it is actually for monitoring and health checks, which are incredibly useful and normal features for production server software to have. Caddy, for example, comes with similar features built in.

If you want to omit them by default, that feels like it should be a separate PR.

I think the confusion here comes from the term "telemetry" when it is actually for monitoring and health checks, which are incredibly useful and normal features for production server software to have. Caddy, for example, comes with similar features built in. If you want to omit them by default, that feels like it should be a separate PR.
Author
Contributor

Some of what I said was wrong and you were right that --no-default-features works the same as this branch. But that seems like a bit obscure of a flag, and by default a LOT more is included, i.e., I'm not sure what tracing-flame is but it's 2nd slowest for me after RocksDB, and this PR cuts it out by default. I just monitor with journalctl and sar. Feels like we can maybe revisit adding flag(s) for 3rd party library inclusion. I noticed the anti spam is included even with no-default. I don't use any of the "plugins" so I can't say how they're best organized behind feature flags. And I need to look a bit more how they're connected and the binary size and compile times they add.

What is the difference then with full and default?

I guess in the interests of getting this thru I will revert that part for now, since you told me about the no default features flag. That should speed my builds up on its own 👍

Some of what I said was wrong and you were right that `--no-default-features` works the same as this branch. But that seems like a bit obscure of a flag, and by default a LOT more is included, i.e., I'm not sure what `tracing-flame` is but it's 2nd slowest for me after RocksDB, and this PR cuts it out by default. I just monitor with `journalctl` and `sar`. Feels like we can maybe revisit adding flag(s) for 3rd party library inclusion. I noticed the anti spam is included even with no-default. I don't use any of the "plugins" so I can't say how they're best organized behind feature flags. And I need to look a bit more how they're connected and the binary size and compile times they add. What is the difference then with full and default? I guess in the interests of getting this thru I will revert that part for now, since you told me about the no default features flag. That should speed my builds up on its own 👍
build(docs): old compilers may need: CXXFLAGS="-Wno-error=unused-parameter"
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
1a6e84c0fc
update env vars defs for building. see below note on DISABLE_JEMALLOC=1,
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
594b39080e
```shell
git clone https://forgejo.ellis.link/continuwuation/rust-rocksdb-zaidoon1.git
cd rust-rocksdb-zaidoon1
git checkout 61d9d23872197e9ace4a477f2617d5c9f50ecb23
git submodule update --init --recursive
cd librocksdb-sys/rocksdb
PORTABLE=0 DISABLE_JEMALLOC=1 CXXFLAGS="-Wno-error=unused-parameter" make shared_lib -j$(nproc)
sudo make install-shared INSTALL_PATH=/usr/local
sudo ldconfig
```
nex changed title from Build(perf): Option to use precompiled RocksDB. Isolate spurious debug crates from release to perf(build): Option to use precompiled RocksDB. Isolate spurious debug crates from release 2026-03-09 16:06:59 +00:00
revised commands to build rocksdb static assembly:

```shell
DISABLE_JEMALLOC=1 EXTRA_CXXFLAGS="-Wno-error=unused-parameter" make shared_lib static_lib -j$(nproc)
sudo make install-shared INSTALL_PATH=/usr/local
sudo make install-static INSTALL_PATH=/usr/local
sudo ldconfig
```
document tech debt: wasteful presence DB i/o and a need for LRU caching
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
2a2c513205
leaner runtime 560 crates. take tracers out of default features. give the people choice
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
9e76153ac0
42 megabytes. 8 minutes. release-max-perf.

we need to improve our documentation, so admins understand how to get non-default features.

i suspect nothing additional is needed to run the full Complement Test suite and serve all basic homeserver functions.

not even URL preview zero overhead: extreme cases may leak IPs or allow forgery attacks.
there's no harm in recompiling once you find you want it.

testing: add WIP engage target for complement tests (not everyone has buildkit or buildx)
add cargo config to use mold, faster linker than ld or cc
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
9f6fd04507
this should also apply to RustRover by default, and you will notice less redundant rebuilding of built static imports.
direnv auto-detect sccache, faster builds from CLI
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
2fa147c9ec
add redundant config since RustRover has the IQ of a carrot. TODO: prevent this breaking if sccache or mold aren't found
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
e9ec975f03
RustRover default to prebuilt RocksDB if found in/usr/local
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
f767a1174d
help RustRover see. You may need below commands. And i may nest things eventually in /usr/loca/uwu for isolation.
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
78e24ffc48
This will tell your linker to look in `/usr/local`, but it may also cause conflicts or unexpected behavior:

```bash
echo "/usr/local/lib" | sudo tee /etc/ld.so.conf.d/usr-local.conf
sudo ldconfig
```
add docs on debugging over SSH with gdb or RustRover
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
b5565021e9
gamesguru force-pushed guru/perf/build-optimizations-dep-tree from b5565021e9
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
to 33d71edec2
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
2026-03-11 04:36:11 +00:00
Compare
add a proof of concept justfile for profiling cpu/mem usage and heat maps
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
7c61c8e09a
justfile targets to prebuild jemalloc, lz4, rocksdb, and zstd. down to 553 crates.
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
2b66ae9ad0
cargo configs so RustRover uses prebuilt libraries
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
d6ab95705a
update "debugging over SSH" docs to use rust-gdb
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
2ec6cafaa3
TODO: investigate dep. inversion w/ ruma, since it rebuilds on trivial/unrelated changes it shouldn't. [empty-commit]
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
40f9517824
gitignore the .tmp/ in case not everyone echoed a gitignore in it with '**'
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
9422b5f761
add some build/compile profiling targets to justfile
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
714fdec98d
update justfile targets (fix). format files/whitespace.
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
bc90eb5e19
prune makefile from this branch
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
a6d377f8d1
from this branch (origin is Jade's, github is mine), I ran:

```shell
FILES=$(gds origin/main --name-only)
BASE=$(git merge-base HEAD github/main)
git diff $BASE..github/main  -- $FILES | git apply --3way
```
prune makefile; fix my stupid slowdowns/regressions, but not quite as good as 9e76153a nor even a6d377f8 yet again.
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
820f1d5d1f
tidy/arrange justfile fix rockdb glitch
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
497dbf9164
Author
Contributor

Guys, I think what i'm gonna do here is open another PR that makes non-breaking, emergency changes that by default will use the following tools (if installed and available). But this new PR will otherwise change nothing about --features stuff and things that might completely break DevOps pipelines and require a bit more collaboration and thinking things through.

So my new PR will configure at the project level, i.e., thru cargo not direnv or anything else shell-level based that has a tendency to not be picked up by my cron jobs and five other tools on 3 different machines.

  • mold
  • clang (think we do this already)
  • sccache
  • pre-built/pre-installed: rocksdb, jemalloc, snappy, zstd, lz4, bzip2, and liburing

The feature flag changes are less important to me.

My laptop and poor VPS simply cannot handle building RocksDB concurrently with the rust project, and they benefit too immensely from these speedups as to neglect expediency here on my part.

I'm not able to quickly build my PRs without first merging this branch (which contains other changes in src/ and therefore arguably pollutes my tests of the PRs).

Guys, I think what i'm gonna do here is open another PR that makes non-breaking, emergency changes that by default will use the following tools (if installed and available). But this new PR will otherwise change nothing about `--features` stuff and things that might completely break DevOps pipelines and require a bit more collaboration and thinking things through. So my new PR will configure at the project level, i.e., thru `cargo` not `direnv` or anything else shell-level based that has a tendency to not be picked up by my `cron` jobs and five other tools on 3 different machines. - `mold` - `clang` (think we do this already) - `sccache` - pre-built/pre-installed: `rocksdb, jemalloc, snappy, zstd, lz4, bzip2, and liburing` The feature flag changes are less important to me. My laptop and poor VPS simply cannot handle building RocksDB concurrently with the rust project, and they benefit too immensely from these speedups as to neglect expediency here on my part. I'm not able to quickly build my PRs without first merging this branch (which contains other changes in `src/` and therefore arguably pollutes my tests of the PRs).
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Required
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Required
Details
Update flake hashes / update-flake-hashes (pull_request) Blocked by required conditions
This pull request is blocked because it's outdated.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u guru/perf/build-optimizations-dep-tree:gamesguru-guru/perf/build-optimizations-dep-tree
git switch gamesguru-guru/perf/build-optimizations-dep-tree
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
continuwuation/continuwuity!1509
No description provided.