WIP: feat: stateless sync and orphan all sync tokens #1456
Draft
gamesguru
wants to merge 188 commits from
gamesguru/continuwuity:guru/experiment/stateless-sync-purge-sync-tokens into main
pull from: gamesguru/continuwuity:guru/experiment/stateless-sync-purge-sync-tokens
merge into: continuwuation:main
continuwuation:main
continuwuation:nex/fix/informative-startup-errs
continuwuation:renovate/tokio-1.x-lockfile
continuwuation:aranje/illegal-car-mods
continuwuation:ginger/password-reset
continuwuation:ginger/no-left-room-initial-sync
continuwuation:nex/feat/policy-servers-2-electric-boogaloo
continuwuation:renovate/rust-patch-updates
continuwuation:jade/docker-entrypoint
continuwuation:jade/dehydrated-devices
continuwuation:ginger/complement-fixes
continuwuation:nex/fix/stale-destination-cache
continuwuation:nex/experiment/sync-mutex
continuwuation:tcpipuk/docker-docs
continuwuation:jade/snafu
continuwuation:renovate/serde_html_form-0.x
continuwuation:jade/rand-update
continuwuation:renovate/reqwest-0.x
continuwuation:nex/stateres-refactor
continuwuation:ginger/779-in-troubleshooting
continuwuation:jade/liveit-guide
continuwuation:jade/http3
continuwuation:nex/feat/admin-hide-empty-rooms
continuwuation:ginger/oobe
continuwuation:nex/fix/debian-thingy
continuwuation:jade/ldap-admin-check
continuwuation:nex/fix/remote-restricted-joins
continuwuation:nex/feat/msc4406-sender-ignored
continuwuation:jade/deadlock-detection
continuwuation:nex/feat/room-shutdown
continuwuation:jade/get-started
continuwuation:jade/docs-guide
continuwuation:ginger/fix-local-invites
continuwuation:nex/fix/tpi
continuwuation:nex/feat/room-deletion
continuwuation:nex/feat/msc4322-media-redaction
continuwuation:ginger/stitched-order
continuwuation:jade/build-info
continuwuation:ginger/deps/update-rspress
continuwuation:jade/admin-announce-improvements
continuwuation:ginger/xtask-improvements
continuwuation:jade/improve-admin-config-display
continuwuation:nex/fix/better-stateres-error-logs
continuwuation:jade/sender-timeouts
continuwuation:nex/feat/custom-v12-room-ids
continuwuation:ginger/update-metadata
continuwuation:nex/feat/admin-force-logout
continuwuation:tom/max-perf-docs
continuwuation:nex/fix/invalid-appservice-reg
continuwuation:nex/feat/antispam
continuwuation:nex/feat/account-locking
continuwuation:jade/logging-cleanup
continuwuation:jade/remove-legacy-appservice-auth
continuwuation:nex/fix/key-query
continuwuation:jade/update-prek
continuwuation:nex/fix/room-summaries
continuwuation:ginger/restrict-admin-commands
continuwuation:ginger/enable-console-by-default
continuwuation:jade/tag-fixes
continuwuation:jade/otlp
continuwuation:nex/meta/pull-req-template
continuwuation:nex/fix/fed-invite-compliance
continuwuation:nex/feat/build-commit
continuwuation:nex/feat/join-logging
continuwuation:jade/mailmap-updates
continuwuation:jade/hack-ci-tmp
continuwuation:jade/v12-stable
continuwuation:jade/relations
continuwuation:ginger/database-refactor
continuwuation:jade/fix-ldap-uiaa
continuwuation:nex/fix/validation
continuwuation:ginger/nuke-invalid-msc4133-fields-in-migration
continuwuation:ginger/downgrade-artifact-actions
continuwuation:oddlid/reload-fix
continuwuation:jade/fix-assert
continuwuation:ginger/sync-v3-cleanup
continuwuation:ginger/remove-absolute-action-urls
continuwuation:jade/website
continuwuation:nex/fix/backoff
continuwuation:ginger/fix-mdbook-for-0.5
continuwuation:ginger/no-docker-on-prs
continuwuation:backport/v0.5.0-rc.8-1
continuwuation:nex/fed-improvements
continuwuation:jade/rust-1.90
continuwuation:jade/mirror-dockerhub
continuwuation:jade/clippy-fixes
continuwuation:jade/fix-support
continuwuation:jade/clean-images
continuwuation:jade/wal-compression-type
continuwuation:jade/flake-clone
continuwuation:ginger/upload-rpms-on-schedule
continuwuation:nex/fix/incoming-fetch
continuwuation:nex/fix/upgrade
continuwuation:tom/ci-fedora-rpm
continuwuation:jade/ci-release-fix
continuwuation:jade/rocksdb-10-5
continuwuation:ginger/fix-msc4133-migration
continuwuation:ginger/migrate-busted-tz
continuwuation:hydra/public
continuwuation:nex/feat/manual-extremities
continuwuation:nex/feat/async-media
continuwuation:nex/feat/fast-joins-hack-do-not-use-DO-NOT-USE
continuwuation:nex/feat/better-logging
continuwuation:trigger-ci-so-latest-isnt-on-illegal-car-mods
continuwuation:nex/feat/pins-backfill
continuwuation:jade/tuwunel-2025-06-old
continuwuation:jade/ai-slop-db-docs
continuwuation:nex/fix-create-auth
continuwuation:jade/version-stats
continuwuation:jade/read-receipts
continuwuation:jade/rust-toolchain-no-targets
continuwuation:jade/logging-features
continuwuation:jade/syncv5-typing
continuwuation:jade/msc2815
continuwuation:jade/purge-sync-tokens
continuwuation:morguldir/see-eye
continuwuation:jade/css-small-screen
continuwuation:nex/wip-751
continuwuation:tuwunel-rebase
continuwuation:test
continuwuation:oddlid/rename-admin-room-bot
continuwuation:strawberry/nix-ci-stuff
continuwuation:strawberry/valgrind
continuwuation:phonemain
continuwuation:strawberry/morgs-snake-sync-jason-main
continuwuation:newer-media-endpoints
continuwuation:folly-coroutines-async-io
continuwuation:federation-retry-timer-port
continuwuation:bad-attempt-at-extracting-homeserver-signing-key
continuwuation:room-deletion-attempt-do-not-use
No reviewers
Labels
Clear labels
This pull request or issue is currently blocked from being merged/closed
Something isn't working as intended
Commits picked from other conduit projects
This requires or includes changes to the database
Something dependency related
Automatic dependency upgrades by Renovate
Low difficulty to implement - touches few parts of the codebase, low complexity
High difficulty to implement - touches many parts of the codebase, high complexity
Medium difficulty to implement - touches more parts of the codebase, higher complexity
Improvements or additions to documentation
New feature or request
Good for newcomers
Additional eyes and keyboards are required for this one
Issues that have been inhereted from the project pre-fork
Features pertaining to homeserver administration
Features pertaining to the appservice API
Features pertaining to authentication
Features pertaining to client-to-server interactions
Issues relating to core matrix functionality, such as state resolution and PDU formats
Features pertaining to server-to-server interactions
Issues related to room version 12 and related changes (temporary label)
Features pertaining to unstable matrix features
Features pertaining to media interactions
Changes or issues related to trust & safety tooling
Related to housekeeping, maintenance, or other repo-meta.
Issues related to CI changes
Packaging
This issue is blocking the next release
This issue is very important
This issue is of a rather low priority
This item is related to general security
This issue has enough information and is confirmed
This issue or pull request already exists
This issue doesn't seem right
This issue needs further investigation
Questions or support requests
This will not be worked on
Ci/CD
Pull requests that update Rust code
Blocked
This pull request or issue is currently blocked from being merged/closed
Bug
Something isn't working as intended
Cherry-picking
Commits picked from other conduit projects
Database
This requires or includes changes to the database
Dependencies
Something dependency related
Dependencies/Renovate
Automatic dependency upgrades by Renovate
Difficulty
Easy
Low difficulty to implement - touches few parts of the codebase, low complexity
Difficulty
Hard
High difficulty to implement - touches many parts of the codebase, high complexity
Difficulty
Medium
Medium difficulty to implement - touches more parts of the codebase, higher complexity
Documentation
Improvements or additions to documentation
Enhancement
New feature or request
Good first issue
Good for newcomers
Help wanted
Additional eyes and keyboards are required for this one
Inherited
Issues that have been inhereted from the project pre-fork
Matrix/Administration
Features pertaining to homeserver administration
Matrix/Appservices
Features pertaining to the appservice API
Matrix/Auth
Features pertaining to authentication
Matrix/Client
Features pertaining to client-to-server interactions
Matrix/Core
Issues relating to core matrix functionality, such as state resolution and PDU formats
Matrix/E2EE
Matrix/Federation
Features pertaining to server-to-server interactions
Matrix/Hydra
Issues related to room version 12 and related changes (temporary label)
Matrix/MSC
Features pertaining to unstable matrix features
Matrix/Media
Features pertaining to media interactions
Matrix/T&S
Changes or issues related to trust & safety tooling
Meta
Related to housekeeping, maintenance, or other repo-meta.
Meta/CI
Issues related to CI changes
Meta/Packaging
Packaging
Priority
Blocking
This issue is blocking the next release
Priority
High
This issue is very important
Priority
Low
This issue is of a rather low priority
Security
This item is related to general security
Status
Confirmed
This issue has enough information and is confirmed
Status
Duplicate
This issue or pull request already exists
Status
Invalid
This issue doesn't seem right
Status
Needs Investigation
This issue needs further investigation
Support
Questions or support requests
To-Merge
Wont fix
This will not be worked on
old/ci/cd
Ci/CD
Archived
old/rust
Pull requests that update Rust code
Archived
No labels
Blocked
Bug
Cherry-picking
Database
Dependencies
Dependencies/Renovate
Difficulty
Easy
Difficulty
Hard
Difficulty
Medium
Documentation
Enhancement
Good first issue
Help wanted
Inherited
Matrix/Administration
Matrix/Appservices
Matrix/Auth
Matrix/Client
Matrix/Core
Matrix/E2EE
Matrix/Federation
Matrix/Hydra
Matrix/MSC
Matrix/Media
Matrix/T&S
Meta
Meta/CI
Meta/Packaging
Priority
Blocking
Priority
High
Priority
Low
Security
Status
Confirmed
Status
Duplicate
Status
Invalid
Status
Needs Investigation
Support
To-Merge
Wont fix
old/ci/cd
old/rust
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
continuwuation/continuwuity!1456
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "gamesguru/continuwuity:guru/experiment/stateless-sync-purge-sync-tokens"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Link to Complement Test Suite incremental diffs per commit: https://github.com/gamesguru/continuwuity/pull/17#issuecomment-3980966155
This pull request addresses !917, dropping
roomsynctoken_shortstatehashvia v19 RocksDB migration and allowing its space to be reclaimed viaquery raw compact --exhaustive. It also introduces an LRU cache and numerous other speedups.NOTICE: This PR is ready for beta testers. Please run with
log_level=INFOand report any race conditions or bugs in the development channel or in a comment here!If you wish to review it, stay focused on what appears to be core functionality. I am still siphoning off features to other branches and deciding what can stay.
Pull request checklist:
mainbranch, and the branch is named something other thanmain.myself, if applicable. This includes ensuring code compiles.
I am running a modified/tweaked version of this branch on a development account. I have joined Matrix HQ and many large or allegedly or reportedly "broken" rooms without ill-effect.
I believe I have fixed some things by brute force that were previously assumed to be solely under the purview of transactions... (Although, it seems I have also introduced some more caching issues. But I would personally prefer to clear my cache and get into rooms than never get into them at all).
Briefly, I had issues on this branch, which caused the accumulation of
eventid_outlierpduand possibly others? (See DB files output below log).Sample logs:
DB files:
Matrix Complement Test Suite results
/csapi2026-02-27T14:15:51.0050286Z DONE 739 tests, 8 skipped, 244 failures in 672.761shttps://github.com/gamesguru/continuwuity/actions/runs/22488841324/job/65145319770
I merged in the transaction branch (at least on my GitHub remote).
Some initial findings:
This error occurred fairly early in the Complement logs: ```log === NAME TestDeviceManagement/DELETE_/device/{deviceId} apidoc_device_management_test.go:134: MatchResponse got status 400 want 401 - http://127.0.0.1:32820/_matrix/client/v3/devices/7hTYOfr7hy => {"errcode":"M_NOT_JSON","error":"M_NOT_JSON: Not json."} === RUN TestDeviceManagement/DELETE_/device/{deviceId}_requires_UI_auth_user_to_match_device_owner ```WIP: feat: stateless sync and orphan all sync tokensto feat: stateless sync and orphan all sync tokensIt's working a lot better now.
wide_methods to parallelize server discovery and remote PDU fetchingThe PR is in a good state after the preceding commit (which fixes the loading symbol seen in the video aboe with large rooms), but i've added even more improvements that i'll soon push.
Currently trying to squash a bug i've identified where joining, leaving, and re-joining prevents you from leaving a 2nd time (keeping you locked in) with a 403 error, "M_FORBIDDEN: Event is not authorized."
But overall the branch is stable and very quick at handling large rooms. If you don't mind a few state bugs that we solve together, I encourage you to install it and beta test v19 of the schema.
Going to push up the remaining commits, and then hopefully squash the aforementioned 403 degenerate state bug, before finally pulling a few more features off onto separate PRs.
update config key descriptions in config/mod.rs add speculative fixes for the following 404 ORB errors in SchildiChat web: {"errcode":"M_NOT_FOUND","error":"M_NOT_FOUND: Fetching media failed: Answer from matrix.sp-codes.de: [404 / M_UNRECOGNIZED] Unrecognized request mxc=mxc://matrix.sp-codes.de/SpHUPhCGRnsYTFlyRYCDZhDD authenticated=true"} {"errcode":"M_NOT_FOUND","error":"M_NOT_FOUND: Fetching media failed: Answer from hampai.ch: [401 / M_UNAUTHORIZED] no auth provided (required) mxc=mxc://hampai.ch/b291e6908914b64277b7ed0e1de87df5d52cac241958243044855644160 authenticated=true"} enforce freeze_legacy_media on all legacy remote fetch paths fix: fallback to unauthenticated media on more errors and safely return NotFound for empty mediafailed to derive last_sync_end_shortstatehashandoldest event is before last sync3e7bc3b6b3authorizationwas missing" aa74ee56eb```log 2026-03-03T02:36:25.790498Z INFO conduwuit_api::client::sync::v3::joined: user joined since last sync: last_sync_end_shortstatehash=Some(3316807), membership=Some(RoomMemberEventContent { avatar_url: Some("mxc://mdev.nutra.tk/Pb87ocP5NsGs7xvS9ZbIDhI30aU7EZLO"), displayname: Some("ggdev 🏴🚩"), is_direct: None, membership: "leave", third_party_invite: None, blurhash: None, reason: None, join_authorized_via_users_server: None, redact_events: None }) ```feat: stateless sync and orphan all sync tokensto WIP: feat: stateless sync and orphan all sync tokensI've gotten through 61% of this PR and I'm just not going to continue further. This diff is impossible to review and there are way too many commits. Please squash some commits down (150 commits with 97 changed files is an infeasible number to review, forgejo even struggles to render this) and remove anything that isn't actually related to this PR's scope.
@ -31,2 +31,4 @@indent_size = 4indent_style = space[Makefile]This seems unnecessary, we don't use makefiles
@ -1,6 +1,9 @@#!/usr/bin/env bashdotenv_if_existswatch_file .envWhat is
watch_file? Should be in its own PR anywaypart of
direnv's spec for.envrcfiles, see attached:Document need to use watch_file · Issue #293 · direnv/direnv
https://github.com/direnv/direnv/issues/293
@ -0,0 +1,76 @@---Workflows need to go under
.forgejo/workflows, we don't use github for CI. CI changes should also be in another PR.@ -0,0 +1,116 @@---Workflows need to go under
.forgejo/workflows, we don't use github for CI. CI changes should also be in another PR.@ -0,0 +1,174 @@---Workflows need to go under
.forgejo/workflows, we don't use github for CI. CI changes should also be in another PR.@ -0,0 +1,255 @@SHELL=/bin/bashWe don't use makefiles, you should ignore this in your VCS locally if you want to have your own
I tried that, but unfortunately it ended up getting deleted or falling out of sync between machines.
Until I familiarize myself with engage/Justfiles, such configuration changes as this are inevitable temporary artifacts on larger PRs. Please don't spend too much time on them, and know my focus is mostly on getting the code changes in prod code ironed out and streamlining the feedback process for rust changes. Even if it goes in, we can revert or even do a quick force push back
@ -6,10 +6,10 @@ set -euo pipefailCOMPLEMENT_SRC="${COMPLEMENT_SRC:-$1}"Complement changes are unrelated to this PR, please make them in a separate PR
@ -51,2 +61,2 @@go test -tags="conduwuit_blacklist" -skip="$SKIPPED_COMPLEMENT_TESTS" -v -timeout 1h -json ./tests/... | tee "$LOG_FILE"set -o pipefailif [ "$HAS_GOTESTSUM" = true ]; then# gotestsum writes full JSON logsHow is this any different to
go test -json?gotestsumdiffers primarily by producing more human-readable output, which displays in color-coded ANSI log format, at least on GitHub.We probably do not want to run them with verbose logging by default, since people can do that in Docker. Verbose logging in CI results in log files 50 MB or more, and quite naturally both GitHub and Forge struggle with rendering that, and will often truncate it after extreme lag in the browser tab.
I am conscious of the fragile, ugly nature of the
if-elsestatement in thebashscript, however given the convenience it confers, I am inclined to believe its merits must exceed the risks.@ -0,0 +1 @@Implemented transactional wrappers around room joins and other key database operations to guarantee atomicity. This mitigates previously silent race conditions where partial state was written (fixes #1142 "Room has no state" sync error).This file is named wrong, and this file is also unrelated to this PR
@ -0,0 +1 @@The `roomsynctoken_shortstatehash` table — responsible for 60% to 90% of database bloat — has been removed. Sync state is now computed statelessly from the timeline on each request, eliminating the per-room write on every sync response. Existing table data is orphaned on disk and can be reclaimed by a future database compaction. Ref: [#917](https://forgejo.ellis.link/continuwuation/continuwuity/issues/917)This is probably a bit too technical for a changelog. Also the file is named wrong.
@ -6,12 +6,18 @@# ============================================= #Complement changes are unrelated to this PR, please make them in a separate PR
@ -0,0 +1,6 @@{"ImportPath":"./tests/...","Action":"build-output","Output":"# ./tests/...\n"}Complement changes are unrelated to this PR, please make them in a separate PR. Also I don't think this needs including in VCS
@ -1286,3 +1290,3 @@#allow_legacy_media = true# This item is undocumented. Please contribute documentation for it.# Disallow remote legacy media downloading. If set to true, requests forThis change seems unrelated to this PR.
@ -320,0 +319,4 @@.config.yes_i_am_very_very_sure_i_want_an_open_registration_server_prone_to_abuse;if services.firstrun.is_first_run() && !open_registration {This change is unrelated to this PR.
@ -115,3 +114,3 @@content_type,content_disposition,} = fetch_thumbnail(&services, &mxc, user, body.timeout_ms, &dim).await?;} = match fetch_thumbnail(&services, &mxc, Some(user), body.timeout_ms, &dim).await {This change is unrelated to this PR
@ -164,3 +164,3 @@| _ =>if !services.globals.server_is_ours(&body.server_name) && body.allow_remote {let response = servicesdebug_info!(%mxc, "Fetching remote media via authenticated federation fallback");This change is unrelated to this PR
@ -383,0 +380,4 @@// Parallel discovery: Kick off make_join to multiple servers at once.// The first one to return a valid stub wins and we try to finish the join with// them.let last_error = Arc::new(tokio::sync::Mutex::new(None));This change seems unrelated to this PR
@ -288,3 +288,3 @@let make_leave_response = services.sending.send_federation_request(.send_synapse_request(This change seems unrelated to this PR
@ -141,3 +141,3 @@.wide_filter_map(|item| visibility_filter(&services, item, sender_user)).take(limit).then(async |mut pdu| {.wide_then(move |mut pdu| async move {This change seems unrelated to this PR
@ -85,3 +85,3 @@})},| _ => Err!(Request(NotFound("Presence state for this user was not found"))),| _ => {This change seems unrelated to this PR
@ -413,3 +409,1 @@warn!(%user_id, %room_id, "Failed to update/send new profile join membership update in room: {e}");}}all_joined_roomsThis change seems unrelated to this PR and is also bad idea
@ -31,3 +31,3 @@if !visible || is_ignored_pdu(services, &event, body.sender_user()).await? {return Err!(Request(Forbidden("You don't have permission to view this event.")));return Err!(Request(NotFound("Event not found.")));This change seems unrelated to this PR
@ -349,3 +349,3 @@.await.map_err(|e| {err!(Request(Unknown("Failed resolving alias \"{alias}\": {e}")))err!(Request(BadAlias("Failed resolving alias \"{alias}\": {e}")))This change seems unrelated to this PR
@ -298,11 +319,26 @@ async fn build_state_and_timeline(// historylet prev_batch = timeline.pdus.front().map(at!(0));let user_has_join_event_in_sync = timelinethe way you calculate this means profile updates will cause limited timelines
@ -628,0 +703,4 @@.ok();// If we can resolve the previous membership event, check if it was Join.// If we couldn't resolve it (None), default to false (not a fresh join)None => Joinis a fresh join@ -628,0 +712,4 @@);if joined_since_last_sync && membership_during_previous_sync.is_some() {warn!(There's no log content here
@ -272,2 +273,2 @@.await.ok().broad_filter_map(|event_id: OwnedEventId| async move {services.rooms.timeline.get_pdu(&event_id).await.ok()This can return outlier PDUs which should never be sent to the client
@ -149,12 +149,12 @@ pub fn build(router: Router<State>, server: &Server) -> Router<State> {.ruma_route(&client::search_events_route)This change seems unrelated to this PR
@ -93,1 +93,4 @@// if the body is not empty and not media, but json parsing failed, it is// invalid JSONif json_body.is_none()This change seems unrelated to this PR
@ -366,3 +366,2 @@let msg = match e.reason() {| TypedHeaderRejectionReason::Missing => "Missing Authorization header.",| TypedHeaderRejectionReason::Error(_) => "Invalid X-Matrix signatures.",| TypedHeaderRejectionReason::Missing => "Missing Authorization header",This change seems unrelated to this PR
@ -21,2 +21,4 @@whereH: RumaHandler<T>;fn ruma_route_path<H, T>(self, handler: &'static H, path: &str) -> SelfThis change seems unrelated to this PR
@ -259,3 +259,3 @@}},| AllowRule::UnstableSpamChecker =>| AllowRule::UnstableSpamChecker => {This change seems unrelated to this PR
@ -359,6 +359,9 @@ async fn handle_room(}async fn handle_edu(services: &Services, client: &IpAddr, origin: &ServerName, edu: Edu) {if services.server.check_running().is_err() {This change seems unrelated to this PR
@ -46,15 +46,28 @@ pub(super) async fn check(&self) -> Result {join!(world_readable, server_in_room, server_can_see, acl_check);if !acl_check {This change seems unrelated to this PR
@ -14,1 +12,3 @@# name = "conduwuit_build_metadata"[[bin]]name = "conduwuit-version"This change seems unrelated to this PR
@ -1,25 +1,17 @@use std::process::Command;#[path = "src/git.rs"]mod git;This change seems unrelated to this PR
@ -27,3 +23,2 @@pub static GIT_REMOTE_COMMIT_URL: Option<&str> = option_env!("GIT_REMOTE_COMMIT_URL");// TODO: Mark dirty builds within the version stringpub static GIT_REMOTE_URL: Option<&str> = option_env!("GIT_REMOTE_URL");This change seems unrelated to this PR
@ -0,0 +1,11 @@#[path = "../git.rs"]This change seems unrelated to this PR
@ -0,0 +1,54 @@use std::process::Command;This change seems unrelated to this PR
@ -2787,3 +2798,3 @@fn default_client_response_timeout() -> u64 { 120 }fn default_client_shutdown_timeout() -> u64 { 15 }fn default_client_shutdown_timeout() -> u64 { 10 }This change seems unrelated to this PR
@ -157,6 +157,14 @@ macro_rules! err_lev {}};(info) => {This change seems unrelated to this PR
@ -70,3 +70,3 @@#[inline]pub fn inspect_debug_log<E: fmt::Debug>(error: &E) {inspect_debug_log_level(error, Level::ERROR);inspect_debug_log_level(error, Level::WARN);This change seems unrelated to this PR
@ -132,2 +132,4 @@Uiaa(ruma::api::client::uiaa::UiaaInfo),// federation / remote#[error("Federation timeout: {0}")]This change seems unrelated to this PR
@ -16,3 +16,3 @@fn into_response(self) -> axum::response::Response {let status = self.status_code();if status.is_server_error() {if status == StatusCode::INTERNAL_SERVER_ERROR {This change seems unrelated to this PR
@ -28,2 +30,4 @@#[inline]#[must_use]pub fn git_remote_commit_url() -> &'static str {This change seems unrelated to this PR
@ -467,3 +467,3 @@if !matches!(membership_state, MembershipState::Join) {warn!(info!(This change seems unrelated to this PR
@ -692,3 +691,3 @@);} else {warn!(event_id = aid.as_str(), "missing auth event");info!(event_id = aid.as_str(), "missing auth event");This change seems unrelated to this PR
I have added a method to rescue outlier
PDUs. I am just pushing things here still as I fix or create them. (This branch is still pending cleanup and the resolution of two or three serious bugs).I also naturally added methods to filter by
roomandsender, however, the database was implemented without indexes as a simple Key-Value store, so this is just a full linear scan (quite slow when cold and not cached or warm).I had a lot of outliers on my development/test account, and I wanted a mechanism to clean them up. Example commands here. Second rescue is shorter due to being in the same room, very close to the first one.
Lack of index on timestamp makes ordering of the chat difficult. So i also added the ability to index a room, playing with that.
make downloadannoyance, avx512 vs. avx2 runner 8a1b36de2emake download-listde1890b89arelease-high-perfprofile for skylake cpu build ef34ba449drelease-max-perfnot justrelease-high-perfe3c910a95c--format pkgnamefor less verbose CI log piping faf04f8e87is_historicalfor prev_events soft-failing 0d46dfcaa7is_historicalfor prev_events soft-failing 5eb54d9c85View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.