continuwuation/continuwuity
continuwuation/continuwuity
14
108
Fork 117
Code Issues 129 Pull requests 23 Releases 35 Packages 3 Activity Actions

WIP: refactor: New remote membership handling #1978

Draft
nex wants to merge 5 commits from nex/feat/centralise-remote-memberships into main
pull from: nex/feat/centralise-remote-memberships
merge into: continuwuation:main
Conversation 1 Commits 5 Files changed 13 +1026 -478
nex commented 2026-07-12 22:41:50 +00:00
Owner
Copy link

This pull request updates the remote membership code to improve performance, reliability, and maintainability. Part of the bob the builder project.

Pull request checklist:

  • This pull request targets the main branch, and the branch is named something other than
    main.
  • I have written an appropriate pull request title and my description is clear.
  • I understand I am responsible for the contents of this pull request.
  • I have followed the contributing guidelines:
This pull request updates the remote membership code to improve performance, reliability, and maintainability. Part of the bob the builder project. **Pull request checklist:** <!-- You need to complete these before your PR can be considered. If you aren't sure about some, feel free to ask for clarification in #dev:continuwuity.org. --> - [x] This pull request targets the `main` branch, and the branch is named something other than `main`. - [x] I have written an appropriate pull request title and my description is clear. - [x] I understand I am responsible for the contents of this pull request. - I have followed the [contributing guidelines][c1]: - [x] My contribution follows the [code style][c2], if applicable. - [x] I ran [pre-commit checks][c1pc] before opening/drafting this pull request. - [ ] I have [tested my contribution][c1t] (or proof-read it for documentation-only changes) myself, if applicable. This includes ensuring code compiles. - [x] My commit messages follow the [commit message format][c1cm] and are descriptive. <!-- Notes on these requirements: - While not required, we encourage you to sign your commits with GPG or SSH to attest the authenticity of your changes. - While we allow LLM-assisted contributions, we do not appreciate contributions that are low quality, which is typical of machine-generated contributions that have not had a lot of love and care from a human. Please do not open a PR if all you have done is asked ChatGPT to tidy up the codebase with a +-100,000 diff. - In the case of code style violations, reviewers may leave review comments/change requests indicating what the ideal change would look like. For example, a reviewer may suggest you lower a log level, or use `match` instead of `if/else` etc. - In the case of code style violations, pre-commit check failures, minor things like typos/spelling errors, and in some cases commit format violations, reviewers may modify your branch directly, typically by making changes and adding a commit. Particularly in the latter case, a reviewer may rebase your commits to squash "spammy" ones (like "fix", "fix", "actually fix"), and reword commit messages that don't satisfy the format. - Pull requests MUST pass the `Checks` CI workflows to be capable of being merged. This can only be bypassed in exceptional circumstances. If your CI flakes, let us know in matrix:r/dev:continuwuity.org. - Pull requests have to be based on the latest `main` commit before being merged. If the main branch changes while you're making your changes, you should make sure you rebase on main before opening a PR. Your branch will be rebased on main before it is merged if it has fallen behind. - We typically only do fast-forward merges, so your entire commit log will be included. Once in main, it's difficult to get out cleanly, so put on your best dress, smile for the cameras! --> [c1]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md [c2]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/docs/development/code_style.mdx [c1pc]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#pre-commit-checks [c1t]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#running-tests-locally [c1cm]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#commit-messages
nex self-assigned this 2026-07-12 22:41:50 +00:00
feat: Write new make_join and send_join handlers
Some checks failed
Auto Labeler / Apply labels based on changed files (pull_request_target) Successful in 2s
Details
Documentation / Build and Deploy Documentation (pull_request) Successful in 1m2s
Details
Checks / Prek / Check changed files (pull_request) Successful in 5s
Details
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m12s
Details
Checks / Changelog / Check changelog is added (pull_request_target) Failing after 6s
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 3m49s
Details
a24019a494
refactor: Use new make_join and send_join functions in membership service
Some checks failed
Checks / Changelog / Check changelog is added (pull_request_target) Failing after 6s
Details
Documentation / Build and Deploy Documentation (pull_request) Successful in 59s
Details
Checks / Prek / Check changed files (pull_request) Successful in 6s
Details
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m13s
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 12m57s
Details
21918e7ba4
fix: Don't allow event ID-less events to make their way into the DB
Some checks failed
Checks / Prek / Check changed files (pull_request) Successful in 4s
Details
Checks / Changelog / Check changelog is added (pull_request_target) Failing after 6s
Details
Documentation / Build and Deploy Documentation (pull_request) Successful in 56s
Details
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m16s
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 4m6s
Details
142e95b5a1
fix: Ensure events received from federated invites have event IDs
Some checks failed
Checks / Changelog / Check changelog is added (pull_request_target) Failing after 5s
Details
Documentation / Build and Deploy Documentation (pull_request) Successful in 59s
Details
Checks / Prek / Check changed files (pull_request) Successful in 5s
Details
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m8s
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 5m57s
Details
6273c7b371
nex force-pushed nex/feat/centralise-remote-memberships from 6273c7b371
Some checks failed
Checks / Changelog / Check changelog is added (pull_request_target) Failing after 5s
Details
Documentation / Build and Deploy Documentation (pull_request) Successful in 59s
Details
Checks / Prek / Check changed files (pull_request) Successful in 5s
Details
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m8s
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 5m57s
Details
to 64c0964102
Some checks failed
Checks / Prek / Check changed files (pull_request) Successful in 4s
Details
Checks / Changelog / Check changelog is added (pull_request_target) Failing after 6s
Details
Documentation / Build and Deploy Documentation (pull_request) Successful in 1m0s
Details
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 2m7s
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 5m24s
Details
2026-07-13 01:33:06 +00:00 Compare
ginger reviewed 2026-07-13 18:14:17 +00:00
src/service/rooms/membership/join.rs
@ -0,0 +43,4 @@
pub enum MakeJoinResult {
/// A make_join request returned and the response was validated
/// successfully.
Success((CanonicalJsonObject, RoomVersionId)),
ginger commented 2026-07-13 18:08:06 +00:00
Owner
Copy link

why's this a canonicaljsonobject and not a proper struct of some kind?

why's this a canonicaljsonobject and not a proper struct of some kind?
nex commented 2026-07-13 18:20:32 +00:00
Author
Owner
Copy link

Because it doesn't need to be. It hands this back to the caller, who then calls seed_local_membership_pdu to make it safe, and then this gets passed through to send_join, which converts the canonical json into federation format. Then, it's possible this template value gets dropped in favour of the one returned during send_join, but the end result is it only gets properly parsed into a PDU after sending the join.

Also keep in mind Pdu drops unknown fields, I figured that was out of scope for this PR in particular (although I can probably squeeze it in if you think it'd be appropriate)

Because it doesn't need to be. It hands this back to the caller, who then calls `seed_local_membership_pdu` to make it safe, and then this gets passed through to `send_join`, which converts the canonical json into federation format. Then, it's possible this template value gets dropped in favour of the one returned during send_join, but the end result is it only gets properly parsed into a PDU after sending the join. Also keep in mind `Pdu` drops unknown fields, I figured that was out of scope for this PR in particular (although I can probably squeeze it in if you think it'd be appropriate)
src/service/rooms/membership/join.rs
@ -0,0 +189,4 @@
via: &ServerName,
room_version_rules: &RoomVersionRules,
state_lock: &RoomMutexGuard,
) -> Result<Option<()>> {
ginger commented 2026-07-13 18:08:32 +00:00
Owner
Copy link

this return type should be a new enum

this return type should be a new enum
👍 1
src/service/rooms/membership/join.rs
@ -0,0 +365,4 @@
&room_id,
)
.await
.inspect(|_| info!("Finished joining {room_id}"))?;
ginger commented 2026-07-13 18:10:07 +00:00
Owner
Copy link

why's this trace in an inspect?

why's this trace in an `inspect`?
nex commented 2026-07-13 18:21:43 +00:00
Author
Owner
Copy link

why not

why not
ginger commented 2026-07-13 20:38:54 +00:00
Owner
Copy link

no point for it to be

no point for it to be
src/service/rooms/membership/join.rs
@ -0,0 +409,4 @@
// value.insert("event_id".to_owned(), event_id.to_string().into());
if self.services.pdu_metadata.is_event_rejected(&event_id).await {
// Rejection will be re-evaluated later
trace!(%event_id, "Un-rejecting event");
ginger commented 2026-07-13 18:10:49 +00:00
Owner
Copy link

🤨I thought that was impossible?

🤨I thought that was impossible?
nex commented 2026-07-13 18:23:23 +00:00
Author
Owner
Copy link

It's not, but in this case we're replacing what we know locally with whatever the remote says. Since it's (as far as im aware) impossible for a rejected event to honestly be returned during the join process, we initially just trust that the remote is correct in case we accidentally incorrectly rejected the event locally before, which means "just leave and rejoin" is also a valid workaround to a federation issue

Why event rejection can be inconsistent is still unknown to me, but it seems to happen in the wild, so. something something v13 except maybe v14 instead

It's not, but in this case we're *replacing* what we know locally with whatever the remote says. Since it's (as far as im aware) impossible for a rejected event to honestly be returned during the join process, we initially just trust that the remote is correct in case we accidentally incorrectly rejected the event locally before, which means "just leave and rejoin" is also a valid workaround to a federation issue Why event rejection can be inconsistent is still unknown to me, but it seems to happen in the wild, so. something something v13 except maybe v14 instead
nex commented 2026-07-13 18:25:56 +00:00
Author
Owner
Copy link

Rejection during the join process also isn't the proper way to go about it. I asked in spec, and it seems what you're supposed to do upon receiving an invalid event while joining is simply drop it entirely. If it's critical to the join, authenticating our own membership will fail, and thus the whole join.
Events that get dropped during join may end up being picked up through standard backfill means, such as the missing event fetches performed by handle_incoming_pdu etc, which can properly evaluate rejection.

Rejection during the join process also isn't the proper way to go about it. I asked in spec, and it seems what you're supposed to do upon receiving an invalid event while joining is simply drop it entirely. If it's critical to the join, authenticating our own membership will fail, and thus the whole join. Events that get dropped during join *may* end up being picked up through standard backfill means, such as the missing event fetches performed by handle_incoming_pdu etc, which *can* properly evaluate rejection.
👍 1
Some checks failed
Checks / Prek / Check changed files (pull_request) Successful in 4s
Required
Details
Checks / Changelog / Check changelog is added (pull_request_target) Failing after 6s
Required
Details
Documentation / Build and Deploy Documentation (pull_request) Successful in 1m0s
Details
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 2m7s
Required
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 5m24s
Required
Details
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin nex/feat/centralise-remote-memberships:nex/feat/centralise-remote-memberships
git switch nex/feat/centralise-remote-memberships
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
Abandoned
This pull request appears to be abandoned by its author.

  
Blocked
This pull request or issue is currently blocked from being merged/closed

  
Bug
Something isn't working as intended

  
Changelog
Added
 Changelog entry added

  
Changelog
Missing
 No changelog when one is needed

  
Changelog
None
 Changelog is unnecesary for this change

  
Cherry-picking
Commits picked from other conduit projects

  
Database
This requires or includes changes to the database

  
Dependencies
Something dependency related

  
Dependencies/Renovate
Automatic dependency upgrades by Renovate

  
Difficulty
Easy
 Low difficulty to implement - touches few parts of the codebase, low complexity

  
Difficulty
Hard
 High difficulty to implement - touches many parts of the codebase, high complexity

  
Difficulty
Medium
 Medium difficulty to implement - touches more parts of the codebase, higher complexity

  
Documentation
Improvements or additions to documentation

  
Enhancement
New feature or request

  
Good first issue
Good for newcomers

  
Help wanted
Additional eyes and keyboards are required for this one

  
Inherited
Issues that have been inhereted from the project pre-fork

  
Matrix/Administration
Features pertaining to homeserver administration

  
Matrix/Appservices
Features pertaining to the appservice API

  
Matrix/Auth
Features pertaining to authentication

  
Matrix/Client
Features pertaining to client-to-server interactions

  
Matrix/Core
Issues relating to core matrix functionality, such as state resolution and PDU formats

  
Matrix/E2EE
Issues related to end to end encryption

  
Matrix/Federation
Features pertaining to server-to-server interactions

  
Matrix/Hydra
Issues related to room version 12 and related changes (temporary label)

  
Matrix/MSC
Features pertaining to unstable matrix features

  
Matrix/Media
Features pertaining to media interactions

  
Matrix/T&S
Changes or issues related to trust & safety tooling

  
Merge
This PR is ready to be merged

  
Merge/Manual
This PR should be manually merged

  
Merge/Squash
This PR should be squashed when it is merged

  
Meta
Related to housekeeping, maintenance, or other repo-meta.

  
Meta/CI
Issues related to CI changes

  
Meta/Packaging
Packaging

  
Priority
Blocking
 This issue is blocking the next release

  
Priority
High
 This issue is very important

  
Priority
Low
 This issue is of a rather low priority

  
Security
This item is related to general security

  
Status
Confirmed
 This issue has enough information and is confirmed

  
Status
Duplicate
 This issue or pull request already exists

  
Status
Invalid
 This issue doesn't seem right

  
Status
Needs Investigation
 This issue needs further investigation

  
Support
Questions or support requests

  
bob the builder
pdu logic rewrite starring nexus nicholson

No labels
Abandoned
Blocked
Bug
Changelog
Added
Changelog
Missing
Changelog
None
Cherry-picking
Database
Dependencies
Dependencies/Renovate
Difficulty
Easy
Difficulty
Hard
Difficulty
Medium
Documentation
Enhancement
Good first issue
Help wanted
Inherited
Matrix/Administration
Matrix/Appservices
Matrix/Auth
Matrix/Client
Matrix/Core
Matrix/E2EE
Matrix/Federation
Matrix/Hydra
Matrix/MSC
Matrix/Media
Matrix/T&S
Merge
Merge/Manual
Merge/Squash
Meta
Meta/CI
Meta/Packaging
Priority
Blocking
Priority
High
Priority
Low
Security
Status
Confirmed
Status
Duplicate
Status
Invalid
Status
Needs Investigation
Support
bob the builder
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
nex
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
continuwuation/continuwuity!1978
No description provided.