Ask remote servers for individual unknown events, take 2 #976

Open
ginger wants to merge 5 commits from ginger/continuwuity:single-event-backfill into main
Contributor

From #899:

This change allows us to use /_matrix/federation/v1/get_event/{eventId} to fetch individual events that we may not have. This is similar to backfill, except it only retrieves an event by its ID, rather than fetching history.

This PR also makes use of this by making /_matrix/client/v3/rooms/{roomId}/event/{eventId} endpoint try to fetch the event locally, falling back to a backfill request if it fails. This means users can fetch pins from before they joined, in the event their server doesn't already have them locally.

From #899: > This change allows us to use [`/_matrix/federation/v1/get_event/{eventId}`](https://spec.matrix.org/unstable/server-server-api/#get_matrixfederationv1eventeventid) to fetch individual events that we may not have. This is similar to backfill, except it only retrieves an event by its ID, rather than fetching history. > > This PR also makes use of this by making [`/_matrix/client/v3/rooms/{roomId}/event/{eventId}`](https://spec.matrix.org/unstable/client-server-api/#get_matrixclientv3roomsroomideventeventid) endpoint try to fetch the event locally, falling back to a backfill request if it fails. This means users can fetch pins from before they joined, in the event their server doesn't already have them locally.
feat: Ask remote servers for individual unknown events
All checks were successful
Release Docker Image / define-variables (push) Successful in 12s
Checks / Prefligit / prefligit (push) Successful in 20s
Checks / Rust / Format (push) Successful in 51s
Checks / Rust / Clippy (push) Successful in 3m32s
Checks / Rust / Cargo Test (push) Successful in 4m57s
Checks / Prefligit / prefligit (pull_request) Successful in 33s
Documentation / Build and Deploy Documentation (pull_request) Successful in 47s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (push) Successful in 13m10s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (push) Successful in 13m28s
Release Docker Image / merge (push) Successful in 31s
89e2faaa8e
feat: Move packaging scripts into dist/ and consolidate the service files
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
19c9a896a6
fix: Update debian systemd unit path
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
2e687e0451
fix: Remove unnecessary user and directory modifications
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
c0c772639f
systemd creates a dynamic user for
continuwuity and manages directories for
it automatically, so the debian postinst
script no longer needs to do that.
fix: Move packaging files from dist/ to pkg/
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
e3072c4392
fix: Use handle_incoming_pdu directly to keep remote PDUs as outliers
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
39f92f1070
ginger force-pushed single-event-backfill from 39f92f1070 to 07d0e672af 2025-09-02 19:20:03 +00:00 Compare
ginger force-pushed single-event-backfill from 07d0e672af to 21dd09a6b6 2025-09-02 19:20:58 +00:00 Compare
feat: Do not persist remote PDUs fetched with admin commands
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
8d353c90f3
@ -21,2 +3,2 @@
}
}
/// Parse a pagination token
pub(crate) fn pagination_token_to_count(token: &str) -> Result<PduCount> { token.parse() }
Author
Contributor

these are just wrappers around the functions on the PduCount struct. why even keep this file if these are the only functions in it?

these are just wrappers around the functions on the `PduCount` struct. why even keep this file if these are the only functions in it?
@ -135,0 +140,4 @@
if local.is_ok() {
// We already have this PDU, no need to backfill
debug!("We already have {event_id} in {room_id}, no need to backfill.");
return local;
Author
Contributor

strange that a function with remote in the name can return local PDUs sometimes

strange that a function with `remote` in the name can return local PDUs sometimes
Author
Contributor

also, this function is similar in implementation and purpose to fetch_and_handle_outliers

also, this function is similar in implementation and purpose to [`fetch_and_handle_outliers`](https://forgejo.ellis.link/continuwuation/continuwuity/src/commit/95610499c7df2d6d1ebe16650431673369975a54/src/service/rooms/event_handler/fetch_and_handle_outliers.rs)
Author
Contributor

This entire system needs to be revisited once continuwuity has better support for holes in the local event timeline. Outlier events work here but they're kind of a hack, and the context route is pretty much impossible to implement.

This entire system needs to be revisited once continuwuity has better support for holes in the local event timeline. Outlier events _work_ here but they're kind of a hack, and the context route is pretty much impossible to implement.
ginger changed title from WIP: Ask remote servers for individual unknown events, take 2 to Ask remote servers for individual unknown events, take 2 2025-09-04 15:25:36 +00:00
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Checks / *
Required
Some required checks are missing.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u single-event-backfill:ginger-single-event-backfill
git switch ginger-single-event-backfill
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: continuwuation/continuwuity#976
No description provided.