Ask remote servers for individual unknown events, take 2 #976

Merged
nex merged 8 commits from ginger/continuwuity:single-event-backfill into main 2025-09-07 22:23:39 +00:00
Member

From #899:

This change allows us to use /_matrix/federation/v1/get_event/{eventId} to fetch individual events that we may not have. This is similar to backfill, except it only retrieves an event by its ID, rather than fetching history.

This PR also makes use of this by making /_matrix/client/v3/rooms/{roomId}/event/{eventId} endpoint try to fetch the event locally, falling back to a backfill request if it fails. This means users can fetch pins from before they joined, in the event their server doesn't already have them locally.

From #899: > This change allows us to use [`/_matrix/federation/v1/get_event/{eventId}`](https://spec.matrix.org/unstable/server-server-api/#get_matrixfederationv1eventeventid) to fetch individual events that we may not have. This is similar to backfill, except it only retrieves an event by its ID, rather than fetching history. > > This PR also makes use of this by making [`/_matrix/client/v3/rooms/{roomId}/event/{eventId}`](https://spec.matrix.org/unstable/client-server-api/#get_matrixclientv3roomsroomideventeventid) endpoint try to fetch the event locally, falling back to a backfill request if it fails. This means users can fetch pins from before they joined, in the event their server doesn't already have them locally.
feat: Ask remote servers for individual unknown events
All checks were successful
Release Docker Image / define-variables (push) Successful in 12s
Checks / Prefligit / prefligit (push) Successful in 20s
Checks / Rust / Format (push) Successful in 51s
Checks / Rust / Clippy (push) Successful in 3m32s
Checks / Rust / Cargo Test (push) Successful in 4m57s
Checks / Prefligit / prefligit (pull_request) Successful in 33s
Documentation / Build and Deploy Documentation (pull_request) Successful in 47s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (push) Successful in 13m10s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (push) Successful in 13m28s
Release Docker Image / merge (push) Successful in 31s
89e2faaa8e
feat: Move packaging scripts into dist/ and consolidate the service files
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
19c9a896a6
fix: Update debian systemd unit path
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
2e687e0451
fix: Remove unnecessary user and directory modifications
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
c0c772639f
systemd creates a dynamic user for
continuwuity and manages directories for
it automatically, so the debian postinst
script no longer needs to do that.
fix: Move packaging files from dist/ to pkg/
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
e3072c4392
ginger force-pushed single-event-backfill from 39f92f1070
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
to 07d0e672af
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
2025-09-02 19:20:03 +00:00
Compare
ginger force-pushed single-event-backfill from 07d0e672af
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
to 21dd09a6b6
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
2025-09-02 19:20:58 +00:00
Compare
@ -21,2 +3,2 @@
}
}
/// Parse a pagination token
pub(crate) fn pagination_token_to_count(token: &str) -> Result<PduCount> { token.parse() }
Author
Member

these are just wrappers around the functions on the PduCount struct. why even keep this file if these are the only functions in it?

these are just wrappers around the functions on the `PduCount` struct. why even keep this file if these are the only functions in it?
ginger marked this conversation as resolved
@ -135,0 +140,4 @@
if local.is_ok() {
// We already have this PDU, no need to backfill
debug!("We already have {event_id} in {room_id}, no need to backfill.");
return local;
Author
Member

strange that a function with remote in the name can return local PDUs sometimes

strange that a function with `remote` in the name can return local PDUs sometimes
Author
Member

also, this function is similar in implementation and purpose to fetch_and_handle_outliers

also, this function is similar in implementation and purpose to [`fetch_and_handle_outliers`](https://forgejo.ellis.link/continuwuation/continuwuity/src/commit/95610499c7df2d6d1ebe16650431673369975a54/src/service/rooms/event_handler/fetch_and_handle_outliers.rs)
Author
Member

This entire system needs to be revisited once continuwuity has better support for holes in the local event timeline. Outlier events work here but they're kind of a hack, and the context route is pretty much impossible to implement.

This entire system needs to be revisited once continuwuity has better support for holes in the local event timeline. Outlier events _work_ here but they're kind of a hack, and the context route is pretty much impossible to implement.
ginger changed title from WIP: Ask remote servers for individual unknown events, take 2 to Ask remote servers for individual unknown events, take 2 2025-09-04 15:25:36 +00:00
ginger force-pushed single-event-backfill from 8d353c90f3
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
to a82470633e
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Release Docker Image / define-variables (pull_request) Successful in 4s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (pull_request) Has been cancelled
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (pull_request) Has been cancelled
Release Docker Image / merge (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Failing after 24s
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 36s
2025-09-06 14:33:28 +00:00
Compare
Jade approved these changes 2025-09-06 17:43:23 +00:00
ginger force-pushed single-event-backfill from a82470633e
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Release Docker Image / define-variables (pull_request) Successful in 4s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (pull_request) Has been cancelled
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (pull_request) Has been cancelled
Release Docker Image / merge (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Failing after 24s
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 36s
to 8e7ee1e210
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Release Docker Image / define-variables (pull_request) Successful in 1s
Checks / Prek / Pre-commit & Formatting (pull_request) Failing after 21s
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 45s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (pull_request) Failing after 1m24s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (pull_request) Failing after 1m20s
Release Docker Image / merge (pull_request) Has been skipped
2025-09-06 18:06:07 +00:00
Compare
nex left a comment
Owner

Just re-running CI then I think we're good for merge

Just re-running CI then I think we're good for merge
nex approved these changes 2025-09-07 20:50:18 +00:00
nex left a comment
Owner

That was supposed to be an approval

That was supposed to be an approval
nex force-pushed single-event-backfill from 8e7ee1e210
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Release Docker Image / define-variables (pull_request) Successful in 1s
Checks / Prek / Pre-commit & Formatting (pull_request) Failing after 21s
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 45s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (pull_request) Failing after 1m24s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (pull_request) Failing after 1m20s
Release Docker Image / merge (pull_request) Has been skipped
to 350aa040a2
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Release Docker Image / define-variables (pull_request) Successful in 15s
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 44s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (pull_request) Successful in 6m1s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (pull_request) Successful in 6m23s
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 8m27s
Release Docker Image / merge (pull_request) Successful in 15s
2025-09-07 20:55:38 +00:00
Compare
nex force-pushed single-event-backfill from 445b8d030f
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m0s
Release Docker Image / define-variables (pull_request) Successful in 2s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (pull_request) Successful in 5m57s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (pull_request) Successful in 5m38s
Release Docker Image / merge (pull_request) Successful in 4s
Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 11m8s
to 90fd92977e
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m11s
Release Docker Image / define-variables (pull_request) Successful in 8s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (pull_request) Successful in 5m31s
Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 8m28s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (pull_request) Failing after 10m34s
Release Docker Image / merge (pull_request) Has been cancelled
2025-09-07 21:20:27 +00:00
Compare
fix: Make RA use the full feature
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Release Docker Image / define-variables (pull_request) Successful in 20s
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m9s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (pull_request) Successful in 9m13s
Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 10m24s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (pull_request) Successful in 5m3s
Release Docker Image / merge (pull_request) Successful in 3s
Documentation / Build and Deploy Documentation (push) Successful in 28s
Release Docker Image / define-variables (push) Successful in 14s
Checks / Prek / Pre-commit & Formatting (push) Successful in 42s
Checks / Prek / Clippy and Cargo Tests (push) Successful in 4m12s
Release Docker Image / build-image (linux/amd64, release, linux-amd64, base) (push) Successful in 6m11s
Release Docker Image / build-image (linux/arm64, release, linux-arm64, base) (push) Successful in 6m25s
Release Docker Image / merge (push) Successful in 9s
5d3e10a048
nex merged commit 5d3e10a048 into main 2025-09-07 22:23:39 +00:00
ginger deleted branch single-event-backfill 2025-09-07 22:23:56 +00:00
nex added this to the v0.5.0-rc.8 milestone 2025-09-21 17:12:28 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
continuwuation/continuwuity!976
No description provided.