wip: feat: Implement MSC3030: Jump to date API endpoint #1535

Draft
gamesguru wants to merge 2 commits from gamesguru/continuwuity:guru/feat/impl/msc3030-go-to-time-in-chat into main
Contributor

May cause anachronisms in incoming events due to modifications in timestamp measuring, and consequently the write logic (if a bug exists causing miscalculation of incoming times). Should not impact previous events (relatively safe in that regard, I hope).

Adds a column family (index) to the DB and an admin command to index an individual room's messages by timestamp.
The column can be dropped, but is harmless either way.
You cannot necessarily undo the indexing, however.

Having admins manually request per-room indexing avoids the potentially massive overhead of a one-time startup migration.
The algorithm should be safe enough to not cause CPU spikes, even against non-indexed rooms.

Suggested improvements or additional test cases are always welcome.


Commit message log summary:

fix: propagate errors, fix bounds updating

fix for backfilled rooms

fix potential bug of "not found" truncating pdu list, potentially to zero

prevent critical database errors from being swallowed in loop

chore: use simplified ruma syntax via helper method

feat: admin command to index a room's messages by timestamp, command to list outliers

This pull request...

Pull request checklist:

  • This pull request targets the main branch, and the branch is named something other than
    main.
  • I have written an appropriate pull request title and my description is clear.
  • I understand I am responsible for the contents of this pull request.
  • I have followed the contributing guidelines:
May cause anachronisms in incoming events due to modifications in timestamp measuring, and consequently the write logic (if a bug exists causing miscalculation of incoming times). Should not impact previous events (relatively safe in that regard, I hope). Adds a column family (index) to the DB and an admin command to index an individual room's messages by timestamp. The column can be dropped, but is harmless either way. You cannot necessarily undo the indexing, however. Having admins manually request per-room indexing avoids the potentially massive overhead of a one-time startup migration. The algorithm should be safe enough to not cause CPU spikes, even against non-indexed rooms. Suggested improvements or additional test cases are always welcome. ---------------------------------------------------- **Commit message log summary:** ``` fix: propagate errors, fix bounds updating fix for backfilled rooms fix potential bug of "not found" truncating pdu list, potentially to zero prevent critical database errors from being swallowed in loop chore: use simplified ruma syntax via helper method feat: admin command to index a room's messages by timestamp, command to list outliers ``` <!-- In order to help reviewers know what your pull request does at a glance, you should ensure that 1. Your PR title is a short, single sentence describing what you changed 2. You have described in more detail what you have changed, why you have changed it, what the intended effect is, and why you think this will be beneficial to the project. If you have made any potentially strange/questionable design choices, but didn't feel they'd benefit from code comments, please don't mention them here - after opening your pull request, go to "files changed", and click on the "+" symbol in the line number gutter, and attach comments to the lines that you think would benefit from some clarification. --> This pull request... <!-- Example: This pull request allows us to warp through time and space ten times faster than before by double-inverting the warp drive with hyperheated jump fluid, both making the drive faster and more efficient. This resolves the common issue where we have to wait more than 10 milliseconds to engage, use, and disengage the warp drive when travelling between galaxies. --> <!-- Closes: #... --> <!-- Fixes: #... --> <!-- Uncomment the above line(s) if your pull request fixes an issue or closes another pull request by superseding it. Replace `#...` with the issue/pr number, such as `#123`. --> **Pull request checklist:** <!-- You need to complete these before your PR can be considered. If you aren't sure about some, feel free to ask for clarification in #dev:continuwuity.org. --> - [x] This pull request targets the `main` branch, and the branch is named something other than `main`. - [x] I have written an appropriate pull request title and my description is clear. - [x] I understand I am responsible for the contents of this pull request. - I have followed the [contributing guidelines][c1]: - [x] My contribution follows the [code style][c2], if applicable. - [x] I ran [pre-commit checks][c1pc] before opening/drafting this pull request. - [ ] I have [tested my contribution][c1t] (or proof-read it for documentation-only changes) myself, if applicable. This includes ensuring code compiles. - [x] My commit messages follow the [commit message format][c1cm] and are descriptive. - [ ] I have written a [news fragment][n1] for this PR, if applicable<!--(can be done after hitting open!)-->. <!-- Notes on these requirements: - While not required, we encourage you to sign your commits with GPG or SSH to attest the authenticity of your changes. - While we allow LLM-assisted contributions, we do not appreciate contributions that are low quality, which is typical of machine-generated contributions that have not had a lot of love and care from a human. Please do not open a PR if all you have done is asked ChatGPT to tidy up the codebase with a +-100,000 diff. - In the case of code style violations, reviewers may leave review comments/change requests indicating what the ideal change would look like. For example, a reviewer may suggest you lower a log level, or use `match` instead of `if/else` etc. - In the case of code style violations, pre-commit check failures, minor things like typos/spelling errors, and in some cases commit format violations, reviewers may modify your branch directly, typically by making changes and adding a commit. Particularly in the latter case, a reviewer may rebase your commits to squash "spammy" ones (like "fix", "fix", "actually fix"), and reword commit messages that don't satisfy the format. - Pull requests MUST pass the `Checks` CI workflows to be capable of being merged. This can only be bypassed in exceptional circumstances. If your CI flakes, let us know in matrix:r/dev:continuwuity.org. - Pull requests have to be based on the latest `main` commit before being merged. If the main branch changes while you're making your changes, you should make sure you rebase on main before opening a PR. Your branch will be rebased on main before it is merged if it has fallen behind. - We typically only do fast-forward merges, so your entire commit log will be included. Once in main, it's difficult to get out cleanly, so put on your best dress, smile for the cameras! --> [c1]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md [c2]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/docs/development/code_style.mdx [c1pc]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#pre-commit-checks [c1t]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#running-tests-locally [c1cm]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#commit-messages [n1]: https://towncrier.readthedocs.io/en/stable/tutorial.html#creating-news-fragments
feat: impl msc3030 room endpoint goto-time via index seek algo
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
dcb02d0012
May cause anachronisms in incoming events due to modifications in timestamp measuring, and consequently the write logic (if a bug exists causing miscalculation of incoming times). Should not impact previous events (relatively safe in that regard, I hope).

Adds a column family (index) to the DB and an admin command to index an individual room's messages by timestamp.
The column can be dropped, but is harmless either way.
You cannot necessarily undo the indexing, however.

Having admins manually request per-room indexing avoids the potentially massive overhead of a one-time startup migration.
The algorithm should be safe enough to not cause CPU spikes, even against non-indexed rooms.

Suggested improvements or additional test cases are always welcome.

----------------------------------------------------

**Commit message log summary:**

```
fix: propagate errors, fix bounds updating

fix for backfilled rooms

fix potential bug of "not found" truncating pdu list, potentially to zero

prevent critical database errors from being swallowed in loop

chore: use simplified ruma syntax via helper method

feat: admin command to index a room's messages by timestamp, command to list outliers
```
@ -0,0 +52,4 @@
// Look for first visible event
while let Some(res) = stream.next().await {
// If the event is not found, skip it (the question mark is meaningful here)
let pdu = res?;
Owner

That's not ignoring the error, it will return with the error

That's not ignoring the error, it will return with the error
@ -73,3 +73,3 @@
#[inline]
pub(crate) fn deserialize<'a, K, V>(kv: KeyVal<'a>) -> Result<KeyVal<'a, K, V>>
pub fn deserialize<'a, K, V>(kv: KeyVal<'a>) -> Result<KeyVal<'a, K, V>>
Owner

Shouldn't be necessary

Shouldn't be necessary
@ -878,1 +881,4 @@
}
#[admin_command]
pub(super) async fn list_outliers(&self, room_id: OwnedRoomId) -> Result {
Owner

Is this a part of msc3030?

Is this a part of msc3030?
nex changed title from feat: impl msc3030 room endpoint goto-time via index seek algo to feat: Implement MSC3030: Jump to date API endpoint 2026-03-13 21:38:48 +00:00
gamesguru changed title from feat: Implement MSC3030: Jump to date API endpoint to wip: feat: Implement MSC3030: Jump to date API endpoint 2026-03-14 14:53:18 +00:00
Merge branch 'main' into guru/feat/impl/msc3030-go-to-time-in-chat
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
d83be540ec
Some checks are pending
Documentation / Build and Deploy Documentation (pull_request) Blocked by required conditions
Checks / Prek / Pre-commit & Formatting (pull_request) Blocked by required conditions
Required
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Blocked by required conditions
Required
Details
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u guru/feat/impl/msc3030-go-to-time-in-chat:gamesguru-guru/feat/impl/msc3030-go-to-time-in-chat
git switch gamesguru-guru/feat/impl/msc3030-go-to-time-in-chat
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
continuwuation/continuwuity!1535
No description provided.