WIP: feat: Implement /_matrix/client/v3/notifications #1345

Draft
gamesguru wants to merge 1 commit from gamesguru/continuwuity:feat/notification-endpoint-cinny into main
Contributor

This pull request adds an endpoint for Cinny's notification panel. WIP.

Closes: #707

Pull request checklist:

  • This pull request targets the main branch, and the branch is named something other than
    main.
  • I have written an appropriate pull request title and my description is clear.
  • I understand I am responsible for the contents of this pull request.
  • I have followed the contributing guidelines:
<!-- In order to help reviewers know what your pull request does at a glance, you should ensure that 1. Your PR title is a short, single sentence describing what you changed 2. You have described in more detail what you have changed, why you have changed it, what the intended effect is, and why you think this will be beneficial to the project. If you have made any potentially strange/questionable design choices, but didn't feel they'd benefit from code comments, please don't mention them here - after opening your pull request, go to "files changed", and click on the "+" symbol in the line number gutter, and attach comments to the lines that you think would benefit from some clarification. --> This pull request adds an endpoint for Cinny's notification panel. WIP. <!-- Example: This pull request allows us to warp through time and space ten times faster than before by double-inverting the warp drive with hyperheated jump fluid, both making the drive faster and more efficient. This resolves the common issue where we have to wait more than 10 milliseconds to engage, use, and disengage the warp drive when travelling between galaxies. --> <!-- Closes: #... --> Closes: #707 <!-- Fixes: #... --> <!-- Uncomment the above line(s) if your pull request fixes an issue or closes another pull request by superseding it. Replace `#...` with the issue/pr number, such as `#123`. --> **Pull request checklist:** <!-- You need to complete these before your PR can be considered. If you aren't sure about some, feel free to ask for clarification in #dev:continuwuity.org. --> - [x] This pull request targets the `main` branch, and the branch is named something other than `main`. - [x] I have written an appropriate pull request title and my description is clear. - [x] I understand I am responsible for the contents of this pull request. - I have followed the [contributing guidelines][c1]: - [x] My contribution follows the [code style][c2], if applicable. - [x] I ran [pre-commit checks][c1pc] before opening/drafting this pull request. - [x] I have [tested my contribution][c1t] (or proof-read it for documentation-only changes) myself, if applicable. This includes ensuring code compiles. - [x] My commit messages follow the [commit message format][c1cm] and are descriptive. - [x] I have written a [news fragment][n1] for this PR, if applicable<!--(can be done after hitting open!)-->. <!-- Notes on these requirements: - While not required, we encourage you to sign your commits with GPG or SSH to attest the authenticity of your changes. - While we allow LLM-assisted contributions, we do not appreciate contributions that are low quality, which is typical of machine-generated contributions that have not had a lot of love and care from a human. Please do not open a PR if all you have done is asked ChatGPT to tidy up the codebase with a +-100,000 diff. - In the case of code style violations, reviewers may leave review comments/change requests indicating what the ideal change would look like. For example, a reviewer may suggest you lower a log level, or use `match` instead of `if/else` etc. - In the case of code style violations, pre-commit check failures, minor things like typos/spelling errors, and in some cases commit format violations, reviewers may modify your branch directly, typically by making changes and adding a commit. Particularly in the latter case, a reviewer may rebase your commits to squash "spammy" ones (like "fix", "fix", "actually fix"), and reword commit messages that don't satisfy the format. - Pull requests MUST pass the `Checks` CI workflows to be capable of being merged. This can only be bypassed in exceptional circumstances. If your CI flakes, let us know in matrix:r/dev:continuwuity.org. - Pull requests have to be based on the latest `main` commit before being merged. If the main branch changes while you're making your changes, you should make sure you rebase on main before opening a PR. Your branch will be rebased on main before it is merged if it has fallen behind. - We typically only do fast-forward merges, so your entire commit log will be included. Once in main, it's difficult to get out cleanly, so put on your best dress, smile for the cameras! --> [c1]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md [c2]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/docs/development/code_style.mdx [c1pc]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#pre-commit-checks [c1t]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#running-tests-locally [c1cm]: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/CONTRIBUTING.md#commit-messages [n1]: https://towncrier.readthedocs.io/en/stable/tutorial.html#creating-news-fragments
nex added this to the 0.5.5 milestone 2026-02-10 03:14:55 +00:00
nex requested review from nex 2026-02-10 03:15:00 +00:00
nex requested changes 2026-02-10 03:18:02 +00:00
nex left a comment
Owner

Looks good, thanks for opening this!

Looks good, thanks for opening this!
@ -0,0 +28,4 @@
) -> Result<get_notifications::v3::Response> {
// Extract the `limit` and `from` query parameters
let limit = body.limit.unwrap_or(uint!(10));
let _from = body.from.as_deref();
Owner

Does this need declaring when it's unused?

Does this need declaring when it's unused?
gamesguru marked this conversation as resolved
@ -0,0 +39,4 @@
let mut rooms_stream = std::pin::pin!(services.rooms.user.stream_notification_counts(sender_user));
while let Some((room_id, count)) = rooms_stream.next().await {
let Ok(room_id) = room_id else { continue };
Owner

Should probably warn!() if there's an error encountered here rather than silently swallowing it

Should probably `warn!()` if there's an error encountered here rather than silently swallowing it
gamesguru marked this conversation as resolved
nex changed title from feat/notification-endpoint-cinny to feat: Implement /_matrix/client/v3/notifications 2026-02-10 03:18:12 +00:00
@ -0,0 +21,4 @@
///
/// Get notifications for the user.
///
/// Currently just returns an empty response.
Owner

doesn't look like an empty response to me

doesn't look like an empty response to me
gamesguru marked this conversation as resolved
@ -0,0 +30,4 @@
let limit = body.limit.unwrap_or(uint!(10));
let _from = body.from.as_deref();
let sender_user = body.sender_user.as_ref().expect("user is authenticated");
Owner

this can be replaced with a call to body.sender_user()

this can be replaced with a call to `body.sender_user()`
gamesguru marked this conversation as resolved
@ -0,0 +127,4 @@
// Sort by timestamp descending (newest first)
notifications.sort_by(|a, b| b.ts.cmp(&a.ts));
// Apply limit
Owner

it would probably be more efficient to use take() on the pdus() stream instead of limiting them here

it would probably be more efficient to use `take()` on the `pdus()` stream instead of limiting them here
Author
Contributor

please see latest implementation and fully critique. Config changes and git history will be cleaned up pending general approval but comments here are welcome as well

please see latest implementation and fully critique. Config changes and git history will be cleaned up pending general approval but comments here are welcome as well
gamesguru marked this conversation as resolved
@ -61,0 +70,4 @@
.stream_prefix::<(OwnedUserId, OwnedRoomId), u64, _>(&prefix)
.map(|res| match res {
| Ok(((_user_id, room_id), count)) => (Ok(room_id), count),
| Err(e) => (Err(e), 0),
Author
Contributor

@nex Would you like a info/warn or debug at least here, too? Thoughts?

@nex Would you like a info/warn or debug at least here, too? Thoughts?
Owner

A warn for the Err arm is probably a good idea, since afaik errors aren't an expected operating condition here

A warn for the Err arm is probably a good idea, since afaik errors aren't an expected operating condition here
gamesguru marked this conversation as resolved
Author
Contributor

uh oh, "Complete Job" has been running ~15 minutes on the "prek / clippy and cargo" check. I'm unsure if I've broken something, or not?

uh oh, "Complete Job" has been running ~15 minutes on the "prek / clippy and cargo" check. I'm unsure if I've broken something, or not?
Owner

Don't worry too much about CI, sometimes it takes a little while to clean up. Hopefully we'll get an upgrade in the next couple months.

Don't worry too much about CI, sometimes it takes a little while to clean up. Hopefully we'll get an upgrade in the next couple months.
Author
Contributor

@Jade wrote in #1345 (comment):

Don't worry too much about CI, sometimes it takes a little while to clean up. Hopefully we'll get an upgrade in the next couple months.

Everything was fast imo, except that last step, "Complete job." Not sure what it's doing

@Jade wrote in https://forgejo.ellis.link/continuwuation/continuwuity/pulls/1345#issuecomment-23700: > Don't worry too much about CI, sometimes it takes a little while to clean up. Hopefully we'll get an upgrade in the next couple months. Everything was fast imo, except that last step, "Complete job." Not sure what it's doing
Owner

it just does that sometimes

it just does that sometimes
nex requested review from nex 2026-02-11 18:53:02 +00:00
Author
Contributor

[C10y] Video Test of notification paging & mark as read
https://imgur.com/a/wiTHdDP#

_[C10y] Video Test of notification paging & mark as read_ https://imgur.com/a/wiTHdDP# <video src="https://i.imgur.com/r5XoHud.mp4" controls width="100%"></video>
Author
Contributor

ok, i think this is good on my end. Ready to be merged in whenever.

there is a small initial delay in my video/stress test, but I think that is due to Cinny and the UI. The /notification network calls all return in under 100 ms. Seeing as my test involved over 1,500 notifications (close to 100 pages), some lag is expected.

i'm not sure what a "news fragment is" or if my PR summary counts? I may need some guidance there.

ok, i think this is good on my end. Ready to be merged in whenever. there is a small initial delay in my video/stress test, but I think that is due to Cinny and the UI. The `/notification` network calls all return in under 100 ms. **Seeing as my test involved over 1,500 notifications (close to 100 pages), some lag is expected.** i'm not sure what a "news fragment is" or if my PR summary counts? I may need some guidance there.
Author
Contributor

There's a runaway memory leak.

I'm assuming it's due to my branch. STOPSHIP

There's a runaway memory leak. I'm assuming it's due to my branch. STOPSHIP
Author
Contributor

image

there's also this bug poking its head around this branch..., You might have to re-dismiss every chat, since now everything is a new notification for some reason.

![image](/attachments/6f45f58b-d3dd-4eb4-8081-2f356d5418f0) there's also this bug poking its head around this branch..., You might have to re-dismiss every chat, since now everything is a new notification for some reason.
Cargo.toml Outdated
@ -644,6 +644,15 @@ path = "src/main"
strip = "symbols"
lto = "thin"
# Hybrid dev/release: "fast" here refers to compile times, not runtime speed
Owner

This should be part of a separate PR.

This should be part of a separate PR.
gamesguru marked this conversation as resolved
@ -85,3 +80,1 @@
// if let Some(ref_path) = run_git_command(&["symbolic-ref", "--quiet", "HEAD"])
// { println!("cargo:rerun-if-changed=.git/{ref_path}");
// }
// Rerun if the git HEAD changes
Owner

These should be part of a separate PR, if they're now working correctly.

These should be part of a separate PR, if they're now working correctly.
gamesguru marked this conversation as resolved
@ -121,3 +121,1 @@
let last_state = self.services.state.summary_stripped(pdu, room_id).await;
self.mark_as_invited(user_id, room_id, pdu.sender(), Some(last_state), None)
.await?;
let sender = pdu.sender();
Owner

What is the purpose of this change and the change to mark_as_invited?

What is the purpose of this change and the change to `mark_as_invited`?
Author
Contributor

unrelated change. speculative fix moved to another branch

unrelated change. speculative fix moved to another branch
gamesguru marked this conversation as resolved
ginger force-pushed feat/notification-endpoint-cinny from b2f0331c4b
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Update flake hashes / update-flake-hashes (pull_request) Successful in 18s
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 2m39s
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 19m37s
to 703f5dbae7
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 4m8s
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 9m3s
Update flake hashes / update-flake-hashes (pull_request) Successful in 33s
2026-02-14 19:34:44 +00:00
Compare
Owner

@gamesguru wrote in #1345 (comment):

There's a runaway memory leak.

I'm assuming it's due to my branch. STOPSHIP

For future reference, if you mark your PR as WIP, we won't review it (unless you ask us to) or consider it for merge. You can always mark it as WIP after opening it by editing the title and prepending WIP:

@gamesguru wrote in https://forgejo.ellis.link/continuwuation/continuwuity/pulls/1345#issuecomment-23959: > There's a runaway memory leak. > > I'm assuming it's due to my branch. STOPSHIP For future reference, if you mark your PR as WIP, we won't review it (unless you ask us to) or consider it for merge. You can always mark it as WIP after opening it by editing the title and prepending `WIP: `
nex removed this from the 0.5.5 milestone 2026-02-14 19:44:27 +00:00
Contributor

@gamesguru wrote in #1345 (comment):

image

there's also this bug poking its head around this branch..., You might have to re-dismiss every chat, since now everything is a new notification for some reason.

I have had that on server restarts sometimes, not on this branch.

@gamesguru wrote in https://forgejo.ellis.link/continuwuation/continuwuity/pulls/1345#issuecomment-24046: > [![image](/attachments/6f45f58b-d3dd-4eb4-8081-2f356d5418f0)](/continuwuation/continuwuity/attachments/6f45f58b-d3dd-4eb4-8081-2f356d5418f0) > > there's also this bug poking its head around this branch..., You might have to re-dismiss every chat, since now everything is a new notification for some reason. I have had that on server restarts sometimes, not on this branch.
gamesguru changed title from feat: Implement /_matrix/client/v3/notifications to WIP: feat: Implement /_matrix/client/v3/notifications 2026-02-14 22:19:50 +00:00
Author
Contributor

@Henry-Hiles wrote in #1345 (comment):

@gamesguru wrote in #1345 (comment):

image
there's also this bug poking its head around this branch..., You might have to re-dismiss every chat, since now everything is a new notification for some reason.

I have had that on server restarts sometimes, not on this branch.

yeah, just happened to me again on the 0.5.4 (196...) release. Is there an open bug for it i wonder

@Henry-Hiles wrote in https://forgejo.ellis.link/continuwuation/continuwuity/pulls/1345#issuecomment-24099: > @gamesguru wrote in #1345 (comment): > > > [![image](/attachments/6f45f58b-d3dd-4eb4-8081-2f356d5418f0)](/continuwuation/continuwuity/attachments/6f45f58b-d3dd-4eb4-8081-2f356d5418f0) > > there's also this bug poking its head around this branch..., You might have to re-dismiss every chat, since now everything is a new notification for some reason. > > I have had that on server restarts sometimes, not on this branch. yeah, just happened to me again on the `0.5.4 (196...)` release. Is there an open bug for it i wonder
Author
Contributor

The memory leak fwiw is no joke. After 24 hours, it had virtually seized up. A message took 5 minutes to send.

I logged into my VPS and had to rub the crush out of my eyes.

... 17 gigs deep in the swap space? I thought i only had 2 Gigs of swap, what?

My initial hypothesis is a memory leak exists in master/main but was made far worse either by greatly increased database or federation dns resolver activity.

Currently I'm looking into briefly diagnosing the memory leak, addressing unnecessary network activity on this branch, and potentially drawing a compromise (query only first 24 notifications per user) to avoid any massive unexpected overhead while simultaneously shipping a relatively important feature.

The memory leak fwiw is no joke. After 24 hours, it had virtually seized up. A message took 5 minutes to send. I logged into my VPS and had to rub the crush out of my eyes. ... 17 gigs deep in the swap space? I thought i only had 2 Gigs of swap, what? My initial hypothesis is a memory leak exists in master/main but was made far worse either by greatly increased database or federation dns resolver activity. Currently I'm looking into briefly diagnosing the memory leak, addressing unnecessary network activity on this branch, and potentially drawing a compromise (query only first 24 notifications per user) to avoid any massive unexpected overhead while simultaneously shipping a relatively important feature.
gamesguru force-pushed feat/notification-endpoint-cinny from 703f5dbae7
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 4m8s
Checks / Prek / Clippy and Cargo Tests (pull_request) Failing after 9m3s
Update flake hashes / update-flake-hashes (pull_request) Successful in 33s
to 51f7a2bb77
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
2026-02-15 04:42:05 +00:00
Compare
Owner

FWIW there may be a leak in existing code but it does not seem to be triggered in main:
image

FWIW there may be a leak in existing code but it does not seem to be triggered in main: ![image](/attachments/da07f8fb-e371-4936-b18b-219299c0ed53)
154 KiB
gamesguru force-pushed feat/notification-endpoint-cinny from 51f7a2bb77
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
Update flake hashes / update-flake-hashes (pull_request) Has been cancelled
to 5dc749d2c1
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
2026-02-24 03:37:28 +00:00
Compare
Author
Contributor

Ok. Seems to be healthy after stress testing. Seems a heap was appropriate here. Now I'll add a news fragment and remove the noisy info log.

here's a log of the initial query vs. the cached run times, as well as a depth probe to ~4000 notifications showing only minimal attrition.

built notification heap: 24 items for ... in 15.882s (used 3264 bytes, scanned 55 PDUs in 13 rooms)
built notification heap: 24 items for ... in 27.095s (used 3264 bytes, scanned 55 PDUs in 13 rooms)
built notification heap: 24 items for ... in 20.096s (used 3264 bytes, scanned 55 PDUs in 13 rooms)
built notification heap: 24 items for ... in 1.884s (used 3264 bytes, scanned 55 PDUs in 13 rooms)
built notification heap: 24 items for ... in 8.902s (used 3264 bytes, scanned 55 PDUs in 13 rooms)
built notification heap: 24 items for ... in 15.863s (used 3264 bytes, scanned 55 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 77 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 55 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 79 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 79 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.041s (used 3264 bytes, scanned 116 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.042s (used 3264 bytes, scanned 175 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.009s (used 3264 bytes, scanned 211 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.036s (used 3264 bytes, scanned 237 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.067s (used 3264 bytes, scanned 265 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.051s (used 3264 bytes, scanned 292 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.008s (used 3264 bytes, scanned 320 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 345 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.018s (used 3264 bytes, scanned 370 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.007s (used 3264 bytes, scanned 392 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.006s (used 3264 bytes, scanned 407 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.037s (used 3264 bytes, scanned 447 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.027s (used 3264 bytes, scanned 465 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.039s (used 3264 bytes, scanned 495 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 519 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.035s (used 3264 bytes, scanned 545 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.031s (used 3264 bytes, scanned 571 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.006s (used 3264 bytes, scanned 591 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.007s (used 3264 bytes, scanned 615 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 641 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.055s (used 3264 bytes, scanned 671 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.010s (used 3264 bytes, scanned 710 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.076s (used 3264 bytes, scanned 780 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.038s (used 3264 bytes, scanned 827 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.009s (used 3264 bytes, scanned 844 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 877 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 911 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.052s (used 3264 bytes, scanned 953 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.053s (used 3264 bytes, scanned 995 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.031s (used 3264 bytes, scanned 1013 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.043s (used 3264 bytes, scanned 1030 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 1078 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 1103 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.077s (used 3264 bytes, scanned 1152 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.097s (used 3264 bytes, scanned 1201 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.042s (used 3264 bytes, scanned 1227 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.065s (used 3264 bytes, scanned 1253 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.027s (used 3264 bytes, scanned 1269 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.112s (used 3264 bytes, scanned 1326 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.088s (used 3264 bytes, scanned 1370 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.040s (used 3264 bytes, scanned 1394 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.059s (used 3264 bytes, scanned 1423 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.408s (used 3264 bytes, scanned 1621 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.022s (used 3264 bytes, scanned 1631 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.135s (used 3264 bytes, scanned 1679 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.112s (used 3264 bytes, scanned 1712 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.056s (used 3264 bytes, scanned 1740 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.038s (used 3264 bytes, scanned 1764 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 1791 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.051s (used 3264 bytes, scanned 1819 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.058s (used 3264 bytes, scanned 1856 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.043s (used 3264 bytes, scanned 1880 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.067s (used 3264 bytes, scanned 1907 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.014s (used 3264 bytes, scanned 1934 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.081s (used 3264 bytes, scanned 1964 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.063s (used 3264 bytes, scanned 1992 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.025s (used 3264 bytes, scanned 2026 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.074s (used 3264 bytes, scanned 2059 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.022s (used 3264 bytes, scanned 2087 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.026s (used 3264 bytes, scanned 2104 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.122s (used 3264 bytes, scanned 2160 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.026s (used 3264 bytes, scanned 2179 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.058s (used 3264 bytes, scanned 2194 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.034s (used 3264 bytes, scanned 2217 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.035s (used 3264 bytes, scanned 2242 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.087s (used 3264 bytes, scanned 2272 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.017s (used 3264 bytes, scanned 2289 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.026s (used 3264 bytes, scanned 2316 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.016s (used 3264 bytes, scanned 2342 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.037s (used 3264 bytes, scanned 2380 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.055s (used 3264 bytes, scanned 2418 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.111s (used 3264 bytes, scanned 2462 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.040s (used 3264 bytes, scanned 2497 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.022s (used 3264 bytes, scanned 2511 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.046s (used 3264 bytes, scanned 2543 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.058s (used 3264 bytes, scanned 2567 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.047s (used 3264 bytes, scanned 2609 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.044s (used 3264 bytes, scanned 2646 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.130s (used 3264 bytes, scanned 2671 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.062s (used 3264 bytes, scanned 2704 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.034s (used 3264 bytes, scanned 2751 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.151s (used 3264 bytes, scanned 2803 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.127s (used 3264 bytes, scanned 2831 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.090s (used 3264 bytes, scanned 2869 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.155s (used 3264 bytes, scanned 2912 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.070s (used 3264 bytes, scanned 2956 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.022s (used 3264 bytes, scanned 2960 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.067s (used 3264 bytes, scanned 2979 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.159s (used 3264 bytes, scanned 3018 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.039s (used 3264 bytes, scanned 3047 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.069s (used 3264 bytes, scanned 3094 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.056s (used 3264 bytes, scanned 3118 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.060s (used 3264 bytes, scanned 3148 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.045s (used 3264 bytes, scanned 3174 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.177s (used 3264 bytes, scanned 3213 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.028s (used 3264 bytes, scanned 3245 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.046s (used 3264 bytes, scanned 3263 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.102s (used 3264 bytes, scanned 3295 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.057s (used 3264 bytes, scanned 3319 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.048s (used 3264 bytes, scanned 3346 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.075s (used 3264 bytes, scanned 3374 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.037s (used 3264 bytes, scanned 3409 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.058s (used 3264 bytes, scanned 3474 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.064s (used 3264 bytes, scanned 3504 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.067s (used 3264 bytes, scanned 3530 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.061s (used 3264 bytes, scanned 3572 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.063s (used 3264 bytes, scanned 3597 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.093s (used 3264 bytes, scanned 3627 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.087s (used 3264 bytes, scanned 3669 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.083s (used 3264 bytes, scanned 3694 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.056s (used 3264 bytes, scanned 3722 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.028s (used 3264 bytes, scanned 3728 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.070s (used 3264 bytes, scanned 3761 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.031s (used 3264 bytes, scanned 3791 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.054s (used 3264 bytes, scanned 3823 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.050s (used 3264 bytes, scanned 3851 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.051s (used 3264 bytes, scanned 3881 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.029s (used 3264 bytes, scanned 3915 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.046s (used 3264 bytes, scanned 3939 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.027s (used 3264 bytes, scanned 3958 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.028s (used 3264 bytes, scanned 3985 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 4011 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.146s (used 3264 bytes, scanned 4052 PDUs in 13 rooms)
built notification heap: 24 items for ... in 0.077s (used 3264 bytes, scanned 4088 PDUs in 13 rooms)
Ok. Seems to be healthy after stress testing. Seems a heap was appropriate here. Now I'll add a news fragment and remove the noisy info log. here's a log of the initial query vs. the cached run times, as well as a depth probe to ~4000 notifications showing only minimal attrition. ```log built notification heap: 24 items for ... in 15.882s (used 3264 bytes, scanned 55 PDUs in 13 rooms) built notification heap: 24 items for ... in 27.095s (used 3264 bytes, scanned 55 PDUs in 13 rooms) built notification heap: 24 items for ... in 20.096s (used 3264 bytes, scanned 55 PDUs in 13 rooms) built notification heap: 24 items for ... in 1.884s (used 3264 bytes, scanned 55 PDUs in 13 rooms) built notification heap: 24 items for ... in 8.902s (used 3264 bytes, scanned 55 PDUs in 13 rooms) built notification heap: 24 items for ... in 15.863s (used 3264 bytes, scanned 55 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 77 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 55 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 79 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 53 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.004s (used 3264 bytes, scanned 79 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.041s (used 3264 bytes, scanned 116 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.042s (used 3264 bytes, scanned 175 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.009s (used 3264 bytes, scanned 211 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.036s (used 3264 bytes, scanned 237 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.067s (used 3264 bytes, scanned 265 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.051s (used 3264 bytes, scanned 292 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.008s (used 3264 bytes, scanned 320 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 345 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.018s (used 3264 bytes, scanned 370 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.007s (used 3264 bytes, scanned 392 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.006s (used 3264 bytes, scanned 407 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.037s (used 3264 bytes, scanned 447 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.027s (used 3264 bytes, scanned 465 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.039s (used 3264 bytes, scanned 495 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 519 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.035s (used 3264 bytes, scanned 545 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.031s (used 3264 bytes, scanned 571 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.006s (used 3264 bytes, scanned 591 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.007s (used 3264 bytes, scanned 615 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 641 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.055s (used 3264 bytes, scanned 671 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.010s (used 3264 bytes, scanned 710 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.076s (used 3264 bytes, scanned 780 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.038s (used 3264 bytes, scanned 827 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.009s (used 3264 bytes, scanned 844 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 877 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 911 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.052s (used 3264 bytes, scanned 953 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.053s (used 3264 bytes, scanned 995 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.031s (used 3264 bytes, scanned 1013 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.043s (used 3264 bytes, scanned 1030 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 1078 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 1103 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.077s (used 3264 bytes, scanned 1152 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.097s (used 3264 bytes, scanned 1201 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.042s (used 3264 bytes, scanned 1227 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.065s (used 3264 bytes, scanned 1253 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.027s (used 3264 bytes, scanned 1269 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.112s (used 3264 bytes, scanned 1326 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.088s (used 3264 bytes, scanned 1370 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.040s (used 3264 bytes, scanned 1394 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.059s (used 3264 bytes, scanned 1423 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.408s (used 3264 bytes, scanned 1621 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.022s (used 3264 bytes, scanned 1631 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.135s (used 3264 bytes, scanned 1679 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.112s (used 3264 bytes, scanned 1712 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.056s (used 3264 bytes, scanned 1740 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.038s (used 3264 bytes, scanned 1764 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.015s (used 3264 bytes, scanned 1791 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.051s (used 3264 bytes, scanned 1819 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.058s (used 3264 bytes, scanned 1856 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.043s (used 3264 bytes, scanned 1880 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.067s (used 3264 bytes, scanned 1907 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.014s (used 3264 bytes, scanned 1934 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.081s (used 3264 bytes, scanned 1964 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.063s (used 3264 bytes, scanned 1992 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.025s (used 3264 bytes, scanned 2026 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.074s (used 3264 bytes, scanned 2059 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.022s (used 3264 bytes, scanned 2087 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.026s (used 3264 bytes, scanned 2104 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.122s (used 3264 bytes, scanned 2160 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.026s (used 3264 bytes, scanned 2179 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.058s (used 3264 bytes, scanned 2194 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.034s (used 3264 bytes, scanned 2217 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.035s (used 3264 bytes, scanned 2242 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.087s (used 3264 bytes, scanned 2272 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.017s (used 3264 bytes, scanned 2289 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.026s (used 3264 bytes, scanned 2316 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.016s (used 3264 bytes, scanned 2342 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.037s (used 3264 bytes, scanned 2380 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.055s (used 3264 bytes, scanned 2418 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.111s (used 3264 bytes, scanned 2462 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.040s (used 3264 bytes, scanned 2497 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.022s (used 3264 bytes, scanned 2511 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.046s (used 3264 bytes, scanned 2543 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.058s (used 3264 bytes, scanned 2567 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.047s (used 3264 bytes, scanned 2609 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.044s (used 3264 bytes, scanned 2646 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.130s (used 3264 bytes, scanned 2671 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.062s (used 3264 bytes, scanned 2704 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.034s (used 3264 bytes, scanned 2751 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.151s (used 3264 bytes, scanned 2803 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.127s (used 3264 bytes, scanned 2831 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.090s (used 3264 bytes, scanned 2869 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.155s (used 3264 bytes, scanned 2912 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.070s (used 3264 bytes, scanned 2956 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.022s (used 3264 bytes, scanned 2960 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.067s (used 3264 bytes, scanned 2979 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.159s (used 3264 bytes, scanned 3018 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.039s (used 3264 bytes, scanned 3047 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.069s (used 3264 bytes, scanned 3094 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.056s (used 3264 bytes, scanned 3118 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.060s (used 3264 bytes, scanned 3148 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.045s (used 3264 bytes, scanned 3174 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.177s (used 3264 bytes, scanned 3213 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.028s (used 3264 bytes, scanned 3245 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.046s (used 3264 bytes, scanned 3263 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.102s (used 3264 bytes, scanned 3295 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.057s (used 3264 bytes, scanned 3319 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.048s (used 3264 bytes, scanned 3346 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.075s (used 3264 bytes, scanned 3374 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.037s (used 3264 bytes, scanned 3409 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.058s (used 3264 bytes, scanned 3474 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.064s (used 3264 bytes, scanned 3504 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.067s (used 3264 bytes, scanned 3530 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.061s (used 3264 bytes, scanned 3572 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.063s (used 3264 bytes, scanned 3597 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.093s (used 3264 bytes, scanned 3627 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.087s (used 3264 bytes, scanned 3669 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.083s (used 3264 bytes, scanned 3694 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.056s (used 3264 bytes, scanned 3722 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.028s (used 3264 bytes, scanned 3728 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.070s (used 3264 bytes, scanned 3761 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.031s (used 3264 bytes, scanned 3791 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.054s (used 3264 bytes, scanned 3823 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.050s (used 3264 bytes, scanned 3851 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.051s (used 3264 bytes, scanned 3881 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.029s (used 3264 bytes, scanned 3915 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.046s (used 3264 bytes, scanned 3939 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.027s (used 3264 bytes, scanned 3958 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.028s (used 3264 bytes, scanned 3985 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.021s (used 3264 bytes, scanned 4011 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.146s (used 3264 bytes, scanned 4052 PDUs in 13 rooms) built notification heap: 24 items for ... in 0.077s (used 3264 bytes, scanned 4088 PDUs in 13 rooms) ```
gamesguru force-pushed feat/notification-endpoint-cinny from 5dc749d2c1
Some checks failed
Documentation / Build and Deploy Documentation (pull_request) Has been cancelled
Checks / Prek / Pre-commit & Formatting (pull_request) Has been cancelled
Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled
to c0865b42dc
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 17m12s
Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 53m32s
2026-02-24 03:55:37 +00:00
Compare
gamesguru changed title from WIP: feat: Implement /_matrix/client/v3/notifications to feat: Implement /_matrix/client/v3/notifications 2026-02-24 03:56:16 +00:00
Contributor

small bystander nitpick here: force pushing makes it difficult to follow the changes that were made.

small bystander nitpick here: force pushing makes it difficult to follow the changes that were made.
Owner

You can compare between force pushes, squashing to remove pointless commits is good. Forgejo not bbeing good at seperating out the rebased branch from changes in force pushes is something that can be improved, but it's not really a git issue

You can compare between force pushes, squashing to remove pointless commits is good. Forgejo not bbeing good at seperating out the rebased branch from changes in force pushes is something that can be improved, but it's not really a git issue
Author
Contributor

As far as the implementation, I'm not sure how well it scales to larger servers. I can only say I tested it with a 1-person server, although up to ~10-15K notifications/unread messages.

I'm not fully sold. This PR might actually need a 3rd evolution. Perhaps, we are better off just making another table/column in the database to store notifications? Thoughts? Preferences?

As far as the implementation, I'm not sure how well it scales to larger servers. I can only say I tested it with a 1-person server, although up to ~10-15K notifications/unread messages. I'm not fully sold. This PR might actually need a 3rd evolution. Perhaps, we are better off just making another table/column in the database to store notifications? Thoughts? Preferences?
Contributor

considering how well implementing sync tokens to persist in the db worked, im not convinced that's a good idea.
then again; you got to store notifications somewhere, and in-memory they'd get lost on restarts, so i suppose a new column family for notifications would be a good idea. maybe delete notifications from the table once they've been marked as read?

considering how well implementing sync tokens to persist in the db worked, im not convinced that's a good idea. then again; you got to store notifications *somewhere*, and in-memory they'd get lost on restarts, so i suppose a new column family for notifications would be a good idea. maybe delete notifications from the table once they've been marked as read?
Owner

I mean the main issue with this is that it's quite big and quite a lot to review. It also has some non-related things that have made their way in (src/service/sending/stats.rs). From what I can see it's roughly the right approach. Adding a new table would probably be best for performance but comes with a few issues that'd have to be solved, so I'd rather not do that yet (for example, cleanup when room access changes, and cleanup of old notifications).

If this was broken down into smaller changes, that would be much easier to review and merge.

I mean the main issue with this is that it's quite big and quite a lot to review. It also has some non-related things that have made their way in (`src/service/sending/stats.rs`). From what I can see it's roughly the right approach. Adding a new table would probably be best for performance but comes with a few issues that'd have to be solved, so I'd rather not do that yet (for example, cleanup when room access changes, and cleanup of old notifications). If this was broken down into smaller changes, that would be much easier to review and merge.
Author
Contributor

@rooot You're exactly right, the performance concerns may be significant. You can see multiple cold calls taking on the order of 1 to 10 seconds before the cache is warm. Might that completely lock up a server with 200 users? Not sure. At least if it did, they should hopefully be able to configure lower limits.

But actually "synapse"-spec notifications are far less common. You might have 10,000 unread messages and only 100 or 500 meaningful notifications. My branch is was more verbose. It just gives you all 10,000 or however many unread messages your push rules and read receipts dictate.

This branch is probably cheaper in terms of developer effort, but more expensive in terms of CPU usage, than a table-based implementation.

I do not think the comparison to sync state tokens is necessarily accurate. I hope it won't take up anywhere near that much disc space, but I more or less agree with Jade that the current implementation on this branch is adequate (pending rigorous time/space complexity analysis).

@Jade wrote in #1345 (comment):

I mean the main issue with this is that it's quite big and quite a lot to review. It also has some non-related things that have made their way in (src/service/sending/stats.rs). From what I can see it's roughly the right approach. Adding a new table would probably be best for performance but comes with a few issues that'd have to be solved, so I'd rather not do that yet (for example, cleanup when room access changes, and cleanup of old notifications).

If this was broken down into smaller changes, that would be much easier to review and merge.

Hi Jade, I tried my best to not have any scope creep. But the metric/logging stuff was relevant given the severe memory leak at one point.

I'm not sure which other "smaller changes" you're referring to. I can try to separate some out, and perhaps we can compromise on some? If it satisfies a legitimate logging need, doesn't leak anything potentially secret to the console, and supports (perhaps slightly indirectly) the development efforts at hand?

@rooot You're exactly right, the performance concerns may be significant. You can see multiple cold calls taking on the order of 1 to 10 seconds before the cache is warm. Might that completely lock up a server with 200 users? Not sure. At least if it did, they should hopefully be able to configure lower limits. But actually "synapse"-spec notifications are far less common. You might have 10,000 unread messages and only 100 or 500 meaningful notifications. My branch is was more verbose. It just gives you all 10,000 or however many unread messages your push rules and read receipts dictate. This branch is probably cheaper in terms of developer effort, but more expensive in terms of CPU usage, than a table-based implementation. I do not think the comparison to sync state tokens is necessarily accurate. I hope it won't take up anywhere near that much disc space, but I more or less agree with Jade that the current implementation on this branch is adequate (pending rigorous time/space complexity analysis). @Jade wrote in https://forgejo.ellis.link/continuwuation/continuwuity/pulls/1345#issuecomment-25121: > I mean the main issue with this is that it's quite big and quite a lot to review. It also has some non-related things that have made their way in (`src/service/sending/stats.rs`). From what I can see it's roughly the right approach. Adding a new table would probably be best for performance but comes with a few issues that'd have to be solved, so I'd rather not do that yet (for example, cleanup when room access changes, and cleanup of old notifications). > > If this was broken down into smaller changes, that would be much easier to review and merge. Hi Jade, I tried my best to not have any scope creep. But the metric/logging stuff was relevant given the severe memory leak at one point. I'm not sure which other "smaller changes" you're referring to. I can try to separate some out, and perhaps we can compromise on some? If it satisfies a legitimate logging need, doesn't leak anything potentially secret to the console, and supports (perhaps slightly indirectly) the development efforts at hand?
Jade changed title from feat: Implement /_matrix/client/v3/notifications to WIP: feat: Implement /_matrix/client/v3/notifications 2026-03-03 18:31:30 +00:00
nex requested changes 2026-03-03 18:37:55 +00:00
nex left a comment
Owner

Actual notification impl looks good but there's a lot of scope creep, potential performance issues, and a few (opinionated) codestyle issues

Actual notification impl looks good but there's a lot of scope creep, potential performance issues, and a few (opinionated) codestyle issues
@ -0,0 +3,4 @@
use futures::StreamExt;
use ruma::{
MilliSecondsSinceUnixEpoch, UInt,
api::client::push::{get_notifications, get_notifications::v3 as r},
Owner

r is not a descriptive import name

`r` is not a descriptive import name
@ -0,0 +27,4 @@
// Wrapper to order notifications by timestamp
#[derive(Debug)]
struct NotificationItem(r::Notification);
Owner

It'd be cleaner to define this struct and associated impls outside of the route function

It'd be cleaner to define this struct and associated impls outside of the route function
@ -0,0 +109,4 @@
.is_joined(sender_user, &room_id)
.await
{
continue;
Owner

Might be a good idea to clean up the counts for "stale rooms" here, otherwise this work will keep being repeated for each subsequent call

Might be a good idea to clean up the counts for "stale rooms" here, otherwise this work will keep being repeated for each subsequent call
@ -0,0 +163,4 @@
}
// Skip events sent by the user themselves
if pdu.sender == *sender_user {
Owner

I think some of these if continue/break conditions can be combined

I think some of these `if` `continue/break` conditions can be combined
@ -0,0 +175,4 @@
.get_actions(sender_user, &ruleset, &power_levels, &pdu_raw, &room_id)
.await;
let mut notify = false;
Owner

Any reason for mutability? would actions.iter().any(|v| matches!(v, &Action::Notify)) not suffice?

Any reason for mutability? would `actions.iter().any(|v| matches!(v, &Action::Notify))` not suffice?
@ -0,0 +208,4 @@
}
// Capture heap stats before consuming
let heap_count = notifications.len();
Owner

This probably isn't worth including especially when it's still executed even in release mode despite no relevant log

This probably isn't worth including especially when it's still executed even in release mode despite no relevant log
Author
Contributor

hi, thanks for the detailed feedback. i read all your other comments and am agreed with implementing them. Adding it to my list now.

I agree that the calculation here feels a bit clumsy and awkwardly placed, intruding in the code. iirc size was constant, so not worth the effort.

if depth, time or total count are easier to introspect (i.e., 2 lines or less) i would like to see it as info level on some subservice. Haven't looked into whether the repo supports database-specific log_level or filtering. If not, i can remove this i guess.

hi, thanks for the detailed feedback. i read all your other comments and am agreed with implementing them. Adding it to my list now. I agree that the calculation here feels a bit clumsy and awkwardly placed, intruding in the code. iirc `size` was constant, so not worth the effort. if depth, time or total count are easier to introspect (i.e., 2 lines or less) i would like to see it as info level on some subservice. Haven't looked into whether the repo supports database-specific log_level or filtering. If not, i can remove this i guess.
@ -56,22 +55,69 @@ impl crate::Service for Service {
}
async fn worker(self: Arc<Self>) -> Result<()> {
use std::collections::HashMap;
Owner

I think changes to presence should go in their own PR as they affect different areas

I think changes to presence should go in their own PR as they affect different areas
Author
Contributor

I definitely agree. Not sure how this stuck around here. Think I planned to include it w the server startup optimizations or a related PR.

I definitely agree. Not sure how this stuck around here. Think I planned to include it w the server startup optimizations or a related PR.
@ -119,2 +119,4 @@
},
| MembershipState::Invite => {
// Check invite filter before expensive stripped state computation
let sender = pdu.sender();
Owner

This should go in its own PR as it is unrelated to notifications

This should go in its own PR as it is unrelated to notifications
@ -121,6 +125,19 @@ impl crate::Service for Service {
joinset
});
// Periodic federation stats reporter (tracked via JoinSet so it
Owner

This should be its own PR as this isn't related to notifications

This should be its own PR as this isn't related to notifications
@ -843,6 +848,15 @@ impl Service {
return Ok(Destination::Federation(server));
}
// Track federation stats
Owner

This should be its own PR as this isn't related to notifications

This should be its own PR as this isn't related to notifications
@ -0,0 +5,4 @@
/// Lightweight atomic counters for federation activity.
/// Logged periodically and reset after each report.
#[derive(Default)]
pub struct FederationStats {
Owner

This should be its own PR as this isn't related to notifications

This should be its own PR as this isn't related to notifications
All checks were successful
Documentation / Build and Deploy Documentation (pull_request) Has been skipped
Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 17m12s
Required
Details
Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 53m32s
Required
Details
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u feat/notification-endpoint-cinny:gamesguru-feat/notification-endpoint-cinny
git switch gamesguru-feat/notification-endpoint-cinny
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
6 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
continuwuation/continuwuity!1345
No description provided.