perf: Continuwuity does not attempt to automatically squash room extremities #1844

New issue

Closed

opened 2026-06-15 14:35:30 +00:00 by nex · 1 comment

nex commented

2026-06-15 14:35:30 +00:00

Owner

Extremity buildup is an expected condition when network partitions are involved, especially in heavily federated rooms like the continuwuity ones, where typically there's 500+ homeservers in the room at any given time, and >50% of the room's population is decentralised (i.e. not on the same server). Synapse has a behaviour to deal with this: dummy events.

A quick database lookup revealed that three privileged Synapse deployments were already occasionally sending dummy events, however since the rest of the room's members aren't allowed to send dummy events, they were not, and were simply aggregating more and more extremities over time, which may never decrease if the server doesn't have an active member sending events.

Conduwuit consciously did not implement any sort of dummy event sending, from memory this was from a misguided belief that they were a crutch for Synapse performance issues, but I've now got evidence to show that they may actually help us in healing some partitioned graphs that end up resulting in an elevated level of erroneous soft-fails.

Theoretically, if we're correctly fetching all missing events after a partition, the number of extremities should naturally reduce back down to near enough one in due time. #1818 helps with this, but is not yet merged nor in a release, and does not appear to resolve the problem completely. As such, I think we should implement some behaviour similar to Synapse's dummy events, to aid in healing partitions and squashing high extremity counts. More investigation is required.

While investigating some strange state divergence, I realised that the continuwuity community rooms were heavily forked between servers, with some servers having 20+ extremities in some of the rooms. This is particularly bad as this means we have a higher tendency to soft-fail actually valid events as they come in if we're retaining lots of extremities. My own server has *thirty five* extremities at the time of writing in the offtopic room alone. Extremity buildup is an expected condition when network partitions are involved, especially in heavily federated rooms like the continuwuity ones, where typically there's 500+ homeservers in the room at any given time, and >50% of the room's population is decentralised (i.e. not on the same server). Synapse has a behaviour to deal with this: [dummy events](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#dummy_events_threshold). A quick database lookup revealed that three privileged Synapse deployments were already occasionally sending dummy events, however since the rest of the room's members aren't allowed to send dummy events, they were not, and were simply aggregating more and more extremities over time, which may never decrease if the server doesn't have an active member sending events. Conduwuit consciously did not implement any sort of dummy event sending, from memory this was from a misguided belief that they were a crutch for Synapse performance issues, but I've now got evidence to show that they may actually help us in healing some partitioned graphs that end up resulting in an elevated level of erroneous soft-fails. Theoretically, if we're correctly fetching all missing events after a partition, the number of extremities should naturally reduce back down to near enough one in due time. #1818 helps with this, but is not yet merged nor in a release, and does not appear to *resolve* the problem completely. As such, I think we should implement some behaviour similar to Synapse's dummy events, to aid in healing partitions and squashing high extremity counts. More investigation is required.

nex added the

labels

2026-06-15 14:35:30 +00:00

nex referenced this issue from a commit

2026-06-16 17:12:45 +00:00

feat: Automatically squash extremities when they exceed a threshold

nex referenced this issue

2026-06-16 17:37:33 +00:00