bug: Federation senders sometimes mark remote servers as permenantly in-flight and stop sending until the server is restarted #2168

New issue

Open

opened 2026-08-22 20:41:20 +00:00 by nex · 0 comments

nex commented

2026-08-22 20:41:20 +00:00

Owner

What is the bug?

#2097 introduced a new "health tracking" system that globalised the federation backoff system, however it appears that the integration with the sender service may not have been clean - I have observed remote servers being stuck in the "sending transaction in process" state at least twice since merging that PR, although can't see how this could happen.

Additional logging has been added to my own local build to try and narrow down the problem if it happens again, if others encounter this same issue I can push that upstream. Knowing if one has the same issue requires enabling trace logs for the sender service conduwuit_service::sending::sender=trace.

Steps to reproduce

Not yet known - appears to be possibly related to the remote catchup system

Server name and version

nexy7574.co.uk 26.8.0-alpha.1 (2b03a251f)

Additional context

No response

### What is the bug? #2097 introduced a new "health tracking" system that globalised the federation backoff system, however it appears that the integration with the sender service may not have been clean - I have observed remote servers being stuck in the "sending transaction in process" state at least twice since merging that PR, although can't see how this could happen. Additional logging has been added to my own local build to try and narrow down the problem if it happens again, if others encounter this same issue I can push that upstream. Knowing if one has the same issue requires enabling trace logs for the sender service `conduwuit_service::sending::sender=trace`. ### Steps to reproduce Not yet known - appears to be possibly related to the remote catchup system ### Server name and version nexy7574.co.uk 26.8.0-alpha.1 (2b03a251f) ### Additional context _No response_

nex added the

labels

2026-08-22 20:41:20 +00:00