feat: Server health tracking #2097

Merged

Aranjedeath merged 28 commits from nex/feat/server-health-tracking into main

2026-08-20 00:57:12 +00:00

Author	SHA1	Message	Date
timedout	84343001a4	chore: Re-generate example config All checks were successful Checks / Changelog / Check changelog is added (pull_request_target) Successful in 11s Details Documentation / Build and Deploy Documentation (pull_request) Successful in 1m5s Details Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m8s Details Checks / Prek / Check changed files (pull_request) Successful in 7s Details Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 8m0s Details	2026-08-19 14:39:18 +01:00
timedout	331025e0fd	chore: Add newsfrag Some checks failed Checks / Changelog / Check changelog is added (pull_request_target) Successful in 7s Details Documentation / Build and Deploy Documentation (pull_request) Successful in 1m5s Details Checks / Prek / Check changed files (pull_request) Successful in 6s Details Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m9s Details Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled Details	2026-08-19 14:36:55 +01:00
timedout	a108208274	fix: Address review comments Some checks failed Checks / Changelog / Check changelog is added (pull_request_target) Failing after 6s Details Documentation / Build and Deploy Documentation (pull_request) Successful in 1m5s Details Checks / Prek / Check changed files (pull_request) Successful in 5s Details Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m10s Details Checks / Prek / Clippy and Cargo Tests (pull_request) Has been cancelled Details	2026-08-19 14:34:15 +01:00
timedout	c04c732b22	feat: Lower default backoff base for a gentler backoff curve Some checks failed Checks / Changelog / Check changelog is added (pull_request_target) Failing after 7s Details Documentation / Build and Deploy Documentation (pull_request) Successful in 1m3s Details Checks / Prek / Check changed files (pull_request) Successful in 6s Details Checks / Prek / Pre-commit & Formatting (pull_request) Successful in 1m9s Details Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 8m11s Details	2026-08-19 14:07:08 +01:00
timedout	7e3e01e731	fix: Don't set retrying status when retrying is illegal Some checks failed Checks / Changelog / Check changelog is added (pull_request_target) Failing after 6s Details Documentation / Build and Deploy Documentation (pull_request) Successful in 1m3s Details Checks / Prek / Check changed files (pull_request) Successful in 5s Details Checks / Prek / Pre-commit & Formatting (pull_request) Failing after 1m10s Details Update flake hashes / update-flake-hashes (pull_request) Successful in 1m22s Details Checks / Prek / Clippy and Cargo Tests (pull_request) Successful in 8m10s Details	2026-08-19 13:48:51 +01:00
timedout	7e41071b8c	fix: Treat any non-ok response as a server error in sender Otherwise we never back off sending failed requests	2026-08-19 13:48:51 +01:00
timedout	abf7333a09	fix: Mark even more error codes as stale based on observed data	2026-08-19 13:48:51 +01:00
timedout	ad0c1b9eb5	fix: Don't stop tracking servers when there's nothing more to send despite being inflight	2026-08-19 13:48:51 +01:00
timedout	1b3a543302	fix: Correctly insert running status only if the sender should attempt to send	2026-08-19 13:48:50 +01:00
timedout	e5e40a347e	style: Reformat	2026-08-19 13:48:50 +01:00
timedout	dfd14f3429	fix(sender): retry_after duration can be negative	2026-08-19 13:48:50 +01:00
timedout	b6280617af	fix(sender): Don't attempt to send concurrent transactions to a remote	2026-08-19 13:48:50 +01:00
timedout	a321fd10ee	style: Centralise decision for stale marker	2026-08-19 13:48:50 +01:00
timedout	e6273e0e15	feat: Update servers-in-backoff command to annotate stale info	2026-08-19 13:48:50 +01:00
timedout	6a62a377e6	feat: Clear destinations cache when a remote is potentially stale	2026-08-19 13:48:50 +01:00
timedout	091f0ddf15	feat: Add debug command to extract backoff info	2026-08-19 13:48:49 +01:00
timedout	e7278199d7	feat: Use an explicit backoff base value	2026-08-19 13:48:49 +01:00
timedout	fd1218a793	chore: Tidy up backoff helper code	2026-08-19 13:48:49 +01:00
timedout	d75981e011	feat: Hook transaction senders into health tracking	2026-08-19 13:48:49 +01:00
timedout	1ec0f76643	style: Remove unsafe arithmetic	2026-08-19 13:48:49 +01:00
timedout	e297093f28	fix: Don't honour unhealthy hits unless the block period has already expired	2026-08-19 13:48:49 +01:00
timedout	86045b3965	feat: Clear remote health tracking map when cache is cleared	2026-08-19 13:48:49 +01:00
timedout	7e1a513557	feat: Trace why remotes get health marked	2026-08-19 13:48:48 +01:00
timedout	93259ed291	feat: Refuse to send requests to unhealthy servers Also marks them as online if a request succeeds	2026-08-19 13:48:48 +01:00
timedout	9e330da4ba	chore: Move health tracking to federation service Otherwise federation would have to depend on sender	2026-08-19 13:48:48 +01:00
timedout	5845741f2e	feat: Mark servers as online/offline based on transaction responses	2026-08-19 13:48:48 +01:00
timedout	bd8239f285	feat: Mark servers as healthy when they successfully make an inbound request	2026-08-19 13:48:48 +01:00
timedout	487c992883	feat: Add server health tracking	2026-08-19 13:48:47 +01:00