continuwuation/continuwuity

Fork 81

WARN conduwuit_api::client::sync::v3::joined: timeline for newly joined room is empty spam (caused by Nheko?) #1362

New issue

Closed

opened 2026-02-12 16:54:19 +00:00 by fingon · 8 comments

fingon commented

2026-02-12 16:54:19 +00:00

The configuration is quite vanilla:

-e CONDUIT_SERVER_NAME=fingon.iki.fi \
-e CONDUIT_ALLOW_REGISTRATION=false \
-e CONDUIT_ALLOW_FEDERATION=true \
-e CONDUIT_DATABASE_PATH=/var/lib/matrix-conduit/ \
-e CONDUIT_PORT=6167 \
-e CONDUIT_ADDRESS=0.0.0.0 \
-e CONDUIT_CONFIG="" \
-e CONDUIT_NEW_USER_DISPLAYNAME_SUFFIX="" \
-e CONDUIT_ROCKSDB_DIRECT_IO=false \
-e CONDUWUIT_ALLOW_DEVICE_NAME_FEDERATION=false \
-e CONDUWUIT_WELL_KNOWN__SERVER=matrix.fingon.iki.fi:443 \
-e CONDUWUIT_WELL_KNOWN__CLIENT=https://matrix.fingon.iki.fi \
-v {DATA_DIR}:/var/lib/matrix-conduit/ \
-p 16167:6167""",

Unfortunate thing is that the log message does not even say what room it is for.

I think it might be caused by Nheko (see attached graph), as when I was traveling I didn't see it as I was only using Element X during it. It might be also just coincidence though.

Now that I killed my clients, and restart continuwuity, it still uses 100% of one core but log messages do not occur.

On unrelated note, it would be good to have some visibility to what is going on in the release build though - I have no idea what is consuming the CPU, I am in handful of federated low-traffic rooms.

I am not sure what causes it exactly, but it is basically happening constantly (both with current latest and main docker tags) for me, and it is using 2 CPU cores presumably for that (99% of log messages are that). The configuration is quite vanilla: ``` -e CONDUIT_SERVER_NAME=fingon.iki.fi \ -e CONDUIT_ALLOW_REGISTRATION=false \ -e CONDUIT_ALLOW_FEDERATION=true \ -e CONDUIT_DATABASE_PATH=/var/lib/matrix-conduit/ \ -e CONDUIT_PORT=6167 \ -e CONDUIT_ADDRESS=0.0.0.0 \ -e CONDUIT_CONFIG="" \ -e CONDUIT_NEW_USER_DISPLAYNAME_SUFFIX="" \ -e CONDUIT_ROCKSDB_DIRECT_IO=false \ -e CONDUWUIT_ALLOW_DEVICE_NAME_FEDERATION=false \ -e CONDUWUIT_WELL_KNOWN__SERVER=matrix.fingon.iki.fi:443 \ -e CONDUWUIT_WELL_KNOWN__CLIENT=https://matrix.fingon.iki.fi \ -v {DATA_DIR}:/var/lib/matrix-conduit/ \ -p 16167:6167""", ``` Unfortunate thing is that the log message does not even say what room it is for. I think it _might_ be caused by Nheko (see attached graph), as when I was traveling I didn't see it as I was only using Element X during it. It might be also just coincidence though. Now that I killed my clients, and restart continuwuity, it still uses 100% of one core but log messages do not occur. On unrelated note, it would be good to have some visibility to what is going on in the release build though - I have no idea what is consuming the CPU, I am in handful of federated low-traffic rooms.

Screenshot 2026-02-12 at 17.22.50.png

76 KiB

ginger commented

2026-02-12 16:56:01 +00:00

Owner

Usually that log line appears as a consequence of the same race conditions that cause !779, but it doesn't really need to be a warning in release builds, especially as it's not really actionable by homeserver admins. It'll be demoted to only warn in debug mode in the next release.

nex added the

Bug

Status

Confirmed

labels

2026-02-12 16:56:32 +00:00

fingon commented

2026-02-12 17:02:06 +00:00

Author

Thanks, good to know.

Is there some way to figure out what is using the CPU (if it is not related to this message)? It has gotten worse over the last couple of releases and the number of rooms and their traffic hasn't really changed.

Thanks, good to know. Is there some way to figure out what is using the CPU (if it is not related to this message)? It has gotten worse over the last couple of releases and the number of rooms and their traffic hasn't really changed.

👀 1

nex commented

2026-02-12 17:02:43 +00:00

Owner

This message is directly related to elevated CPU usage - check how many requests per second you're getting to /_matrix/client/v3/sync

This message is directly related to elevated CPU usage - check how many requests per second you're getting to `/_matrix/client/v3/sync`

fingon commented

2026-02-12 17:07:57 +00:00

Author

About 10 rps if I have client active (to the sync endpoint), and way less if not. But even without client running, the CPU usage stays at 100% (for single core - it spikes higher if there's serious traffic going on).

There's some federation traffic but not a lot.

fingon commented

2026-02-12 17:18:51 +00:00

Author

Network traffic for whole node is <200kb/s, and there is almost no i/o, in case that helps (most of the network traffic is not related to Continuwuity).

fingon commented

2026-02-13 06:03:59 +00:00

Author

fwiw, I checked with 0.5.1. If Nhekos are connected it uses CPU, but otherwise not. With :main or :latest (as of yesterday) there is 100% single core CPU usage so there's some regression in that range, not quite sure what.

fingon commented

2026-02-13 15:05:53 +00:00

Author

Even without clients it eventually started using all the cpu so perhaps the version was just red herring.

Fun fact: current main-maxpedf works still entirely fine after I limited it to 0,2 cores ( so 1/5 or less than what it wanted to use ). No idea what exactly it is doing as network or I/o is not involved much.

Even without clients it eventually started using all the cpu so perhaps the version was just red herring. Fun fact: current main-maxpedf works still entirely fine after I limited it to 0,2 cores ( so 1/5 or less than what it wanted to use ). No idea what exactly it is doing as network or I/o is not involved much.

fingon commented

2026-02-18 17:40:48 +00:00

Author

Ah, looks like current main fixes that too - the culprit was presumably federated presence. Sigh. ( f458f6ab76 )

Ah, looks like current main fixes that too - the culprit was presumably federated presence. Sigh. ( https://forgejo.ellis.link/continuwuation/continuwuity/commit/f458f6ab763e71480bc85964f77aaba8a866b191 )