resolvematrix adds a secondary level of DNS caching TTL which is currently not configurable #2103

Closed
opened 2026-08-05 02:51:56 +00:00 by ianthetechie · 8 comments
Contributor

Recently a friend moved their c10y server to a new host. Their DNS TTLs were reasonable values, but my c10y server failed to federate for some time until I unraveled the layers of DNS caching.

Within the main codebase, Hickory DNS is used directly and enforces a (configurable) minimum TTL, ignoring the TTL in the DNS record if it's below this value (default: 3h).

Additionally, c10y delegates to resolvematrix. Unless a resolution cache value is explicitly set, it defaults to 24h (I am not sure this is the right default, but that's not the point of this issue). The current c10y usage of MatrixResolver does not explicitly set a value, so 24h gets used.

If I'm reading this correctly, the way c10y uses resolvematrix currently creates a shadow cache of 24h which ignores the min TTL configuration of the server. This cache is separate from the hickory one.

I'm opening this as an issue because I'm not familiar with the thinking around these 2 crates interaction, but I'd suggest one of the caches should go away.

Recently a friend moved their c10y server to a new host. Their DNS TTLs were reasonable values, but my c10y server failed to federate for some time until I unraveled the layers of DNS caching. Within the main codebase, Hickory DNS is used directly and enforces a ([configurable](https://forgejo.ellis.link/continuwuation/continuwuity/src/commit/80e98a2299b9d9657fba31c570d43c4887ee3c04/src/core/config/mod.rs#L299)) minimum TTL, ignoring the TTL in the DNS record if it's below this value (default: 3h). Additionally, c10y delegates to `resolvematrix`. Unless a resolution cache value is explicitly set, it [defaults to 24h](https://forgejo.ellis.link/continuwuation/resolvematrix/src/commit/434f56b4f90552e8f79f8716f8d49a513f1a2f81/src/server.rs#L178) (I am not sure this is the right default, but that's not the point of this issue). The [current c10y usage of `MatrixResolver`](https://forgejo.ellis.link/continuwuation/continuwuity/src/commit/fe9f718c6e355ee5bff0dfc275b9fa5a4479506b/src/service/client/mod.rs#L46-L50) does not explicitly set a value, so 24h gets used. If I'm reading this correctly, the way c10y uses `resolvematrix` currently creates a shadow cache of 24h which ignores the min TTL configuration of the server. This cache is separate from the hickory one. I'm opening this as an issue because I'm not familiar with the thinking around these 2 crates interaction, but I'd suggest one of the caches should go away.
Owner

This is a known issue and appropriately honouring relevant cache values is a planned change: continuwuation/resolvematrix#21

I'm going to close this issue since it's more relevant to resolvematrix instead of continuwuity directly. Feel free to re-open if you disagree!

This is a known issue and appropriately honouring relevant cache values is a planned change: https://forgejo.ellis.link/continuwuation/resolvematrix/issues/21 I'm going to close this issue since it's more relevant to resolvematrix instead of continuwuity directly. Feel free to re-open if you disagree!
Member

Please keep open for tracking

Please keep open for tracking
Owner

@stratself I think we're intending to follow the DNS TTL, and then nex is working on code which will track down/alive systems so when the system comes alive after IP change things update appropriately.

@stratself I think we're intending to follow the DNS TTL, and then nex is working on code which will track down/alive systems so when the system comes alive after IP change things update appropriately.
Owner

@stratself wrote in #2103 (comment):

Please keep open for tracking

the issue for tracking is at continuwuation/resolvematrix#21. resolvematrix is separate software

@stratself wrote in https://forgejo.ellis.link/continuwuation/continuwuity/issues/2103#issuecomment-33843: > Please keep open for tracking the issue for tracking is at https://forgejo.ellis.link/continuwuation/resolvematrix/issues/21. resolvematrix is separate software
Member

The current c10y usage of MatrixResolver does not explicitly set a value, so 24h gets used.

Except that c10y explicitly passes in a DNS resolver to use, which comes from the get_dns_resolver function, which does utilize the system config for DNS TTL values:

.dns_resolver(dns_resolver.clone())

It is correct that resolvematrix has a cache that defaults to 24 hours, however this is a full resolution cache, not just DNS. It can be cleared with the !admin query resolver flush-cache --all admin command. This clears both the 24h internal cache, as well as the DNS cache from hickory.

The duration for the cache was discussed here in the dev room, a few weeks ago. As nex linked to, continuwuation/resolvematrix#21 is the issue for utilizing DNS TTL and Cache-Control headers. I have taken a stab at this issue in the past, but it is a lot more complex than it would initially seem.

> The current c10y usage of MatrixResolver does not explicitly set a value, so 24h gets used. Except that c10y explicitly passes in a DNS resolver to use, which comes from the [`get_dns_resolver`](https://forgejo.ellis.link/continuwuation/continuwuity/src/commit/fe9f718c6e355ee5bff0dfc275b9fa5a4479506b/src/service/client/mod.rs#L259-L312) function, which does utilize the system config for DNS TTL values: https://forgejo.ellis.link/continuwuation/continuwuity/src/commit/fe9f718c6e355ee5bff0dfc275b9fa5a4479506b/src/service/client/mod.rs#L49 It is correct that resolvematrix has a cache that defaults to 24 hours, however this is a full resolution cache, not just DNS. It can be cleared with the `!admin query resolver flush-cache --all` admin command. This clears both the 24h internal cache, as well as the DNS cache from hickory. The duration for the cache was discussed [here](https://matrix.to/#/!ksTlboXVgcyWjv5GrlEeKyQuJ8ZCprnwQx2b6-BQ44Q/%240vgcSMqCgyoAyv5DsurrMHNWXNtotnJ3I3VK4WuhMV8?via=wolfgirl.pet&via=explodie.org&via=matrix.org) in the dev room, a few weeks ago. As nex linked to, continuwuation/resolvematrix#21 is the issue for utilizing DNS TTL and Cache-Control headers. I have taken a stab at this issue in the past, but it is a lot more complex than it would initially seem.
Author
Contributor

Thanks for opening the bug on resolvematrix @nex! However, I consider it a bug in c10y that we have two separate DNS caches, one of which is configurable and one of which is not 🤔

Additionally, it appears that there is a bug in the path for !admin query resolver flush-cache <server>. While --all does clear everything correctly, the server-specific path only clears the resolvematrix side. Any DNS caches from hickory would remain.

I suppose that is probably be trackable as a separate issue. I'd be happy to work on these btw but I opened this issue to start a discussion on what the end state should be.

Thanks for opening the bug on resolvematrix @nex! However, I consider it a bug in c10y that we have two separate DNS caches, one of which is configurable and one of which is not 🤔 Additionally, it appears that there is a bug in the path for `!admin query resolver flush-cache <server>`. While `--all` does clear everything correctly, the server-specific path only clears the resolvematrix side. Any DNS caches from hickory would remain. I suppose that is probably be trackable as a separate issue. I'd be happy to work on these btw but I opened this issue to start a discussion on what the end state should be.
Owner

I guess it's technically a bug that hickory's dns cache exists at all, we should probably just remove that

I guess it's technically a bug that hickory's dns cache exists at all, we should probably just remove that
Author
Contributor

I support that conclusion.

I support that conclusion.
Sign in to join this conversation.
No milestone
No project
No assignees
5 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
continuwuation/continuwuity#2103
No description provided.