Limit read of well-known files

Jade commented

2026-03-19 21:13:32 +00:00

Owner

CWE-409 or similar

s1lv3r commented

2026-04-14 23:31:55 +00:00

Member

with "limit read" here do you mean like limit how much data is read?

Henry-Hiles commented

2026-04-14 23:34:24 +00:00

Member

@s1lv3r wrote in #1 (comment):

with "limit read" here do you mean like limit how much data is read?

I think so, to avoid e.g. zip bombs

@s1lv3r wrote in https://forgejo.ellis.link/continuwuation/resolvematrix/issues/1#issuecomment-28052: > with "limit read" here do you mean like limit how much data is read? I think so, to avoid e.g. zip bombs

Henry-Hiles commented

2026-04-14 23:34:38 +00:00

Member

Something similar to spider size for URL previews could be used I guess?

s1lv3r commented

2026-04-14 23:39:29 +00:00

Member

or set a hardcoded max of like 1MB, no reasonable well-known file will ever be above that size

👍 1

Jade commented

2026-04-14 23:40:09 +00:00

Author

Owner

It's just an option that needs to be passed to reqwest iirc, not too hard to do.

s1lv3r commented

2026-04-15 00:07:25 +00:00

Member

Looked into it, would have to be done using either:

Response::content_length(): Not always known
Chunking, reading e.g. 8096 bytes at a time, adding to a Vec, counting the total (as described in https://github.com/seanmonstar/reqwest/issues/848)

Probably the easiest to do the former, and fall back to the latter if the former fails. Will start an implementation.

Looked into it, would have to be done using either: - [`Response::content_length()`](https://docs.rs/reqwest/latest/reqwest/struct.Response.html#method.content_length): Not always known - Chunking, reading e.g. 8096 bytes at a time, adding to a `Vec`, counting the total (as described in https://github.com/seanmonstar/reqwest/issues/848) Probably the easiest to do the former, and fall back to the latter if the former fails. Will start an implementation.

Henry-Hiles commented

2026-04-15 00:08:50 +00:00

Member

@s1lv3r wrote in #1 (comment):

Looked into it, would have to be done using either:

* [`Response::content_length()`](https://docs.rs/reqwest/latest/reqwest/struct.Response.html#method.content_length): Not always known

One of the reasons this isnt known is

The response is gzipped and automatically decoded (thus changing the actual decoded length).

Given the CWE this is trying to avoid is with GZIP/compressed content, I would say that way is not helpful really.

@s1lv3r wrote in https://forgejo.ellis.link/continuwuation/resolvematrix/issues/1#issuecomment-28061: > Looked into it, would have to be done using either: > > * [`Response::content_length()`](https://docs.rs/reqwest/latest/reqwest/struct.Response.html#method.content_length): Not always known One of the reasons this isnt known is > The response is gzipped and automatically decoded (thus changing the actual decoded length). > Given the CWE this is trying to avoid is with GZIP/compressed content, I would say that way is not helpful really.

Henry-Hiles commented

2026-04-15 00:09:26 +00:00

Member

Anyways, I thought jade said it was just one param

s1lv3r commented

2026-04-15 00:09:58 +00:00

Member

I thought so too, doesn't look like it from what I could find

Jade commented

2026-04-15 00:13:33 +00:00

Author

Owner

It's already done in Continuwuity, I was just going to copy from there

s1lv3r commented

2026-04-15 00:14:33 +00:00

Member

oh

good idea, might as well yeah

oh good idea, might as well yeah

s1lv3r referenced this issue from continuwuation/continuwuity

2026-04-19 12:20:34 +00:00

Rewrite resolver service #1505

nex added a new dependency

2026-04-28 03:52:11 +00:00

continuwuation/continuwuity#1505 - Rewrite resolver service

nex commented

2026-04-28 03:52:59 +00:00

Owner

@Jade wrote in #1 (comment):

It's already done in Continuwuity, I was just going to copy from there

Relevant file: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/src/core/utils/response.rs

@Jade wrote in https://forgejo.ellis.link/continuwuation/resolvematrix/issues/1#issuecomment-28065: > It's already done in Continuwuity, I was just going to copy from there Relevant file: https://forgejo.ellis.link/continuwuation/continuwuity/src/branch/main/src/core/utils/response.rs

s1lv3r referenced this issue

2026-06-21 23:38:27 +00:00

Rewrite to use builder pattern #11

s1lv3r referenced this issue

2026-06-22 23:58:46 +00:00

Move to builder pattern and split up library #12

s1lv3r self-assigned this

2026-06-23 21:47:52 +00:00

s1lv3r commented

2026-06-26 22:13:20 +00:00

Member

Closed in #13