6 ms·
And if you set your own content length header, most http servers will respect it and not chunk. That way, you can stream a 4-gig file that you know the size of
by pkulak 2y ago
And if you set your own content length header, most http servers will respect it and not chunk. That way, you can stream a 4-gig file that you know the size of per the metadata. This makes downloading nicer because browsers and such will then show a progress bar and time estimate.
However, you better be right! I just found a bug in some really old code that was gzipping every response when it was appropriate (ie, asked for, textual, etc). But it was ignoring the content-length header! So, if it was set manually, it would then be wrong after compression. That caused insidious bugs for years. The fix, obviously, was to just delete that manual header if the stream was going to be compressed.
- bugtodiffer 2y agoDid you see if you could turn this into HTTP Request Smuggling? Or something else with security impact? Sounds like a powerful bug you have, potentially.
- knallfrosch 2y agoTo me it sounds like the server handled the request just fine and reused the header (which was wrong.) The client then had the problem of a wrong response.
- guappa 2y agoIf you say to read 1000 bytes from memory and then pass a 900 bytes long array, that's a security bug that can cause crash, corrupt data, and leaking stuff that shouldn't have leaked.
- jpc0 2y agoThe size of the buffer and how many bytes are written have nothing intrinsically linked to what the header says. It's a bug sure but does not mean there's any security issue on the server.
- guappa 2y agoIt will likely generate corrupt files on the client as well.
- Aachen 2y agoNot very. The system might allocate that length ahead of time (I've seen that option in torrent clients and iirc ftp systems) but, latest by the time a FIN comes in, it'll know the file is finished and can truncate it. If finished-early downloads are not implemented despite it doing preallocation, that's still not a security bug
- guappa 2y agoif a FIN comes, the client will mark the file as partially downloaded. but it might not come, since decades http sends more than one file per connection, so it might just get the beginning of the next reply, write that and the next reply will be corrupt as well.
- michaelmior 2y agoIt's a bug for sure, but I think whether it's a security issue could depend on the language. If the callee is able to determine the length of the array, it can just return an error instead of a potential buffer overrun.
- stickfigure 2y agoIn this case, the 1000 bytes aren't being read from memory, they're being read from a socket. If you try to over-read from a socket the worst you'll get is a blocking call or an error (depending what mode you're in).
- bugtodiffer 2y agomaybe response based trickery then? :D What happens to the response after that one, are the first 100 bytes cut of, or what? I'm pretty sure something like this can cause some form of HTTP desync in a loadbalancer/proxy setup.
- dotancohen 2y ago> That caused insidious bugs for years. A lot of people here could probably benefit professionally from hearing about what the bugs were. Knowing what to identify in the future could be really helpful. Thanks.
- pkulak 2y agoWell, for years our logs would fill up with these nasty warnings of "connection dropped", something like that. Naturally, you think that's just some badly configured client, mobile connection, something. But then why would it be configured to log that as a warning (or it may have even triggered an error). I think that was because when payloads are small the compression overhead makes them larger, which means the Content-Length is too small and clients would be terminating the connection early. And getting garbage or truncated responses. Ouch!
- dotancohen 2y agoInsidious was the right word. Thank you!