6 ms·
It looks pretty difficult to exploit. Untrusted 4GB input with non-chunked update.
by bvrmn 4y ago
It looks pretty difficult to exploit. Untrusted 4GB input with non-chunked update.
- nequo 4y agoThe purpose of running SHA is to determine whether the input can be trusted though. But it is true that relatively few files circulate at or above that size.
- thayne 4y agoAnd even if they are, you probably aren't going to do a digest on the whole thing all at once.
- kevincox 4y agoIDK, I see a lot of "non-production" code that likes to read whole files into a in-memory buffer rather than bothering with streaming. With modern memory sizes it is completely possible that you could read a 4GiB file into memory without having issues.
- Too 4y agoThinking about it, it's strange that hashlib doesn't have a function that accepts a file-object as input, to make the efficient method default pit of success. Naive implementations like hash(f.read()) are all too common to see. Such an api would also allow the entirety of the tight loop to be done outside of slow interpreted code. Guessing the rationale is that if you read a file, you also want to use the content for something else, not only compute its hash.
- mike_d 4y ago4+ GB inputs are often fed to hashing functions. Verifying the integrity of file downloads before running them for example.
- hot_gril 4y agoBut that'll probably run in chunks. Not gonna load 4GB into memory. Edit: Turns out there was another comment like this, and a response was that the file could be mmap'd.
- Cthulhu_ 4y agoYou're optimistic, which is commendable, but unfortunately a naive implementation would probably just read it all. I probably would, because the happy path doesn't deal with 4 GB inputs.
- hot_gril 4y agoI guess if your test machine always has a lot of contiguous* free memory or you never test it with a large enough file. Otherwise, it's impossible to ignore. One large download, and it wouldn't just be slow, it'd not work at all. * Which is usually much harder than just free mem. I recall macOS always being able to give a huge chunk of memory, I guess by rearranging on the fly, but it's very slow if you ask for a lot. And Linux just says no.
- kevingadd 4y agoIt doesn't need to be contiguous on any modern operating system I'm aware of. You just need 4gb of free contiguous address space, which virtually every 64-bit application will have.
- hot_gril 4y agoOh, then I ought to try my experiment again sometime. Last time was 8 years ago on Ubuntu Server, and maybe I'm missing a detail, like unknowingly running a 32-bit OS. But yeah the behavior you describe is more what I'd expect.
- someweirdperson 4y ago> And Linux just says no. Linux might say yes, even if the memory isn't available. See notes of malloc: "This means that when malloc() returns non-NULL there is no guarantee that the memory really is available." [0] [0] https://man7.org/linux/man-pages/man3/malloc.3.html https://man7.org/linux/man-pages/man3/malloc.3.html