5 ms·

Fun with gzip bombs and email clients

155 points by bundie 1y ago

jerf 1y ago
Another fun one is the .zip or .tar.gz file that decompresses to itself: https://research.swtch.com/zip https://research.swtch.com/zip If you are processing emails for security reasons, and want to find viruses even if they are in archive files, it's easy to write the code to "just keep unarchiving until we're out of things to unarchive", but not only can that lead to quite astonishing expansions, it can actually be a process that never terminates at all. I remember when I first read about these, and "a small file that decompresses to a gigabyte" was also "a small file that decompresses to several multiples of your entire hard disk space" and even servers couldn't handle it. Now I read articles like this one talking about "oh yeah Evolution filled up 100GB of space" like that's no big deal. If you have a recursive decompressor you can still make small files that uncompress to large amounts even by 2025 standards, because the symbols the compressor will use to represent "as many zeros as I can have" will themselves be redundant. The rule that you can't compress already-compressed content doesn't necessarily apply to these sorts of files.
- cyanydeez 1y ago
  I'd be curious if there's an LLM prompt equivalent of a zip bomb that will explode the context window. I know there's deterministic limits on context window, but future LLMs _are_ going to have strange loops and going to be very susceptible to circular reasoning. Before AGI, there will be a untenable gullible general intelligence.
  - colechristensen 1y ago
    One of the things that will likely _characterize_ AGI are nondeterministic loops. My bet is that if AGI is possible it will take a form that looks something like x_(n+1) = A * x_n (1 - x_n) Where x is a billions long vector and the parameters in A (sizeof(x)^2 ?) are trained and also tuned to have period 3 or nearly period three for a meta-stable near chaotic progression of x. "Period three implies chaos" https://www.its.caltech.edu/~matilde/LiYorke.pdf https://www.its.caltech.edu/~matilde/LiYorke.pdf That is if AGI is possible at all without wetware.
    - Y_Y 1y ago
      Chaos isn't intelligence. Chaos is unmanageable growth in your solution space, the oppisite of what you want.
      - colechristensen 1y ago
        There are theories and evidence that your brain operates hovering on the edge of the phase transition to chaos https://en.m.wikipedia.org/wiki/Critical_brain_hypothesis https://en.m.wikipedia.org/wiki/Critical_brain_hypothesis
        mindesc 1y ago
        If you have any system that tries to gravitate to a local minimum it is almost impossible to not make Newton's fractal with it. Classical feed forward network learning does pretty much look like newtons method to me. Please take a look into https://en.m.wikipedia.org/wiki/Newton%27s_method https://en.m.wikipedia.org/wiki/Newton%27s_method
      - cyanydeez 1y ago
        Whats confusing to me is the dual use of the word entropy in both the physical science and in communication. The local minimums are some how stable in a world of increasing entropy. How do these local minimums ever form when there's such a large arrow of entropy. Certainly intelligence is a reduction of entropy, but it's also certainly not stable. Just like cellular automata (https://record.umich.edu/articles/simple-rules-can-produce-complex-behavior-physicist-wolfram-says/ https://record.umich.edu/articles/simple-rules-can-produce-c...), loops that are stable can't evolve, but loops that are unstable have too much entropy. So, we're likely searching for a system thats meta stable within a small range of input entropy (physical) and output entropy (information).
  - JoshTriplett 1y ago
    I've seen LLMs get into loops because they forgot what they were trying to do. For instance, I asked an LLM to write some code to search for certain types of wordplay, and it started making a word list (rather than writing code to pull in a standard dictionary), and then it got distracted and just kept listing words until it ran out of time.
- masklinn 1y ago
  A few years ago David Fitfield invented a technique which provides a million-to-one non-recursive expansion, by overlapping the file streams: https://www.bamsoftware.com/hacks/zipbomb/ https://www.bamsoftware.com/hacks/zipbomb/
  - panarky 1y ago
    Might be fun to respond with one of these to malicious requests for /.env, /.git/config and /.aws/credentials instead of politely returning 404s.
    - bspammer 1y ago
      It’s definitely tempting, but I prefer not to piss off people who are already being actively malicious.
      - philodeon 1y ago
        Isn’t this how a court system works?
      - Twirrim 1y ago
        It's all just spray-and-pray crap. You're extremely unlikely to be their target, they're just looking for a convenient shell for a botnet. The most likely way they'll handle it if you do actually break them is just blacklist your address. You're not going to be worth the effort.
    - Twirrim 1y ago
      I've been sending a nice 10GB gzip bomb (12MB after compression, rate limited download speed) to people that send various malicious requests. I think I might update it tonight with this other approach.
      - 1718627440 1y ago
        Can't you just server /dev/urandom?
- ac29 1y ago
  > Now I read articles like this one talking about "oh yeah Evolution filled up 100GB of space" like that's no big deal. Is this actually a practical issue though? Windows, Mac and Linux all support transparent compression at the filesystem level, so 100GB of /dev/zero isnt actually going to fill much space at all.
  - kiwijamo 1y ago
    That's not switched on by default unless you use a filesytsem like ZFS.
metalrain 1y ago
So can you construct valid image that would also act as zip bomb? Jpeg and other lossy compression images should allow some of that, but dependens on compatibility of compression between gzip and image format. There is that example where you have "zero image" of big dimensions, but can you actually conflate gzip and image compression?
- cogman10 1y ago
  I don't think you can do it with Jpeg, but you could probably do it with PNG which is basically using the same compression algorithm as zip.
  - masklinn 1y ago
    Deflate allows a maximum compression ratio of 1000:1 or thereabouts. Considering I’ve seen real world JPEGs above 300:1 (https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73909/world.topo.bathy.200412.3x21600x21600.A2.jpg https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/739...) I would not be surprised if you could craft a jpeg getting very close to or exceeding 4 digits.
    - cogman10 1y ago
      The reason it doesn't work with JPEG is JPEG isn't a description of individual pixels but rather how you'd calculate what the individual pixel should be. That's part of the reason you can progressively load jpeg data. PNG is actually a description of the RGB value for the individual pixels. That's why I believe you could png bomb, you could have a 2 billion by 2 billion black pixel image which would ultimately eat up a bunch of space in your GPU and memory to decode. Perhaps something similar is possible with a JPEG, but it's really nothing to do with the compression info. JPEGs have a max size of 65,535×65,535, which would keep you from exploding them.
      - LegionMammal978 1y ago
        DEFLATE can only obtain a best-case compression ratio approaching 1032:1. (Put the byte to repeat in a preceding block, and set "0" = 256 and "1" = 285 for the literal/length code and "0" = 0 for the distance code. Then "10" will output 258 bytes.) This means a 2 Gpx × 2 Gpx PNG image will still be at least ~3.875 PB. If you send it compressed over the wire, you could get another factor of 1032, or perhaps more depending on which algorithms the client supports. Also, you could generate it on demand as a data stream. Bit these run the risk of the client stopping the transfer before ever trying to process the image.
albinowax_ 1y ago
That Evolution mail caching behaviour is really sketchy. I wonder if it could be used for an exploit in the right scenario. If nothing else, it’s a good way to make an email that looks completely different depending on which client it’s opened in.
- johannes1234321 1y ago
  > it’s a good way to make an email that looks completely different depending on which client it’s opened in. Well, for that use the differences in HTML&CSS support and filtering ... I guess the reason they added this was that they noticed many mails contain same tracking images and decided to cut of tracking data that way.
  - mike-cardwell 1y ago
    I don't think this was done on purpose. If the query string is "?a=b" that's fine, and it's used in the cache filename. But if the query string is "?a" then it's excluded from the cache filename. Either way, the correct full URL is fetched with the full query string. It's just how it's cached that is affected.
    - deleted 1y ago
      [deleted]
jokoon 1y ago
Isn't that trivial to prevent zip bombs?
- masklinn 1y ago
  Most things are trivial to prevent if you know of and think to check for them.
anthk 1y ago
How does it work with Claws Mail/Sylpheed?
upofadown 1y ago
Yet another reason to prevent emails from downloading stuff from remote servers... It appears that you can't do these sorts of things with with CID embedded images...
austinjp 1y ago
Hmm that reminds me, it idly crossed my mind recently about whether AIs with online RAG have decent zip-bomb protection. This thought was provoked when I realised Perplexity would find and download and (apparently) analyse spreadsheet content. I'm sure there are zip-bomb equivalents in binary formats like .xlsx, PDF, .docx, etc.
- mxmlnkn 1y ago
  > I'm sure there are zip-bomb equivalents in binary formats like .xlsx, PDF, .docx, etc. Yes. Both, docx and xlsx are literally just a zip of XML files with a different extension. PDF can contain zlib streams, which use deflate compression just as gzip, so all the mentioned methods apply to all three formats.
- mk_stjames 1y ago
  In addition to zip bombing AIs with file parsers, I've wondered about 'context bombs' in the sense of trigger phrases that trip up LLMs into getting stuck into repeating phrases or reasoning evaluations without ever hitting an end of sequence (EOS) token, thus running a system up against API call limits / burning credits / effectively ddosing services etc. Due to the inherent fuzziness/diversity in all models right now I don't think there is a universal approach to this idea but it is something people deploying these systems may want to try and detect.
grahameb 1y ago
I ran into one of these in the very early 00s; was working at a university (back in the days when a couple of people would run all the central servers, running Linux on beige PCs.) We had some anti-spam/AV software that looked at every incoming email hooked into Postfix, and the server kept running out of disk space. Eventually tracked it down to an email which contained a zip of stock trading data – just the three letter stock code and the shift. It wasn't malicious, it just had an extraordinarily high compression ratio!
imcritic 1y ago
The owner of this site can just fuck off with their nationalism.

155 points by bundie 1y ago

jerf 1y ago

Another fun one is the .zip or .tar.gz file that decompresses to itself: https://research.swtch.com/zip https://research.swtch.com/zip If you are processing emails for security reasons, and want to find viruses even if they are in archive files, it's easy to write the code to "just keep unarchiving until we're out of things to unarchive", but not only can that lead to quite astonishing expansions, it can actually be a process that never terminates at all. I remember when I first read about these, and "a small file that decompresses to a gigabyte" was also "a small file that decompresses to several multiples of your entire hard disk space" and even servers couldn't handle it. Now I read articles like this one talking about "oh yeah Evolution filled up 100GB of space" like that's no big deal. If you have a recursive decompressor you can still make small files that uncompress to large amounts even by 2025 standards, because the symbols the compressor will use to represent "as many zeros as I can have" will themselves be redundant. The rule that you can't compress already-compressed content doesn't necessarily apply to these sorts of files.

cyanydeez 1y ago
I'd be curious if there's an LLM prompt equivalent of a zip bomb that will explode the context window. I know there's deterministic limits on context window, but future LLMs _are_ going to have strange loops and going to be very susceptible to circular reasoning. Before AGI, there will be a untenable gullible general intelligence.
- colechristensen 1y ago
  One of the things that will likely _characterize_ AGI are nondeterministic loops. My bet is that if AGI is possible it will take a form that looks something like x_(n+1) = A * x_n (1 - x_n) Where x is a billions long vector and the parameters in A (sizeof(x)^2 ?) are trained and also tuned to have period 3 or nearly period three for a meta-stable near chaotic progression of x. "Period three implies chaos" https://www.its.caltech.edu/~matilde/LiYorke.pdf https://www.its.caltech.edu/~matilde/LiYorke.pdf That is if AGI is possible at all without wetware.
  - Y_Y 1y ago
    Chaos isn't intelligence. Chaos is unmanageable growth in your solution space, the oppisite of what you want.
    - colechristensen 1y ago
      There are theories and evidence that your brain operates hovering on the edge of the phase transition to chaos https://en.m.wikipedia.org/wiki/Critical_brain_hypothesis https://en.m.wikipedia.org/wiki/Critical_brain_hypothesis
      - mindesc 1y ago
        If you have any system that tries to gravitate to a local minimum it is almost impossible to not make Newton's fractal with it. Classical feed forward network learning does pretty much look like newtons method to me. Please take a look into https://en.m.wikipedia.org/wiki/Newton%27s_method https://en.m.wikipedia.org/wiki/Newton%27s_method
    - cyanydeez 1y ago
      Whats confusing to me is the dual use of the word entropy in both the physical science and in communication. The local minimums are some how stable in a world of increasing entropy. How do these local minimums ever form when there's such a large arrow of entropy. Certainly intelligence is a reduction of entropy, but it's also certainly not stable. Just like cellular automata (https://record.umich.edu/articles/simple-rules-can-produce-complex-behavior-physicist-wolfram-says/ https://record.umich.edu/articles/simple-rules-can-produce-c...), loops that are stable can't evolve, but loops that are unstable have too much entropy. So, we're likely searching for a system thats meta stable within a small range of input entropy (physical) and output entropy (information).
- JoshTriplett 1y ago
  I've seen LLMs get into loops because they forgot what they were trying to do. For instance, I asked an LLM to write some code to search for certain types of wordplay, and it started making a word list (rather than writing code to pull in a standard dictionary), and then it got distracted and just kept listing words until it ran out of time.
masklinn 1y ago
A few years ago David Fitfield invented a technique which provides a million-to-one non-recursive expansion, by overlapping the file streams: https://www.bamsoftware.com/hacks/zipbomb/ https://www.bamsoftware.com/hacks/zipbomb/
- panarky 1y ago
  Might be fun to respond with one of these to malicious requests for /.env, /.git/config and /.aws/credentials instead of politely returning 404s.
  - bspammer 1y ago
    It’s definitely tempting, but I prefer not to piss off people who are already being actively malicious.
    - philodeon 1y ago
      Isn’t this how a court system works?
    - Twirrim 1y ago
      It's all just spray-and-pray crap. You're extremely unlikely to be their target, they're just looking for a convenient shell for a botnet. The most likely way they'll handle it if you do actually break them is just blacklist your address. You're not going to be worth the effort.
  - Twirrim 1y ago
    I've been sending a nice 10GB gzip bomb (12MB after compression, rate limited download speed) to people that send various malicious requests. I think I might update it tonight with this other approach.
    - 1718627440 1y ago
      Can't you just server /dev/urandom?
ac29 1y ago
> Now I read articles like this one talking about "oh yeah Evolution filled up 100GB of space" like that's no big deal. Is this actually a practical issue though? Windows, Mac and Linux all support transparent compression at the filesystem level, so 100GB of /dev/zero isnt actually going to fill much space at all.
- kiwijamo 1y ago
  That's not switched on by default unless you use a filesytsem like ZFS.

metalrain 1y ago

So can you construct valid image that would also act as zip bomb? Jpeg and other lossy compression images should allow some of that, but dependens on compatibility of compression between gzip and image format. There is that example where you have "zero image" of big dimensions, but can you actually conflate gzip and image compression?

cogman10 1y ago
I don't think you can do it with Jpeg, but you could probably do it with PNG which is basically using the same compression algorithm as zip.
- masklinn 1y ago
  Deflate allows a maximum compression ratio of 1000:1 or thereabouts. Considering I’ve seen real world JPEGs above 300:1 (https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73909/world.topo.bathy.200412.3x21600x21600.A2.jpg https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/739...) I would not be surprised if you could craft a jpeg getting very close to or exceeding 4 digits.
  - cogman10 1y ago
    The reason it doesn't work with JPEG is JPEG isn't a description of individual pixels but rather how you'd calculate what the individual pixel should be. That's part of the reason you can progressively load jpeg data. PNG is actually a description of the RGB value for the individual pixels. That's why I believe you could png bomb, you could have a 2 billion by 2 billion black pixel image which would ultimately eat up a bunch of space in your GPU and memory to decode. Perhaps something similar is possible with a JPEG, but it's really nothing to do with the compression info. JPEGs have a max size of 65,535×65,535, which would keep you from exploding them.
    - LegionMammal978 1y ago
      DEFLATE can only obtain a best-case compression ratio approaching 1032:1. (Put the byte to repeat in a preceding block, and set "0" = 256 and "1" = 285 for the literal/length code and "0" = 0 for the distance code. Then "10" will output 258 bytes.) This means a 2 Gpx × 2 Gpx PNG image will still be at least ~3.875 PB. If you send it compressed over the wire, you could get another factor of 1032, or perhaps more depending on which algorithms the client supports. Also, you could generate it on demand as a data stream. Bit these run the risk of the client stopping the transfer before ever trying to process the image.

albinowax_ 1y ago

That Evolution mail caching behaviour is really sketchy. I wonder if it could be used for an exploit in the right scenario. If nothing else, it’s a good way to make an email that looks completely different depending on which client it’s opened in.

johannes1234321 1y ago
> it’s a good way to make an email that looks completely different depending on which client it’s opened in. Well, for that use the differences in HTML&CSS support and filtering ... I guess the reason they added this was that they noticed many mails contain same tracking images and decided to cut of tracking data that way.
- mike-cardwell 1y ago
  I don't think this was done on purpose. If the query string is "?a=b" that's fine, and it's used in the cache filename. But if the query string is "?a" then it's excluded from the cache filename. Either way, the correct full URL is fetched with the full query string. It's just how it's cached that is affected.
  - deleted 1y ago
    [deleted]

jokoon 1y ago

Isn't that trivial to prevent zip bombs?

masklinn 1y ago
Most things are trivial to prevent if you know of and think to check for them.

anthk 1y ago

How does it work with Claws Mail/Sylpheed?

upofadown 1y ago

Yet another reason to prevent emails from downloading stuff from remote servers... It appears that you can't do these sorts of things with with CID embedded images...

austinjp 1y ago

Hmm that reminds me, it idly crossed my mind recently about whether AIs with online RAG have decent zip-bomb protection. This thought was provoked when I realised Perplexity would find and download and (apparently) analyse spreadsheet content. I'm sure there are zip-bomb equivalents in binary formats like .xlsx, PDF, .docx, etc.

mxmlnkn 1y ago
> I'm sure there are zip-bomb equivalents in binary formats like .xlsx, PDF, .docx, etc. Yes. Both, docx and xlsx are literally just a zip of XML files with a different extension. PDF can contain zlib streams, which use deflate compression just as gzip, so all the mentioned methods apply to all three formats.
mk_stjames 1y ago
In addition to zip bombing AIs with file parsers, I've wondered about 'context bombs' in the sense of trigger phrases that trip up LLMs into getting stuck into repeating phrases or reasoning evaluations without ever hitting an end of sequence (EOS) token, thus running a system up against API call limits / burning credits / effectively ddosing services etc. Due to the inherent fuzziness/diversity in all models right now I don't think there is a universal approach to this idea but it is something people deploying these systems may want to try and detect.

grahameb 1y ago

I ran into one of these in the very early 00s; was working at a university (back in the days when a couple of people would run all the central servers, running Linux on beige PCs.) We had some anti-spam/AV software that looked at every incoming email hooked into Postfix, and the server kept running out of disk space. Eventually tracked it down to an email which contained a zip of stock trading data – just the three letter stock code and the shift. It wasn't malicious, it just had an extraordinarily high compression ratio!

imcritic 1y ago

The owner of this site can just fuck off with their nationalism.