6 ms·

Yet Another Zip Trick

105 points by todsacerdoti 1y ago

o11c 1y ago
I don't see anything "another" about this; this problem is well known by $((CURRENTYEAR-10)) or so.
- sp0rk 1y ago
  The author explains in the article that they previously gave a presentation outlining various techniques to achieve a "schizophrenic" zip file. The blog post discusses an additional technique that was not present in their previous presentation.
- amelius 1y ago
  Ok, so how do you handle them? Do you take screenshots of your PDFs before you forward them? Other ideas?
  - dbdr 1y ago
    You can forward the PDF itself, not the zip file.
    - amelius 1y ago
      I don't know much about the subject but I do know that PDF is an executable format, and I suppose it could use fingerprinting (based on e.g. screen size or resolution) to become schizophrenic (?)
- gynvael 1y ago
  Author here :) Are you saying that parser discrepancy is a well known problem OR that this exact technical issue described (CD offset vs CD size used to derive CD location) is a well known problem? If the prior, you are of course right — parser discrepancy and the existence of these kinds of "multiple personality" files isn't a new thing. I'm pretty sure this can be traced back to at least the '80 :) However, if you mean the latter, I would appreciate a link if you can recall where you've seen it before / read about it — I can then add it to the article.
soupfordummies 1y ago
Obviously it sucks in the real world but I do always appreciate the cleverness of exploits like these.
- netsharc 1y ago
  The described exploit seems theoretical. In order to create the schizophrenic ZIP, the attacker would have to figure out what ZIP stacks are being used and ensure they act differently - if the 2 departments use the same stack, then the exploit can't work, can it?
  - B1FF_PSUVM 1y ago
    Like spam, the exploit would still be profitable if only a small fraction worked.
  - wat10000 1y ago
    A more realistic attack would be something like, slipping a malicious payload past a scanner by emailing a zip file that appears innocent when unpacked with the scanner’s zip implementation but produces malware when unpacked with the email client’s implementation. There’s a decent chance they’ll be different, and it wouldn’t be too hard to guess which ones a target might be using.
    - o11c 1y ago
      Often you don't have to guess, just use how the software responds as an oracle.
  - JdeBP 1y ago
    None of this stuff is theoretical. It's just old. There was a time when passing ZIP files around was a very popular method of software distribution, and things like this were gotchas that had to be watched for. It was widely known, at least amongst sysops, that the varied toolsets that handled ZIP archives were functionally different. And there were scanners and sanity checkers, and bugfixes to PKUNZIP, that dealt in this stuff for uploaded files and FREQ responses. Did people exploit the differences? Yes. Although it was mainly on the level of creating prank ZIP files on non-Microsoft operating systems with 8.3 filenames such as "PRN" or "CLOCK$". * https://groups.google.com/g/alt.comp.virus/c/zLV-Y2a71gs/m/Un3c5fFXnVoJ https://groups.google.com/g/alt.comp.virus/c/zLV-Y2a71gs/m/U... However, the truly terrible idea of self-extracting archives was popular, which meant that archives with "interesting" arrangements of the archive within the overall file were widespread. ZIP comments were also liberally applied and altered by pretty much every BBS that passed an archive along. And the Unix people wanted to be able to use pipes, something that the MS-DOS original never had to cater for. Also, there were people who exploited the fact that different tools took different things as gospel. Even within the past decade one can find people still being caught out by the fact that there's a header field that instructs what the pathname separator character(s) used are; and that ZIP tools that expect non-seekable streams operate differently to ZIP tools that expect seekable regular files.
wingmanjd 1y ago
Since docx files are similar to a zip file with the extension changed, could this trick fake out Microsoft Word?
- mlyle 1y ago
  The trick depends upon different implementations doing different things. Not likely for Word (though I suppose it is -possible- across different versions or different OSes).
  - netsharc 1y ago
    To respond to Grandfather comment, modern Office files are really just ZIPs with different extensions, they even have the magic string "PK" at the very beginning of the file. I do wonder, since a lot of tools outside of the MS ecosystem can read Office files (e.g. LibreOffice and Google Docs as well as plenty of other online tools), if indeed the hack as described by the article is possible. One would just need to figure out the ZIP stacks used by said tools.
    - saghm 1y ago
      You can even just rename a docx file to use the zip extension and then manually unzip it for those curious. If I remember correctly, the contents are XML files with structure encoding the formatting around the content.
  - larschdk 1y ago
    The Office365 online and desktop implementations of zip could be different.
  - hnlmorg 1y ago
    It’s very common for organisations to only give expensive MS Office licenses to a subset of employees while the rest rely on O365 or Google Docs. Then you have people on Linux or macOS who might also use LibreOffice, Apples Office suite, or something else entirely. And given MS Office is the de facto standard, you’ll often see people open OOXML documents within non-MS office suites. After all, OOXML is an open standard (sarcasm). ODF (the document formats favoured by most other office suites) is also ZIP-based XML. So they too could be vulnerable.
socalgal2 1y ago
It is not a schizophrenic zip file to have inline headers that are not referenced in the TOC. the TOC is the only source of truth in a zip file. It was designed this way in purpose so that you can add new versions of files on your 20 disk zip without having to re-write all 20 disks. Pkzip would read the TOC from disk 20, append your new file to disk 20 (or 21 if there was not enough space) and then write a new TOC at the end that does not reference the old file still in the zip. That is by design. Reading anything other than the files in the TOC is an invalid zip reader
- masklinn 1y ago
  While that is true (even if some people really want to stream read zips and will thus trigger tar-style conflicts) that is not what the article is about. The article is really more about software using the "size of central directory" field to find the central directory (a sin Python's standard library apparently commits) versus the "offset of central directory". A commenter on an other post of that article noted that technically there's no reason for the central directory to be right before the EOCD so seeking backwards from the EOCD by the size of CD is just incorrect. In fact zip was designed such that the central directory could be split across multiple disks (and later files), so it was not possible to guarantee a simple backwards jump from the EOCD to the start of CD.
  - cxr 1y ago
    > technically there's no reason for the central directory to be right before the EOCD so seeking backwards from the EOCD by the size of CD is just incorrect Yes, and in fact APPNOTE.TXT expects implementations to deal with discontiguous CD/EOCD records; it actually mandates that some records appear between them. (It's where ZIP64-specific records go, for example.) The whole thing seems to stem from a belief by the author of this post that Info-ZIP is the canonical implementation. (This is of course wrong. That would be PKZIP.) If Info-ZIP is exhibiting the behavior described, then the Info-ZIP implementation is flat out wrong. (This has happened before. See <https://news.ycombinator.com/item?id=27925393 https://news.ycombinator.com/item?id=27925393> where the author of that post provides pushback to Mark Adler, receives pushback from Adler in response, and then just stops pushing and defers to Adler's reading. Adler is an expert in his domain, and he's made valuable contributions to free software, but his expertise is in compression, and he's not the authority on the spec or the format; Adler didn't create ZIP or write APPNOTE.TXT—Phil Katz did.)
jonathanlydall 1y ago
PDFs in .zip files is in itself a huge smell to begin with. I don't think I've actually ever received a zipped up .pdf, so it's clearly not legitimately necessary for anyone to do so and should you ever see it you should treat it highly suspiciously. I get admin@ emails for our company domain and there is a somewhat steady stream of run of the mill fraud attempts. A trick I've seen happen quite a lot recently is emails with .svg attachments, which have some lightly obfuscated JavaScript in them and which ultimately redirects you to some dodgy looking URL (which I never visited). I simply made a rule to outright reject all .svg files from external sources and I get a report any time it's attempted. In about the last 12 months this has been running, it's probably blocked about 20 incoming emails and only one of those was a false positive and even the false positive was a weird case as we were sending a .svg file to a creative company who for some reason had our .svg attachment in their reply back to us.
- Joker_vD 1y ago
  Well, there is a case for PDFs in a .zip, but it's when you're e.g. downloading some set of documents from your e-government site, and it's like 4 PDFs, all cryptographically signed: so you get a .zip with 4 PDFs in it and 4 .sig files with matching names. But in emails? Just attach however many PDFs you need and send, they don't really compress anyway; and I think most web-mail fronts actually allow you to download all the attachments as a single .zip — but obviously those .zip-files are not maliciously crafted (I hope, at least). Also, now that I think of it, forwarding the PDF you've extracted and visually reviewed instead of the original .zip-file would defeat this attack (unless, of course, it's the PDF file that's schizophrenic).
  - b112 1y ago
    The other reason is to use zip's incredibly mega super secure password protection, to keep super secret docs safe. (I see it all the time. Along with the password in the very same email)
    - gcarvalho 1y ago
      I’m not saying this is good security hygiene, but not necessarily everyone with access to the file will also have access to the password in the email. e.g. someone downloaded the password protected zip on a public computer, logged out of their email, but forgot to delete the file.
deleted 1y ago
[deleted]

105 points by todsacerdoti 1y ago

o11c 1y ago

I don't see anything "another" about this; this problem is well known by $((CURRENTYEAR-10)) or so.

sp0rk 1y ago
The author explains in the article that they previously gave a presentation outlining various techniques to achieve a "schizophrenic" zip file. The blog post discusses an additional technique that was not present in their previous presentation.
amelius 1y ago
Ok, so how do you handle them? Do you take screenshots of your PDFs before you forward them? Other ideas?
- dbdr 1y ago
  You can forward the PDF itself, not the zip file.
  - amelius 1y ago
    I don't know much about the subject but I do know that PDF is an executable format, and I suppose it could use fingerprinting (based on e.g. screen size or resolution) to become schizophrenic (?)
gynvael 1y ago
Author here :) Are you saying that parser discrepancy is a well known problem OR that this exact technical issue described (CD offset vs CD size used to derive CD location) is a well known problem? If the prior, you are of course right — parser discrepancy and the existence of these kinds of "multiple personality" files isn't a new thing. I'm pretty sure this can be traced back to at least the '80 :) However, if you mean the latter, I would appreciate a link if you can recall where you've seen it before / read about it — I can then add it to the article.

soupfordummies 1y ago

Obviously it sucks in the real world but I do always appreciate the cleverness of exploits like these.

netsharc 1y ago
The described exploit seems theoretical. In order to create the schizophrenic ZIP, the attacker would have to figure out what ZIP stacks are being used and ensure they act differently - if the 2 departments use the same stack, then the exploit can't work, can it?
- B1FF_PSUVM 1y ago
  Like spam, the exploit would still be profitable if only a small fraction worked.
- wat10000 1y ago
  A more realistic attack would be something like, slipping a malicious payload past a scanner by emailing a zip file that appears innocent when unpacked with the scanner’s zip implementation but produces malware when unpacked with the email client’s implementation. There’s a decent chance they’ll be different, and it wouldn’t be too hard to guess which ones a target might be using.
  - o11c 1y ago
    Often you don't have to guess, just use how the software responds as an oracle.
- JdeBP 1y ago
  None of this stuff is theoretical. It's just old. There was a time when passing ZIP files around was a very popular method of software distribution, and things like this were gotchas that had to be watched for. It was widely known, at least amongst sysops, that the varied toolsets that handled ZIP archives were functionally different. And there were scanners and sanity checkers, and bugfixes to PKUNZIP, that dealt in this stuff for uploaded files and FREQ responses. Did people exploit the differences? Yes. Although it was mainly on the level of creating prank ZIP files on non-Microsoft operating systems with 8.3 filenames such as "PRN" or "CLOCK$". * https://groups.google.com/g/alt.comp.virus/c/zLV-Y2a71gs/m/Un3c5fFXnVoJ https://groups.google.com/g/alt.comp.virus/c/zLV-Y2a71gs/m/U... However, the truly terrible idea of self-extracting archives was popular, which meant that archives with "interesting" arrangements of the archive within the overall file were widespread. ZIP comments were also liberally applied and altered by pretty much every BBS that passed an archive along. And the Unix people wanted to be able to use pipes, something that the MS-DOS original never had to cater for. Also, there were people who exploited the fact that different tools took different things as gospel. Even within the past decade one can find people still being caught out by the fact that there's a header field that instructs what the pathname separator character(s) used are; and that ZIP tools that expect non-seekable streams operate differently to ZIP tools that expect seekable regular files.

wingmanjd 1y ago

Since docx files are similar to a zip file with the extension changed, could this trick fake out Microsoft Word?

mlyle 1y ago
The trick depends upon different implementations doing different things. Not likely for Word (though I suppose it is -possible- across different versions or different OSes).
- netsharc 1y ago
  To respond to Grandfather comment, modern Office files are really just ZIPs with different extensions, they even have the magic string "PK" at the very beginning of the file. I do wonder, since a lot of tools outside of the MS ecosystem can read Office files (e.g. LibreOffice and Google Docs as well as plenty of other online tools), if indeed the hack as described by the article is possible. One would just need to figure out the ZIP stacks used by said tools.
  - saghm 1y ago
    You can even just rename a docx file to use the zip extension and then manually unzip it for those curious. If I remember correctly, the contents are XML files with structure encoding the formatting around the content.
- larschdk 1y ago
  The Office365 online and desktop implementations of zip could be different.
- hnlmorg 1y ago
  It’s very common for organisations to only give expensive MS Office licenses to a subset of employees while the rest rely on O365 or Google Docs. Then you have people on Linux or macOS who might also use LibreOffice, Apples Office suite, or something else entirely. And given MS Office is the de facto standard, you’ll often see people open OOXML documents within non-MS office suites. After all, OOXML is an open standard (sarcasm). ODF (the document formats favoured by most other office suites) is also ZIP-based XML. So they too could be vulnerable.

socalgal2 1y ago

It is not a schizophrenic zip file to have inline headers that are not referenced in the TOC. the TOC is the only source of truth in a zip file. It was designed this way in purpose so that you can add new versions of files on your 20 disk zip without having to re-write all 20 disks. Pkzip would read the TOC from disk 20, append your new file to disk 20 (or 21 if there was not enough space) and then write a new TOC at the end that does not reference the old file still in the zip. That is by design. Reading anything other than the files in the TOC is an invalid zip reader

masklinn 1y ago
While that is true (even if some people really want to stream read zips and will thus trigger tar-style conflicts) that is not what the article is about. The article is really more about software using the "size of central directory" field to find the central directory (a sin Python's standard library apparently commits) versus the "offset of central directory". A commenter on an other post of that article noted that technically there's no reason for the central directory to be right before the EOCD so seeking backwards from the EOCD by the size of CD is just incorrect. In fact zip was designed such that the central directory could be split across multiple disks (and later files), so it was not possible to guarantee a simple backwards jump from the EOCD to the start of CD.
- cxr 1y ago
  > technically there's no reason for the central directory to be right before the EOCD so seeking backwards from the EOCD by the size of CD is just incorrect Yes, and in fact APPNOTE.TXT expects implementations to deal with discontiguous CD/EOCD records; it actually mandates that some records appear between them. (It's where ZIP64-specific records go, for example.) The whole thing seems to stem from a belief by the author of this post that Info-ZIP is the canonical implementation. (This is of course wrong. That would be PKZIP.) If Info-ZIP is exhibiting the behavior described, then the Info-ZIP implementation is flat out wrong. (This has happened before. See <https://news.ycombinator.com/item?id=27925393 https://news.ycombinator.com/item?id=27925393> where the author of that post provides pushback to Mark Adler, receives pushback from Adler in response, and then just stops pushing and defers to Adler's reading. Adler is an expert in his domain, and he's made valuable contributions to free software, but his expertise is in compression, and he's not the authority on the spec or the format; Adler didn't create ZIP or write APPNOTE.TXT—Phil Katz did.)

jonathanlydall 1y ago

PDFs in .zip files is in itself a huge smell to begin with. I don't think I've actually ever received a zipped up .pdf, so it's clearly not legitimately necessary for anyone to do so and should you ever see it you should treat it highly suspiciously. I get admin@ emails for our company domain and there is a somewhat steady stream of run of the mill fraud attempts. A trick I've seen happen quite a lot recently is emails with .svg attachments, which have some lightly obfuscated JavaScript in them and which ultimately redirects you to some dodgy looking URL (which I never visited). I simply made a rule to outright reject all .svg files from external sources and I get a report any time it's attempted. In about the last 12 months this has been running, it's probably blocked about 20 incoming emails and only one of those was a false positive and even the false positive was a weird case as we were sending a .svg file to a creative company who for some reason had our .svg attachment in their reply back to us.

Joker_vD 1y ago
Well, there is a case for PDFs in a .zip, but it's when you're e.g. downloading some set of documents from your e-government site, and it's like 4 PDFs, all cryptographically signed: so you get a .zip with 4 PDFs in it and 4 .sig files with matching names. But in emails? Just attach however many PDFs you need and send, they don't really compress anyway; and I think most web-mail fronts actually allow you to download all the attachments as a single .zip — but obviously those .zip-files are not maliciously crafted (I hope, at least). Also, now that I think of it, forwarding the PDF you've extracted and visually reviewed instead of the original .zip-file would defeat this attack (unless, of course, it's the PDF file that's schizophrenic).
- b112 1y ago
  The other reason is to use zip's incredibly mega super secure password protection, to keep super secret docs safe. (I see it all the time. Along with the password in the very same email)
  - gcarvalho 1y ago
    I’m not saying this is good security hygiene, but not necessarily everyone with access to the file will also have access to the password in the email. e.g. someone downloaded the password protected zip on a public computer, logged out of their email, but forgot to delete the file.

deleted 1y ago

[deleted]