8 ms·

Hackers use ZIP file concatenation to evade detection

207 points by sandwichsphinx 2y ago

kencausey 2y ago
Related: https://isc.sans.edu/forums/diary/zipdump+Evasive+ZIP+Concatenation/31426/ https://isc.sans.edu/forums/diary/zipdump+Evasive+ZIP+Concat... https://isc.sans.edu/forums/diary/zipdump+PKZIP+Records/31428/ https://isc.sans.edu/forums/diary/zipdump+PKZIP+Records/3142...
avidiax 2y ago
This is sometimes used non-maliciously to concatenate zipped eBooks to a JPEG of the cover art. 4Chan's /lit/ board used to do this, but I can't find any reference to it anymore. https://entropymine.wordpress.com/2018/11/01/about-that-jpeg-zip-shakespeare-hybrid-file/ https://entropymine.wordpress.com/2018/11/01/about-that-jpeg... https://github.com/Anti-Forensics/tucker https://github.com/Anti-Forensics/tucker
- theoreticalmal 2y ago
  I think the reason it’s not used anymore is because it was used maliciously to share CSAM on other boards and 4chan banned uploading anything that looks like a concatenated zip
  - yapyap 2y ago
    :/
- bsimpson 2y ago
  That first read should be an HN post in its own right.
  - bblb 2y ago
    https://news.ycombinator.com/item?id=18342042 https://news.ycombinator.com/item?id=18342042
- 47282847 2y ago
  FYI, the blog post describes a zip file embedded in the ICC profile data of a JPEG in order to survive web image transformations, whereas the linked Tucker script is just appending the zip to the image.
486sx33 2y ago
Encrypted ZIP files have long been a way to evade any sort of malware detection during transmission.
- mmcgaha 2y ago
  I doubt it still works but things I needed to get through email I would embed in word documents.
  - mplewis9z 2y ago
    A modern word document file (.docx) is literally just a Zip archive with a special folder structure, so unless your company is scanning word document contents I can’t imagine there’s any issue.
  - cogman10 2y ago
    Would probably still work. There's just too many formats which makes it very hard for a content blocker to really stop. I pity the programmer that has to decode the 1000 versions of xls to find the binary blob that could be a virus.
    - telgareith 2y ago
      1000? No. There's two. Openxml and the original xls. OpenXML can be scanned for issues like any other XML file. Alas, it's more difficult to get excel to accept that it shouldn't delete leading zeros than it is to check a spreadsheet's sus-o-scale.
      - cogman10 2y ago
        1000 is an over exaggeration, but it is not just 2 standards. xls morphed with every version of Microsoft Excel. MS Excel has pretty good backwards compatibility, but making an xls parser is notoriously hard because of the many differences between versions.
- 0cf8612b2e1e 2y ago
nh2 2y ago
WinRAR does it right, 7zip and Windows Explorer do it wrong according to https://en.m.wikipedia.org/wiki/ZIP_(file_format) https://en.m.wikipedia.org/wiki/ZIP_(file_format) > only files specified in the central directory at the end of the file are valid. Scanning a ZIP file for local file headers is invalid (except in the case of corrupted archives)
- leni536 2y ago
  This contradicts the specification, which explicitly supports stream-processing zip files, which necessarily can't happen if your source of truth is the central directory record. Unless if you can wrap your stream processing in some kind of transaction that you can drop once you discover foul play.
  - nh2 2y ago
    Source what you're referring to / explanation? In ZIP, later info wins. I don't see how that isn't always streamable.
    - leni536 2y ago
      Hmm, it appears that you are right. I vaguely remembered that zip was streamable, but it appears that it only means that it's stream writable, as in you can write zip file contents from a datastream of unknown size, and append the checksum and file size later in the zip file stream. However such a zip file is definitely not stream readable, as the local file header no longer contains the size of the following file data, so you can't know where to end a given file. So for reading you definitely have to locate the central directory record. In my defense the spec says[1]. > 4.3.2 Each file placed into a ZIP file MUST be preceded by a "local file header" record for that file. Each "local file header" MUST be accompanied by a corresponding "central directory header" record within the central directory section of the ZIP file. Then in 4.3.6 it describes the file format, which seems to be fundamentally incompatible to altering zip files by appending data, as the resulting file would not conform to this format. So basically some implementations (maybe opportunistically, relying on compressed sizes being available in the local file headers) stream read from the zip file, assuming that it's a valid zip file, but not validating. Some other implementations only use the central directory record at the end, but don't validate the file format either. A validating zip file parser should be possible, by locating the central directory record and checking that the referred files with their metadata fully cover the file contents of the zip files, without gaps and overlaps. But this probably won't win any benchmarks. [1] https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
wahern 2y ago
From a security perspective, and as a programmer, I've never liked ZIP files precisely because there are two mechanisms to identify the contents, the per-file header and the central directory. When you're defining a format, protocol, or w'ever, ideally there should be a single source of truth, a single valid & useable parse, etc; basically, the structure of the data or process should be intrinsically constraining. There shouldn't be a pathway for multiple implementations to produce different functional results, and ZIP archives are in my mind the archetype for getting this wrong. tar files aren't ideal, but in the abstract (ignoring issues with long file names) they don't have this problem. (tar files don't support random access, either, but better to rely on something suboptimal than something that's fundamentally broken.) A similar security problem, though not as fundamentally baked into the format, is MIME parsing. The header section is supposed to be delimited from the body by an empty line (likewise for nested entities). But what if it's not? For better or worse, Sendmail was tolerant of the absence of an empty line and treated as headers everything up to the first line that didn't parse as a header or header continuation.[1] Later systems, like Postfix, originally copied this behavior. But Microsoft Exchange and Outlook are even more tolerant, yet in a much more horrendous way, by parsing as a header anything that looks like a Content-Type or related header immediately after the first empty line. They have similar hacks for other, similar violations. So today, depending on the receiving software, you can send messages that appear differently, including having different attachments. It's a security nightmare! I not a Postel's Law hater, but ZIP archives and Microsoft's MIME parsing behaviors are just egregiously wrong and indefensible. And even if you think the Robustness Principle is inherently bad policy, you still have to design your formats, protocols, and systems to be as intrinsically constraining as possible. You can't rely on vendors adhering to a MUST rule in an RFC, unless it's unquestioningly crystal clear what the repercussions will be--everybody else will (because it's the natural and convenient thing to do) reject your output as trash and drop it on the floor immediately so violations never have a chance to get a foothold. [1] MTAs don't necessarily need to care about MIME parsing, but Sendmail eventually gained features where parsing message contents mattered, setting the de facto norm (for those paying attention) until Microsoft came along.
- failbuffer 2y ago
  In a similar vein, HTTP header smuggling attacks exploit differences in header parsing. For instance, a reverse proxy and a web server might handle repetition of headers or the presence of whitespace differently.
pjdkoch 2y ago
Now?
waltbosz 2y ago
I've done similar stuff. Concat a zip (that keeps throwing false positives) to a jpg and scanners will treat it like a jpg. Then write a script that chops off the jpg to access the zip. All this so I could automate a web a app deploy.
- ttyprintk 2y ago
  Or attach a zip through Exhange
Retr0id 2y ago
Related, my two favourite ZIP parser issues: https://bugzilla.mozilla.org/show_bug.cgi?id=1534483 https://bugzilla.mozilla.org/show_bug.cgi?id=1534483 "Ambiguous zip parsing allows hiding add-on files from linter and reviewers" https://issues.chromium.org/issues/40082940 https://issues.chromium.org/issues/40082940 "Security: Crazy Linker on Android allows modification of Chrome APK without breaking signature" The big problem with the ZIP format is that although the "spec" says what a ZIP file looks like, it does not tell you in concrete terms how to parse it, leading to all sorts of ambiguities and divergent implementations. Someone needs to write a "strict ZIP" spec that has explicit and well-defined parsing rules, and then we need to get every existing ZIP implementation to agree to follow said spec.
- wyldfire 2y ago
  Or: better yet, just use an archive format for archival and a compression layer for compression. Don't use zip at all.
  - Retr0id 2y ago
    What non-compressing archive format would you suggest? tar doesn't support random access which is a non-starter for many use cases.
    - deleted 2y ago
      [deleted]
    - acka 2y ago
      DAR (Disk ARchiver)[1] looks to be a good alternative. It supports random access, encryption, and individual file compression within the archive. [1] http://dar.linux.free.fr/ http://dar.linux.free.fr/
      - Retr0id 2y ago
        That seems counter to GP's suggestion of doing compression at a separate layer
_pdp_ 2y ago
This attack vector has been known for at least 17 years, if not longer. https://gnucitizen.org/blog/java-jar-attacks-and-features/ https://gnucitizen.org/blog/java-jar-attacks-and-features/ https://gnucitizen.org/blog/gifars-and-other-issues/ https://gnucitizen.org/blog/gifars-and-other-issues/
- tedk-42 2y ago
  I'm with you. I've evaded all sorts of scanning tools by base64 encoding data (i.e. binary data) to and copy pasting the text from insecure to highly secured environments. At the end of the day, these malware databases rely on hashing and detecting for known bad hashes and there are lots of command line tools to help get over that sort of thing like zip/tar etc.
  - athrowaway3z 2y ago
    rot13 must be outlawed for its use by cyber-criminals!
  - trollbridge 2y ago
    I used to have a workflow for updating code inside a very highly secure environment that relied on exactly this: Run build of prior version, run build of current version, run diff against them, compress with xz -9, base64 encode, generate output, base64 encode, e-mail it to myself, copy text of email, type "openssl base64 -d | unxz | bash", right click. E-mailing this was completely fine according to the stringent security protocols but e-mailing a zip of the code, etc. was absolutely 100% not. That would have to go on the vendor's formal portal. (Eventually I just opened my own "portal" to upload binaries to, put the vendor I worked for's logo on it, and issued a statement saying it was an official place to download binaries from the vendor. But sometimes their WAF would still mangle downloads or flag them as a risk, so I made sure builds had options of coming in an obfuscated base64 format.)
- cyberax 2y ago
  17 years? We played tricks with zip bombs that used this approach during 90-s.
mtnGoat 2y ago
Nice remix of an old technique. I remember file packing exes together for fun and profit back in the day.
unnouinceput 2y ago
Quote: "To defend against concatenated ZIP files, Perception Point suggests that users and organizations use security solutions that support recursive unpacking." That's the worse advice actually. You want the hidden shit to stay there unable to be seen by default programs. That's how you got all the crap in Windows mail starting from 90's when Outlook started to trying to be "smart" and automatically detect and run additional content. Be dumb and don't discover anything, let it rot in there. The only one that should do this is the antivirus, rest of unpackers/readers/whatever stay dumb.
- brabel 2y ago
  I agree. The ZIP definition is extremely clear that the contents of the ZIP are defined by the single Central Directory at the end of the file. Local headers are only valid if pointed to by the Central Directory. Any other local headers are just supposed to be treated as garbage, except by software that is specifically meant to recover corrupted ZIP archive's contents.
Jerrrrrrry 2y ago
Remember, secure encryption, good compression, and truely random data are indistinguishable. It's best to paste that encrypted payload into a JPG with some bullshit magic headers and upload that to a trusted Exfil pivot instead. Or, to get SuperMarioKart.rom to work with your chromeApp-XEMU emulator to play during down-time at work, just rename it to SMB.png, and email it you yourself.
- seanhunter 2y ago
  > Remember, secure encryption, good compression, and truely random data are indistinguishable. Yes, and the only reason the bad guys get away with this is the people who trust signature-based scanning at the perimeter to detect all threats. One of the hacks I'm most proud of in my whole career was when we were doing a proof of concept at an enterprise client we were being deliberately obstructed by the internal IT group due to politics between their boss and the boss who sponsored our POC. For unrelated trademark-related reasons we were prevented by a third party from having the software on physical media but we had a specific contractual clause agreeing to let us download it for install. So while we had been contractually engaged to provide this software and we had a strict deadline to prove value, the enterprise IT group were preventing us from actually getting it through the virus-scanning firewall to get it installed. What to do? The scanner looked for the signature of executable or zipped files and blocked them. It would also block any files larger than a certain size. So what I did was write two shell scripts called "shred" and "unshred". "Shred" would take any files you gave it as input, make them into a tarball, encrypt that to confuse the virus scanner and then split it up into chunks small enough to get through the firewall, and "unshred" would reverse this. This almost worked, but I found that the first chunk was always failing to transmit through the firewall. The scanner noticed some signature that openssl was putting at the front of the file when encrypting it. The solution? Change shred to add 1k of random noise to the front of the file and unshred to remove it. Job done. Our files were transmitted perfectly (I got the scripts to check the md5sum on both sides to be sure), and though the process was slow, we could continue. The funny thing was the POC was a bake-off versus another (more established) vendor and they couldn't get their software installed until they had done a couple of weeks of trench warfare with enterprise IT. "To keep things fair" the people organising the POC decided to delay to let them have time to install, and eventually the person blocking us from installing was persuaded to change their mind (by being fired), so "shred" and "unshred" could be retired.
deleted 2y ago
[deleted]
solatic 2y ago
Not really a new technique. A long time ago in a galaxy far far away, I needed to get libraries from the internet onto an air-gapped network, and the supported way was to burn them to a disk and bring the disk to be scanned. The scanner never allowed executables so it would always reject the libraries. Can't make this up, but the supported way that InfoSec (!!) explained to us to get past the scanner was to take advantage of WinRAR being available on the network, so split the rar archive a bunch of times (foo.r01, foo.r02,...) and the scanner, being unable to parse them nor execute them, would happily rubber-stamp them and pass them along. As long as the process was followed, InfoSec was happy. Thankfully this was before the days when people were really worried about supply chain security. Glad to see this bit of security theater recognized as such.
PreInternet01 2y ago
> To defend against concatenated ZIP files, Perception Point suggests that users and organizations use security solutions that support recursive unpacking Yeah, or, you know, just outright reject any ZIP file that doesn't start with a file entry, where a forward-scan of the file entries doesn't match the result of the central-directory-based walk. There is just so much malicious crud coming in via email that you just want to instantly reject anything that doesn't look 'normal', and you definitely don't want to descend into the madness of recursive unpacking, 'cuz that enables another class of well-known attacks. And no, "but my precious use-case" simply doesn't apply, as you're practically limited to a whole 50MB per attachment anyway. Sure, "this ZIP file is also a PDF is also a PNG is also a NES cartridge which displays its own MD5" (viz https://github.com/angea/pocorgtfo/tree/master/writeups/19 https://github.com/angea/pocorgtfo/tree/master/writeups/19) has a place (and should definitely be required study material for anyone writing mail filters!), but business email ain't it.
- exmadscientist 2y ago
  That's fair, but do realize that sometimes people do have to send around archives from the last century (they got archived for a reason!) or created by eldritch-horror tools that just make weird files (which, sometimes, are the gold masters for certain very important outputs...). And it's kind of annoying when these weird but standard files get silently dropped. Especially when that same file went through just fine yesterday, before the duly zealous security settings changed for whatever reason. All I'm saying is, don't drop my stuff silently because your code couldn't be arsed to deal with (ugly) standard formats. At least give me a warning ("file of type not scannable" or whatever, the actual words are not so important). And then when I have to yell at the Shanghai people I can yell at them for the correct reasons.
  - PreInternet01 2y ago
    Oh, nothing gets dropped silently, but bounced right back with `550 5.7.1 Message rejected due to content (Attachment refused: MATCH-code)`. And for anything oversized, funny or otherwise non-standard, we offer a very convenient file transfer service.
rty32 2y ago
I checked the time to make sure today is in the year of 2024. I swear this has been widely known at least since the Win 98 era.
nxnxhxnxhz 2y ago
[flagged]
alkonaut 2y ago
Why do scanners need to look inside compressed archives at all? If the malicious file is extracted, it can be detected then. If it’s not extracted and instead used directly from the archive, then the offending code that executes malicious payloads from inside archives should be detectable? Is that the role of AutoIt in this scenario?
- rocqua 2y ago
  Because they want to check on the wire, before it hits an endpoint. Common situation is detecting malware sent through a phishing mail. You want to intercept those before a user can unpack the file.
  - canucker2016 2y ago
    And "Defense In Depth" - the more layers that bad actors have to avoid/circumvent reduces the chance of their success. see https://www.fortinet.com/resources/cyberglossary/defense-in-depth https://www.fortinet.com/resources/cyberglossary/defense-in-...
  - alkonaut 2y ago
    Yes that was my question: "why, do you want to do that when it sounds like a futile job to do?" If you have any kind of transformation (whatever trivial ROT or XOR) of the payload then there is no chance of detecting it via pattern matching, and if you need a "program" to process it into its malicious form, then how does detection work? I understand you want to keep malicious payload from reaching endpoints in a form that would risk being consumed (malformed documents, images causing buffer overflows, executable and script files). But beyond that?
yieldcrv 2y ago
What year is it
user070223 2y ago
Last I checked virustotal doesn't test nested archives for viruses even though it's an issue old as modern computing
- canucker2016 2y ago
  VirusTotal just passes on the file to the actual virus scanners (a for-loop with an api as a front-end) - it's up to each individual virus scanner to scan as they see fit (including scanning any unreferenced holes and compressed zip entries in a zip archive). I have no idea why those virus scanners don't check nested archives. Maybe time/cpu constraints?

Related: https://isc.sans.edu/forums/diary/zipdump+Evasive+ZIP+Concatenation/31426/ https://isc.sans.edu/forums/diary/zipdump+Evasive+ZIP+Concat... https://isc.sans.edu/forums/diary/zipdump+PKZIP+Records/31428/ https://isc.sans.edu/forums/diary/zipdump+PKZIP+Records/3142...

avidiax 2y ago

This is sometimes used non-maliciously to concatenate zipped eBooks to a JPEG of the cover art. 4Chan's /lit/ board used to do this, but I can't find any reference to it anymore. https://entropymine.wordpress.com/2018/11/01/about-that-jpeg-zip-shakespeare-hybrid-file/ https://entropymine.wordpress.com/2018/11/01/about-that-jpeg... https://github.com/Anti-Forensics/tucker https://github.com/Anti-Forensics/tucker

theoreticalmal 2y ago
I think the reason it’s not used anymore is because it was used maliciously to share CSAM on other boards and 4chan banned uploading anything that looks like a concatenated zip
- yapyap 2y ago
  :/
bsimpson 2y ago
That first read should be an HN post in its own right.
- bblb 2y ago
  https://news.ycombinator.com/item?id=18342042 https://news.ycombinator.com/item?id=18342042
47282847 2y ago
FYI, the blog post describes a zip file embedded in the ICC profile data of a JPEG in order to survive web image transformations, whereas the linked Tucker script is just appending the zip to the image.

486sx33 2y ago

Encrypted ZIP files have long been a way to evade any sort of malware detection during transmission.

mmcgaha 2y ago
I doubt it still works but things I needed to get through email I would embed in word documents.
- mplewis9z 2y ago
  A modern word document file (.docx) is literally just a Zip archive with a special folder structure, so unless your company is scanning word document contents I can’t imagine there’s any issue.
- cogman10 2y ago
  Would probably still work. There's just too many formats which makes it very hard for a content blocker to really stop. I pity the programmer that has to decode the 1000 versions of xls to find the binary blob that could be a virus.
  - telgareith 2y ago
    1000? No. There's two. Openxml and the original xls. OpenXML can be scanned for issues like any other XML file. Alas, it's more difficult to get excel to accept that it shouldn't delete leading zeros than it is to check a spreadsheet's sus-o-scale.
    - cogman10 2y ago
      1000 is an over exaggeration, but it is not just 2 standards. xls morphed with every version of Microsoft Excel. MS Excel has pretty good backwards compatibility, but making an xls parser is notoriously hard because of the many differences between versions.
0cf8612b2e1e 2y ago

nh2 2y ago

WinRAR does it right, 7zip and Windows Explorer do it wrong according to https://en.m.wikipedia.org/wiki/ZIP_(file_format) https://en.m.wikipedia.org/wiki/ZIP_(file_format) > only files specified in the central directory at the end of the file are valid. Scanning a ZIP file for local file headers is invalid (except in the case of corrupted archives)

leni536 2y ago
This contradicts the specification, which explicitly supports stream-processing zip files, which necessarily can't happen if your source of truth is the central directory record. Unless if you can wrap your stream processing in some kind of transaction that you can drop once you discover foul play.
- nh2 2y ago
  Source what you're referring to / explanation? In ZIP, later info wins. I don't see how that isn't always streamable.
  - leni536 2y ago
    Hmm, it appears that you are right. I vaguely remembered that zip was streamable, but it appears that it only means that it's stream writable, as in you can write zip file contents from a datastream of unknown size, and append the checksum and file size later in the zip file stream. However such a zip file is definitely not stream readable, as the local file header no longer contains the size of the following file data, so you can't know where to end a given file. So for reading you definitely have to locate the central directory record. In my defense the spec says[1]. > 4.3.2 Each file placed into a ZIP file MUST be preceded by a "local file header" record for that file. Each "local file header" MUST be accompanied by a corresponding "central directory header" record within the central directory section of the ZIP file. Then in 4.3.6 it describes the file format, which seems to be fundamentally incompatible to altering zip files by appending data, as the resulting file would not conform to this format. So basically some implementations (maybe opportunistically, relying on compressed sizes being available in the local file headers) stream read from the zip file, assuming that it's a valid zip file, but not validating. Some other implementations only use the central directory record at the end, but don't validate the file format either. A validating zip file parser should be possible, by locating the central directory record and checking that the referred files with their metadata fully cover the file contents of the zip files, without gaps and overlaps. But this probably won't win any benchmarks. [1] https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT

wahern 2y ago

From a security perspective, and as a programmer, I've never liked ZIP files precisely because there are two mechanisms to identify the contents, the per-file header and the central directory. When you're defining a format, protocol, or w'ever, ideally there should be a single source of truth, a single valid & useable parse, etc; basically, the structure of the data or process should be intrinsically constraining. There shouldn't be a pathway for multiple implementations to produce different functional results, and ZIP archives are in my mind the archetype for getting this wrong. tar files aren't ideal, but in the abstract (ignoring issues with long file names) they don't have this problem. (tar files don't support random access, either, but better to rely on something suboptimal than something that's fundamentally broken.) A similar security problem, though not as fundamentally baked into the format, is MIME parsing. The header section is supposed to be delimited from the body by an empty line (likewise for nested entities). But what if it's not? For better or worse, Sendmail was tolerant of the absence of an empty line and treated as headers everything up to the first line that didn't parse as a header or header continuation.[1] Later systems, like Postfix, originally copied this behavior. But Microsoft Exchange and Outlook are even more tolerant, yet in a much more horrendous way, by parsing as a header anything that looks like a Content-Type or related header immediately after the first empty line. They have similar hacks for other, similar violations. So today, depending on the receiving software, you can send messages that appear differently, including having different attachments. It's a security nightmare! I not a Postel's Law hater, but ZIP archives and Microsoft's MIME parsing behaviors are just egregiously wrong and indefensible. And even if you think the Robustness Principle is inherently bad policy, you still have to design your formats, protocols, and systems to be as intrinsically constraining as possible. You can't rely on vendors adhering to a MUST rule in an RFC, unless it's unquestioningly crystal clear what the repercussions will be--everybody else will (because it's the natural and convenient thing to do) reject your output as trash and drop it on the floor immediately so violations never have a chance to get a foothold. [1] MTAs don't necessarily need to care about MIME parsing, but Sendmail eventually gained features where parsing message contents mattered, setting the de facto norm (for those paying attention) until Microsoft came along.

failbuffer 2y ago
In a similar vein, HTTP header smuggling attacks exploit differences in header parsing. For instance, a reverse proxy and a web server might handle repetition of headers or the presence of whitespace differently.

pjdkoch 2y ago

Now?

waltbosz 2y ago

I've done similar stuff. Concat a zip (that keeps throwing false positives) to a jpg and scanners will treat it like a jpg. Then write a script that chops off the jpg to access the zip. All this so I could automate a web a app deploy.

ttyprintk 2y ago
Or attach a zip through Exhange

Retr0id 2y ago

Related, my two favourite ZIP parser issues: https://bugzilla.mozilla.org/show_bug.cgi?id=1534483 https://bugzilla.mozilla.org/show_bug.cgi?id=1534483 "Ambiguous zip parsing allows hiding add-on files from linter and reviewers" https://issues.chromium.org/issues/40082940 https://issues.chromium.org/issues/40082940 "Security: Crazy Linker on Android allows modification of Chrome APK without breaking signature" The big problem with the ZIP format is that although the "spec" says what a ZIP file looks like, it does not tell you in concrete terms how to parse it, leading to all sorts of ambiguities and divergent implementations. Someone needs to write a "strict ZIP" spec that has explicit and well-defined parsing rules, and then we need to get every existing ZIP implementation to agree to follow said spec.

wyldfire 2y ago
Or: better yet, just use an archive format for archival and a compression layer for compression. Don't use zip at all.
- Retr0id 2y ago
  What non-compressing archive format would you suggest? tar doesn't support random access which is a non-starter for many use cases.
  - deleted 2y ago
    [deleted]
  - acka 2y ago
    DAR (Disk ARchiver)[1] looks to be a good alternative. It supports random access, encryption, and individual file compression within the archive. [1] http://dar.linux.free.fr/ http://dar.linux.free.fr/
    - Retr0id 2y ago
      That seems counter to GP's suggestion of doing compression at a separate layer

_pdp_ 2y ago

This attack vector has been known for at least 17 years, if not longer. https://gnucitizen.org/blog/java-jar-attacks-and-features/ https://gnucitizen.org/blog/java-jar-attacks-and-features/ https://gnucitizen.org/blog/gifars-and-other-issues/ https://gnucitizen.org/blog/gifars-and-other-issues/

tedk-42 2y ago
I'm with you. I've evaded all sorts of scanning tools by base64 encoding data (i.e. binary data) to and copy pasting the text from insecure to highly secured environments. At the end of the day, these malware databases rely on hashing and detecting for known bad hashes and there are lots of command line tools to help get over that sort of thing like zip/tar etc.
- athrowaway3z 2y ago
  rot13 must be outlawed for its use by cyber-criminals!
- trollbridge 2y ago
  I used to have a workflow for updating code inside a very highly secure environment that relied on exactly this: Run build of prior version, run build of current version, run diff against them, compress with xz -9, base64 encode, generate output, base64 encode, e-mail it to myself, copy text of email, type "openssl base64 -d | unxz | bash", right click. E-mailing this was completely fine according to the stringent security protocols but e-mailing a zip of the code, etc. was absolutely 100% not. That would have to go on the vendor's formal portal. (Eventually I just opened my own "portal" to upload binaries to, put the vendor I worked for's logo on it, and issued a statement saying it was an official place to download binaries from the vendor. But sometimes their WAF would still mangle downloads or flag them as a risk, so I made sure builds had options of coming in an obfuscated base64 format.)
cyberax 2y ago
17 years? We played tricks with zip bombs that used this approach during 90-s.

mtnGoat 2y ago

Nice remix of an old technique. I remember file packing exes together for fun and profit back in the day.

unnouinceput 2y ago

Quote: "To defend against concatenated ZIP files, Perception Point suggests that users and organizations use security solutions that support recursive unpacking." That's the worse advice actually. You want the hidden shit to stay there unable to be seen by default programs. That's how you got all the crap in Windows mail starting from 90's when Outlook started to trying to be "smart" and automatically detect and run additional content. Be dumb and don't discover anything, let it rot in there. The only one that should do this is the antivirus, rest of unpackers/readers/whatever stay dumb.

brabel 2y ago
I agree. The ZIP definition is extremely clear that the contents of the ZIP are defined by the single Central Directory at the end of the file. Local headers are only valid if pointed to by the Central Directory. Any other local headers are just supposed to be treated as garbage, except by software that is specifically meant to recover corrupted ZIP archive's contents.

Jerrrrrrry 2y ago

Remember, secure encryption, good compression, and truely random data are indistinguishable. It's best to paste that encrypted payload into a JPG with some bullshit magic headers and upload that to a trusted Exfil pivot instead. Or, to get SuperMarioKart.rom to work with your chromeApp-XEMU emulator to play during down-time at work, just rename it to SMB.png, and email it you yourself.

seanhunter 2y ago
> Remember, secure encryption, good compression, and truely random data are indistinguishable. Yes, and the only reason the bad guys get away with this is the people who trust signature-based scanning at the perimeter to detect all threats. One of the hacks I'm most proud of in my whole career was when we were doing a proof of concept at an enterprise client we were being deliberately obstructed by the internal IT group due to politics between their boss and the boss who sponsored our POC. For unrelated trademark-related reasons we were prevented by a third party from having the software on physical media but we had a specific contractual clause agreeing to let us download it for install. So while we had been contractually engaged to provide this software and we had a strict deadline to prove value, the enterprise IT group were preventing us from actually getting it through the virus-scanning firewall to get it installed. What to do? The scanner looked for the signature of executable or zipped files and blocked them. It would also block any files larger than a certain size. So what I did was write two shell scripts called "shred" and "unshred". "Shred" would take any files you gave it as input, make them into a tarball, encrypt that to confuse the virus scanner and then split it up into chunks small enough to get through the firewall, and "unshred" would reverse this. This almost worked, but I found that the first chunk was always failing to transmit through the firewall. The scanner noticed some signature that openssl was putting at the front of the file when encrypting it. The solution? Change shred to add 1k of random noise to the front of the file and unshred to remove it. Job done. Our files were transmitted perfectly (I got the scripts to check the md5sum on both sides to be sure), and though the process was slow, we could continue. The funny thing was the POC was a bake-off versus another (more established) vendor and they couldn't get their software installed until they had done a couple of weeks of trench warfare with enterprise IT. "To keep things fair" the people organising the POC decided to delay to let them have time to install, and eventually the person blocking us from installing was persuaded to change their mind (by being fired), so "shred" and "unshred" could be retired.

deleted 2y ago

[deleted]

solatic 2y ago

Not really a new technique. A long time ago in a galaxy far far away, I needed to get libraries from the internet onto an air-gapped network, and the supported way was to burn them to a disk and bring the disk to be scanned. The scanner never allowed executables so it would always reject the libraries. Can't make this up, but the supported way that InfoSec (!!) explained to us to get past the scanner was to take advantage of WinRAR being available on the network, so split the rar archive a bunch of times (foo.r01, foo.r02,...) and the scanner, being unable to parse them nor execute them, would happily rubber-stamp them and pass them along. As long as the process was followed, InfoSec was happy. Thankfully this was before the days when people were really worried about supply chain security. Glad to see this bit of security theater recognized as such.

PreInternet01 2y ago

> To defend against concatenated ZIP files, Perception Point suggests that users and organizations use security solutions that support recursive unpacking Yeah, or, you know, just outright reject any ZIP file that doesn't start with a file entry, where a forward-scan of the file entries doesn't match the result of the central-directory-based walk. There is just so much malicious crud coming in via email that you just want to instantly reject anything that doesn't look 'normal', and you definitely don't want to descend into the madness of recursive unpacking, 'cuz that enables another class of well-known attacks. And no, "but my precious use-case" simply doesn't apply, as you're practically limited to a whole 50MB per attachment anyway. Sure, "this ZIP file is also a PDF is also a PNG is also a NES cartridge which displays its own MD5" (viz https://github.com/angea/pocorgtfo/tree/master/writeups/19 https://github.com/angea/pocorgtfo/tree/master/writeups/19) has a place (and should definitely be required study material for anyone writing mail filters!), but business email ain't it.

exmadscientist 2y ago
That's fair, but do realize that sometimes people do have to send around archives from the last century (they got archived for a reason!) or created by eldritch-horror tools that just make weird files (which, sometimes, are the gold masters for certain very important outputs...). And it's kind of annoying when these weird but standard files get silently dropped. Especially when that same file went through just fine yesterday, before the duly zealous security settings changed for whatever reason. All I'm saying is, don't drop my stuff silently because your code couldn't be arsed to deal with (ugly) standard formats. At least give me a warning ("file of type not scannable" or whatever, the actual words are not so important). And then when I have to yell at the Shanghai people I can yell at them for the correct reasons.
- PreInternet01 2y ago
  Oh, nothing gets dropped silently, but bounced right back with `550 5.7.1 Message rejected due to content (Attachment refused: MATCH-code)`. And for anything oversized, funny or otherwise non-standard, we offer a very convenient file transfer service.

rty32 2y ago

I checked the time to make sure today is in the year of 2024. I swear this has been widely known at least since the Win 98 era.

nxnxhxnxhz 2y ago

[flagged]

alkonaut 2y ago

Why do scanners need to look inside compressed archives at all? If the malicious file is extracted, it can be detected then. If it’s not extracted and instead used directly from the archive, then the offending code that executes malicious payloads from inside archives should be detectable? Is that the role of AutoIt in this scenario?

rocqua 2y ago
Because they want to check on the wire, before it hits an endpoint. Common situation is detecting malware sent through a phishing mail. You want to intercept those before a user can unpack the file.
- canucker2016 2y ago
  And "Defense In Depth" - the more layers that bad actors have to avoid/circumvent reduces the chance of their success. see https://www.fortinet.com/resources/cyberglossary/defense-in-depth https://www.fortinet.com/resources/cyberglossary/defense-in-...
- alkonaut 2y ago
  Yes that was my question: "why, do you want to do that when it sounds like a futile job to do?" If you have any kind of transformation (whatever trivial ROT or XOR) of the payload then there is no chance of detecting it via pattern matching, and if you need a "program" to process it into its malicious form, then how does detection work? I understand you want to keep malicious payload from reaching endpoints in a form that would risk being consumed (malformed documents, images causing buffer overflows, executable and script files). But beyond that?

yieldcrv 2y ago

What year is it

user070223 2y ago

Last I checked virustotal doesn't test nested archives for viruses even though it's an issue old as modern computing

canucker2016 2y ago
VirusTotal just passes on the file to the actual virus scanners (a for-loop with an api as a front-end) - it's up to each individual virus scanner to scan as they see fit (including scanning any unreferenced holes and compressed zip entries in a zip archive). I have no idea why those virus scanners don't check nested archives. Maybe time/cpu constraints?