6 ms·
From the top of my head, most are to make it as clear as possible that the file is binary and NOT text: > MUST be the very first N bytes in the file For every
by rickdeckard 2y ago
From the top of my head, most are to make it as clear as possible that the file is binary and NOT text:
> MUST be the very first N bytes in the file
For every system to be able to parse it without loading the entire file
> MUST be at least four bytes long, eight is better
To reduce risk of two different binary files on the same system having the same magic number
> MUST include at least one byte with the high bit set
To avoid wrongful identification as an ASCII file (ASCII doesn't use the high bit)
> MUST include a byte sequence that is invalid UTF-8
To avoid wrongful identification as UTF-8 text file
> SHOULD include a zero byte
To avoid wrongful identification as ANY text file
- gus_massa 2y ago>> MUST be the very first N bytes in the file > For every system to be able to parse it without loading the entire file It also solves the ambiguity problem, zip files have the magic numbers at the end, and most other files like pdf have the magic numbers at the beginning, so you can have a file that is both a pdf and a zip file.
- account42 2y agoFor ZIP files this is a design GOAL to a allow things like self-extracting archives.
- silvestrov 2y agoIt is a security nightmare.
- bongodongobob 2y agoCan you explain why?
- MyOutfitIsVague 2y agoDon't know their rationale, but back in the day it was really popular to cat a zip file onto the end of a JPEG or PNG and upload it onto 4chan, to be able to smuggle zip files where only images were supposed to be allowed. I remember back in the day seeing the "mods are asleep, post sinks" threads where people would share pictures of sinks, and I thought it was people being goofy, but later somebody told me that people were sharing zips of CP in those threads. I don't know if it's true or not, and never cared to find out.
- bongodongobob 2y agoThat's not really security though, that's just bad behavior.
- MyOutfitIsVague 2y agoPreventing smuggling of data where you don't want it is definitely an aspect of security.
- account42 2y agoIf your security depends on no format with undesirable properties existing then you have no security. The problem here is not the zip format but insufficient validation for the images you accept - the hidden data could be any ad-hoc format. Message smuggling in image files in particular is only something you can prevent if you re-encode the image -- and even then it's possible to hide messages in the image data in ways that will survive re-encodes.
- snackbroken 2y agoAny time two parts of a system disagree on how to interpret a given input, there's an exploit waiting to happen. One of the more famous examples of this is HTTP request smuggling. As a more concrete example of how file type confusion can bite you, you can imagine a hypothetical photo sharing service that lets users upload both individual images and zip files containing images; The basic structure of the server looks something like function user_upload_hook(file): if(is_zipfile(file)): extract(file, tempdir) else: move(file, tempdir/file) for image in tempdir: create_thumbnail(image) ... The developers are aware that zip files can contain zip bombs, so they decide to place some off the shelf ZipCop middleware in front of their application. ZipCop rejects all "bad" zip files, including files that aren't zip files at all. That's almost what they want, so they glue it all together with a shell script that first runs `file` (the POSIX command) on the user-supplied files and only feeds them through ZipCop if the file type isn't on a whitelist of image files. ZipCop rejects bad zip files, and image files are treated properly. All is well and there is much rejo- BANG! A zip bomb blows up in production. A malicious user has concatenated a JPEG of a cute kitten with a zip bomb. `file` reports that the uploaded file is a JPEG, so it's fed through unchecked to the server. The application's `is_zipfile()` correctly identifies that the file is a valid zip file, so the application extracts it and DOSes the server. The two different layers of the stack disagreeing on how to classify the offending file directly lead to an exploitable vulnerability.
- Aachen 2y agoI thought security people found it fun to do polyglots, as in the magazine POC||GTFO
- foxglacier 2y agoIsn't a bigger risk wrongly identifying a text file as the specific type of binary you're trying to process?