5 ms·

Anybody have any insight into what kind of trickery goes into this? I know a decent amount about both of the compression algorithms, but I've forgotten the stru

Anybody have any insight into what kind of trickery goes into this? I know a decent amount about both of the compression algorithms, but I've forgotten the structure of the file formats so I can't tell if this required any really funny business.

DavidBuchanan 8y ago
It isn't massively complex, but it did require some custom encoding/decoding logic. I plan to write a blog post on this some time soon...
danielvf 8y ago
I've written my own zip and unzipping code. Basically the "top level" zip file section lives at the end of the zip file and contains pointers back into the file for the actual data. This is so that you can keep adding to a zip file without having to rewrite anything you have done previously - just append a few more files and then your updated directory at the end. JPEG segments contain two special opening bytes, maybe a length, and then the actual data. So it would be trivial to have the last jpeg section in a jpeg file end with zip directory bytes. This directory could point back to data in other sections of the jpeg. ..my guess, anyway.
- plaguuuuuu 8y ago
  I can't exactly remember what JPG and ZIP look like, but yeah pretty much, I've done it before. What's surprising is that it works on Twitter. I tried the same thing with image hosts and the data got scrubbed. I guess they don't re-encode small images but I dunno.
  - Laforet 8y ago
    In other words twitter's backend did not sanitize image uploads properly.
ThrustVectoring 8y ago
The only trickery involved is making sure it survives twitter's compression and image cleaning algorithms. Other than that, programs that read images will ignore garbage data after what should be there for a jpg or whatever, and programs that open archives will ignore anything that comes before the archive header, so it's literally as straightforward as appending a .rar file at the end of a .jpg file.
comboy 8y ago
cat file.jpg file.zip > magic.jpg that's all there is to it (tested under linux, file compressed and decompressed with zip command)
- Retr0spectrum 8y ago
  That's not all there is to it. Did you try uploading the resulting file to twitter?
  - comboy 8y ago
    I did not. This is just a response to the parent comment not to the post itself. (Although if they do something with this file and you still are able to do this trick that would seem even more .. interesting on their side than leaving images below certain size untouched)
- tyingq 8y ago
  The zip offset fields would be wrong, so your unzip must have some error correction. Use "zip -A zipfile" on Linux to fix them. As mentioned though, image upload sites would likely scrub out the extra data.
charonn0 8y ago
The JPG format ignores data that comes after the EOF marker, and the zip format ignores data that comes before the first file entry.

DavidBuchanan 8y ago

It isn't massively complex, but it did require some custom encoding/decoding logic. I plan to write a blog post on this some time soon...

danielvf 8y ago

I've written my own zip and unzipping code. Basically the "top level" zip file section lives at the end of the zip file and contains pointers back into the file for the actual data. This is so that you can keep adding to a zip file without having to rewrite anything you have done previously - just append a few more files and then your updated directory at the end. JPEG segments contain two special opening bytes, maybe a length, and then the actual data. So it would be trivial to have the last jpeg section in a jpeg file end with zip directory bytes. This directory could point back to data in other sections of the jpeg. ..my guess, anyway.

plaguuuuuu 8y ago
I can't exactly remember what JPG and ZIP look like, but yeah pretty much, I've done it before. What's surprising is that it works on Twitter. I tried the same thing with image hosts and the data got scrubbed. I guess they don't re-encode small images but I dunno.
- Laforet 8y ago
  In other words twitter's backend did not sanitize image uploads properly.

ThrustVectoring 8y ago

The only trickery involved is making sure it survives twitter's compression and image cleaning algorithms. Other than that, programs that read images will ignore garbage data after what should be there for a jpg or whatever, and programs that open archives will ignore anything that comes before the archive header, so it's literally as straightforward as appending a .rar file at the end of a .jpg file.

comboy 8y ago

cat file.jpg file.zip > magic.jpg that's all there is to it (tested under linux, file compressed and decompressed with zip command)

Retr0spectrum 8y ago
That's not all there is to it. Did you try uploading the resulting file to twitter?
- comboy 8y ago
  I did not. This is just a response to the parent comment not to the post itself. (Although if they do something with this file and you still are able to do this trick that would seem even more .. interesting on their side than leaving images below certain size untouched)
tyingq 8y ago
The zip offset fields would be wrong, so your unzip must have some error correction. Use "zip -A zipfile" on Linux to fix them. As mentioned though, image upload sites would likely scrub out the extra data.

charonn0 8y ago

The JPG format ignores data that comes after the EOF marker, and the zip format ignores data that comes before the first file entry.