8 ms·
How Hackerman would create an image just by typing 0 and 1 – deep dive into GIF
- mikecx 3y agoLove learning about image/video formats and I enjoy how you broke it all down. Just a heads up, pretty sure you've got a typo: "Next is the Global Packed Field, which in this case is 70 which in binary form is 00000000." 70 in binary would be 100110. (64 + 0 + 0 + 4 + 2 + 0)
- jsf01 3y agoThis pops up again later when he says 81 is 10000001. Or that width 8 corresponds to 04. I don’t know enough about the gif format to know if I just misunderstood these parts or if they were written incorrectly, but it was a bit confusing.
- happybits 3y ago81 in hex is 10000001 in binary, so I think this part is correct
- happybits 3y agoThanks mikecx, it was a typo, I've changed the text to "Next is the Global Packed Field, which in this case is 00..."
- jtaft 3y agoReminds me of graphic designer vs css programmer https://youtube.com/shorts/YWT8Dqd-AmQ?feature=shared https://youtube.com/shorts/YWT8Dqd-AmQ?feature=shared
- dfox 3y agoNote that the article glances over the first byte of image data which specifies how many bits each pixel occupies and sets it 07, which makes first 128 symbols of the resulting LZW stream byte aligned. This is probably never the case in practice (for the presented images it should be 1 respectively 2).
- gpvos 3y agoIt is mentioned near the end though.
- snoopsnopp 3y agoSomeone needs to release bit for bit break downs of common image formats. It’s always fun to draw by hand.
- pjc50 3y agoSome years ago (1996) I wrote a GIF decoder from scratch based entirely on http://qzx.com/pc-gpe/gif.txt http://qzx.com/pc-gpe/gif.txt (and a description of LZW bundled with my copy of pcgpe that seems to be missing from that page) . I remember it being quite a struggle against off-by-one errors. I do wish there was a really good format for describing binary file formats in a way that was amenable to codegen. Kaitai https://kaitai.io/ https://kaitai.io/ seems to be the state of the art.
- yjftsjthsd-h 3y agoKaitai looks nice - have you used it enough to review how it handles? I'm just starting a project to deal with somewhat involved on-disk formats[0] and this might be helpful. [0] The other day, someone was asking for a "tar2ext4" tool, and I thought "hey, that should exist, and I need a side project!". I was prepared to use an annotated hex viewer ( https://hachoir.readthedocs.io/en/latest/wx.html https://hachoir.readthedocs.io/en/latest/wx.html ) and hand roll the encoder/decoder, but I'll happily take tool assistance:)
- pjc50 3y agoKaitai is good if: - your format is fully known (it's less helpful if you're trying to incrementally build a parser while reverse engineering) - you want to read files, but don't care about writing - you don't mind that the development is not very active For writing "tar2ext4" I would genuinely look at how much work it would be to run the ext4 code from the kernel in a different context; there's a lot of it to consider. Or do what the Apple "dmg" tooling does and make a ramdisk.
- yjftsjthsd-h 3y agoOkay, so of interest but maybe not applicable to my usecase. Thanks:) Yeah, it remains to be seen how complex the actual format/code is. Would need to balance the difficulty of recreating it (which I assume to be quite high) against difficulty of extracting kernel code... although https://github.com/lkl/linux https://github.com/lkl/linux exists so for all I know maybe it's easy¯\_(ツ)_/¯ And yes, if I needed to actually write a "tar2ext4" tool today - like, start working in the morning and have it done by EOD - I would absolutely use... actually probably a loopback device rather than a true ramdisk, but yeah. But that requires root access and fiddling with loopback config, which seems excessive for what is, ultimately, just another archive format (from a certain point of view). And honestly some of it is just that it sounds fun to get my hands dirty with filesystem code in userspace:)
- charlieyu1 3y agoI like reading articles like this. Compression is always a hard part for me to understand, just feels very unintuitive
- seiferteric 3y agoNetpbm https://en.wikipedia.org/wiki/Netpbm https://en.wikipedia.org/wiki/Netpbm is even easier
- 1-6 3y agoThanks, I wonder if I can now train an AI to create exquisite GIFs using binary using these instructions.
- mock-possum 3y ago> Next is the Global Packed Field, which in this case is 70 which in binary form is 00000000 … what? 70 should be 1000110 surely?
- charlieyu1 3y ago70 in hex. So 01110000. The 111 bits are unused according to the article so it can be anything.
- jszymborski 3y agoDefeats the purpose a bit, but its far easier to type out a PBM [0] file by hand and then just convert it to gif using e.g. imagemagick [0] https://en.wikipedia.org/wiki/Netpbm https://en.wikipedia.org/wiki/Netpbm
- fwip 3y agoThere's a cute plugin[0] for Vim which converts any image to XPM, which is a similar format that Vim has syntax-coloring for. You can edit the text, and then on save, it will get converted back to the original format. I've used it a few times to quickly preview an image or edit a favicon. It's more of party trick than seriously useful, though. [0]https://github.com/tpope/vim-afterimage https://github.com/tpope/vim-afterimage
- yjftsjthsd-h 3y agoNot even just manually typing... Last time I wanted to have a program save a picture [0] it was easiest to write PPM and then convert that to a real format. Super inefficient file sizes, but good tradeoff for a hobby project. I can take some big intermediate files in exchange for not needing a graphics file format library:) [0] I was playing with the Linux framebuffer and wrote - among other things - a screenshot tool.
- TacticalCoder 3y agoWait, TFA doesn't even contain a link to this vid from Hackerman hacking time!? https://youtu.be/KEkrWRHCDQU https://youtu.be/KEkrWRHCDQU (my favorite part is when he goes into hardcore hacking mode while putting a Nintendo glove on)
- Moru 3y agoEh, yes it did :-)
- kristopolous 3y agoWhat was the follow up content that's no longer available?
- matheusmoreira 3y agoI assume it's these: https://youtu.be/fQGbXmkSArs https://youtu.be/fQGbXmkSArs https://youtu.be/ZTidn2dBYbY https://youtu.be/ZTidn2dBYbY https://youtu.be/bS5P_LAqiVg https://youtu.be/bS5P_LAqiVg
- kristopolous 3y agoAt some point the writer showed someone the script and said "I'm going to make this" and a bunch of people looked over it and said "thor, dinosaurs, hitler ... bullets going through the phone? right, ok, sure."
- 1letterunixname 3y agoBah. Should've used PCX files for that extra steganographic space goodness. (JPG is too easy.) If you can't create punchcards or hex blindfolded, there are always tools: [pdf] https://www.pedramhayati.com/images/docs/survey_of_steganography_and_steganalytic_tools.pdf https://www.pedramhayati.com/images/docs/survey_of_steganogr... [zip] https://ftp.funet.fi/pub/crypt/archive/idea.sec.dsi.unimi.it/code/wnstorm.zip https://ftp.funet.fi/pub/crypt/archive/idea.sec.dsi.unimi.it... [zip] https://web.archive.org/web/20230828124101/https://dl.packetstormsecurity.net/crypt/stego/DOS/dc-stego.zip https://web.archive.org/web/20230828124101/https://dl.packet...
- boneitis 3y ago> In Visual Studio Code, there is an extension called Hex Editor, which lets you view and edit the binary file. I'll take this opportunity to bring up a method to patch binary data in True Scots^H^H^H^H^HHackerman fashion, using nothing more than vim and xxd, which are already installed everywhere (for some definition of "everywhere"). LiveOverflow describes it between 5:02-7:46 in: https://www.youtube.com/watch?v=LyNyf3UM9Yc&t=302s https://www.youtube.com/watch?v=LyNyf3UM9Yc&t=302s It is the `:%!xxd` and `:%!xxd -r` trick. (Trying it out again before commenting here, it seems like one might need to `:set nofixeol` beforehand so as not to append any nonexistent newline at the end of file). My mind is blown by this trick, and I've still never got around to understanding wtf is happening here. (ETA: Upon some thought, I reckon `xxd -r` can just chug along happily by completely ignoring the ascii rendering columns.)
- themk 3y agoThis isn't so much a "trick" as it is the main purpose of xxd. xxd is distributed with vim (as in, I'm pretty sure if you want to send a patch to xxd, you send it to the vim repo). One of it's primary purposes is to allow for the editing of binary files in Vim.
- boneitis 3y agoI had no idea!
- follower 3y agoOne great resource for GIF-related explorations is Matthew Flickinger's "What's In A GIF" project: * https://www.matthewflickinger.com/lab/whatsinagif/index.html https://www.matthewflickinger.com/lab/whatsinagif/index.html The original version is apparently from ~2005 and is used as the basis of the giflib docs referenced by the original article[0]. (The giflib docs do expand on the content of the original, so are still worth reading.) But Matthew Flickinger's original version has continued to be updated as recently as 2022[1] and now includes two helpful browser-based GIF tools: * GIF Explorer: https://www.matthewflickinger.com/lab/whatsinagif/gif_explorer.asp https://www.matthewflickinger.com/lab/whatsinagif/gif_explor... * GIF Encoder: https://www.matthewflickinger.com/lab/whatsinagif/gif_encoder.asp https://www.matthewflickinger.com/lab/whatsinagif/gif_encode... GIF Explorer displays the "interpreted" bytes of any GIF file in an almost "literate" style and has an UI/UX which I'd be really interested to see used in a generic reverse-engineering/binary viewer tool. GIF Encoder enables you to create an image in the browser & see how it is GIF encoded. I have a rant about how modern GIF usage could be so much better than it is (and still be within the original specification) but instead of subjecting you to that I'll subject you to this project of mine instead: https://audiogif.rancidbacon.com https://audiogif.rancidbacon.com [0] https://giflib.sourceforge.net/whatsinagif/index.html https://giflib.sourceforge.net/whatsinagif/index.html [1] https://github.com/MrFlick/whats-in-a-gif https://github.com/MrFlick/whats-in-a-gif
- akdas 3y agoI absolutely love the "What's In A GIF" series. It's what inspired me to write my own GIF decoder while learning Erlang at the same time: https://github.com/avik-das/giferly https://github.com/avik-das/giferly The first time around, I struggled a lot with decoding errors. Many years later, after being a more experienced developer, I wrote the LZW decompression with unit tests. Doing so forced me to think about each edge case, and fix issues without breaking existing functionality. Very quickly, I was able to open pretty much any GIF file I threw at it!
- happybits 3y agoThanks follower I've read his posts about GIF and referred to it at the end of my article. But I didn't know about "GIF Encoder" and "GIF Explorer" - interesting!
- happybits 3y agoCUSTOMER Hackerman, I need an impressive icon for my website. It should be 5x5 pixels big and look like a rabbit. Can you please draw it for me? HACKERMAN Draw!? Bah! I don’t need any graphic program for that. I am Hackerman. I will code it for you. You will get the image next week. CUSTOMER Next week?? But… HACKERMAN No buts! I just need to read about how the GIF file format works, then I can create the image in no time. [TIME PASSES] After spending some evenings, Hackerman gets the main idea of how the GIF file format works and the compression algorithm called LZW. With that knowledge, he succeeded in creating the image within an hour. Hackerman calculated that the binary of the image should be as follows: 47 49 46 38 39 61 00 00 00 00 70 00 00 2c 00 00 00 00 05 00 05 00 81 11 11 11 FF FF FF D5 D7 D9 00 00 00 07 0F 80 01 00 83 01 82 84 85 88 82 8A 85 02 85 81 00 3b So he just opened his code editor, saved the file as rabbit.gif, and sent it to his customer. Boom! Easy-peasy! Do you want understand the GIF-file format and be as cool as Hackerman?
- bluejekyll 3y agoYes? Is that such a bad thing? Is this trying to say there is no value in learning something so low level. Exploration leads to learning, and learning leads to innovation. Perhaps Hackerman will go on to create the greatest image encoding library for images so that they can easily scale from 5x5 to 25x25 or more and fit in the same space as the 5x5. Who knows.
- loloquwowndueo 3y agoHe’s quoting from TFA.
- atoav 3y agoI teach a foundational media technology course at on of the bigger european art universities — I do the same thing with the students using a broadcast wave file. The goal of the thing isn't to turn them into hackers, it is to give them a feeling what the stuff they work with is made of, what a file is. This is also a great introduction to talk about compression, metadata, encoding, decoding, sample rate, bitdepth and so on. If you dive that deep into it, the settings in a typical media conversion program will suddenly become much less intimidating. My motto always was: this was made by humans so it should be possible for humans to understand it as well. And this is maybe the "hidden" lesson: If you bring enough patience you can go into the depth of nearly every topic.