4 ms·

I'm not in this world at all, but I have to wonder: is this really a thing? Binary tools, "binary editors, libraries to encode and decode data, parser generator

I'm not in this world at all, but I have to wonder: is this really a thing? Binary tools, "binary editors, libraries to encode and decode data, parser generators, binary data description languages and frameworks, binary formats and encodings, assemblers, debuggers, reverse engineering suites, and the like." Don't binary tools find their home more naturally in a larger language/runtime ecosystem, and would be talked about in that context? E.g. you'd get a talk on gdb at a BSD conference. Or Java-specific binary tools at a Java conference. This makes sense too since, for a lot of programmers, including myself, binary tools are "weapons of last resort" in a debug session - and they are typically highly specialized, and we're not comparing different binary tools so much as picking from the (paltry few) available to us. I'd like to learn something about a part of the programming world that I know little about. I've been getting more and more interested in lower level mutable state, like CPU microcode and UEFI firmware and (I suppose) the contents of HP's iLO little-connected-computer-that-runs-alongside-the-CPU-and-we-should-really-have-a-better-word-for [0]. But then, I mean the premise of the talk is so broad you could conceivably have a talk about Wireshark. Or Java bytecode manipulation libraries and tools. Or even code that does streaming transforms on video in nodejs. "Libraries to encode and decode data" places no limit to the valid subjects of such a talk. [0] https://isc.sans.edu/diary/Over+20+thousand+servers+have+their+iLO+interfaces+exposed+to+the+internet%2C+many+with+outdated+and+vulnerable+versions+of+FW/28276 https://isc.sans.edu/diary/Over+20+thousand+servers+have+the...

jcranmer 5y ago
Earlier today, I was playing around with doing some video game save file editing, and any of the tools that would have been useful in that endeavor is what I think they would be looking for. That is to say: * Start with a hex editor--you need to read and write (but mostly read) the binary data of a file. * Fast exploratory decoding: this sequence of bytes might be a uint8_t, int64_t, null-terminated string, double, etc... tell me all of the possible values that might be lurking there, so that I can guess where precisely a value is stored. * Structure editing: binary files are usually structured, so I want to be able to say what the structure is for this data. Like maybe there's an int count of entries, and the entries are variably sized (so grab the full size from the middle of the struct), and let me code that into the editor so I can see where entry #342 is. Notably, this also lets me check if what I think is a size is actually a size. * Structure guessing: of course, part of the issue with reverse engineering the format is that you're often unsure of much of the data, so you also need to be able to fill on only small sections of the struct. I know that a struct is, say, 540 bytes, and that byte 148 was something I was interested, but I don't know what's going on for the preceding 100 bytes, so don't make me fill anything in. Or when I do find something interesting in there, make it easy for me to designate something there without trying too hard. One of the issues you run into is there's not really any good way to convey the structure format of binary data--C structs are just way, way too underpowered (as they are fixed size, so there's no way to convey "this array has <field #5> elements in it" or worse "this array has as many elements until <end-of-stream>").
pabs3 5y ago
A couple of examples of the tools they are talking about: GNU poke, Kaitai Struct: https://www.jemarch.net/poke https://www.jemarch.net/poke https://kaitai.io/ https://kaitai.io/ Its mostly about tools surrounding the binary file format ecosystem, for reverse engineering, creating readers/writers, debugging edge cases etc.

jcranmer 5y ago

Earlier today, I was playing around with doing some video game save file editing, and any of the tools that would have been useful in that endeavor is what I think they would be looking for. That is to say: * Start with a hex editor--you need to read and write (but mostly read) the binary data of a file. * Fast exploratory decoding: this sequence of bytes might be a uint8_t, int64_t, null-terminated string, double, etc... tell me all of the possible values that might be lurking there, so that I can guess where precisely a value is stored. * Structure editing: binary files are usually structured, so I want to be able to say what the structure is for this data. Like maybe there's an int count of entries, and the entries are variably sized (so grab the full size from the middle of the struct), and let me code that into the editor so I can see where entry #342 is. Notably, this also lets me check if what I think is a size is actually a size. * Structure guessing: of course, part of the issue with reverse engineering the format is that you're often unsure of much of the data, so you also need to be able to fill on only small sections of the struct. I know that a struct is, say, 540 bytes, and that byte 148 was something I was interested, but I don't know what's going on for the preceding 100 bytes, so don't make me fill anything in. Or when I do find something interesting in there, make it easy for me to designate something there without trying too hard. One of the issues you run into is there's not really any good way to convey the structure format of binary data--C structs are just way, way too underpowered (as they are fixed size, so there's no way to convey "this array has <field #5> elements in it" or worse "this array has as many elements until <end-of-stream>").

pabs3 5y ago

A couple of examples of the tools they are talking about: GNU poke, Kaitai Struct: https://www.jemarch.net/poke https://www.jemarch.net/poke https://kaitai.io/ https://kaitai.io/ Its mostly about tools surrounding the binary file format ecosystem, for reverse engineering, creating readers/writers, debugging edge cases etc.