4 ms·

Unstripping Stripped Binaries

189 points by taviso 4y ago

saagarjha 4y ago
> What I really want is to take the names and types I’ve figured out from my disassembler, and make them visible to gdb. You may also want to check out https://github.com/mahaloz/decomp2dbg https://github.com/mahaloz/decomp2dbg. (Disclaimer: I will never not shill Zion's code.)
- arbruijn 4y ago
  And for Ghidra https://github.com/cesena/ghidra2dwarf https://github.com/cesena/ghidra2dwarf
  - mahaloz 4y ago
    Wow this was actually just the code reference I was looking for to extend decomp2dbg into Ghidra. I had seen ida2dwarf before, but not this sleek Python approach to it. It does still seem though that you need to run this script after every time you change something in Ghidra, which is less than ideal for me. With this, I think it will be possible now for seamless Ghidra support. Ty ty, I'll be sure to give credit where credit is due. Ofc Robert Xiao worked on this, lol.
- mahaloz 4y ago
  Ty for the shoutout <3
082349872349872 4y ago
worth reading to the end: I knew about various arcane symbol formats, but TIL that cross-architecture CFG-based diffing is a thing.
- archi42 4y ago
  I am only little surprised that it works; after all, compiling a program for two platforms results in a mostly equivalent control flow graphs, except for optimizations and different code gen strategies (*); and of course platform specific stuff (but that was probably already abstracted for such a project back then). BUT I was very (pleasantly) surprised someone did go through the effort of implementing this and made it available for everyone as FOSS <3 (*, rant) I think inlining and tail call optimizations (especially calls into libc or the equivalent of the era) might cause the most trouble. Also loop generation can have subtle differences reflected in the CFG (not only loop unrolling, but also transforming a while-loop into a do-while-loop causes differences in the CFGs). Back then I'd also expect compilers not always perform equally well for "simple" things like dead code elimination. I wish I had the time to check out bindiff in detail, it's really interesting (personal bias: In the past I did work on static code analysis and had a supporting role in a compiler dev team).
- tdullien 4y ago
  Original author of the BinDiff tool (ca. 2004) here. BinDiff uses a good number of different algorithms to narrow the set of candidates for CFG-based matching -- so things like identical string references may be used to match a large function, and then basic blocks in that large function may be matched, and then BinDiff may identify that the functions called from these basic blocks will be identical, too. The entire field has exploded, and there are dozens of tools for function matching nowadays, some with very sophisticated ML approaches etc. - it is somewhat surprising that BinDiff still does it's job, considering that the core engine has not been modified or improved since 2010. I'd also like to extend kudos to https://twitter.com/admvonschneider https://twitter.com/admvonschneider -- he is the sole Googler sacrificing his 20% time to keep the tool alive...
  - saagarjha 4y ago
    Did Google ever end up open sourcing BinDiff? I've always been curious at what goes under the hood.
    - tdullien 4y ago
      So that has been "in the works" for 7+ years, and I think it repeatedly stalls because someone needs to really drive it against the bureaucratic internal obstacles ... There's no reason they aren't doing it, except "nobody has time to do it".
      - saagarjha 4y ago
        Do you think the result, if open sourced, would be worth it? Or is the code not particularly amenable to maintenance and integration?
        tdullien 4y ago
        Not sure if it'd be worth it, but it's reasonably clean C++ code, which is not terribly difficult to maintain & integrate.
      - 4y ago
david_draco 4y ago
Maybe machine learning techniques for translating text could produce unstripped binaries based on stripped binaries, given a large training set?
magic_hamster 4y ago
You have a really great blog. Kudos. I just read through a bunch of your adventures and it was great fun.
ntauthority 4y ago
For Windows, there once was a similar tool that synthesizes a .pdb file using the DIA API from a list of names: https://pefrm-units.osdn.jp/pe_debug.html https://pefrm-units.osdn.jp/pe_debug.html

189 points by taviso 4y ago

> What I really want is to take the names and types I’ve figured out from my disassembler, and make them visible to gdb. You may also want to check out https://github.com/mahaloz/decomp2dbg https://github.com/mahaloz/decomp2dbg. (Disclaimer: I will never not shill Zion's code.)

arbruijn 4y ago
And for Ghidra https://github.com/cesena/ghidra2dwarf https://github.com/cesena/ghidra2dwarf
- mahaloz 4y ago
  Wow this was actually just the code reference I was looking for to extend decomp2dbg into Ghidra. I had seen ida2dwarf before, but not this sleek Python approach to it. It does still seem though that you need to run this script after every time you change something in Ghidra, which is less than ideal for me. With this, I think it will be possible now for seamless Ghidra support. Ty ty, I'll be sure to give credit where credit is due. Ofc Robert Xiao worked on this, lol.
mahaloz 4y ago
Ty for the shoutout <3

082349872349872 4y ago

worth reading to the end: I knew about various arcane symbol formats, but TIL that cross-architecture CFG-based diffing is a thing.

archi42 4y ago
I am only little surprised that it works; after all, compiling a program for two platforms results in a mostly equivalent control flow graphs, except for optimizations and different code gen strategies (*); and of course platform specific stuff (but that was probably already abstracted for such a project back then). BUT I was very (pleasantly) surprised someone did go through the effort of implementing this and made it available for everyone as FOSS <3 (*, rant) I think inlining and tail call optimizations (especially calls into libc or the equivalent of the era) might cause the most trouble. Also loop generation can have subtle differences reflected in the CFG (not only loop unrolling, but also transforming a while-loop into a do-while-loop causes differences in the CFGs). Back then I'd also expect compilers not always perform equally well for "simple" things like dead code elimination. I wish I had the time to check out bindiff in detail, it's really interesting (personal bias: In the past I did work on static code analysis and had a supporting role in a compiler dev team).
tdullien 4y ago
Original author of the BinDiff tool (ca. 2004) here. BinDiff uses a good number of different algorithms to narrow the set of candidates for CFG-based matching -- so things like identical string references may be used to match a large function, and then basic blocks in that large function may be matched, and then BinDiff may identify that the functions called from these basic blocks will be identical, too. The entire field has exploded, and there are dozens of tools for function matching nowadays, some with very sophisticated ML approaches etc. - it is somewhat surprising that BinDiff still does it's job, considering that the core engine has not been modified or improved since 2010. I'd also like to extend kudos to https://twitter.com/admvonschneider https://twitter.com/admvonschneider -- he is the sole Googler sacrificing his 20% time to keep the tool alive...
- saagarjha 4y ago
  Did Google ever end up open sourcing BinDiff? I've always been curious at what goes under the hood.
  - tdullien 4y ago
    So that has been "in the works" for 7+ years, and I think it repeatedly stalls because someone needs to really drive it against the bureaucratic internal obstacles ... There's no reason they aren't doing it, except "nobody has time to do it".
    - saagarjha 4y ago
      Do you think the result, if open sourced, would be worth it? Or is the code not particularly amenable to maintenance and integration?
      - tdullien 4y ago
        Not sure if it'd be worth it, but it's reasonably clean C++ code, which is not terribly difficult to maintain & integrate.
    - 4y ago

david_draco 4y ago

Maybe machine learning techniques for translating text could produce unstripped binaries based on stripped binaries, given a large training set?

magic_hamster 4y ago

You have a really great blog. Kudos. I just read through a bunch of your adventures and it was great fun.

ntauthority 4y ago

For Windows, there once was a similar tool that synthesizes a .pdb file using the DIA API from a list of names: https://pefrm-units.osdn.jp/pe_debug.html https://pefrm-units.osdn.jp/pe_debug.html