4 ms·

Wow, I really could have used this for my Ph.D. research (deep learning for obfuscated code). I ditched Ghidra in my experiments in favor of angr early on beca

Wow, I really could have used this for my Ph.D. research (deep learning for obfuscated code). I ditched Ghidra in my experiments in favor of angr early on because Ghidra did not play nicely with multiprocessing and I had a lot of data to process. Well maybe it does but it was much easier for me to achieve the same thing with angr. Love the name! Although I feel compelled to point out that Compiler Explorer is the name of the project and Godbolt is its author's last name, but I suppose if people are to the point of using Godbolt as a verb the ship has sailed.

psifertex 3y ago
We know! Similarly, the GH repo is actually the Decompiler Explorer: https://github.com/decompiler-explorer/decompiler-explorer/ https://github.com/decompiler-explorer/decompiler-explorer/
- account42 3y ago
  I like the name, it's cute and a nice homage.
tomcam 3y ago
Sometimes we must look back in angr
- hoosieree 3y ago
  That better be a Bowie reference and not an Oasis reference.
  - tomcam 3y ago
    John Osborne actually. But never Oasis ;0
mvelbaum 3y ago
Has there been any good progress in deobfuscating/decompiling machine code using Machine Learning techniques?
- hoosieree 3y ago
  Short answer: not where it counts. My work focuses on recognizing known functions in obfuscated binaries, but there are some papers you might want to check out related to deobfuscation, if not necessarily using ML for deobfuscation or decompilation. My take is that ML can soundly defeat the "easy" and more static obfuscation types (encodings, control flow flattening, splitting functions). It's low hanging fruit, and it's what I worked on most, but adoption is slow. On the other hand, "hard" obfuscations like virtualized functions or programs which embed JIT compilers to obfuscate at runtime... as far as I know, those are still unsolved problems. This is a good overview of the subject, but pretty old and doesn't cover "hard" obfuscations: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1566145 https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1566145. https://www.jinyier.me/papers/DATE19_Obf.pdf https://www.jinyier.me/papers/DATE19_Obf.pdf uses deobfuscation for RTL logic (FGPA/ASIC domain) with SAT solvers. Might be useful for a point of view from a fairly different domain. https://advising.cs.arizona.edu/~debray/Publications/generic-deobf.pdf https://advising.cs.arizona.edu/~debray/Publications/generic... uses "semantics-preserving transformations" to shed obfuscation. I think this approach is the way to go, especially when combined with dynamic/symbolic analysis to mitigate virt/jit types of transformations. I'll mention this one as a cautionary tale: https://dl.acm.org/doi/pdf/10.1145/2886012 https://dl.acm.org/doi/pdf/10.1145/2886012 has some good general info but glosses over the machine learning approach. It considers Hex-rays' FLIRT to be "machine learning", but FLIRT just hashes signatures, can be spoofed (i.e. https://siliconpr0n.org/uv/issues_with_flirt_aware_malware.pdf https://siliconpr0n.org/uv/issues_with_flirt_aware_malware.p...), and is useless against obfuscation. Eventually I think SBOM tools like Black Duck[1] and SLSA[2] will incorporate ML to improve the accuracy of even figuring out what dependencies a piece of software actually has. [1]: https://www.synopsys.com/software-integrity/software-composition-analysis-tools/black-duck-sca.html https://www.synopsys.com/software-integrity/software-composi... [2]: https://slsa.dev/ https://slsa.dev/
  - mvelbaum 3y ago
    Very cool - thank you very much! > My take is that ML can soundly defeat the "easy" and more static obfuscation types (encodings, control flow flattening, splitting functions). It's low hanging fruit, and it's what I worked on most, but adoption is slow. If I wanted to implement my own toy HexRays-like decompiler using a few of these techniques to decompile x86-64 binaries is there any high quality up-to-date paper/resource you would recommend? Or do you think that "A Generic Approach to Automatic Deobfuscation of Executable Code" paper is a good enough start? Also, what do you think about https://tigress.wtf/ https://tigress.wtf/ ?

psifertex 3y ago

We know! Similarly, the GH repo is actually the Decompiler Explorer: https://github.com/decompiler-explorer/decompiler-explorer/ https://github.com/decompiler-explorer/decompiler-explorer/

account42 3y ago
I like the name, it's cute and a nice homage.

tomcam 3y ago

Sometimes we must look back in angr

hoosieree 3y ago
That better be a Bowie reference and not an Oasis reference.
- tomcam 3y ago
  John Osborne actually. But never Oasis ;0

mvelbaum 3y ago

Has there been any good progress in deobfuscating/decompiling machine code using Machine Learning techniques?

hoosieree 3y ago
Short answer: not where it counts. My work focuses on recognizing known functions in obfuscated binaries, but there are some papers you might want to check out related to deobfuscation, if not necessarily using ML for deobfuscation or decompilation. My take is that ML can soundly defeat the "easy" and more static obfuscation types (encodings, control flow flattening, splitting functions). It's low hanging fruit, and it's what I worked on most, but adoption is slow. On the other hand, "hard" obfuscations like virtualized functions or programs which embed JIT compilers to obfuscate at runtime... as far as I know, those are still unsolved problems. This is a good overview of the subject, but pretty old and doesn't cover "hard" obfuscations: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1566145 https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1566145. https://www.jinyier.me/papers/DATE19_Obf.pdf https://www.jinyier.me/papers/DATE19_Obf.pdf uses deobfuscation for RTL logic (FGPA/ASIC domain) with SAT solvers. Might be useful for a point of view from a fairly different domain. https://advising.cs.arizona.edu/~debray/Publications/generic-deobf.pdf https://advising.cs.arizona.edu/~debray/Publications/generic... uses "semantics-preserving transformations" to shed obfuscation. I think this approach is the way to go, especially when combined with dynamic/symbolic analysis to mitigate virt/jit types of transformations. I'll mention this one as a cautionary tale: https://dl.acm.org/doi/pdf/10.1145/2886012 https://dl.acm.org/doi/pdf/10.1145/2886012 has some good general info but glosses over the machine learning approach. It considers Hex-rays' FLIRT to be "machine learning", but FLIRT just hashes signatures, can be spoofed (i.e. https://siliconpr0n.org/uv/issues_with_flirt_aware_malware.pdf https://siliconpr0n.org/uv/issues_with_flirt_aware_malware.p...), and is useless against obfuscation. Eventually I think SBOM tools like Black Duck[1] and SLSA[2] will incorporate ML to improve the accuracy of even figuring out what dependencies a piece of software actually has. [1]: https://www.synopsys.com/software-integrity/software-composition-analysis-tools/black-duck-sca.html https://www.synopsys.com/software-integrity/software-composi... [2]: https://slsa.dev/ https://slsa.dev/
- mvelbaum 3y ago
  Very cool - thank you very much! > My take is that ML can soundly defeat the "easy" and more static obfuscation types (encodings, control flow flattening, splitting functions). It's low hanging fruit, and it's what I worked on most, but adoption is slow. If I wanted to implement my own toy HexRays-like decompiler using a few of these techniques to decompile x86-64 binaries is there any high quality up-to-date paper/resource you would recommend? Or do you think that "A Generic Approach to Automatic Deobfuscation of Executable Code" paper is a good enough start? Also, what do you think about https://tigress.wtf/ https://tigress.wtf/ ?