5 ms·

Decompiling 2024: A Year of Resurgance in Decompilation Research

147 points by matt_d 2y ago

loloquwowndueo 2y ago
“Resurgence” not “resurgance”. I wanted to leave a comment in the article itself but it wants me to sign in with GitHub, which: yuk, so I’m commenting here instead.
- mahaloz 2y ago
  Welp, that's a really sad typo... I've made a post-publication edit to the article now, but my shame is immortalized in Git: https://github.com/mahaloz/mahaloz.re/commit/90b760f53ef51b71cdc6b1f5d41f179dc7b60468 https://github.com/mahaloz/mahaloz.re/commit/90b760f53ef51b7...
  - sitkack 2y ago
    Proof you didn't use AI? :)
    - nayuki 2y ago
      Spell checkers existed before AI.
    - wakawaka28 2y ago
      "Please answer this question but misspell about one percent of the words in the answer."
benob 2y ago
> If you’ve ever talked to me in person, you’d know that I’m a disbeliever of AI replacing decompilers any time soon Decompilation, seen as a translation problem, is by any means a job that suits AI methods. Give time to researchers to gather enough mappings between source code and machine code, get used to training large predictive models, and you shall see top notch decompilers that beat all engineered methods.
- kachapopopow 2y ago
  Typical obfuscation sure. vms, obfuscation and everything in between is just noise to AI.
- donatj 2y ago
  It's pattern matching, plain and simple, An area where AI excels. AI driven decomp is absolutely on its way
  - dartos 2y ago
    Maybe in conjunction with a deterministic decompiler. precision wrt translation, especially when the translation is not 1-to-1, is not excellent with LLMs. In fact, their lack of precision is what makes them so good at translating natural languages!
  - ChrisKnott 2y ago
    It's also perfect for RL because it can compile it's output and check it against the input. It's a translation exercise where there's already a perfect machine translator in one direction. It probably just hasn't happened because decompilation is not a particularly useful thing for the vast majority of people.
  - thesz 2y ago
    Let me parrot you, it's fun. "It's pattern matching, plain and simple, an area where pattern matching algorithms excel. Pattern matching driven decomp absolutely leads" Decompilation is a dependence graph problem, one can formulate decompilation as a graph transformation/rewrite. Neural networks are notoriously bad at graphs.
    - gsam 2y ago
      > Neural networks are notoriously bad at graphs. AlphaFold is based on graph neural networks. The biggest issue is that we still do not know how to best encode graph problems in ways neural networks can exploit. Current graph neural network techniques exploit certain invariants but cannot distinguish between various similar graphs. And yet, they're still generating meaningful insights.
mips_avatar 2y ago
Decompilers aren’t just for security research they’re a key part of data compression of software updates. Delta compressors make deltas between decompiled code. So an improvement in mapping of decompiled files could have as much as a 20x improvement in software update size.
- mahaloz 2y ago
  I love this use case! Do you have any public links acknowledging/mentioning/showing this use case? Including it in the Applications portion of the Dec Wiki would be great.
  - deleted 2y ago
    [deleted]
- auguzanellato 2y ago
  Don’t delta updates usually use some binary diffing algorithm? Even if they didn’t I can understand why would they make deltas using decompiled code and not the disassembled one.
- pjc50 2y ago
  ? I thought delta compressors operated on the binary directly, can you provide an example of one which does actual decompilation/recompilation?
  - mips_avatar 2y ago
    Pretty much only Microsoft msdelta does it. Google courgette only does a disassembly.
rgovostes 2y ago
This was an informative article and I hope the author continues the series. Regarding AI-assisted renaming of variables, the author calls this "a strict improvement over traditional decompilation." But looking at the example: struct IxpMsg { struct Buffer { char* data; uint8_t* buffer; char* pos; uint8_t* pos; char* end; => uint8_t* streamPos; _ixpuint size; uint32_t bufferSize; _ixpuint mode; uint32_t type; }; } Ground Truth ReSym (AI) I am reluctant to allow the decompiler to influence my judgment about the meaning of variables. `streamPos` is not equivalent to `end`. Consider the issue multiplied by 20 or 100 as many incorrect assumptions, and it would severel cloud your understanding of the decompiled code. Combining this with reasoning models that can justify their labels would be very helpful. UX improvements could also be made to indicate confidence or progressively disclose these assumptions.
- summerlight 2y ago
  It's a strict improvement over the technology before LLM, which usually just assigns random cryptic symbols.
  - rgovostes 2y ago
    I disagree with it as a blanket statement. It’s related to the problem of hallucinations in LLMs; sometimes they come up with plausible, misleading bullshit. The user quickly relies on the automatic labeling as a crutch, while exhausting mental effort trying to distinguish the bullshit from the accurate labels. The cryptic symbols do not convey meaning but consequently can’t mislead you. I don’t reject the whole concept and am bullish on AI-assisted decompilation. But the UX needs to help the user have confidence in the results, just like source code-level static analyzers generate proofs.
  - RolfRolles 2y ago
    No, it's really not a strict improvement. A meaningless name like `v2` does at least convey that you, as the analyst, haven't understood the role of the variable well enough to rename it to something more fitting to its inferred purpose. If the LLM comes up with an "informative" variable name that is not very well-suited towards what it actually does, the name can waste your time by misleading you as to the role of the variable.
makz 2y ago
As a total beginner in this field, how can I begin experimenting with this?
- fulafel 2y ago
  The article covers several themes in decompilation, but for academic work in decompilation just take some papers, study them, study references, try to reproduce the experiments I guess. For the bare basics, you can get a disassembly of a random binary on Linux with objdump -S
ellis0n 2y ago
I remember working on DCC, a decompiler for C created by Cristina Cifuentes in 1990. It felt like magic and the future, but it was incredibly difficult and interesting. I used it for decompiling firmware and it was hard to convince my boss that we needed it.
FusspawnUK 2y ago
Ive had supprisingly good results feeding ghidras decomp output to chat gpt and having it simplify and explain it it seems to be very capable of having some understanding of what the original code would do. for instance i was feeding it some game decomp. a function looking for an entity in a 3d array of tiles. It somehow inferred it was an array of tiles and that it was hunting for a specific entity. None of the decomp I fed it had any variable/function names or comments, just the usual var1,var2 ect. How did it know what the underlying code was doing?

147 points by matt_d 2y ago

“Resurgence” not “resurgance”. I wanted to leave a comment in the article itself but it wants me to sign in with GitHub, which: yuk, so I’m commenting here instead.

mahaloz 2y ago
Welp, that's a really sad typo... I've made a post-publication edit to the article now, but my shame is immortalized in Git: https://github.com/mahaloz/mahaloz.re/commit/90b760f53ef51b71cdc6b1f5d41f179dc7b60468 https://github.com/mahaloz/mahaloz.re/commit/90b760f53ef51b7...
- sitkack 2y ago
  Proof you didn't use AI? :)
  - nayuki 2y ago
    Spell checkers existed before AI.
  - wakawaka28 2y ago
    "Please answer this question but misspell about one percent of the words in the answer."

benob 2y ago

> If you’ve ever talked to me in person, you’d know that I’m a disbeliever of AI replacing decompilers any time soon Decompilation, seen as a translation problem, is by any means a job that suits AI methods. Give time to researchers to gather enough mappings between source code and machine code, get used to training large predictive models, and you shall see top notch decompilers that beat all engineered methods.

kachapopopow 2y ago
Typical obfuscation sure. vms, obfuscation and everything in between is just noise to AI.
donatj 2y ago
It's pattern matching, plain and simple, An area where AI excels. AI driven decomp is absolutely on its way
- dartos 2y ago
  Maybe in conjunction with a deterministic decompiler. precision wrt translation, especially when the translation is not 1-to-1, is not excellent with LLMs. In fact, their lack of precision is what makes them so good at translating natural languages!
- ChrisKnott 2y ago
  It's also perfect for RL because it can compile it's output and check it against the input. It's a translation exercise where there's already a perfect machine translator in one direction. It probably just hasn't happened because decompilation is not a particularly useful thing for the vast majority of people.
- thesz 2y ago
  Let me parrot you, it's fun. "It's pattern matching, plain and simple, an area where pattern matching algorithms excel. Pattern matching driven decomp absolutely leads" Decompilation is a dependence graph problem, one can formulate decompilation as a graph transformation/rewrite. Neural networks are notoriously bad at graphs.
  - gsam 2y ago
    > Neural networks are notoriously bad at graphs. AlphaFold is based on graph neural networks. The biggest issue is that we still do not know how to best encode graph problems in ways neural networks can exploit. Current graph neural network techniques exploit certain invariants but cannot distinguish between various similar graphs. And yet, they're still generating meaningful insights.

mips_avatar 2y ago

Decompilers aren’t just for security research they’re a key part of data compression of software updates. Delta compressors make deltas between decompiled code. So an improvement in mapping of decompiled files could have as much as a 20x improvement in software update size.

mahaloz 2y ago
I love this use case! Do you have any public links acknowledging/mentioning/showing this use case? Including it in the Applications portion of the Dec Wiki would be great.
- deleted 2y ago
  [deleted]
auguzanellato 2y ago
Don’t delta updates usually use some binary diffing algorithm? Even if they didn’t I can understand why would they make deltas using decompiled code and not the disassembled one.
pjc50 2y ago
? I thought delta compressors operated on the binary directly, can you provide an example of one which does actual decompilation/recompilation?
- mips_avatar 2y ago
  Pretty much only Microsoft msdelta does it. Google courgette only does a disassembly.

rgovostes 2y ago

This was an informative article and I hope the author continues the series. Regarding AI-assisted renaming of variables, the author calls this "a strict improvement over traditional decompilation." But looking at the example: struct IxpMsg { struct Buffer { char* data; uint8_t* buffer; char* pos; uint8_t* pos; char* end; => uint8_t* streamPos; _ixpuint size; uint32_t bufferSize; _ixpuint mode; uint32_t type; }; } Ground Truth ReSym (AI) I am reluctant to allow the decompiler to influence my judgment about the meaning of variables. `streamPos` is not equivalent to `end`. Consider the issue multiplied by 20 or 100 as many incorrect assumptions, and it would severel cloud your understanding of the decompiled code. Combining this with reasoning models that can justify their labels would be very helpful. UX improvements could also be made to indicate confidence or progressively disclose these assumptions.

summerlight 2y ago
It's a strict improvement over the technology before LLM, which usually just assigns random cryptic symbols.
- rgovostes 2y ago
  I disagree with it as a blanket statement. It’s related to the problem of hallucinations in LLMs; sometimes they come up with plausible, misleading bullshit. The user quickly relies on the automatic labeling as a crutch, while exhausting mental effort trying to distinguish the bullshit from the accurate labels. The cryptic symbols do not convey meaning but consequently can’t mislead you. I don’t reject the whole concept and am bullish on AI-assisted decompilation. But the UX needs to help the user have confidence in the results, just like source code-level static analyzers generate proofs.
- RolfRolles 2y ago
  No, it's really not a strict improvement. A meaningless name like `v2` does at least convey that you, as the analyst, haven't understood the role of the variable well enough to rename it to something more fitting to its inferred purpose. If the LLM comes up with an "informative" variable name that is not very well-suited towards what it actually does, the name can waste your time by misleading you as to the role of the variable.

makz 2y ago

As a total beginner in this field, how can I begin experimenting with this?

fulafel 2y ago
The article covers several themes in decompilation, but for academic work in decompilation just take some papers, study them, study references, try to reproduce the experiments I guess. For the bare basics, you can get a disassembly of a random binary on Linux with objdump -S

ellis0n 2y ago

I remember working on DCC, a decompiler for C created by Cristina Cifuentes in 1990. It felt like magic and the future, but it was incredibly difficult and interesting. I used it for decompiling firmware and it was hard to convince my boss that we needed it.

FusspawnUK 2y ago

Ive had supprisingly good results feeding ghidras decomp output to chat gpt and having it simplify and explain it it seems to be very capable of having some understanding of what the original code would do. for instance i was feeding it some game decomp. a function looking for an entity in a 3d array of tiles. It somehow inferred it was an array of tiles and that it was hunting for a specific entity. None of the decomp I fed it had any variable/function names or comments, just the usual var1,var2 ect. How did it know what the underlying code was doing?