3 ms·

Kuna: Decompiler Development in the Age of Coding Agents

81 points by matt_d 3mo ago

fishfasell 3mo ago
It would be cool to see agentic interpretation of function and variable names. It can see and track the flow of data a lot faster than a human can so if given some context, maybe it could synthesize names for them.
- snoopen 3mo ago
  I've used Claude with Ghidra MCP and had good success. It correctly identified and named MD5 functions, file IO functions, and the whole chain from a UI dialogue function. And when prompted it named all the interesting variables too.
  - hackcasual 3mo ago
    I was working on adding better WASM support to ghidra, but now I can just point Claude to binaryen
    - bpavuk 3mo ago
      I wonder, how many such interesting and universally beneficial developments did emergence of tools like Claude Code kill?
    - nneonneo 3mo ago
      Well, that's too bad; as the main developer of https://github.com/nneonneo/ghidra-wasm-plugin https://github.com/nneonneo/ghidra-wasm-plugin it would have been really nice to get better support for all the new features that have been added into the spec over the last few years. I also think that, for larger projects, it's still useful to have real decompilation support; for example, Il2CppDumper works in tandem with ghidra-wasm-plugin to enable decompilation of Unity Il2Cpp projects, which are often enormous (100MB wasm files are not uncommon) and for which the symbols + types are invaluable.
  - ChickeNES 3mo ago
    I'm not sure you even need the MCP tbh, I just have Claude use pyghidra scripting.
- billypilgrim 3mo ago
  idalib with Claude Code already works really well. But honestly, despite what people have been saying, LLMs have been very good at decompiling for at least two years now, I have been using it for that purpose regularly. Even just copying disassembly from the current function and all nested called functions from IDA into ChatGPT is already unexpectedly good.
DobarDabar 3mo ago
[dead]
saidnooneever 3mo ago
this is really amazing work thank you and in my opinion (as its explained) a very good example of how to use AI powered development and research to advance the tools we have. decompilation is really hard, and better algorithms for it are very valuable contributions for many areas of tech.
ur-whale 3mo ago
Not sure why someone would want to use an LLM to build a new decompiler instead of training an LLM to BE a decompiler. Especially given the fact that you have an infinite training set to train that LLM from (compilers can generate as much training data as you could possibly want).
- gopalraja 3mo ago
  Yeah, fair question. I read this less as “build a decompiler with an LLM instead of training one to decompile,” and more as a different loop: you still get an artifact you can measure and improve against IDA / Ghidra / angr. Training on infinite compiler data is powerful for the first path; this post is mostly about the second one (i think, but i could wrong).
- Tepix 3mo ago
  LLMs are famously inefficient, expensive and unreliable.
  - ur-whale 3mo ago
    > LLMs are famously inefficient, expensive and unreliable. Maybe, but here I am suggesting an LLM specifically trained to do only machine code to high-level code reverse engineering. I suspect this can be made both efficient and very reliable. Especially given the fact that there is very often a path the verify if what the LLM produces is a valid answer (by testing if the high level code produced by the LLM, when compiled, behaves like the original machine code).
    - mahaloz 3mo ago
      Answered above, but the gist is people have tried: https://decompilation.wiki/fundamentals/neural-decompilation/ https://decompilation.wiki/fundamentals/neural-decompilation..., and yet general LLMs from Anthropic and OpenAI appear to match or beat all of these works :(
- Rexxar 3mo ago
  There is no point to have an LLM do what can be done faster and deterministically by standard algorithms.
deleted 3mo ago
[deleted]
gopalraja 3mo ago
[flagged]
comandillos 3mo ago
Haven't used IDA much lately, but after looking at the screenshot with that IDA PRO decompiled code in their website I feel like Ghidra is already ahead of them in this area :D
- Orphis 3mo ago
  I have seen some code where either Ghidra or IDA is producing the better output. It's not cut and dry, but in general, I do prefer Ghidra.
- mahaloz 3mo ago
  Interesting. A lot of data shows IDA Pro is significantly better than Ghidra: https://decbench.com/ https://decbench.com/ Based on today's results, IDA Pro is ahead by 15 percentage points, which would mean, statistically, IDA Pro will recover perfect source code for 15% more functions than Ghidra on average.
mahaloz 3mo ago
Creator here! Since I saw some questions below, I'd like to clarify some things. Kuna is originally Ghidra ported to Rust, but it has changed quite a bit since. The results shown in the screenshot on the site are not a feature in Ghidra, which includes the Switch layout and some of the variable simplification. It's from Ghidra, previously talked about here [1]. > It would be cool to see agentic interpretation of function and variable names. On DecBench, which I also created last week, you can see just that for Codex and Claude when given only assembly [2]. They dominate, but also have some significant recall problems. > Why not train an LLM to be a decompiler instead of make one Over the last two decades, this has been attempted _many_ times [3]. To date, there has been no successful project/paper that can compete with traditional decompilers on fundamental metrics. I speculate this is due to their inability to abstract the many-to-one mapping of optimization to code. This can change, but I desired to take a different approach. If frontier labs continue to get better, they will become the de facto decompilation users. Why not get LLMs to help LLMs? Design a core decompiler that helps them more than others :). [1]: https://www.youtube.com/watch?v=VP29biKLoSw https://www.youtube.com/watch?v=VP29biKLoSw [2]: https://decbench.com/leaderboard/?dataset=sample-set https://decbench.com/leaderboard/?dataset=sample-set [3]: https://decompilation.wiki/fundamentals/neural-decompilation/ https://decompilation.wiki/fundamentals/neural-decompilation...

81 points by matt_d 3mo ago

It would be cool to see agentic interpretation of function and variable names. It can see and track the flow of data a lot faster than a human can so if given some context, maybe it could synthesize names for them.

snoopen 3mo ago
I've used Claude with Ghidra MCP and had good success. It correctly identified and named MD5 functions, file IO functions, and the whole chain from a UI dialogue function. And when prompted it named all the interesting variables too.
- hackcasual 3mo ago
  I was working on adding better WASM support to ghidra, but now I can just point Claude to binaryen
  - bpavuk 3mo ago
    I wonder, how many such interesting and universally beneficial developments did emergence of tools like Claude Code kill?
  - nneonneo 3mo ago
    Well, that's too bad; as the main developer of https://github.com/nneonneo/ghidra-wasm-plugin https://github.com/nneonneo/ghidra-wasm-plugin it would have been really nice to get better support for all the new features that have been added into the spec over the last few years. I also think that, for larger projects, it's still useful to have real decompilation support; for example, Il2CppDumper works in tandem with ghidra-wasm-plugin to enable decompilation of Unity Il2Cpp projects, which are often enormous (100MB wasm files are not uncommon) and for which the symbols + types are invaluable.
- ChickeNES 3mo ago
  I'm not sure you even need the MCP tbh, I just have Claude use pyghidra scripting.
billypilgrim 3mo ago
idalib with Claude Code already works really well. But honestly, despite what people have been saying, LLMs have been very good at decompiling for at least two years now, I have been using it for that purpose regularly. Even just copying disassembly from the current function and all nested called functions from IDA into ChatGPT is already unexpectedly good.

DobarDabar 3mo ago

[dead]

saidnooneever 3mo ago

this is really amazing work thank you and in my opinion (as its explained) a very good example of how to use AI powered development and research to advance the tools we have. decompilation is really hard, and better algorithms for it are very valuable contributions for many areas of tech.

ur-whale 3mo ago

Not sure why someone would want to use an LLM to build a new decompiler instead of training an LLM to BE a decompiler. Especially given the fact that you have an infinite training set to train that LLM from (compilers can generate as much training data as you could possibly want).

gopalraja 3mo ago
Yeah, fair question. I read this less as “build a decompiler with an LLM instead of training one to decompile,” and more as a different loop: you still get an artifact you can measure and improve against IDA / Ghidra / angr. Training on infinite compiler data is powerful for the first path; this post is mostly about the second one (i think, but i could wrong).
Tepix 3mo ago
LLMs are famously inefficient, expensive and unreliable.
- ur-whale 3mo ago
  > LLMs are famously inefficient, expensive and unreliable. Maybe, but here I am suggesting an LLM specifically trained to do only machine code to high-level code reverse engineering. I suspect this can be made both efficient and very reliable. Especially given the fact that there is very often a path the verify if what the LLM produces is a valid answer (by testing if the high level code produced by the LLM, when compiled, behaves like the original machine code).
  - mahaloz 3mo ago
    Answered above, but the gist is people have tried: https://decompilation.wiki/fundamentals/neural-decompilation/ https://decompilation.wiki/fundamentals/neural-decompilation..., and yet general LLMs from Anthropic and OpenAI appear to match or beat all of these works :(
Rexxar 3mo ago
There is no point to have an LLM do what can be done faster and deterministically by standard algorithms.

deleted 3mo ago

[deleted]

gopalraja 3mo ago

[flagged]

comandillos 3mo ago

Haven't used IDA much lately, but after looking at the screenshot with that IDA PRO decompiled code in their website I feel like Ghidra is already ahead of them in this area :D

Orphis 3mo ago
I have seen some code where either Ghidra or IDA is producing the better output. It's not cut and dry, but in general, I do prefer Ghidra.
mahaloz 3mo ago
Interesting. A lot of data shows IDA Pro is significantly better than Ghidra: https://decbench.com/ https://decbench.com/ Based on today's results, IDA Pro is ahead by 15 percentage points, which would mean, statistically, IDA Pro will recover perfect source code for 15% more functions than Ghidra on average.

mahaloz 3mo ago

Creator here! Since I saw some questions below, I'd like to clarify some things. Kuna is originally Ghidra ported to Rust, but it has changed quite a bit since. The results shown in the screenshot on the site are not a feature in Ghidra, which includes the Switch layout and some of the variable simplification. It's from Ghidra, previously talked about here [1]. > It would be cool to see agentic interpretation of function and variable names. On DecBench, which I also created last week, you can see just that for Codex and Claude when given only assembly [2]. They dominate, but also have some significant recall problems. > Why not train an LLM to be a decompiler instead of make one Over the last two decades, this has been attempted _many_ times [3]. To date, there has been no successful project/paper that can compete with traditional decompilers on fundamental metrics. I speculate this is due to their inability to abstract the many-to-one mapping of optimization to code. This can change, but I desired to take a different approach. If frontier labs continue to get better, they will become the de facto decompilation users. Why not get LLMs to help LLMs? Design a core decompiler that helps them more than others :). [1]: https://www.youtube.com/watch?v=VP29biKLoSw https://www.youtube.com/watch?v=VP29biKLoSw [2]: https://decbench.com/leaderboard/?dataset=sample-set https://decbench.com/leaderboard/?dataset=sample-set [3]: https://decompilation.wiki/fundamentals/neural-decompilation/ https://decompilation.wiki/fundamentals/neural-decompilation...