4 ms·
One of my side projects is very similar: I built a recursive descent disassembler for Gameboy that I'm working on building into a decompiler. I took a slightly
by T-R 12y ago
One of my side projects is very similar: I built a recursive descent disassembler for Gameboy that I'm working on building into a decompiler. I took a slightly different approach - rather than generate assembly, I generate a call graph - each instruction is a node with zero or more potential following nodes.
The key word, though, is 'potential' - as the author mentions, one of the issues you run into with disassembly is that you sometimes run into code that jumps into garbage (dynamic jumps with insufficiently constrained inputs, or code that should've been unreachable), and sometimes you just hit an infinite loop waiting for an interrupt. It really needs some input/interactivity to determine likely entry points and dead ends/unreachable code.
One nice thing you can do while you have the call graph, though, is propagate constraints (basically type inference/data flow analysis) to get an idea of where dynamic jumps may go, which ones might be under-constrained, and which branch targets are likely dead ends. Haven't gotten as far as implementing it yet, but there's a paper on it here: http://www.cs.rhul.ac.uk/home/kinder/papers/phdthesis.pdf http://www.cs.rhul.ac.uk/home/kinder/papers/phdthesis.pdf
- RodgerTheGreat 12y agoI've done something vaguely along these lines in the disassembler which is part of my Chip8 IDE[1]. I take advantage of the fact that the ISA is register-rich (16 general-purpose registers) but individual registers are only 8 bits wide, making it feasible to propagate value sets per register. This allows me to perform fairly precise analysis for situations like jump tables, identifying unreachable entries and code/data overlap. Like most static analysis techniques it still runs into brick walls for code which relies heavily on self-modification. [1] https://github.com/JohnEarnest/Octo https://github.com/JohnEarnest/Octo