3 ms·
The commercial decompiler they compared to is RetDec, which is not really a competent commercial decompiler. By "the commercial decompiler" I was expecting to s
by rfoo 6y ago
The commercial decompiler they compared to is RetDec, which is not really a competent commercial decompiler. By "the commercial decompiler" I was expecting to see IDA Pro, the de-facto standard of this industry. Or at least Ghidra if they need an open source & free one.
Also, I'd argue their benchmark is unfair. Checkout the appendix F, in the second example RetDec did actually produced correct result (though its output is not the most readable and it didn't de-optimize enough to remove noises introduced by compiler optimization), but they dismissed it as "difficult for human understanding". And in the first example I suspect they hand-picked a case where RetDec failed to perform function signature recovery because the function argument is floating point.
Edit: also, they run their benchmark on MIPS, because apparently their NN performs worse on x86 than MIPS. But no traditional decompiler was heavily optimized for MIPS.
- qayxc 6y agoThere seem to be some copy&paste errors in their NN-decompiled version in appendix F as well, which strikes me as odd. Their version (ii) cannot compile, since the variable names don't match (e.g. they return "c" instead of "v3" and multiply by "b3" instead of "v3"). They did use x86-64 ISA as well, though without any optimisation enabled, i.e. "-O0" (see Table 5 in appendix D). I doubt those results are very useful in practice due to the complete lack of optimisation.