8 ms·

A Sampling of Anti-Decompilation Techniques

120 points by swalsh 9y ago

ttoinou 9y ago
What about using UPX ? https://upx.github.io https://upx.github.io Can it make harder for software crackers to achieve their goal ?
- TACIXAT 9y ago
  upx -d packed.exe UPX is fairly easy to defeat, but many malicious samples will have their own packers / loaders.
- wiz21c 9y ago
  not alone. It's been long time since I've hacked anything but first one had fun by storing the decompression code into the interrupt table (yes, there). Or by making sure the decompression routine used accurate timings (so that if you fiddle with step by step execution it fails). Or calculating jump's based on values on the screen (that is, not the values show by your debugger). That way the decompiler should, at least, be able to execute the code. But that was before the protected mode in intel. My knowledge stops there.
- mar77i 9y ago
  Wow, that copyright text with guys named Molnar and Reiser reads a tad past my bullshit thresholds.
krylon 9y ago
As someone who has only ever written about 20 instructions worth of assembly, I am kind of torn. The kind of cleverness needed to thwart the decompiler's efforts is very impressive. A part of me wishes I had been around at a time when assembly was an acceptable "language" instead of the last resort it is today for most scenarios. Getting this close to the metal (or silicon) and being able to pull off such a stunt must be exhilarating. At the same time, another part of me is very happy I do not have to deal with such low-level details. Debugging code written in assembly (with assembly being the main language, not just inline assembly in a C/C++ program or something like that) must have been exquisite torture.
- nether 9y ago
  I agree, this is pretty leet. Too leet for me.
- jackhack 9y ago
  Debugging was just different. You had to get into the mindset of the original developer and understand what he intended and the impact of poking a memory-mapped location (could cause a graphics page flip, or toggle the speaker, or executing something in ROM). Comments were your friend. Without, yes, it could be a challenge. But decompiling & debugging wasn't just limited to ASM. I once worked for a shop that lost its original C source code and only had a copy of the executable. (major source code control failure - no backups of the server). They decompiled it and worked from the resultant C code. It was as bad as you can imagine, and enormous amounts of time were spent renaming "variable1" to "customerName", or methods from "sub232" to "CalculateTax()", etc. After a few months the code was sprinkled with meaningful names, but it was still very difficult to read and felt very unnatural, formal, and distorted. It also ran slower than the original when recompiled. But as for defeating a disassembler, I recall a few strategies stretching back to the late 70s/early 1980s on the Apple ][, when the "disassembler" was a human being rather than a tool. This is probably more appropriately called obfuscation or even copy protection, but it feels related. There were a variety of approaches taken: * Self-modifying code. This was the most common approach. Code would loop through areas of memory, incrementing/decrementing values by set values or based on an algorithm to produce valid opcodes. At any point in time, much of the code loaded was nonsense, so debugging had to be done in decoded blocks. * Jump tables (and indexed jump tables with offsets calculated at runtime) so that the code would skip all over the place. This was mostly just an irritation, rather than a hard obfuscation. * Burying executable code in resources (sounds/images) and related techniques such as code that calculated its hashvalue and used that to self-modify to 'unlock' executable blocks. *Overlays. Code was loaded on top of unused resources, and resources loaded on top of code. And while it wasn't a decompiler defeat, per se, changing the timing for the floppy disk drivers which read from non-standard disks would prevent one from simply loading data from disk and decompiling -- only that code that is in memory could be examined. Nybble-copiers were created to defeat this approach. One or all of these approaches could be combined to produce code that was a nightmare to debug, but ultimately anything that the computer could execute could be decompiled, eventually. Fun times.
tptacek 9y ago
This is a really cool post. You can also target IDA itself directly, rather than the decompiler, making it difficult to even view the disassembly. It's been awhile since I did any of this kind of work (it's relevant to software security tokens, games, and content protection), but within the last few years people have published IDA RCE memory corruption, so I imagine it hasn't gotten too much harder to hopelessly confuse IDA.
- rjzzleep 9y ago
  Well, same here, it's been very long since I've seen that. But wasn't what got scrambled imports, polymorphic code nasty anti debugging the shareware packers? Asprotect and whatnot.
munin 9y ago
There's an interesting vein of research work here in making software reverse engineering more difficult, and measuring how much more difficult. A precursor to decompilation is control flow analysis, the production of the control flow graph you see in the "before" stages in all of the examples in this post. You can go one step further, on a good day, and make it very difficult (perhaps very very difficult, perhaps impossible) to recover a precise control flow graph for a function. There are a few different ways to do this, and I like these approaches more than targeting specific heuristics in IDA/Hexrays because, on a good day for the obfuscator, you can make a theoretical statement about the work effort required to un-do the obfuscation. If you can make that work effort large, then you start to have a security guarantee that is a shade of the security guarantee you get in cryptography. The methods outlined in the parent blog post are great because you can start using them today, but if they annoy Ilfak enough, he'll fix them and they'll stop working.
DogestFogey 9y ago
I'm surprised the Movfuscator hasn't been mentioned yet. It compiles C code into unconditional MOVs, and if you watch the author's Derbycon 2015 video there are ways you can scramble the MOV instructions, truely making it a decompilation nightmare. 1. Movfuscator page https://github.com/xoreaxeaxeax/movfuscator https://github.com/xoreaxeaxeax/movfuscator 2. Derbycon 2015 video https://www.youtube.com/watch?v=R7EEoWg6Ekk https://www.youtube.com/watch?v=R7EEoWg6Ekk
- akanet 9y ago
  This is because "movfuscation" isn't a practical option for people actually trying to ship binaries that still perform well for customers but resist reverse-engineering. One of the battlegrounds for this sort of thing is the tug of war between game developers and cheat developers, and games still need to perform very well. Things the author mentioned, like address-rewriting at runtime, don't incur a performance penalty.
  - doctorwho 9y ago
    Just rebuild the sensitive portions of your code using movfuscator and leave the performance critical stuff alone. As long as everything is statically linked and you don't do anything stupid like "if (check) unlock()" that can be easily patched, it would make life pretty miserable for the RE crowd.
    - ttoinou 9y ago
      What's wrong with "if (check) unlock()" ? x)
      - emiliobumachar 9y ago
        It can be modified to "if (true) unlock()" relatively very easily, even in binary. No disassembling needed.
  - rasz 9y ago
    tell that to game developers shipping shitty DRM running game code inside a non hardware accelerated VM.
glandium 9y ago
I don't know if decompilers are able to do something about it, but there's a "neat" technique where your machine code can be interpreted as two different sequences of instructions depending where you start instruction decoding. For a simple but artificial example of what I mean: The following sequence of bytes: b8 50 83 ec 10 Decodes as: mov $0x10ec8350, %eax if you start at the first byte, and, if you start at the second byte, as: push %eax sub $0x10, %esp Here I essentially hid an instruction in the mov'ed data, because that's the easiest way to create something like that, but I've seen mind blowing examples of this technique. I unfortunately don't remember where.
- psykotic 9y ago
  That's an issue for static disassembly, which is step one of decompilation. Your snippet is a classic example that can throw off naive linear-scan disassemblers. Recursive disassemblers can handle it easily if (and it's a big if) they can identify the basic block entry points. If you have top-level entry points (which can also be a problem), the main problem for a recursive disassembler are indirect jumps that don't fit standard patterns like switch jump tables. All of this gets easier if you can augment your static analysis with control flow traces from program executions with coverage of the relevant branches, so you don't miss basic block entry points.
  - munin 9y ago
    > All of this gets easier if you can augment your static analysis with control flow traces from program executions with coverage of the relevant branches, so you don't miss basic block entry points. That's only true if the traces you can generate do a reasonable job at covering the states the application can find itself in, which is a big assumption.
    - psykotic 9y ago
      Yes, that's what I meant by coverage of the relevant branches. But the good news is that all these partial, heuristic sources of information can be combined. E.g. for detecting function entry points, you can combine information from ELF/PE export tables (if present), function prologues detected by a linear scan, vtables, static CALL targets, dynamic CALL targets from run-time traces, etc.
kccqzy 9y ago
There are also undocumented instructions and prefixes. For example at this year’s DEFCON someone presented a particular kind of e9 jump prefixed with 66. That instruction is incorrectly disassmbled by all hitherto known disassemblers (IDA, gdb, objdump, VS, etc). And since that will change the length of the instruction, effectively you can make the disassembler produce garbage.
amenghra 9y ago
Reminds me tricks people used to crash debuggers. E.g. https://reverse.put.as/2012/01/31/anti-debug-trick-1-abusing-mach-o-to-crash-gdb/ https://reverse.put.as/2012/01/31/anti-debug-trick-1-abusing..., http://blog.ioactive.com/2012/12/striking-back-gdb-and-ida-debuggers.html http://blog.ioactive.com/2012/12/striking-back-gdb-and-ida-d... and https://xorl.wordpress.com/2009/01/05/more-gdb-anti-debugging/ https://xorl.wordpress.com/2009/01/05/more-gdb-anti-debuggin...
krylon 9y ago
There is an interesting PowerPoint presentation you can find on the internet on how Skype used to evade debuggers. http://www.secdev.org/conf/skype_BHEU06.handout.pdf http://www.secdev.org/conf/skype_BHEU06.handout.pdf The techniques described in that document are different, but in a way they are alike - both use dirty tricks to shield themselves from people trying to reverse engineer them. The only difference is that one takes place "at compile time", in way, while the other works in a dynamic program as it is executing.

120 points by swalsh 9y ago

What about using UPX ? https://upx.github.io https://upx.github.io Can it make harder for software crackers to achieve their goal ?

TACIXAT 9y ago
upx -d packed.exe UPX is fairly easy to defeat, but many malicious samples will have their own packers / loaders.
wiz21c 9y ago
not alone. It's been long time since I've hacked anything but first one had fun by storing the decompression code into the interrupt table (yes, there). Or by making sure the decompression routine used accurate timings (so that if you fiddle with step by step execution it fails). Or calculating jump's based on values on the screen (that is, not the values show by your debugger). That way the decompiler should, at least, be able to execute the code. But that was before the protected mode in intel. My knowledge stops there.
mar77i 9y ago
Wow, that copyright text with guys named Molnar and Reiser reads a tad past my bullshit thresholds.

krylon 9y ago

As someone who has only ever written about 20 instructions worth of assembly, I am kind of torn. The kind of cleverness needed to thwart the decompiler's efforts is very impressive. A part of me wishes I had been around at a time when assembly was an acceptable "language" instead of the last resort it is today for most scenarios. Getting this close to the metal (or silicon) and being able to pull off such a stunt must be exhilarating. At the same time, another part of me is very happy I do not have to deal with such low-level details. Debugging code written in assembly (with assembly being the main language, not just inline assembly in a C/C++ program or something like that) must have been exquisite torture.

nether 9y ago
I agree, this is pretty leet. Too leet for me.
jackhack 9y ago
Debugging was just different. You had to get into the mindset of the original developer and understand what he intended and the impact of poking a memory-mapped location (could cause a graphics page flip, or toggle the speaker, or executing something in ROM). Comments were your friend. Without, yes, it could be a challenge. But decompiling & debugging wasn't just limited to ASM. I once worked for a shop that lost its original C source code and only had a copy of the executable. (major source code control failure - no backups of the server). They decompiled it and worked from the resultant C code. It was as bad as you can imagine, and enormous amounts of time were spent renaming "variable1" to "customerName", or methods from "sub232" to "CalculateTax()", etc. After a few months the code was sprinkled with meaningful names, but it was still very difficult to read and felt very unnatural, formal, and distorted. It also ran slower than the original when recompiled. But as for defeating a disassembler, I recall a few strategies stretching back to the late 70s/early 1980s on the Apple ][, when the "disassembler" was a human being rather than a tool. This is probably more appropriately called obfuscation or even copy protection, but it feels related. There were a variety of approaches taken: * Self-modifying code. This was the most common approach. Code would loop through areas of memory, incrementing/decrementing values by set values or based on an algorithm to produce valid opcodes. At any point in time, much of the code loaded was nonsense, so debugging had to be done in decoded blocks. * Jump tables (and indexed jump tables with offsets calculated at runtime) so that the code would skip all over the place. This was mostly just an irritation, rather than a hard obfuscation. * Burying executable code in resources (sounds/images) and related techniques such as code that calculated its hashvalue and used that to self-modify to 'unlock' executable blocks. *Overlays. Code was loaded on top of unused resources, and resources loaded on top of code. And while it wasn't a decompiler defeat, per se, changing the timing for the floppy disk drivers which read from non-standard disks would prevent one from simply loading data from disk and decompiling -- only that code that is in memory could be examined. Nybble-copiers were created to defeat this approach. One or all of these approaches could be combined to produce code that was a nightmare to debug, but ultimately anything that the computer could execute could be decompiled, eventually. Fun times.

tptacek 9y ago

This is a really cool post. You can also target IDA itself directly, rather than the decompiler, making it difficult to even view the disassembly. It's been awhile since I did any of this kind of work (it's relevant to software security tokens, games, and content protection), but within the last few years people have published IDA RCE memory corruption, so I imagine it hasn't gotten too much harder to hopelessly confuse IDA.

rjzzleep 9y ago
Well, same here, it's been very long since I've seen that. But wasn't what got scrambled imports, polymorphic code nasty anti debugging the shareware packers? Asprotect and whatnot.

munin 9y ago

There's an interesting vein of research work here in making software reverse engineering more difficult, and measuring how much more difficult. A precursor to decompilation is control flow analysis, the production of the control flow graph you see in the "before" stages in all of the examples in this post. You can go one step further, on a good day, and make it very difficult (perhaps very very difficult, perhaps impossible) to recover a precise control flow graph for a function. There are a few different ways to do this, and I like these approaches more than targeting specific heuristics in IDA/Hexrays because, on a good day for the obfuscator, you can make a theoretical statement about the work effort required to un-do the obfuscation. If you can make that work effort large, then you start to have a security guarantee that is a shade of the security guarantee you get in cryptography. The methods outlined in the parent blog post are great because you can start using them today, but if they annoy Ilfak enough, he'll fix them and they'll stop working.

DogestFogey 9y ago

I'm surprised the Movfuscator hasn't been mentioned yet. It compiles C code into unconditional MOVs, and if you watch the author's Derbycon 2015 video there are ways you can scramble the MOV instructions, truely making it a decompilation nightmare. 1. Movfuscator page https://github.com/xoreaxeaxeax/movfuscator https://github.com/xoreaxeaxeax/movfuscator 2. Derbycon 2015 video https://www.youtube.com/watch?v=R7EEoWg6Ekk https://www.youtube.com/watch?v=R7EEoWg6Ekk

akanet 9y ago
This is because "movfuscation" isn't a practical option for people actually trying to ship binaries that still perform well for customers but resist reverse-engineering. One of the battlegrounds for this sort of thing is the tug of war between game developers and cheat developers, and games still need to perform very well. Things the author mentioned, like address-rewriting at runtime, don't incur a performance penalty.
- doctorwho 9y ago
  Just rebuild the sensitive portions of your code using movfuscator and leave the performance critical stuff alone. As long as everything is statically linked and you don't do anything stupid like "if (check) unlock()" that can be easily patched, it would make life pretty miserable for the RE crowd.
  - ttoinou 9y ago
    What's wrong with "if (check) unlock()" ? x)
    - emiliobumachar 9y ago
      It can be modified to "if (true) unlock()" relatively very easily, even in binary. No disassembling needed.
- rasz 9y ago
  tell that to game developers shipping shitty DRM running game code inside a non hardware accelerated VM.

glandium 9y ago

I don't know if decompilers are able to do something about it, but there's a "neat" technique where your machine code can be interpreted as two different sequences of instructions depending where you start instruction decoding. For a simple but artificial example of what I mean: The following sequence of bytes: b8 50 83 ec 10 Decodes as: mov $0x10ec8350, %eax if you start at the first byte, and, if you start at the second byte, as: push %eax sub $0x10, %esp Here I essentially hid an instruction in the mov'ed data, because that's the easiest way to create something like that, but I've seen mind blowing examples of this technique. I unfortunately don't remember where.

psykotic 9y ago
That's an issue for static disassembly, which is step one of decompilation. Your snippet is a classic example that can throw off naive linear-scan disassemblers. Recursive disassemblers can handle it easily if (and it's a big if) they can identify the basic block entry points. If you have top-level entry points (which can also be a problem), the main problem for a recursive disassembler are indirect jumps that don't fit standard patterns like switch jump tables. All of this gets easier if you can augment your static analysis with control flow traces from program executions with coverage of the relevant branches, so you don't miss basic block entry points.
- munin 9y ago
  > All of this gets easier if you can augment your static analysis with control flow traces from program executions with coverage of the relevant branches, so you don't miss basic block entry points. That's only true if the traces you can generate do a reasonable job at covering the states the application can find itself in, which is a big assumption.
  - psykotic 9y ago
    Yes, that's what I meant by coverage of the relevant branches. But the good news is that all these partial, heuristic sources of information can be combined. E.g. for detecting function entry points, you can combine information from ELF/PE export tables (if present), function prologues detected by a linear scan, vtables, static CALL targets, dynamic CALL targets from run-time traces, etc.

kccqzy 9y ago

There are also undocumented instructions and prefixes. For example at this year’s DEFCON someone presented a particular kind of e9 jump prefixed with 66. That instruction is incorrectly disassmbled by all hitherto known disassemblers (IDA, gdb, objdump, VS, etc). And since that will change the length of the instruction, effectively you can make the disassembler produce garbage.

amenghra 9y ago

Reminds me tricks people used to crash debuggers. E.g. https://reverse.put.as/2012/01/31/anti-debug-trick-1-abusing-mach-o-to-crash-gdb/ https://reverse.put.as/2012/01/31/anti-debug-trick-1-abusing..., http://blog.ioactive.com/2012/12/striking-back-gdb-and-ida-debuggers.html http://blog.ioactive.com/2012/12/striking-back-gdb-and-ida-d... and https://xorl.wordpress.com/2009/01/05/more-gdb-anti-debugging/ https://xorl.wordpress.com/2009/01/05/more-gdb-anti-debuggin...

krylon 9y ago

There is an interesting PowerPoint presentation you can find on the internet on how Skype used to evade debuggers. http://www.secdev.org/conf/skype_BHEU06.handout.pdf http://www.secdev.org/conf/skype_BHEU06.handout.pdf The techniques described in that document are different, but in a way they are alike - both use dirty tricks to shield themselves from people trying to reverse engineer them. The only difference is that one takes place "at compile time", in way, while the other works in a dynamic program as it is executing.