4 ms·

Does it work against intentionally obfuscated binary code? Anti-piracy measures and suchlike.

by geographomics 11y ago

tptacek 11y ago
I'd also be curious about how often you see obfuscated application code in either app store.
- NateLawson 11y ago
  It's really common in Android (10-20% of apps), but happens occasionally in iOS too. The most common Android obfuscator is Proguard, but we also see Codeguard and others. The most surprising thing we found is that many developers build their own custom obfuscator. I think it's because they're trying to keep people from ripping off their code or hide secrets on the client side. Because Android allows a variety of techniques (custom classloaders, native code extensions that can overwrite their own code), there's a proliferation of methods there. In iOS, it's less common. You'll see a few frameworks like RNCryptor used to hide client-side secrets. Some apps use UAObfuscatedString to try to hide class names and selectors, but our similarity algorithms are based on a number of independent program features, including control-flow graph structure, which are unaffected by this. There are obfuscating compilers too, but it's very rare for anyone to use them.
  - tedunangst 11y ago
    > our similarity algorithms are based on a number of independent program features, including control-flow graph structure, which are unaffected by this. I wonder if/when that will change. Obscuring CFG fingerprints wasn't a problem before. I suppose it depends if people are truly trying to hide their dependencies, or if the 3rd party code just gets caught up as part of trying to prevent the main code from being decompiled.
    - NateLawson 11y ago
      It's usually the latter, which is that 3rd-party code just gets pulled into what the developer is actually trying to protect. Nobody tries to hide all their dependencies, as far as I can tell. Occasionally, they're trying to hide one particular extension, for example, a banking auth framework.
NateLawson 11y ago
Yes, in two ways. First, it's often useful to target malware or other systems by the common packer code they use. This doesn't require deobfuscating anything. More interestingly, the matching algorithms we developed are by design resistant to many common obfuscation schemes. For example, Proguard renames variables and functions, as well as discarding dead code. But other aspects of the code that we index, including control flow structures and data references survive. We've designed our matching engine to apply a combination of all these factors to be resilient to changes in subsets of them. Compilation is one of the simplest forms of "obfuscation", so started with dealing with different levels of optimization discarding code or changing the opcodes. If a program is self-decrypting, then we have to apply standard unpacking techniques to get back to a reasonable format before ingesting it. Nothing magic there.
- munin 11y ago
  What if the obfuscation removes the ability to construct a control flow graph, but is not self-decrypting?
  - NateLawson 11y ago
    We depend on a huge number of program features so CFG alone isn't enough to throw it off (data references, function layout, linker behavior, and much more are included in our matching). You're right that, at some point, you've destroyed enough of the features that matching will fail. For that case, you'll always need targeted techniques, such as for virtualizing obfuscators. Since there's a big performance loss when you use them, almost no legitimate mobile apps are willing to pay the battery and speed loss.

Does it work against intentionally obfuscated binary code? Anti-piracy measures and suchlike.

tptacek 11y ago

I'd also be curious about how often you see obfuscated application code in either app store.

NateLawson 11y ago
It's really common in Android (10-20% of apps), but happens occasionally in iOS too. The most common Android obfuscator is Proguard, but we also see Codeguard and others. The most surprising thing we found is that many developers build their own custom obfuscator. I think it's because they're trying to keep people from ripping off their code or hide secrets on the client side. Because Android allows a variety of techniques (custom classloaders, native code extensions that can overwrite their own code), there's a proliferation of methods there. In iOS, it's less common. You'll see a few frameworks like RNCryptor used to hide client-side secrets. Some apps use UAObfuscatedString to try to hide class names and selectors, but our similarity algorithms are based on a number of independent program features, including control-flow graph structure, which are unaffected by this. There are obfuscating compilers too, but it's very rare for anyone to use them.
- tedunangst 11y ago
  > our similarity algorithms are based on a number of independent program features, including control-flow graph structure, which are unaffected by this. I wonder if/when that will change. Obscuring CFG fingerprints wasn't a problem before. I suppose it depends if people are truly trying to hide their dependencies, or if the 3rd party code just gets caught up as part of trying to prevent the main code from being decompiled.
  - NateLawson 11y ago
    It's usually the latter, which is that 3rd-party code just gets pulled into what the developer is actually trying to protect. Nobody tries to hide all their dependencies, as far as I can tell. Occasionally, they're trying to hide one particular extension, for example, a banking auth framework.

NateLawson 11y ago

Yes, in two ways. First, it's often useful to target malware or other systems by the common packer code they use. This doesn't require deobfuscating anything. More interestingly, the matching algorithms we developed are by design resistant to many common obfuscation schemes. For example, Proguard renames variables and functions, as well as discarding dead code. But other aspects of the code that we index, including control flow structures and data references survive. We've designed our matching engine to apply a combination of all these factors to be resilient to changes in subsets of them. Compilation is one of the simplest forms of "obfuscation", so started with dealing with different levels of optimization discarding code or changing the opcodes. If a program is self-decrypting, then we have to apply standard unpacking techniques to get back to a reasonable format before ingesting it. Nothing magic there.

munin 11y ago
What if the obfuscation removes the ability to construct a control flow graph, but is not self-decrypting?
- NateLawson 11y ago
  We depend on a huge number of program features so CFG alone isn't enough to throw it off (data references, function layout, linker behavior, and much more are included in our matching). You're right that, at some point, you've destroyed enough of the features that matching will fail. For that case, you'll always need targeted techniques, such as for virtualizing obfuscators. Since there's a big performance loss when you use them, almost no legitimate mobile apps are willing to pay the battery and speed loss.