4 ms·

Hi, I'm happy to discuss how we're finding hidden flaws in millions of apps that even developers didn't know about. We've built a really cool binary code search

Hi, I'm happy to discuss how we're finding hidden flaws in millions of apps that even developers didn't know about. We've built a really cool binary code search engine that has indexed the structure and behavior of apps. Our engine allows us to quickly find apps that exhibit particular problems, such as calling a broken API or using a version of a library that has a vulnerability. I need to write more about how it works. We translate the app code into an intermediate language (like LLVM bitcode) and index features derived from both the structure (callgraph/control flow graph) and syntax (opcodes) of each function. This allows you to search for snippets of code that match particular patterns or discover the relationship between modules by assessing the similarity of each. Since we use an IL, we can match code cross-platform. I'd love to talk about it here if you have questions.

Natsu 11y ago
How many false positives would you get if you ran it against an Oracle product? :)
- NateLawson 11y ago
  While that's a great joke, we actually have a unique way of minimizing false positives compared to the average static analysis tool. We apply code similarity techniques to reuse analysis across apps, so we don't naively treat a binary as a random jumble of code. So if there's a flaw in a particular parameter passed to a single function, we can find all apps that have the relevant code and then zoom in on just calls to this function. The smaller the number of candidates a matching algorithm has to consider, the less chance of a false positive.
  - Natsu 11y ago
    Just how specifically does it identify its findings? Will it just say "this looks like a common buffer overflow/format string bug/etc." or can it say things like "this embeds openssl 0.9.8q, which is horribly insecure"?
    - NateLawson 11y ago
      Yes, the 2nd one. We can tell people: * What components are in the app * Exactly which versions * How they're being used/integrated We give very specific instructions how to fix, telling devs exactly which files or components are causing the issue and why. You might also like the native library dependency graph, which helps you understand the linkage of various libs even in apps that don't have any bugs. We're not going to dump a list of thousands of random potential flaws because we think devs are busy enough as it is. You can see what I mean here (no registration): https://searchlight.sourcedna.com/search https://searchlight.sourcedna.com/search Search for "HBO" or "Modern War" to see a few apps that have an Android M crash bug and the advice we give. [Edited to give more background]
      - tptacek 11y ago
        The chord graphs on the app pages are new since the last time I saw Searchlight. What do they mean? (Maybe Modern War's is just not clear).
        NateLawson 11y ago
        The chord graphs show the linkage relationships amongst all the native libraries in Android apps. Each library has an edge to the other libraries it depends on. To keep it clear, we don't show platform libraries that are too common, like libc. We think it might help devs understand how dependencies are dragged in, especially from code that's not theirs. I'd appreciate feedback on if it actually is helpful or what other things we could make more clear.
        Natsu 11y ago
        It looks like a wormhole and a couple of the edges are hard to read, but it does display in an interesting way which libraries are sucking in all the crap.
        tptacek 11y ago
geographomics 11y ago
Does it work against intentionally obfuscated binary code? Anti-piracy measures and suchlike.
- tptacek 11y ago
  I'd also be curious about how often you see obfuscated application code in either app store.
  - NateLawson 11y ago
    It's really common in Android (10-20% of apps), but happens occasionally in iOS too. The most common Android obfuscator is Proguard, but we also see Codeguard and others. The most surprising thing we found is that many developers build their own custom obfuscator. I think it's because they're trying to keep people from ripping off their code or hide secrets on the client side. Because Android allows a variety of techniques (custom classloaders, native code extensions that can overwrite their own code), there's a proliferation of methods there. In iOS, it's less common. You'll see a few frameworks like RNCryptor used to hide client-side secrets. Some apps use UAObfuscatedString to try to hide class names and selectors, but our similarity algorithms are based on a number of independent program features, including control-flow graph structure, which are unaffected by this. There are obfuscating compilers too, but it's very rare for anyone to use them.
    - tedunangst 11y ago
      > our similarity algorithms are based on a number of independent program features, including control-flow graph structure, which are unaffected by this. I wonder if/when that will change. Obscuring CFG fingerprints wasn't a problem before. I suppose it depends if people are truly trying to hide their dependencies, or if the 3rd party code just gets caught up as part of trying to prevent the main code from being decompiled.
      - NateLawson 11y ago
        It's usually the latter, which is that 3rd-party code just gets pulled into what the developer is actually trying to protect. Nobody tries to hide all their dependencies, as far as I can tell. Occasionally, they're trying to hide one particular extension, for example, a banking auth framework.
bdamm 11y ago
Do you plan to make the tool available to general application development, e.g. outside of apps available on Google Play/AppStore? If I understand correctly, your architecture translates the binary code into an intermediate language (effectively a reverse compiler) so that patterns across different build targets can be recognized as the same API use? Can you translate from platforms like Java? Can your platform recognize a call to a poor native API if done through a platform like Java? (e.g. a Java program calls a function that the JVM translates into a native call, but the parameters are setting up the caller for a vulnerability?) Does your platform handle transitive data identification? My point is that the call itself may be generalized, and specific sources of that call may be the source of the problem, but that may be abstracted by several layers of functions. How does your product differentiate itself from the other static code analyzers available in the market place?
- NateLawson 11y ago
  We're focused on applying our engine to solve problems in the app stores, so no plans to make it a standalone software product. Yes, we track JNI links from Java -> native. Yes, we do data flow analysis as well, but it's targeted via control flow analysis. In other words, first we identify a code path of interest and then target the data values/types that appear on that path. This way we're not wasting time analyzing data that is unimportant. We're not a static code analyzer, we're a service for helping developers improve mobile apps. We'll never tell you "Here are 8488 potential integer overflows to investigate." Instead, we've pre-processed the issues to give targeted recommendations that any dev can understand. Quality over volume, in other words.
rmc 11y ago
Can I ask a silly question? How did you get all the apps? Did you have to buy them all and download them? Or do you have a special deal?
- tptacek 11y ago
  He does not have a special deal.

Natsu 11y ago

How many false positives would you get if you ran it against an Oracle product? :)

NateLawson 11y ago
While that's a great joke, we actually have a unique way of minimizing false positives compared to the average static analysis tool. We apply code similarity techniques to reuse analysis across apps, so we don't naively treat a binary as a random jumble of code. So if there's a flaw in a particular parameter passed to a single function, we can find all apps that have the relevant code and then zoom in on just calls to this function. The smaller the number of candidates a matching algorithm has to consider, the less chance of a false positive.
- Natsu 11y ago
  Just how specifically does it identify its findings? Will it just say "this looks like a common buffer overflow/format string bug/etc." or can it say things like "this embeds openssl 0.9.8q, which is horribly insecure"?
  - NateLawson 11y ago
    Yes, the 2nd one. We can tell people: * What components are in the app * Exactly which versions * How they're being used/integrated We give very specific instructions how to fix, telling devs exactly which files or components are causing the issue and why. You might also like the native library dependency graph, which helps you understand the linkage of various libs even in apps that don't have any bugs. We're not going to dump a list of thousands of random potential flaws because we think devs are busy enough as it is. You can see what I mean here (no registration): https://searchlight.sourcedna.com/search https://searchlight.sourcedna.com/search Search for "HBO" or "Modern War" to see a few apps that have an Android M crash bug and the advice we give. [Edited to give more background]
    - tptacek 11y ago
      The chord graphs on the app pages are new since the last time I saw Searchlight. What do they mean? (Maybe Modern War's is just not clear).
      - NateLawson 11y ago
        The chord graphs show the linkage relationships amongst all the native libraries in Android apps. Each library has an edge to the other libraries it depends on. To keep it clear, we don't show platform libraries that are too common, like libc. We think it might help devs understand how dependencies are dragged in, especially from code that's not theirs. I'd appreciate feedback on if it actually is helpful or what other things we could make more clear.
        Natsu 11y ago
        It looks like a wormhole and a couple of the edges are hard to read, but it does display in an interesting way which libraries are sucking in all the crap.
        tptacek 11y ago

geographomics 11y ago

Does it work against intentionally obfuscated binary code? Anti-piracy measures and suchlike.

tptacek 11y ago
I'd also be curious about how often you see obfuscated application code in either app store.
- NateLawson 11y ago
  It's really common in Android (10-20% of apps), but happens occasionally in iOS too. The most common Android obfuscator is Proguard, but we also see Codeguard and others. The most surprising thing we found is that many developers build their own custom obfuscator. I think it's because they're trying to keep people from ripping off their code or hide secrets on the client side. Because Android allows a variety of techniques (custom classloaders, native code extensions that can overwrite their own code), there's a proliferation of methods there. In iOS, it's less common. You'll see a few frameworks like RNCryptor used to hide client-side secrets. Some apps use UAObfuscatedString to try to hide class names and selectors, but our similarity algorithms are based on a number of independent program features, including control-flow graph structure, which are unaffected by this. There are obfuscating compilers too, but it's very rare for anyone to use them.
  - tedunangst 11y ago
    > our similarity algorithms are based on a number of independent program features, including control-flow graph structure, which are unaffected by this. I wonder if/when that will change. Obscuring CFG fingerprints wasn't a problem before. I suppose it depends if people are truly trying to hide their dependencies, or if the 3rd party code just gets caught up as part of trying to prevent the main code from being decompiled.
    - NateLawson 11y ago
      It's usually the latter, which is that 3rd-party code just gets pulled into what the developer is actually trying to protect. Nobody tries to hide all their dependencies, as far as I can tell. Occasionally, they're trying to hide one particular extension, for example, a banking auth framework.

bdamm 11y ago

Do you plan to make the tool available to general application development, e.g. outside of apps available on Google Play/AppStore? If I understand correctly, your architecture translates the binary code into an intermediate language (effectively a reverse compiler) so that patterns across different build targets can be recognized as the same API use? Can you translate from platforms like Java? Can your platform recognize a call to a poor native API if done through a platform like Java? (e.g. a Java program calls a function that the JVM translates into a native call, but the parameters are setting up the caller for a vulnerability?) Does your platform handle transitive data identification? My point is that the call itself may be generalized, and specific sources of that call may be the source of the problem, but that may be abstracted by several layers of functions. How does your product differentiate itself from the other static code analyzers available in the market place?

NateLawson 11y ago
We're focused on applying our engine to solve problems in the app stores, so no plans to make it a standalone software product. Yes, we track JNI links from Java -> native. Yes, we do data flow analysis as well, but it's targeted via control flow analysis. In other words, first we identify a code path of interest and then target the data values/types that appear on that path. This way we're not wasting time analyzing data that is unimportant. We're not a static code analyzer, we're a service for helping developers improve mobile apps. We'll never tell you "Here are 8488 potential integer overflows to investigate." Instead, we've pre-processed the issues to give targeted recommendations that any dev can understand. Quality over volume, in other words.

rmc 11y ago

Can I ask a silly question? How did you get all the apps? Did you have to buy them all and download them? Or do you have a special deal?

tptacek 11y ago
He does not have a special deal.