5 ms·

Machine learning is pervasive in the new products targeted at cybersecurity and infosec. From malware to anomaly detection among billions of logs, it's been par

by pilooch 10y ago

Machine learning is pervasive in the new products targeted at cybersecurity and infosec. From malware to anomaly detection among billions of logs, it's been part of the demand of my customers for at least four years now. Most of the largest companies are already running and testing a series of novel proprietary or custom in-house systems. The main differences with straight 'supervised' machine learning are the lack of labels, and the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here). So unsupervised learning is king in this domain, and often security operators have to keep an eye on and interpret the results. Another difference with other fields is that datasets are rare, mostly because of privacy as the logs can be very revealing. For this reason the market exhibits a lock that cannot be overcome by everyone I believe. Basically, you sort of need to be in the place already. Here is a recent very high level survey on the topic (a TC report, not that bad for once), https://techcrunch.com/2016/07/01/exploiting-machine-learning-in-cybersecurity/ https://techcrunch.com/2016/07/01/exploiting-machine-learnin... Here are two of our own Open Source tooling we use the most in application: - very efficient C++ map-reduce feature generation for logs (for ML and analytics): https://github.com/soprasteria/cybersecurity-miw https://github.com/soprasteria/cybersecurity-miw - machine learning / deep learning server: https://github.com/beniz/deepdetect https://github.com/beniz/deepdetect The ML cybersecurity + infosec field is still young, but moving very fast, a lot of new startups and (somewhat opaque) products.

kriro 10y ago
Yeah my initial reaction was...wait a second ML is used in info sec. The most obvious examples are intrusion detection and general network analysis. A less glamorous example is pretty much every spam filter (which also filters fishing stuff and is thus very much infosec-relevant). I think false positives are a huge problem in the info sec context and less so in other domains. To give a made up example let's say you have ML-based intrusion detection that is hotwired to some sysadmin pager. Even if your false positive rate is fairly low (at acceptable levels in other domains) with the sheer amount of data a bunch of sysadmin calls will be triggered in sum. That is a huge human factor risk as it most likely will result in teaching admins to ignore certain alerts....which of course opens up the interesting attack vector of understanding/"reversing" the ML and triggering false positives a bunch of times until you want to attack through a channel that will trigger a similar response. Regarding the available data sets: I suppose honeypots/-nets could be created for the purpose of gathering some data sets.
mafribe 10y ago
Is there any evidence that this works in the sense of foiling sophisticated attacks?
- pilooch 10y ago
  AFAIK not really. Detecting C&C is not that easy in unsupervised settings. The MIT paper (link in the TC article) makes claims, but this is mostly PR. There are some reports from Google as well that you may find around, and at least at the time (few years back) they make the same observation we do with my coworkers: the rate of false positive remains high, and thus it is the job of trained security operators to spot the sophisticated attacks. Empowering the operators with more data mining tools is a good step not yet fulfilled I believe.
- hannob 10y ago
  If you ask "Is there any evidence for ..." for so-called IT security products then the answer is practically always "No".
Razengan 10y ago
> the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here). Unrelated, but that gave me a thought to ponder: Will it ever be possible to use computers made inside this universe to model a MORE complex universe? Has that already happened?
- drdeca 10y ago
  Well, what do you mean by more complex? A Turing machine can simulate a python interpreter. Have you seen the xkcd with the rocks in the desert? https://xkcd.com/505/ https://xkcd.com/505/
  - Razengan 10y ago
    I meant create a new, virtual universe that is more complex (i.e. contains more information) than this one.
    - drdeca 10y ago
      Well, in order to store the state of the simulation, it would need to store the amount of information needed to describe the state of the simulation. But the information needed to describe the state of the machine doing the simulation, by being enough to describe the state of the machine storing the information about the simulation (including the part of the state which stores the information describing the simulation) , is therefore enough to store the information describing the state of the simulation. So, therefore, the smallest possible complete description of the state of the simulation is not larger than a complete description of the machine doing the simulating. So if that is what you mean by more complex / what you mean by more information, then, no, the less-information thing cannot completely simulate the more-information thing.
      - MichaelBurge 10y ago
        It could be that our universe's state can be described by the initial input '1234567', 1 page of differential equations, and an amount of time T to simulate. An inner universe's state could have an initial input of some string describing all of human history up to T in the outer universe, the same differential equations, and the same amount of time T to simulate. One might say that the inner universe contains more information, because the smallest unambiguous description of it is larger than the smallest description of its parent universe. However, the outer universe is able to simulate the inner universe.
sp_ 10y ago
Great summary! By nature of my job (eng lead of a major mobile malware detection team) I have a lot of startups pitch their ML solutions to me. A couple of thoughts: - There are no publicly available data sets for training available. There are a few small ones and a few old ones, but they don't reflect the reality of 2016. Companies that approach me and pitch me solutions to the malware of 2012 are not useful. - The majority of mobile malware is based on some kind of social engineering. On a code level these are indistinguishable from legitimate applications (the same APIs are used in the same fashion). The only difference is whether app behavior meets user expectations or not. Making this decision automatically seems intractable so far. - Malware is not really a well-defined term. There is phishing, toll fraud, Trojans, privilege escalation exploits, ... If you generically look for malware, the signals you will look for are going to approach the complete set of APIs made available by your OS. Your results will just be a giant blob where everything is connected. Pick a single malware category and focus on just that at a time. ML signals for priv esc will look very different from those for phishing. - ML is sexy. Malware analysis is not. Startups seem to hire too many ML people and not enough malware analysis people. I've had startups pitch to me that had literally zero people on staff who knew what mobile malware actually looked like. They just did anomaly detection and then tossed the results over to my team to verify the results. That's not how it works. We're not your QA team. :)
- pilooch 10y ago
  Hey, just finished a malware ML custom system for one of the largest european corporations, large enough that some malware is targeted at them. Result is 97% accuracy (they did retrain and check on their own held out dataset). More careful analysis is needed (many malware have high entropy 'zones' that may help the classifier find the right category), but overall it does work. See the Microsoft / Kaggle challenge on classifying malware families, winning solution is > 99% accuracy IIRC.
  - Cybiote 10y ago
    The 99% means little when it suffers from a similar sort of problem that the immune system has with cancer. Adversary's lack of stationarity vs a fixed model.
BrainInAJar 10y ago
Unfortunately the FP rate is just unacceptably high, you end up overloading your SOC (which means your analysts end up flagging a lot of real threats as FP)
netman21 10y ago
And of course Cylance for file artifact analysis and TrustPipe for two way network conversation analysis.

by pilooch 10y ago

kriro 10y ago

Yeah my initial reaction was...wait a second ML is used in info sec. The most obvious examples are intrusion detection and general network analysis. A less glamorous example is pretty much every spam filter (which also filters fishing stuff and is thus very much infosec-relevant). I think false positives are a huge problem in the info sec context and less so in other domains. To give a made up example let's say you have ML-based intrusion detection that is hotwired to some sysadmin pager. Even if your false positive rate is fairly low (at acceptable levels in other domains) with the sheer amount of data a bunch of sysadmin calls will be triggered in sum. That is a huge human factor risk as it most likely will result in teaching admins to ignore certain alerts....which of course opens up the interesting attack vector of understanding/"reversing" the ML and triggering false positives a bunch of times until you want to attack through a channel that will trigger a similar response. Regarding the available data sets: I suppose honeypots/-nets could be created for the purpose of gathering some data sets.

mafribe 10y ago

Is there any evidence that this works in the sense of foiling sophisticated attacks?

pilooch 10y ago
AFAIK not really. Detecting C&C is not that easy in unsupervised settings. The MIT paper (link in the TC article) makes claims, but this is mostly PR. There are some reports from Google as well that you may find around, and at least at the time (few years back) they make the same observation we do with my coworkers: the rate of false positive remains high, and thus it is the job of trained security operators to spot the sophisticated attacks. Empowering the operators with more data mining tools is a good step not yet fulfilled I believe.
hannob 10y ago
If you ask "Is there any evidence for ..." for so-called IT security products then the answer is practically always "No".

Razengan 10y ago

> the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here). Unrelated, but that gave me a thought to ponder: Will it ever be possible to use computers made inside this universe to model a MORE complex universe? Has that already happened?

drdeca 10y ago
Well, what do you mean by more complex? A Turing machine can simulate a python interpreter. Have you seen the xkcd with the rocks in the desert? https://xkcd.com/505/ https://xkcd.com/505/
- Razengan 10y ago
  I meant create a new, virtual universe that is more complex (i.e. contains more information) than this one.
  - drdeca 10y ago
    Well, in order to store the state of the simulation, it would need to store the amount of information needed to describe the state of the simulation. But the information needed to describe the state of the machine doing the simulation, by being enough to describe the state of the machine storing the information about the simulation (including the part of the state which stores the information describing the simulation) , is therefore enough to store the information describing the state of the simulation. So, therefore, the smallest possible complete description of the state of the simulation is not larger than a complete description of the machine doing the simulating. So if that is what you mean by more complex / what you mean by more information, then, no, the less-information thing cannot completely simulate the more-information thing.
    - MichaelBurge 10y ago
      It could be that our universe's state can be described by the initial input '1234567', 1 page of differential equations, and an amount of time T to simulate. An inner universe's state could have an initial input of some string describing all of human history up to T in the outer universe, the same differential equations, and the same amount of time T to simulate. One might say that the inner universe contains more information, because the smallest unambiguous description of it is larger than the smallest description of its parent universe. However, the outer universe is able to simulate the inner universe.

sp_ 10y ago

Great summary! By nature of my job (eng lead of a major mobile malware detection team) I have a lot of startups pitch their ML solutions to me. A couple of thoughts: - There are no publicly available data sets for training available. There are a few small ones and a few old ones, but they don't reflect the reality of 2016. Companies that approach me and pitch me solutions to the malware of 2012 are not useful. - The majority of mobile malware is based on some kind of social engineering. On a code level these are indistinguishable from legitimate applications (the same APIs are used in the same fashion). The only difference is whether app behavior meets user expectations or not. Making this decision automatically seems intractable so far. - Malware is not really a well-defined term. There is phishing, toll fraud, Trojans, privilege escalation exploits, ... If you generically look for malware, the signals you will look for are going to approach the complete set of APIs made available by your OS. Your results will just be a giant blob where everything is connected. Pick a single malware category and focus on just that at a time. ML signals for priv esc will look very different from those for phishing. - ML is sexy. Malware analysis is not. Startups seem to hire too many ML people and not enough malware analysis people. I've had startups pitch to me that had literally zero people on staff who knew what mobile malware actually looked like. They just did anomaly detection and then tossed the results over to my team to verify the results. That's not how it works. We're not your QA team. :)

pilooch 10y ago
Hey, just finished a malware ML custom system for one of the largest european corporations, large enough that some malware is targeted at them. Result is 97% accuracy (they did retrain and check on their own held out dataset). More careful analysis is needed (many malware have high entropy 'zones' that may help the classifier find the right category), but overall it does work. See the Microsoft / Kaggle challenge on classifying malware families, winning solution is > 99% accuracy IIRC.
- Cybiote 10y ago
  The 99% means little when it suffers from a similar sort of problem that the immune system has with cancer. Adversary's lack of stationarity vs a fixed model.

BrainInAJar 10y ago

Unfortunately the FP rate is just unacceptably high, you end up overloading your SOC (which means your analysts end up flagging a lot of real threats as FP)

netman21 10y ago

And of course Cylance for file artifact analysis and TrustPipe for two way network conversation analysis.