5 ms·

I am skeptical of anomaly detection since in my experience anomalies are common and diverse and don't actually matter, so I expect these systems to basically in

I am skeptical of anomaly detection since in my experience anomalies are common and diverse and don't actually matter, so I expect these systems to basically inundate people with false positives. Their offline training accuracy is garbage: 16% precision, so all of the real work is basically being done in the online training portion, which gets it to a respectable 82%+ precision. But they don't tell you how many alerts they had to label to get those numbers. Maybe over the long run you get those numbers, but you really want to know if it takes 10 or 10,000 examples to get there. Also, their dataset distribution is very different to reality: they have 7% of their dataset annotated as real anomalies; I don't think anyone in the real world wants 5% of their log entries to get flagged as anomalies. So I expect their precision numbers to be far worse on more realistically distributed logs.

asavinov 8y ago
Another problem of anomaly detection is that they do not provide any (domain specific) explanation for why the system thinks it is an anomaly. The system also does not say what to do in this situation, which means that such anomalies are not actionable findings. Therefore I think anomaly detection should be used as a pre-processing step which generates input for some other other components of the system.
stephengillie 8y ago
At a previous tech support position, I collaborated with a data scientist to create a predictive alert system based on system notification data. It would monitor the quantity of noise from each interface on the network and alert on anomalies in Slack. The only issue was that it didn't work - we saw only false positive noise, and it sat quiet during actual incidents. It would be interesting to see another team's attempts, and what different design choices they make.
dimitry12 8y ago
Are they doing supervised training for anomalies?
jstarfish 8y ago
It's a good time to make some money baffling people with bullshit in the cybersecurity space. Of course if you let an ML-powered "anomaly detection" engine run rampant on your logs, it's going to find anomalies...just like if you hire a ghost hunter, you'll be informed that your house is haunted. In the end, ghost chasing is all this anomaly nonsense turns out to be-- the justifications for conclusions by ML practitioners and ghost hunters alike tend to be equally mumbly and hand-wavy. Me working from home is technically an anomaly, and one these systems are all too eager to flag. We get random logins from overseas VPSes-- it's an anomaly! Oh, wait, no, we onboarded a client application. Oh, look, a random login from China for a US-based employee with no history of foreign logins! Yeah, that guy just started in a new position with travel requirements. Hey, this IP just tried to log into 5000 user accounts! Congratulations, you just alerted me to the existence of carrier NAT. None of this saves any time and usually wastes it, since it stirs up paranoia where none was otherwise warranted. It's a fun toy that gives the appearance of being productive when all it's actually doing is generating literally endless busywork. Good for justifying your SOC budget I suppose. But in the end nobody wants to pay a quarter-million dollars for a black box that just sits there quietly-- if it's not constantly drawing attention to itself and all the badness it's pretending to find, you're not going to have any reason to renew the license. "Renew it? Why? This thing didn't find anything at all last year."
- noir-york 8y ago
  So what does your organisation use for intrusion detection? Humans eyeballing logs doesn't scale. Rule-based approaches?
  - russh 8y ago
    Mostly user complaints...
  - deleted 8y ago
    [deleted]
- Eridrus 8y ago
  Oh I know, I spent a decade in Security and work in ML now, and I can see how badly people want to put the two together, but it's basically 90% bullshit and 10% same old shit of varying effectiveness.

asavinov 8y ago

Another problem of anomaly detection is that they do not provide any (domain specific) explanation for why the system thinks it is an anomaly. The system also does not say what to do in this situation, which means that such anomalies are not actionable findings. Therefore I think anomaly detection should be used as a pre-processing step which generates input for some other other components of the system.

stephengillie 8y ago

At a previous tech support position, I collaborated with a data scientist to create a predictive alert system based on system notification data. It would monitor the quantity of noise from each interface on the network and alert on anomalies in Slack. The only issue was that it didn't work - we saw only false positive noise, and it sat quiet during actual incidents. It would be interesting to see another team's attempts, and what different design choices they make.

dimitry12 8y ago

Are they doing supervised training for anomalies?

jstarfish 8y ago

It's a good time to make some money baffling people with bullshit in the cybersecurity space. Of course if you let an ML-powered "anomaly detection" engine run rampant on your logs, it's going to find anomalies...just like if you hire a ghost hunter, you'll be informed that your house is haunted. In the end, ghost chasing is all this anomaly nonsense turns out to be-- the justifications for conclusions by ML practitioners and ghost hunters alike tend to be equally mumbly and hand-wavy. Me working from home is technically an anomaly, and one these systems are all too eager to flag. We get random logins from overseas VPSes-- it's an anomaly! Oh, wait, no, we onboarded a client application. Oh, look, a random login from China for a US-based employee with no history of foreign logins! Yeah, that guy just started in a new position with travel requirements. Hey, this IP just tried to log into 5000 user accounts! Congratulations, you just alerted me to the existence of carrier NAT. None of this saves any time and usually wastes it, since it stirs up paranoia where none was otherwise warranted. It's a fun toy that gives the appearance of being productive when all it's actually doing is generating literally endless busywork. Good for justifying your SOC budget I suppose. But in the end nobody wants to pay a quarter-million dollars for a black box that just sits there quietly-- if it's not constantly drawing attention to itself and all the badness it's pretending to find, you're not going to have any reason to renew the license. "Renew it? Why? This thing didn't find anything at all last year."

noir-york 8y ago
So what does your organisation use for intrusion detection? Humans eyeballing logs doesn't scale. Rule-based approaches?
- russh 8y ago
  Mostly user complaints...
- deleted 8y ago
  [deleted]
Eridrus 8y ago
Oh I know, I spent a decade in Security and work in ML now, and I can see how badly people want to put the two together, but it's basically 90% bullshit and 10% same old shit of varying effectiveness.