3 ms·
This is a nicely done article. One of the issues with using a z-score for count data is it assumes the data are normally distributed. This implies values < 0 ar
by RA_Fisher 6y ago
This is a nicely done article. One of the issues with using a z-score for count data is it assumes the data are normally distributed. This implies values < 0 are possible. However, since counts are lower-bounded at zero, they’re asymmetric unlike the normal. This means that practically this method would produce too many false positives for most applications. Whatever threshold is chosen, is likely to have too narrow an envelope between false negative and false positive to be reliable for count data. However, for normally distributed data, this would work perfectly.
- mb7733 6y ago> One of the issues with using a z-score for count data is it assumes the data are normally distributed A z-score is just the number of standard deviations away from the mean a data point lies. It doesn't require any assumption about the distribution of the data. If you wanted to make some inference, like maybe about the likelihood of observing some z-score given a hypothesis, then you would need to assume a distribution. But, nothing the author does assumes a distribution, normal or otherwise. It doesn't mention probability at all as far as I can see.
- teej 6y agoWhen you decide that a value N standard deviations away from the mean is an anomaly that must be investigated, you implicitly make an assumption about the underlying distribution of the data.
- RA_Fisher 6y agoExactly, it’s a smuggled assumption. A standard deviation is the second moment of a normal distribution. A negative binomial is a much better model for count data: https://statwonk.com/poisson-gamma-negative-binomial.html https://statwonk.com/poisson-gamma-negative-binomial.html
- mb7733 6y agoThe simple approach in the article is no different if you imagine it as a negative binomial. The article does not assume any distribution, it's entirely nonparametric.
- conjectures 6y agoThat's not the case. Looking for |z|>t implicitly assumes the distribution is symmetric and unimodal for a start. As in, the method would break if those assumptions were violated. Non-parametric means more than just avoiding talk of distributions.
- mb7733 6y agoGood point about symmetry & unimodality. You could easily construct a bimodal example where the method fails entirely. In my previous post I said "doesn't assume any distribution" I meant it doesn't depend on any particular distribution function. But in the post above that I was sloppy and said "no assumptions about the distribution at all". Good catch.
- mb7733 6y agoThat's looking at it backwards---this is a non parametric approach. The author didn't assume a priori that a point with a z score greater than N was anamolous, for some fixed N. Instead, they determined N by looking at the past data. (Contrast with the parametric approach of assuming normality and concluding that any point lying >3 sd from the mean is an anamoly due to properties of the gaussian distribution.)
- usgroup 6y agoI don’t think this is correct . You can calibrate a threshold for a metric empirically without interpreting the metric probabilistically. Ie that z > N is empirically a good threshold for anomaly detection calls for no necessary further interpretation.
- RA_Fisher 6y agoSure, but for most processes that’ll yield disappointing performance. Any tool can be used without regard for the consequences, but knowing a tool’s statistical properties yields know-how about the consequences of its application. It’s often a matter of the costs of error / stakes of the most problem at hand. Cheap solutions are often the best for cheap problems.
- isoprophlex 6y agoGood point... maybe first transforming the data to be approximately normal could be a solution? (I'm not a statistician)
- gregw2 6y agoMy current take on this is that one should use a P99 (or P99.9, etc) median and anything above that is considered an anomaly. You then are not assuming anything about the distribution. I'm curious if others think this is a bad/good approach.