5 ms·

The book I have my eye on is Nonparametric Statistical Process Control [1] by Chakraborti and Graham. In the past I've also tried the trick of taking the logar

The book I have my eye on is Nonparametric Statistical Process Control [1] by Chakraborti and Graham. In the past I've also tried the trick of taking the logarithm of values to convert a power distribution into a more normal-ish distribution. My project was to see if I could alert on an increased rate of uploads to RubyGems, signaling a possible malware campaign. Time between uploads follows a power law. I used the logarithm trick to convert it -- though not successfully. There are actually multiple distributions -- the "random" one which is prominent, but also other peaks centered around 1 second, 2 seconds, 3 seconds etc. Basically the detection was made difficult because of mass uploads being rate-limited on the uploader's side. [1] https://www.amazon.com/dp/1118456033/ https://www.amazon.com/dp/1118456033/

kqr 3y ago
Thanks, I'll take a look! Seems like it's mainly about other distributions in the Gaussian domain of attraction, though, like Poisson and binomial.
- jacques_chester 3y ago
  Oh, I hadn't noticed that emphasis. I guess I'm still stuck with the logarithm trick. I also just checked Montgomery and he reports that another simple transform is y^0.25 -- ie, square root of square root. I haven't tried it though.
shadowsun7 3y ago
So I've got a dumb question here: what happens when you use vanilla XmR charts with J-curve shaped or sub-exponential distributions? My current simplistic (and very dumb!) solution that I've used for power-law type distributions — like HN virality, for instance — is to count the number of days between viral events, and then subject that to process control.[1] I basically take Wheeler's approach to chunky data and use that for J-curve type data, which tells me if the behaviour of my 'HN virality process' has changed. I'd be very interested to learn of other approaches. [1] HN traffic for commoncog.com displays routine variation most weeks with an Upper Process Limit of 192 and a Lower Process Limit of 0, unless one of my articles hit the front page, at which point I get 11-16k additional uniques).
- jacques_chester 3y ago
  Time between events is an approach Montgomery (8th EMEA) discusses in 7.3.5. The application there is for dealing with very low error/defect rates. I am not familiar with Wheeler's approach.
- kqr 3y ago
  I have an upcoming article on my lack of understanding on how to do this also. It's not finished but you may enjoy a near-finished draft. https://two-wrongs.com/extreme-value-spc https://two-wrongs.com/extreme-value-spc I did forget to bring up the Poisson approximation you mention though. I'll include that too.
  - jacques_chester 3y ago
    The example of performance is interesting because as you say, there are often multiple jostling distributions under the surface (GC is one, but another doozy is CPU frequency scaling). One possible way out is to look for measurements that contribute to running time but which are not affected by other factors. I remember the YJIT folks talking about using CPU instruction counters, but I can't find it on the benchmark website.

kqr 3y ago

Thanks, I'll take a look! Seems like it's mainly about other distributions in the Gaussian domain of attraction, though, like Poisson and binomial.

jacques_chester 3y ago
Oh, I hadn't noticed that emphasis. I guess I'm still stuck with the logarithm trick. I also just checked Montgomery and he reports that another simple transform is y^0.25 -- ie, square root of square root. I haven't tried it though.

shadowsun7 3y ago

So I've got a dumb question here: what happens when you use vanilla XmR charts with J-curve shaped or sub-exponential distributions? My current simplistic (and very dumb!) solution that I've used for power-law type distributions — like HN virality, for instance — is to count the number of days between viral events, and then subject that to process control.[1] I basically take Wheeler's approach to chunky data and use that for J-curve type data, which tells me if the behaviour of my 'HN virality process' has changed. I'd be very interested to learn of other approaches. [1] HN traffic for commoncog.com displays routine variation most weeks with an Upper Process Limit of 192 and a Lower Process Limit of 0, unless one of my articles hit the front page, at which point I get 11-16k additional uniques).

jacques_chester 3y ago
Time between events is an approach Montgomery (8th EMEA) discusses in 7.3.5. The application there is for dealing with very low error/defect rates. I am not familiar with Wheeler's approach.
kqr 3y ago
I have an upcoming article on my lack of understanding on how to do this also. It's not finished but you may enjoy a near-finished draft. https://two-wrongs.com/extreme-value-spc https://two-wrongs.com/extreme-value-spc I did forget to bring up the Poisson approximation you mention though. I'll include that too.
- jacques_chester 3y ago
  The example of performance is interesting because as you say, there are often multiple jostling distributions under the surface (GC is one, but another doozy is CPU frequency scaling). One possible way out is to look for measurements that contribute to running time but which are not affected by other factors. I remember the YJIT folks talking about using CPU instruction counters, but I can't find it on the benchmark website.