I kinda think this article proves too much; similar arguments could be made against any statistical simplification of performance. It's not that it's "wrong", i

I kinda think this article proves too much; similar arguments could be made against any statistical simplification of performance. It's not that it's "wrong", it's that it's "too right". I mostly advocate for the min in the specific context of benchmarking to improve the performance of a specific piece of code, on the theory that it's the best measure to cut away all the random bits you can't control with changes to the algorithm under test. If you've got a 5x variance in performance, and you make a 10% improvement in your algorithm, min seems like one of the better ways to figure that out, but you've still got a fundamental problem that that amount of non-determinism is just too much noise to see through via any mechanism very reliably, without a lot of runs. In terms of measuring "real" performance, yeah, min isn't a very good choice, being equivalent to just measuring the 0%th percentile, which anyone doing real measuring knows isn't all that useful in the "real world". But when I'm optimizing, I use "min". I also, as others have said, try to just eliminate any variance as high as what is discussed in the article because that's difficult-to-impossible to work with no matter how you slice it. (You can in principle always do "enough runs" to get through any random variance, but that doesn't mean you're willing to wait for a 1000 runs of your minute-long computation just to see if this particular improvement was statistically-significant, and then do it another fifty times.)