5 ms·
Never ever use averages in your stats. Use medians and percentiles. Averages are a meaningless stat especially if you benchmark.
by dmitriid 9y ago
Never ever use averages in your stats. Use medians and percentiles.
Averages are a meaningless stat especially if you benchmark.
- mr_toad 9y agoNever use the best linear unbiased estimator? That's a strong statement.
- dmitriid 9y agoIt’s not the best. There’s a Russian joke that goes “One person is cooling in the morgue, the other is running a high fever. The average temperature in the hospital is 36.6 C though” Averages are very susceptible to outliers and don’t give a proper insight into your data. What you really want is: how did half of my requests perform, 90% of requests perform, 95% of requests perform, 99% of requests perform
- screaminghawk 9y agoMedian is a type of average. You are confusing "average" with "mean". To be fair, it's not clear which form of average is used in the article and your assumption that it is the mean is likely correct.
- blueline 9y agoI have never seen median referred to as a "type of average" in any context. Is there any foundation to this?
- niteshade 9y agoQuite literally the first result in Google: http://www.dummies.com/education/math/pre-algebra/the-three-types-of-average-median-mode-and-mean/ http://www.dummies.com/education/math/pre-algebra/the-three-...
- e12e 9y agoAlso, from webster's: "av·er·age\ˈa-v(ə-)rij\ noun : a number that is calculated by adding quantities together and then dividing the total by the number of quantities : a level that is typical of a group, class, or series : a middle point between extremes Full Definition 1 a : a single value (as a mean, mode, or median) that summarizes or represents the general significance of a set of unequal values b : mean 1b 2 a : an estimation of or approximation to an arithmetic mean (...)" It's common to see arithmetic mean and average to be treated as synonyms - but the mean is really just a type of average.
- claytonjy 9y agoYou might like this blog post from John Myles White: http://www.johnmyleswhite.com/notebook/2013/03/22/modes-medians-and-means-an-unifying-perspective/ http://www.johnmyleswhite.com/notebook/2013/03/22/modes-medi... It compares mode, median, and mean as each being specific implementations of a more general concept of discrepancy.
- phamilton 9y agoColloquially, median and mode are used as average more than you'd expect. For example, we might say "the average joe is right handed", which is most likely the mode.
- wilgertvelinga 9y agoI would say leave the average in but also show standard deviation and 95th percentile average. Of course it is still very unlikely that Azure Functions would come out on top in this case.
- currysoup_t 9y agoWhat leads you to believe performance follows a normal distribution? For a given especially slow request would you expect a corresponding especially fast request?
- DenisM 9y agoStandard deviation applies to all distributions, not just the normal distribution.
- teej 9y agoYou are arguing against yourself here. Having the mean and 95th/99th percentile is going to give you a better picture of the distribution than mean alone. Mean alone actually doesn’t give you ANY idea of the distribution. I don’t know why OP is being downvoted, managing to 95th percentile is very common practice and I also thought was a missing addition to the article.