What is bad about median is that a compiler that performs well on most benchmarks but then falls off a cliff for many other, but less than half, performs well a

What is bad about median is that a compiler that performs well on most benchmarks but then falls off a cliff for many other, but less than half, performs well altogether. I think the distance-weighted estimator would serve the purpose of the ranking better. http://en.wikipedia.org/wiki/Distance-weighted_estimator http://en.wikipedia.org/wiki/Distance-weighted_estimator This measure finds a measure of the centre of the distribution that gives less weight to items further away from that centre without discarding them. In this way, the measure is not dominated by outliers, but they still contribute to the final result. I don't say that this is the best measure, only one that is better. A factor analysis might get at the extent to which the different benchmarks are doing the same thing. Some attempt to find measures of how these activities are represented in larger pieces of code might suggest weights for the benchmarks (there's a literature on analysing loads that might be relevant here). But this would make their analysis more complex and more prone to bias leaking into their analysis, so I don't say they ought to do this; my point is that median is simply a bad population measure for the purposes of their survey. You have argued against me without trying to defend the choice of median as the measure used for the ranking. Is that denialism?