3 ms·
This article includes a misleading graph. The "performance improvement" bar graph has bars that extend down into performance degradation, making the improvement
by mrob 8y ago
This article includes a misleading graph. The "performance improvement" bar graph has bars that extend down into performance degradation, making the improvement look bigger than it really is. If you're going to label the bars as "improvement" then the y-axis should start at 1.
- wyldfire 8y agoNo, it's just normalized data. See the legend: "Throughput/latency improvement over glibc 2.27". Rather than show the new metric and the baseline, they showed all metrics as a function of the baseline. All of them are an improvement. This is a common idiom for compiler benchmarks -- IMO it's not misleading at all. > making the improvement look bigger than it really is. They even highlight the 1.0 line. IMO it would be confusing for them to start the y-axis at 1.0.
- mrob 8y agoI don't recall seeing normalized data like this on a bar chart before. It's standard for line charts where the x-axis is sampled from continuous data, not bar charts. The 1.0 value is effectively the zero point, so bars should extend up and down from 1.0 (in this case only up because no performance got worse).
- jcranmer 8y agoThis kind of normalized bar graph is the most common way things like SPEC results are reported, although I think that showing one category of bars that's fixed to 1.0 to label what it's being normalized against is more common. (Annoyingly, throughput and latency are both shoved in the same graph, which gives the misleading impression that you're comparing one against the other).