3 ms·
I think the case for "axis must always go to 0" is overblown. Zero isn't always meaningful, for instance chance performance or performance of trivial algorithms
by glutamate 3y ago
I think the case for "axis must always go to 0" is overblown. Zero isn't always meaningful, for instance chance performance or performance of trivial algorithms is likely >0%. Sometimes if axis must go to zero you can't see small changes. For instance if you plot world population 2014-2024 on an axis going to zero, you won't be able to see if we are growing or shrinking.
- tkellogg 3y agoOTOH having the chart start at zero would REALLY emphasize how saturated this field is, and how little this announcement matters.
- c2occnw 3y agoThe difference between 32% and 70% wouldn't be significant if the chart started at zero?
- generalizations 3y agoIt would be very obvious indeed how small the difference between 73.7,73.0,71.4,and 69.8 actually is.
- nilstycho 3y agoI agree with your general point, but world population is still visibly increasing on that interval. https://ourworldindata.org/explorers/population-and-demography?time=2014..2024&Metric=Population&Projection+Scenario=Medium&country=~OWID_WRL https://ourworldindata.org/explorers/population-and-demograp... Perhaps "global mean temperature in Kelvin" would be a comparable example.
- pandastronaut 3y agoEven starting at 30%, the MMLU graph is false. The four bars are wrong. Even their own 73,7% is not at the right height. The Mixtral 71.4% is below the 70% mark of the axis. This is really the kind of marketing trick that makes me avoid a provider / publisher. I can't build trust this way.
- tylermw 3y agoI believe they are using the percentages as part of the height of the bar chart! I thought I'd seen every way someone could do dataviz wrong (particularly with a bar chart), but this one is new to me.
- pandastronaut 3y agoInteresting! It is probably one of the worst trick I have seen in a while for a bar graph. Never seen this one before. Trust vanishes instantly facing that kind of dataviz.
- radicality 3y agoWow, that is indeed a novel approach haha, took me a moment to even understand what you described since would never imagine someone plotting a bar chart like that.
- spetryk 3y agoThat's really strange and incredibly frustrating - but slightly less so if it's consistent with all of the bars (including their own). I take issue with their choice of bar ordering - they placed the lowest-performing model directly next to theirs to make the gap as visible as possible, and shoved the second-best model (Grok-1) as far from theirs as possible. Seems intentional to me. The more marketing tricks you pile up in a dataviz, the less trust I place in your product for sure.
- occamrazor 3y agoIt‘s more likely to be incompetence than malice: even their 73.7% is closer to 72% than to 74%.
- dskhudia 3y agoIt’s an honest mistake in scaling the bars. It’s getting fixed soon. The percentages are correct though. In the process of converting excel chart to pretty graphs for the blog, scale got messed up.
- tartrate 3y ago
- TZubiri 3y agoThen you can plot it on a greater timescale, or plot the change rate
- patrickthebold 3y agoCertainly a bar chart might not be the best choice to convey the data you have. But if you choose to have a bar chart and have it not start at zero, what do the bars help you convey? For world population you could see if it is increasing or decreasing, which is good but it would be hard to evaluate the rate the population is increasing. Maybe a sparkline would be a better choice?