4 ms·

Sadly this is seen at so many prestigious ML conferences, a trimmed X axis which makes performance seem significant when it’s sometimes incremental

• exe34 2y ago
  I think it's acceptable if you're trying to show subtle differences - but I would probably put the whole plot and then the zoomed version and clearly label it as "zoomed in for highlighting <.....>"
  • nerdponx 2y ago
    You don't need to include 0 on every axis. In this case they really made the numbers smaller than they should be, so it's hard to see that the scale is on the order of single digits. It looks like this is about a 3-4% improvement over GPT-4o-mini and Gemini Pro 1.5. The bigger problem here is not the axis baseline, but the fact that I have no idea (as a non-AI-researcher) what benchmark this is, or if 0 is even the natural minimum. The caption should at least mention what the natural range of the x-axis is.
    • Ukv 2y ago
      > the fact that I have no idea (as a non-AI-researcher) what benchmark this is The figure labels it as as "average score [on] 6 multimodal reasoning benchmarks", and the caption notes that the full results are in table 7 - which lists those benchmarks: MMStar-R, MMBench-R, MMVet-R, MathVista, AI2D, Hallusion I think it's mostly fine as a lead diagram giving an overview before going into detail.
      • nerdponx 2y ago
        Right, I don't need to know what they are, I just need to know what "64" means. Is the baseline actually 0? That detail is enough to avoid actually drawing 0 on the axis.