4 ms·

There are objective metrics, that’s what benchmarks are.

• Retric 4mo ago
  Companies game benchmarks, but sure they are better than nothing.
  • MattRix 4mo ago
    I mean you wanted something objective, and they are. I don’t know why you’re being dismissive of them, they’re a huge element of what drives model development forward. These companies aren’t just making stuff up, they really do want to improve the models, and the models really are improving.
    • Retric 4mo ago
      > I don’t know why you’re being dismissive of them I’m aware of multiple cases of benchmark cheating/“optimization”. So, taking benchmarks a face value seems laughable.