3 ms·
Companies game benchmarks, but sure they are better than nothing.
by Retric 4mo ago
Companies game benchmarks, but sure they are better than nothing.
- MattRix 4mo agoI mean you wanted something objective, and they are. I don’t know why you’re being dismissive of them, they’re a huge element of what drives model development forward. These companies aren’t just making stuff up, they really do want to improve the models, and the models really are improving.
- Retric 4mo ago> I don’t know why you’re being dismissive of them I’m aware of multiple cases of benchmark cheating/“optimization”. So, taking benchmarks a face value seems laughable.