3 ms·
Wasn’t that for human tests, i.e. not specifically AI benchmarks? Benchmarks should generally not be game-able by overfitting.
by lxgr 2y ago
Wasn’t that for human tests, i.e. not specifically AI benchmarks? Benchmarks should generally not be game-able by overfitting.
- matt3210 2y agoThe article shows all the tests against human performance. The math one in particular is the one where small variations reduce the success rate significantly. I can’t find the source but it was pasted here in the last 2 weeks.
- CrazyStat 2y agoYou're probably remembering this: https://news.ycombinator.com/item?id=42565606 https://news.ycombinator.com/item?id=42565606