3 ms·
This study appears to be evidence against that: the model failed the benchmark but arrived at the correct answer.
by letmevoteplease 2mo ago
This study appears to be evidence against that: the model failed the benchmark but arrived at the correct answer.
- fsh 2mo agoHalf of the benchmarks don't have published answers. That's why training data providers have been hiring physicists to solve them.
- deleted 2mo ago[deleted]
- red75prime 2mo agoThis is unconventional benchmaxxing then, when they decrease the benchmark scores to allow models to generalize on correct solutions.