3 ms·
> making the most interesting and challenging LLM benchmark so far. This[1] is currently the most challenging benchmark. I would like to see how O3 handles it,
by lossolo 2y ago
> making the most interesting and challenging LLM benchmark so far.
This[1] is currently the most challenging benchmark. I would like to see how O3 handles it, as O1 solved only 1%.
1. https://epoch.ai/frontiermath/the-benchmark https://epoch.ai/frontiermath/the-benchmark
- pynappo 2y agoApparently o3 scored about 25% https://youtu.be/SKBG1sqdyIU?t=4m40s https://youtu.be/SKBG1sqdyIU?t=4m40s
- FiberBundle 2y agoThis is actually the result that I find way more impressive. Elite mathematicians think these problems are challenging and thought they were years away from being solvable by AI.
- modeless 2y agoYou're right, I was wrong to say "most challenging" as there have been harder ones coming out recently. I think the correct statement would be "most challenging long-standing benchmark" as I don't believe any other test designed in 2019 has resisted progress for so long. FrontierMath is only a month old. And of course the real key feature of ARC is that it is easy for humans. FrontierMath is (intentionally) not.