4 ms·
> If they had, surely they could have gotten themselves an even higher mark than 25%. there is potentially some limitation of LLMs memorizing such complex pro
by riku_iki 2y ago
> If they had, surely they could have gotten themselves an even higher mark than 25%.
there is potentially some limitation of LLMs memorizing such complex proofs
- woopwoop 2y agoThey aren't proofs, they're just numbers. All the questions have numerical answers. That's how they're evaluated.
- riku_iki 2y agoI think those reasoning models are smart enough to not emit memorized answer if they can't come with CoT proof. But OAI could draw any result, no one was checking, they probably were not brave enough to declare math as solved topic.