4 ms·
Such a dramatic name for such a boring set of tests. We need to test whether it can come up with a Nobel Prize-winning scientific breakthrough, a Booker/Pulitze
by rednafi 2y ago
Such a dramatic name for such a boring set of tests. We need to test whether it can come up with a Nobel Prize-winning scientific breakthrough, a Booker/Pulitzer-worthy novel, Ken Thompson-level code that solves a real problem, or a proof for Fermat’s Last Theorem.
- vitiral 2y agoNone of those are easily verifiable
- rednafi 2y agoThen they probably shouldn't have called it "Humanity's last exam." Kinda lame, if you think about it.
- egillie 2y agoThis makes me wonder if you could train an llm without any references to Wiles’ work and see if it can compete Fermat’s last theorem