4 ms·
I would love to see how this performs on ARC-AGI 2, zero-shot, private eval. I hope we get an update from Chollet and team regarding performance.
by daveguy 1y ago
I would love to see how this performs on ARC-AGI 2, zero-shot, private eval. I hope we get an update from Chollet and team regarding performance.
- danbtl 1y ago9.9% on ARC-AGI 2 https://x.com/fchollet/status/1953511631054680085 https://x.com/fchollet/status/1953511631054680085
- daveguy 1y agoHah, that was fast! Thank you. They must have had preview access. It didn't bode well that SimonW [0] had to explicitly tell GPT-5 to use python to get a table sorted correctly (but awesome that in can use python as a tool without any plumbing). It appears we are not quite to AGI yet. [0] https://simonwillison.net/2025/Aug/7/gpt-5/ https://simonwillison.net/2025/Aug/7/gpt-5/