3 ms·
Wow, 30% on ARC-AGI-3 for $20k total. Huge jump from GPT-5.6's 7.8% at $20k per task. I continue to believe ARC-AGI measures something different and important c
by modeless 3mo ago
Wow, 30% on ARC-AGI-3 for $20k total. Huge jump from GPT-5.6's 7.8% at $20k per task. I continue to believe ARC-AGI measures something different and important compared to other benchmarks.
- oh_no 3mo agoseeing a jump this big is not a great sign for the continuing value of a benchmark
- modeless 3mo agoIt will continue to be valuable as a cost and speed benchmark long after it is saturated at the high end. And they are already working on ARC-AGI 4 and thinking about going even farther.
- dominotw 3mo ago> I continue to believe ARC-AGI measures something different why is that? its now being benchmaxxed too