I think this is an error in the leaderboard. Looking at the test logs, they had reasoning effort set to 'high'. So it should be in the CoT category instead of t

I think this is an error in the leaderboard. Looking at the test logs, they had reasoning effort set to 'high'. So it should be in the CoT category instead of the base LLM category. https://huggingface.co/datasets/arcprize/arc_agi_v1_public_eval/tree/main/deepseek-v3.2 https://huggingface.co/datasets/arcprize/arc_agi_v1_public_e... "kwargs": { "max_tokens": 100000, "stream": true, "reasoning_effort": "high", "rate_limit": { "rate": 2, "period": 60 } } The other paper ran Deepseek v3.2 without reasoning as a baseline and got 15.5%, which is much more in line with other base LLMs like GPT-5.2.