4 ms·
> Another interesting question is why the frontier labs are piling on pure maths The reason is that the original scaling axes (parameters, training tokens, tes
by anon373839 2mo ago
> Another interesting question is why the frontier labs are piling on pure maths
The reason is that the original scaling axes (parameters, training tokens, test-time compute) have saturated already, but RLVR (reinforcement learning from verifiable rewards) is still scaling well. And math has this nice property where you can synthetically generate arbitrary volumes of rewards to train the model, because math is self-contained and completely objective. Open-ended reasoning and analysis don't have that convenient property, and that is why progress is much slower outside of math and coding.
- robotpepi 2mo agodo we have any concrete idea of how well the models are scaling now? I agree with these 10 results being impressive, and it is easy to think "wow, and last year the models were barely able to solve IMOs problems". But for me it is perfectly possible that a non sofic group could be found by 100 good IMO students working on all the different strategies that have been proposed (OpenAIs solution was based on "expander graphs", which were introduced to solve the problem some years ago), so it could be that current AI is simply many (say 1000) old models working in parallel. This is linear scaling, not exponential. It could be I'm completely wrong also, the problem is that we have little information.