4 ms·
>> August 2025 https://www.wheresyoured.at/how-to-argue-with-an-ai-booster/ https://www.wheresyoured.at/how-to-argue-with-an-ai-booster/ : "These models have cl
by jefftk 1mo ago
>> August 2025 https://www.wheresyoured.at/how-to-argue-with-an-ai-booster/ https://www.wheresyoured.at/how-to-argue-with-an-ai-booster/ : "These models have clearly hit a wall where training is hitting diminishing returns"
>> Wrong
>It was my understanding—and I'm no expert, so if someone does know better please correct me!—that indeed by the second half of 2025 training, and also post-training reinforcement-learning stuff, both hit seriously diminishing returns, and the thing that is continuing to scale well or pretty well is inference.
I'm not an expert either, but while I do think for a bit it looked like ~all the improvement was inference-time scaling, it hasn't stayed that way. Mythos/Fable is likely a very large model (ex: it knows many things without searching) and this is probably part of its high level of capability, and the companies have started doing very large amounts of RL (which in OpenAI's case led to the HF attack).