3 ms·
> Scaling post-training is all we did for GLM-5.3. Love this opening line. And wow, great results. > As agent capability improves, much of the difficulty in s
by wxw 2mo ago
> Scaling post-training is all we did for GLM-5.3.
Love this opening line. And wow, great results.
> As agent capability improves, much of the difficulty in scaling post-training moves from the model to the environment.
- tjwebbnorfolk 2mo agodoes this suggest 5.3 is the same # of parameters as 5.2?
- fahrradflucht 2mo ago“Today we are releasing GLM-5.3. It uses the same base model as GLM-5.2 — every gain comes from post-training.“
- tjwebbnorfolk 2mo agoI was asking if this implies that 5.3 has the same number of parameters as 5.2. I can, in fact, read. What I didn't do is understand the implication of that statement. Thank you for your copy/paste service.
- fahrradflucht 2mo agoApologies then. The not-reading-the-article crowd was going strong that day and it looks like you caught a stray.
- unrvl22 2mo agowhich is the bigger headline that people don't realize. this is 744b and its head to head with Kimi K3 (2.8T), smashes DS v4 pro (1.5T). even Opus and Sol are rumored to be 1.5T+ this is half the size!
- Havoc 2mo agoYou do need to compare active parameter too though. The total size isn’t a reliable indicator anymore
- AureliusMA 2mo agoin terms of performance the formula seems to be : dense parameters = sqrt(total*active)
- davidlt 2mo agoIt's the same pre-training, they are just adding more (+ better) SFT, RL, etc. (post-training). Model internal knowledge cut-off is still the same. It seems we are doing pre-training every 6 months, and post-training every 4-8 weeks now.
- kleiba2 2mo agoWhat actually is "scaling post-training"?
- FergusArgyll 2mo agoMore RLVR. Give it verifiable problems, if it doesn't find a solution move on, if it does, use that as a reward signal.
- Gecko4072 2mo agoCan’t this be extended quite far? Use a cerebras-served model, use verification techniques to generate and solve millions of problems and then use that as training?
- gvkhna 2mo agoThat’s the whole point, just cost and compute limitations in your way (mostly).
- kevincox 2mo agoThis isn't latency bound, it is trivially parallelize. So you want to run it on the most efficient compute you have, not the fastest.
- andai 2mo agoOpenAI was bragging the other day about how HLE on Cerebras only took 11 hours. But you're saying if they just spun up a separate cheapo VM for each task, it would have finished faster and cheaper?
- dannyw 2mo agoModels are trained in highly distributed and parallel ways these days. It is entirely possible to run multiple concurrent post-training runs. When a frontier lap deploys a 1M RL gym rollout, these 1M environments are absolutely not talking to each other or interconnected. They individually generate traces and movements that can be then combined for post training.