2 ms·
Does this also apply to LLM training at scale? I would be a bit surprised if it does, fwiw.
by zozbot234 10mo ago
Does this also apply to LLM training at scale? I would be a bit surprised if it does, fwiw.
- _0ffh 10mo agoYup, as soon as data is the bottleneck and not compute, diffusion wins. Tested following the Chinchilla scaling strategy from 7M to 2.5B parameters. https://arxiv.org/abs/2507.15857 https://arxiv.org/abs/2507.15857