3 ms·
Chinchilla suggests that most models now are undertrained. It's been the lore that model size was the bottleneck but we've since passed that and now training da
by naasking 4y ago
Chinchilla suggests that most models now are undertrained. It's been the lore that model size was the bottleneck but we've since passed that and now training data is the limiting factor.
- make3 4y agochinchilla only says that it would be more computationally efficient to have more data than to make the model larger, not that making the model larger wouldn't still benefit in performance gain