3 ms·
I think it's often been shown that constraints are what brings out true breakthroughs, so I'm skeptical of scaling up throw brute force - to me it seems like a
by saubeidl 1y ago
I think it's often been shown that constraints are what brings out true breakthroughs, so I'm skeptical of scaling up throw brute force - to me it seems like a lazy way out that will prevent you from challenging your base assumptions and figuring out better ways of doing things.
- Closi 1y agoI think there is a world where you have both - a portion of research sitting under constraints, and also massive computing capability to help build the next big model. Put another way - I doubt OpenAI would have built GPT-3 faster if they had less compute.