3 ms·
Firstly, LLMs are an embarrassingly parallel problem, so yes, you can actually get quite far simply by throwing more hardware at it. The catch is that the gain
by int_19h 4y ago
Firstly, LLMs are an embarrassingly parallel problem, so yes, you can actually get quite far simply by throwing more hardware at it. The catch is that the gain is not linear - e.g. you need 4x more VRAM for 2x inputs / context window size. But if doing that unlocks more useful emergent properties, it may well be a worthwhile trade-off - and it doesn't have to spontaneously become an AGI for that to happen. So I think we'll be playing this game for quite a long time.