2 ms·

I'm assuming that LLMs will hit the plateau that all emerging technology hits. Just because cars used to travel at 25 MPH and their speed increased to 75 MPH d

• atleastoptimal 2y ago
  The point is that the plateau could be 6 months from now or decades from now, you don't know, and can't make a judgement solely on the idea that all tech hits plateaus. Moore's law held steady for decades. Nvidia's chip improvements, the main driver behind exponential gains in AI compute per dollar, are not close to slowing down. Even if the current generation of chips is the last one forever (almost 0% chance this is the case), all the SOTA models today were trained on chips from 1-2 years ago, there's still a 2-4x gain per dollar to be made for free. I'll believe LLM's plateau when they do plateau, but judging on the rate of development in algorithmic advances, chip capacity, multi-modal integration to broaden training data, investment, etc, it seems silly to just assume it will just plateau at a level no more significant that the level we are at now.