3 ms·
History? This isn't the first RAM shortage. When one happens, producers build more fabs. The fabs come online, the availability of memory shoots up, and the
by AnimalMuppet 4mo ago
History? This isn't the first RAM shortage. When one happens, producers build more fabs. The fabs come online, the availability of memory shoots up, and the shortage goes away, usually replaced by a glut.
If you want to argue that this is different from all previous RAM shortages, you can, but the burden of proof is on you to show the difference.
- nok22kon 4mo agothere is a glut if demand stops. this time demand doesn't stop. there is an exponential demand for tokens.
- swiftcoder 4mo ago> there is an exponential demand for tokens [citation needed] There is certainly economic pressure to create an exponential demand for tokens, but we've already seen a pullback from the costly "token maxing" companies were pushing last year. It's also pretty unclear to what degree the RAM shortage is driven by inference (versus by training). We're rapidly approaching the point where frontier models are "good enough" for everyday use, are at some point we're going to hit diminishing returns on training new trillion-parameter models...