4 ms·
Will all these data centers really be needed in 5 or 10 years from now? The technology, compression, etc, for models are getting better and better requiring le
by linuxhansl 23d ago
Will all these data centers really be needed in 5 or 10 years from now?
The technology, compression, etc, for models are getting better and better requiring
less and less H/W and power to run the same inference power.
Perhaps our hunger for ever more powerful models will be enough to offset this, but I am not sure.
- tommica 23d agoProbably yes, and if not, it can most likely be repurposed
- pizzaiolo 22d agoEnough to pay back the investments on a sensible timescale? Press X to doubt.
- deleted 23d ago[deleted]
- ra 23d agoWe will all be happy with our local inference, but how will our overloards be able to surveil and control us en-masse without them?
- iTokio 23d agoThe trillon dollars question is more, will the current GPUs still be relevant in 3 years? 5? 10? Will incredibly efficient new GPUs appear? will custom ASIC, models burned in silicons, take off? Will unified memory systems become usable? The datacenter bets are that things will mostly continue like now, and that these GPUs won’t depreciate too much. For now theses bets are self-fulfilling, causing hardware to even appreciate.
- flippingheck 23d agoIt's a big bet to not have a plan for needing power. I expect as LLMs penetrate further, we'll see much more of it, continually running, and doing more 3D work.
- helsinkiandrew 23d agoThe (naive or forward thinking depending on your opinion) is that cheaper 'inference' will result in more demand, like the more efficient Watt steam engine increasing consumption of coal. https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
- vasco 23d agoEnergy usage for computing by humanity will only go up with time. It should be obvious that most of our energy will go to computing for anyone who's worked with computers. At some point you exhaust the kinds of things you can do in the physical world, but you can compute infinitely. There's limits to where you can go but not to what you can think about.
- blitzar 22d agoI predict that within 100 years, data centers will be twice as powerful, ten thousand times larger, and so expensive that only the 5 richest kings of Europe will own them
- shmel 22d agoIt reminds me of "640KB of RAM is enough for everyone". Look where we are right now. We always find a way to spend resources. I was frustrated with my last laptop because of frequent swapping/OOM, so I built a desktop with 64GB RAM during covid. I thought I would never have this problem again, I even disabled swap. Now I regret I didn't get 128GB instead to fit bigger local LLMs. But even without LLMs, my Firefox crashed with OOM just yesterday. Apparently I open too many tabs too quickly. I used to think "wow, ADSL is so fast, I will never need faster internet connection" and now I need fiber to sync 500GB dataset fast enough so I don't get frustrated. Gosh, I remember the feeling when I upgraded from 14.4k to 56k dialup modem.