3 ms·

Congratulations! You have just realized that the AI data center build out is a total scam, built on both the insurmountable trillions of debt, and the assumptio

by rvz 2mo ago

Congratulations! You have just realized that the AI data center build out is a total scam, built on both the insurmountable trillions of debt, and the assumption that only GPUs are all we need to continue scaling. There exist other AI accelerators (TPUs, ASICs) that perfectly exceed the throughput that LLMs need to scale as well. But the true solution is more software optimizations. There's a tiny handful of them but more needs to be discovered so that we can reduce building hundreds of more data centers as the alternatives mature. As better software becomes more useful for the alternative AI hardware for developers with LLMs running efficiently you then would have more choices of hardware to run your LLMs on rather than just only GPUs.

blovescoffee 2mo ago
TPUs and ASICs run in data centers too. Your argument only holds true if there's some satisfied limit to demand for inference. If not, data centers will continue to spring up to host more and more agents. Even if agents were running on hardware and software as efficient as the human brain, its conceivable we want trillions of them running at any given time which would require data center scale.
- georgeecollins 2mo ago
  Everything has some satisfied limit to demand, often depending on the price. If you assume there will never be any satisfied limit to demand for inference at any price you can justify any investment.
  - aldonius 2mo ago
    Yeah, but there's certainly a part of the curve where price drops by X OOMs and demand increases by much more than X OOMs. (Presumably some of that is substitution and some of that is new use cases.)
  - adventured 2mo ago
    Looking back nearly 80 years, what has been the limit to transistor demand so far? Unlimited. What has been the limit to electricity demand globally? Unlimited. We can't get enough and never will. Costs have to become pretty severe to turn back the demand as well.
  - x-complexity 2mo ago
    So far, at least by Openrouter's weekly numbers, there doesn't seem to be a satisfied limit. https://openrouter.ai/rankings#top-models https://openrouter.ai/rankings#top-models And their market share sits at around 16-20%. At 75.3 trillion tokens for the week ending 10 Aug 2026, that means that up to 450 trillion tokens were plausibly demanded by the whole market for that week. My take: At max saturation, each person on earth could have their demands satiated by an average of 16 agents running concurrently. Sometimes more, often times less, but the average would likely be at 16. At 200 tokens/second for each agent, that would mean 15.48288 quintillion tokens per week. We're currently at about 0.00290643601% of the calculated demand ceiling. Even if the demand limit per person is just 1 agent at 50 tokens/second, the current demand's still 0.186011905% of the theoretical ceiling.
skyberrys 2mo ago
I wonder what this looks like in 5 years... Will there be a massive push to repurpose these giant boxes into housing? Will they get turned back into the farm land from where they came? When a data center goes bust, what happens to the parts left behind?
- gpm 2mo ago
  I'd think the infrastructure would tend towards factories, smelters, and so on. Industrial things that have reasonably high power demands, can use the building, and don't care about the lack of windows. They're typically not built where you want housing, and the buildings are distinctly the wrong shape. If you can't use the power infrastructure profitably my next thought would be warehousing. But also... we've seen a pretty continually increasing demand for compute. Even if AI busts a bit (or becomes a bit more efficient) I bet most data centres stay data centres, just less profitable ones.
jryle70 2mo ago
Huh, why I'm not surprised that HN is full of opinions confidently stated without any numbers or resources to back up? > built on both the insurmountable trillions of debt, and the assumption that only GPUs are all we need to continue scaling. Insurmountable according to whom? And who assume that only GPUs are all we need to continue scaling? Google, Amazon, Microsoft, Meta and OpenAI, all have or plan custom non-GPU AI chips. Do they plan to use them not for scaling?
necovek 2mo ago
Is this article not exactly about ASICs built for LLM inference and training acceleration?

by rvz 2mo ago

blovescoffee 2mo ago

TPUs and ASICs run in data centers too. Your argument only holds true if there's some satisfied limit to demand for inference. If not, data centers will continue to spring up to host more and more agents. Even if agents were running on hardware and software as efficient as the human brain, its conceivable we want trillions of them running at any given time which would require data center scale.

georgeecollins 2mo ago
Everything has some satisfied limit to demand, often depending on the price. If you assume there will never be any satisfied limit to demand for inference at any price you can justify any investment.
- aldonius 2mo ago
  Yeah, but there's certainly a part of the curve where price drops by X OOMs and demand increases by much more than X OOMs. (Presumably some of that is substitution and some of that is new use cases.)
- adventured 2mo ago
  Looking back nearly 80 years, what has been the limit to transistor demand so far? Unlimited. What has been the limit to electricity demand globally? Unlimited. We can't get enough and never will. Costs have to become pretty severe to turn back the demand as well.
- x-complexity 2mo ago
  So far, at least by Openrouter's weekly numbers, there doesn't seem to be a satisfied limit. https://openrouter.ai/rankings#top-models https://openrouter.ai/rankings#top-models And their market share sits at around 16-20%. At 75.3 trillion tokens for the week ending 10 Aug 2026, that means that up to 450 trillion tokens were plausibly demanded by the whole market for that week. My take: At max saturation, each person on earth could have their demands satiated by an average of 16 agents running concurrently. Sometimes more, often times less, but the average would likely be at 16. At 200 tokens/second for each agent, that would mean 15.48288 quintillion tokens per week. We're currently at about 0.00290643601% of the calculated demand ceiling. Even if the demand limit per person is just 1 agent at 50 tokens/second, the current demand's still 0.186011905% of the theoretical ceiling.

skyberrys 2mo ago

I wonder what this looks like in 5 years... Will there be a massive push to repurpose these giant boxes into housing? Will they get turned back into the farm land from where they came? When a data center goes bust, what happens to the parts left behind?

gpm 2mo ago
I'd think the infrastructure would tend towards factories, smelters, and so on. Industrial things that have reasonably high power demands, can use the building, and don't care about the lack of windows. They're typically not built where you want housing, and the buildings are distinctly the wrong shape. If you can't use the power infrastructure profitably my next thought would be warehousing. But also... we've seen a pretty continually increasing demand for compute. Even if AI busts a bit (or becomes a bit more efficient) I bet most data centres stay data centres, just less profitable ones.

jryle70 2mo ago

Huh, why I'm not surprised that HN is full of opinions confidently stated without any numbers or resources to back up? > built on both the insurmountable trillions of debt, and the assumption that only GPUs are all we need to continue scaling. Insurmountable according to whom? And who assume that only GPUs are all we need to continue scaling? Google, Amazon, Microsoft, Meta and OpenAI, all have or plan custom non-GPU AI chips. Do they plan to use them not for scaling?

necovek 2mo ago

Is this article not exactly about ASICs built for LLM inference and training acceleration?