3 ms·

Here's an useful exercise. Suppose someone makes an advance that makes LLMs training require 90% less compute, but does not make them any smarter. (Recent resul

Here's an useful exercise. Suppose someone makes an advance that makes LLMs training require 90% less compute, but does not make them any smarter. (Recent results from China indicate substantial compute reductions are possible.) Demand for GPU time drops. What happens to the economics of the AI boom?

dilyevsky 1y ago
Demand most likely won't drop due to Jevons and the worst will be big labs and Nvidia probably drop significantly in valuation but still pretty good. Also recent china results only show that you could build cheaper model hitting the benchmarks not actually generally competitive irl.
- fluoridation 1y ago
  Agreed, with one caveat: demand for hardware can drop if the optimization is so substantial that deployed hardware is already capable of fulfilling any possible request in a timely fashion. Say, anyone becomes able to run any state-of-the-art model on any desktop computer. 90% is not enough, though. It has to be at least three or four orders of magnitude. The ones driving demand are big datacenters, so they need to be in a position where just throwing more hardware at the problem solves nothing.
thomasdeleeuw 1y ago
I remember something like that happening. Someone optimzed something and boom, one billion of valuation gone. It seems ridiculous on the surface, and definitely smells of a bubble to me.
- Animats 1y ago
  That's happening to the automobile market in China. Electric car and battery manufacturing costs go down with scale. Way down. Huge electric car glut in China. Car prices crashing. Government worried.[1] Profit margin of China car companies is down to around 4%. [1] https://www.cnn.com/2025/09/26/cars/chinese-electric-cars-price-wars-intl-hnk-dst https://www.cnn.com/2025/09/26/cars/chinese-electric-cars-pr...
lern_too_spel 1y ago
The vast majority of energy is already used at inference time. This gap is only going to widen as the number of applications increases and people ask their LLMs to do more complicated tasks. If training costs go down, I expect models will be trained on more video, so training energy usage will not decrease either.
mdhb 1y ago
I think the problem is bigger than that too. If you have a problem set where you cannot afford any kind of mistakes LLMs are never going to be the right tool, they are never going to get to that stage. If you have a problem set where some mistakes are tolerable however, all you need to do is slightly increase your tolerance budget for mistakes and you can run something like deepseek locally at a tiny fraction of the cost. Which means these giant data centers are useful for training but the economics of running ongoing inference for customers doesn’t make long term sense for the vast majority of people. Which leaves you with a huge huge huge financial problem, one which may have devastating large scale impacts on the economy.
kcexn 1y ago
This is true not just in AI but in tech in general. Data centers could be cheaper and more energy efficient. Phones could last longer, etc. Ultimately, companies don't like participating in markets that are races to the bottom (better outcomes at lower prices). Technologies that make LLM training cheaper will only see widespread adoption after its no longer feasible to keep expanding data centers to handle the data processing loads.
aoeusnth1 1y ago
If training is cheaper, then costs go down and models become both cheaper and more profitable. Training be scaled up to make the models smarter and larger. Distillation can bring those benefits to smaller models. Training is a pure cost - inference is wildly profitable if you ignore training. Reducing training costs improves the economics of LLMs significantly.

dilyevsky 1y ago

Demand most likely won't drop due to Jevons and the worst will be big labs and Nvidia probably drop significantly in valuation but still pretty good. Also recent china results only show that you could build cheaper model hitting the benchmarks not actually generally competitive irl.

fluoridation 1y ago
Agreed, with one caveat: demand for hardware can drop if the optimization is so substantial that deployed hardware is already capable of fulfilling any possible request in a timely fashion. Say, anyone becomes able to run any state-of-the-art model on any desktop computer. 90% is not enough, though. It has to be at least three or four orders of magnitude. The ones driving demand are big datacenters, so they need to be in a position where just throwing more hardware at the problem solves nothing.

thomasdeleeuw 1y ago

I remember something like that happening. Someone optimzed something and boom, one billion of valuation gone. It seems ridiculous on the surface, and definitely smells of a bubble to me.

Animats 1y ago
That's happening to the automobile market in China. Electric car and battery manufacturing costs go down with scale. Way down. Huge electric car glut in China. Car prices crashing. Government worried.[1] Profit margin of China car companies is down to around 4%. [1] https://www.cnn.com/2025/09/26/cars/chinese-electric-cars-price-wars-intl-hnk-dst https://www.cnn.com/2025/09/26/cars/chinese-electric-cars-pr...

lern_too_spel 1y ago

The vast majority of energy is already used at inference time. This gap is only going to widen as the number of applications increases and people ask their LLMs to do more complicated tasks. If training costs go down, I expect models will be trained on more video, so training energy usage will not decrease either.

mdhb 1y ago

I think the problem is bigger than that too. If you have a problem set where you cannot afford any kind of mistakes LLMs are never going to be the right tool, they are never going to get to that stage. If you have a problem set where some mistakes are tolerable however, all you need to do is slightly increase your tolerance budget for mistakes and you can run something like deepseek locally at a tiny fraction of the cost. Which means these giant data centers are useful for training but the economics of running ongoing inference for customers doesn’t make long term sense for the vast majority of people. Which leaves you with a huge huge huge financial problem, one which may have devastating large scale impacts on the economy.

kcexn 1y ago

This is true not just in AI but in tech in general. Data centers could be cheaper and more energy efficient. Phones could last longer, etc. Ultimately, companies don't like participating in markets that are races to the bottom (better outcomes at lower prices). Technologies that make LLM training cheaper will only see widespread adoption after its no longer feasible to keep expanding data centers to handle the data processing loads.

aoeusnth1 1y ago

If training is cheaper, then costs go down and models become both cheaper and more profitable. Training be scaled up to make the models smarter and larger. Distillation can bring those benefits to smaller models. Training is a pure cost - inference is wildly profitable if you ignore training. Reducing training costs improves the economics of LLMs significantly.