4 ms·

The stock price had assumptions baked in about the number of units expected to be sold. DeepSeek cut that hardware estimate by as much as 45x. That is that abso

by gleenn 2y ago

The stock price had assumptions baked in about the number of units expected to be sold. DeepSeek cut that hardware estimate by as much as 45x. That is that absolute obvious correlation between that model being very efficient to train and NVDA dropping 18%.

stavros 2y ago
Can someone explain how DeepSeek cut that estimate? Their (fast) API is always down, and the third-party providers on OpenRouter are more expensive than Claude.
- juliuskiesian 2y ago
  They are currently under severe cyberattack from the US.
sampullman 2y ago
It still doesn't make sense to me. If the money for training is still there, wouldn't companies that can afford it use the efficiency gains and also scale up models? Unless AI is a bubble, and it pops, I can't see the demand for compute going down.
- MichaelZuo 2y ago
  They would scale things up slightly slower. And money 5 years from now is simply worth less to markets than money 7 years from now.
- EGreg 2y ago
  I think AI is a bubble. The amount of compute for inference is vastly overestimated, because a lot of caching is coming. It's driven by maniacal statements like Sam Altman's insistence that we must spend Trillions on compute, to achieve AGI, and it's more important than anything else. Project Stargate is some large fraction of that, and of course Softbank is no stranger to losing money on overestimating demand (for example, WeWork). To be fair, China has a lot of overestimation of demand too (for example Evergrande). The other is that rapid competition leads to overinvestment by all parties. https://www.wsj.com/tech/ai/sam-altman-seeks-trillions-of-dollars-to-reshape-business-of-chips-and-ai-89ab3db0 https://www.wsj.com/tech/ai/sam-altman-seeks-trillions-of-do...
  - disgruntledphd2 2y ago
    Which is great for us, we'll have loads of cheap compute and hopefully a bunch more carbon free energy supply, assuming that the AI stuff all ends in tears (for now).
    - EGreg 2y ago
      Yep! Shareholders and capitalists overinvesting in stuff is great if it leaves behind great infrastructure. They take the risk and the public benefits.
- gleenn 2y ago
  There is a belief that we've peaked in terms of bigger model = better results. I think it was GPT 4 is actually smaller in parameter count and better than 3.5 for instance. There is also a finite amount of useful data that people think we have hit, so adding more parameters isn't helpful if you don't have new data to train them on.
Aunche 2y ago
Eli Whitney thought he could reduce slavery by making cotton processing 45x more efficient...
aoanevdus 2y ago
I don’t get it. The labs have regularly made improvements that dramatically lower the cost of training an equal-performing model. When they do this, they also train a larger model with even higher performance. This time, DeepSeek did the first part but didn’t do the second. Now every lab in the world will throw their compute into the effort to replicate and beat DeepSeek’s model with larger scale. It’s not like everyone is just going to say “well I guess AI is smart enough now, no point improving it anymore!” and stop building bigger training clusters. If anything, r1 makes even more GPU demand likely, since it mitigated or at least delayed the risk AI hit a dead end (in which case, ceasing development may actually make sense).
- gleenn 2y ago
  Define dramatically with numbers. From all the sources I've read, it was so significant and also run on a far more limited cluster and the results are as good as the other frontier models. Optimizations have been coming, I think the one or more they found were significantly larger.

by gleenn 2y ago

stavros 2y ago

Can someone explain how DeepSeek cut that estimate? Their (fast) API is always down, and the third-party providers on OpenRouter are more expensive than Claude.

juliuskiesian 2y ago
They are currently under severe cyberattack from the US.

sampullman 2y ago

It still doesn't make sense to me. If the money for training is still there, wouldn't companies that can afford it use the efficiency gains and also scale up models? Unless AI is a bubble, and it pops, I can't see the demand for compute going down.

MichaelZuo 2y ago
They would scale things up slightly slower. And money 5 years from now is simply worth less to markets than money 7 years from now.
EGreg 2y ago
I think AI is a bubble. The amount of compute for inference is vastly overestimated, because a lot of caching is coming. It's driven by maniacal statements like Sam Altman's insistence that we must spend Trillions on compute, to achieve AGI, and it's more important than anything else. Project Stargate is some large fraction of that, and of course Softbank is no stranger to losing money on overestimating demand (for example, WeWork). To be fair, China has a lot of overestimation of demand too (for example Evergrande). The other is that rapid competition leads to overinvestment by all parties. https://www.wsj.com/tech/ai/sam-altman-seeks-trillions-of-dollars-to-reshape-business-of-chips-and-ai-89ab3db0 https://www.wsj.com/tech/ai/sam-altman-seeks-trillions-of-do...
- disgruntledphd2 2y ago
  Which is great for us, we'll have loads of cheap compute and hopefully a bunch more carbon free energy supply, assuming that the AI stuff all ends in tears (for now).
  - EGreg 2y ago
    Yep! Shareholders and capitalists overinvesting in stuff is great if it leaves behind great infrastructure. They take the risk and the public benefits.
gleenn 2y ago
There is a belief that we've peaked in terms of bigger model = better results. I think it was GPT 4 is actually smaller in parameter count and better than 3.5 for instance. There is also a finite amount of useful data that people think we have hit, so adding more parameters isn't helpful if you don't have new data to train them on.

Aunche 2y ago

Eli Whitney thought he could reduce slavery by making cotton processing 45x more efficient...

aoanevdus 2y ago

I don’t get it. The labs have regularly made improvements that dramatically lower the cost of training an equal-performing model. When they do this, they also train a larger model with even higher performance. This time, DeepSeek did the first part but didn’t do the second. Now every lab in the world will throw their compute into the effort to replicate and beat DeepSeek’s model with larger scale. It’s not like everyone is just going to say “well I guess AI is smart enough now, no point improving it anymore!” and stop building bigger training clusters. If anything, r1 makes even more GPU demand likely, since it mitigated or at least delayed the risk AI hit a dead end (in which case, ceasing development may actually make sense).

gleenn 2y ago
Define dramatically with numbers. From all the sources I've read, it was so significant and also run on a far more limited cluster and the results are as good as the other frontier models. Optimizations have been coming, I think the one or more they found were significantly larger.