6 ms·

Hype buyers are also Hype sellers - anything Nvidia was last week is exactly what it is this week - DeepSeek doesn't really have any impact on Nvidia sales - So

Hype buyers are also Hype sellers - anything Nvidia was last week is exactly what it is this week - DeepSeek doesn't really have any impact on Nvidia sales - Some argument could be made that this can shift compute off of cloud and onto end user devices, but that really seems like a stretch given what I've seen running this locally.

zozbot234 2y ago
The full DeepSeek model is ~700B params or so - way too large for most end users to run locally. What some folks are running locally is fine-tuned versions of Llama and Qwen, that are not going to be directly comparable in any way.
- m00x 2y ago
  Many people are missing this due to journalists completely missing the point when presenting these facts. Distilled models are nothing new.
  - mtkd 2y ago
    Ollama calling the distilled models deepseek-r1:7b etc. doesn't help
Analemma_ 2y ago
I agree hype is a big portion of it, but if DeepSeek really has found a way to train models just as good as frontier ones for a hundredth of the hardware investment, that is a substantial material difference for Nvidia's future earnings.
- zozbot234 2y ago
  > if DeepSeek really has found a way to train models just as good as frontier ones for a hundredth of the hardware investment Frontier models are heavily compute constrained - the leading AI model makers have got way more training data already than they could do anything with. Any improvement in training compute-efficiency is great news for them, no matter where it comes from. Especially since the DeepSeek folks have gone into great detail wrt. documenting their approach.
  - rpcope1 2y ago
    > leading AI model makers have got way more training data already than they could do anything with. Citation needed.
    - nullc 2y ago
      If you include multimodal data then I think it's pretty obvious that training is compute limited. Also current SOTA models are good enough that you can generate endless training data by letting the model operate stuff like a C compiler, python interpreter, Sage computer algebra, etc.
- echelon 2y ago
  1. Nobody has replicated their DeepSeek's results on their reported budget yet. Scale.ai's Alexander Wang says they're lying and that they have a huge, clandestine H100 cluster. HuggingFace is assembling an effort to publicly duplicate the paper's claims. 2. Even if DeepSeek's budget claims are true, they trained their model on the outputs of an expensive foundation model built from a massive capital outlay. To truly replicate these results from scratch, it might require an expensive model upstream.
  - tgtweak 2y ago
    https://xyzlabs.substack.com/p/berkeley-researchers-replicate-deepseek?utm_campaign=post&utm_medium=web https://xyzlabs.substack.com/p/berkeley-researchers-replicat... Given they've reproduced earlier model's and vetted it - I think it's probably safe to assume that these new models are not out of thin air - but until somebody reproduces it, it's up in the air.
- throwup238 2y ago
  Is it? Training is only done once, inference requires GPUs to scale, especially for a 685B model. And now, there’s an open source o1 equivalent model that companies can run locally, which means that there’s a much bigger market for underutilized on-prem GPUs.
  - tgtweak 2y ago
    I'd be really curious about the split in hardware for training vs inference - I got the read that it was a very high ratio to the point the training is not a significant portion of the requisite hardware but instead the inference at scale sucks up most of the available datacenter gpu share. Could be entirely wrong here - would love a fact-check by industry insider or journalist.
- anon291 2y ago
  Not really. The training methodology opens up whole new mechanisms that'll make it much easier to train non-language models, which have been very much neglected. Think robot multi-modal models; visual / video question answering; audio processing, etc.
- nullc 2y ago
  I don't see how that follows. Making training more effective makes every unit of compute spent on training more valuable. This should increase demand unless we've reached a point where better models are not valuable. The openness of DeepSeek's approach also means that there will be more smaller entities engaging in training rather than a few massive entities that have more ability to set the price they pay. Plus reasoning models substantially increase inference costs, since for each token of output you may have hundreds of tokens of reasoning. Arguments on the point can go both ways, but I think on the balance I would expect any improvements in efficiency increase demand.
  - diamond559 2y ago
    Unless we get actual AGI I don't honestly care as a non coder. The art is slop and predatory, the chatbots are stilted and pointless, anytime a company uses AI there is huge backlash and there are just no commercial products with any real demand. Make it as cheap as dirt and I still don't see what use it is besides for scammers I guess...
- DennisP 2y ago
  Or Nvidia keeps its earnings and our best frontier models get a hundred times better.

zozbot234 2y ago

The full DeepSeek model is ~700B params or so - way too large for most end users to run locally. What some folks are running locally is fine-tuned versions of Llama and Qwen, that are not going to be directly comparable in any way.

m00x 2y ago
Many people are missing this due to journalists completely missing the point when presenting these facts. Distilled models are nothing new.
- mtkd 2y ago
  Ollama calling the distilled models deepseek-r1:7b etc. doesn't help

Analemma_ 2y ago

I agree hype is a big portion of it, but if DeepSeek really has found a way to train models just as good as frontier ones for a hundredth of the hardware investment, that is a substantial material difference for Nvidia's future earnings.

zozbot234 2y ago
> if DeepSeek really has found a way to train models just as good as frontier ones for a hundredth of the hardware investment Frontier models are heavily compute constrained - the leading AI model makers have got way more training data already than they could do anything with. Any improvement in training compute-efficiency is great news for them, no matter where it comes from. Especially since the DeepSeek folks have gone into great detail wrt. documenting their approach.
- rpcope1 2y ago
  > leading AI model makers have got way more training data already than they could do anything with. Citation needed.
  - nullc 2y ago
    If you include multimodal data then I think it's pretty obvious that training is compute limited. Also current SOTA models are good enough that you can generate endless training data by letting the model operate stuff like a C compiler, python interpreter, Sage computer algebra, etc.
echelon 2y ago
1. Nobody has replicated their DeepSeek's results on their reported budget yet. Scale.ai's Alexander Wang says they're lying and that they have a huge, clandestine H100 cluster. HuggingFace is assembling an effort to publicly duplicate the paper's claims. 2. Even if DeepSeek's budget claims are true, they trained their model on the outputs of an expensive foundation model built from a massive capital outlay. To truly replicate these results from scratch, it might require an expensive model upstream.
- tgtweak 2y ago
  https://xyzlabs.substack.com/p/berkeley-researchers-replicate-deepseek?utm_campaign=post&utm_medium=web https://xyzlabs.substack.com/p/berkeley-researchers-replicat... Given they've reproduced earlier model's and vetted it - I think it's probably safe to assume that these new models are not out of thin air - but until somebody reproduces it, it's up in the air.
throwup238 2y ago
Is it? Training is only done once, inference requires GPUs to scale, especially for a 685B model. And now, there’s an open source o1 equivalent model that companies can run locally, which means that there’s a much bigger market for underutilized on-prem GPUs.
- tgtweak 2y ago
  I'd be really curious about the split in hardware for training vs inference - I got the read that it was a very high ratio to the point the training is not a significant portion of the requisite hardware but instead the inference at scale sucks up most of the available datacenter gpu share. Could be entirely wrong here - would love a fact-check by industry insider or journalist.
anon291 2y ago
Not really. The training methodology opens up whole new mechanisms that'll make it much easier to train non-language models, which have been very much neglected. Think robot multi-modal models; visual / video question answering; audio processing, etc.
nullc 2y ago
I don't see how that follows. Making training more effective makes every unit of compute spent on training more valuable. This should increase demand unless we've reached a point where better models are not valuable. The openness of DeepSeek's approach also means that there will be more smaller entities engaging in training rather than a few massive entities that have more ability to set the price they pay. Plus reasoning models substantially increase inference costs, since for each token of output you may have hundreds of tokens of reasoning. Arguments on the point can go both ways, but I think on the balance I would expect any improvements in efficiency increase demand.
- diamond559 2y ago
  Unless we get actual AGI I don't honestly care as a non coder. The art is slop and predatory, the chatbots are stilted and pointless, anytime a company uses AI there is huge backlash and there are just no commercial products with any real demand. Make it as cheap as dirt and I still don't see what use it is besides for scammers I guess...
DennisP 2y ago
Or Nvidia keeps its earnings and our best frontier models get a hundred times better.