6 ms·

Nvidia -13% in Frankfurt stock market just now. Valuations of private unicorns like OpenAi and Anthropic must be in free fall. DeepSeek spends $6 million in o

by nabla9 2y ago

Nvidia -13% in Frankfurt stock market just now. Valuations of private unicorns like OpenAi and Anthropic must be in free fall. DeepSeek spends $6 million in old H800 hardware to develop open source model that overtakes ChatGPT. AI gets better, but profit margins sink with strong competition. Chinese AI startup DeepSeek overtakes ChatGPT on Apple App Store https://news.ycombinator.com/item?id=42839656 https://news.ycombinator.com/item?id=42839656 Edit: Nvidia now -15% in Frankfurt.

gostsamo 2y ago
Consider that the chinese might be misrepresenting their costs. A newsletter was implying that they might do it to undermine the sanctions justifications. Agree that the AI bubble should pop though and the earlier, the better.
- nabla9 2y ago
  Their model is open and they published paper describing it https://arxiv.org/pdf/2412.19437 https://arxiv.org/pdf/2412.19437 The can't be far off, or it would be noticed. Even if they are heavily government subsidized for energy and hardware, I don see how the cost of training in the US would be more than double.
  - imdsm 2y ago
    Let's wait for reproduction first
    - jredwards 2y ago
      Posted elsewhere: https://xyzlabs.substack.com/p/berkeley-researchers-replicate-deepseek https://xyzlabs.substack.com/p/berkeley-researchers-replicat...
  - KeplerBoy 2y ago
    People on the internet can lie. Especially when such a lie could cause the nasdaq to dip multiple percent points. Not saying they are lying, but there incentives.
    - BoredPositron 2y ago
      Huggingface is reproducing it live on their blog…
    - pizza 2y ago
      Much easier to identify the incentives of the people who just lost a lot of money who were betting on the idea that it was their money that was going to make artificial intelligence intelligent. Everyone’s already begun trying this recipe in-house. Either it works with much less compute, or it doesn’t. For instance, HKUST just did an experiment where small weak base models trained with DeepSeek’s method beat stronger small base models being trained with much more costly RL methods. Already this seems like it is enough to upend the low end models niche market, things like haiku and 4o-mini. Be really skeptical why the people who should be making tons of money by realizing actually it was all a mirage and that they can now get the real stuff for even cheaper, would spend so much effort shouting about this, in order to undercut their own profitability..
  - logicchains 2y ago
    They express their cost in terms of GPU hours, then convert that to USD based on market GPU rental rates, so it's not affected by subsidies. It's possible however they lied about GPU hours, but if that was the case an expert should be able to show they lied by working out how many flops are needed to train based on the amount of tokens they say they used vs the flops of the GPUs they say they used.
    - rfoo 2y ago
      Total training FLOPs can be deduced from model architecture (which they can't hide since they released weights) and how many tokens they trained on. With total training FLOPs and GPU hours you can calculate MFU. And the MFU of their deepseek-v3 train is around 40%, which sounds right. Both Google and Meta reported higher MFU. So the GPU hours should be correct. The only thing they could have lied is on how many tokens they trained the model on. DeepSeek reported 14T which is also similar to what Meta did so nothing crazy here. tl;dr all numbers check up and the winnings come from the model architecture innovations they made.
  - bythreads 2y ago
    Doesnt that say they based it on llama?, sooooo not really a bottoms up training - since the cost of llama is 100% surely not part of their quote.
    - diggan 2y ago
      I did a quick search for "llama" and didn't find anywhere they outright state they just fine-tuned some llama weights. Is it possible that they based their model architecture on the llama model architecture? Rather than just fine-tuned already training llama weights? In that case, they'd still have to do "bottoms up" training.
esperent 2y ago
> DeepSeek spends $6 million in old H800 hardware to develop open source model that overtakes ChatGPT. DeepSeek claims that's what they spent. They're under a trade embargo, and if they had access to any more than that it would have been obtained illegally. They might be telling the truth, but let's wait until someone else replicates it before we fully accept it.
- belter 2y ago
  They have a lot of H100: https://www.reddit.com/r/NVDA_Stock/comments/1iadc0s/evidence_that_h100_nvidia_gpus_are_in_china_dated/ https://www.reddit.com/r/NVDA_Stock/comments/1iadc0s/evidenc...
  - msoad 2y ago
    four GPUs are very convincing indeed! :D
    - belter 2y ago
      You missed the black unwrapped boxes in the background....
    - bayindirh 2y ago
      That's 8 (not 4), on a NVIDIA platform board to start with. You can't buy them as "GPU"s and integrate them to your system. NVIDIA sells your the platform (GPUs + platform board which includes switches and all the support infra), and you integrate that behemoth of a board to your server, as a single unit. So that open server and the wrapped ones at the back are more telling than it looks.
- senko 2y ago
  There are already some (limited) reproductions that suggest they're not completely lying (ie that there are indeed perf benefits).
- sekai 2y ago
  [flagged]
nolok 2y ago
They were massively overhyped though, it feels more like a correction (and a partial one at that) than a fall.
- benterix 2y ago
  Of course they are overhyped but in spite of this Altman is always asking for more money. And we know financially they are just burning money. So when someone finally brings a cheap but good model for the masses, this is where money should go. (This will also help all small AI startups.)
negamax 2y ago
The bigger correction will be in tech stocks that are overly exposed to datacenter investments to accommodate for ever rising AI demands. MSFT, AMZN, META they are all exposed
- myth_drannon 2y ago
  MSFT is down 7%
  - voidfunc 2y ago
    It's kind of silly. It's not like MSFT and the other hyper-scalers dont need the capacity build out for other reasons too. This should be an easy pivot if DeepSeek turns out to be as good as promised.
SebFender 2y ago
Taking out the popcorn early today.
scrlk 2y ago
Not a good day for those who decided to hold 3x Nvidia ETPs - down 40% earlier.
chad1n 2y ago
This is not exactly right, they said they spent $6M on training V3, there aren't numbers out there related to the training of R1, I can feel it will be cheaper than o1, but it's hard to tell how much cheaper. I can guess that overall deepseek spent way less than openai to release the model, because I have the feeling that the R&D part was cheaper too, but we don't have the numbers yet. Anyway, we can assume that deepseek and Alibaba will try to get the most out of their current GPUs however.
thelastgallon 2y ago
[flagged]
- GuB-42 2y ago
  The issue here is not that DeepSeek exists as a competitor to GPT, Claude, Gemini,... The issue is that DeepSeek have shown that you don't need that much raw computing power to run an AI, which means that companies including OpenAI may focus more on efficiency than on throwing more GPUs at the problem, which is not good news for those in the business of making GPUs. At least according to the market.
  - paganel 2y ago
    One of the questions about this is that of the US’s human capital, i.e. does the US (still) have enough capable tech people in order to make that happen?
    - mike_hearn 2y ago
      Lol, yes. The US is still very much at the forefront of this stuff. DeepSeek have presented some neat optimizations, but there have been many such papers and optimizations get implemented quickly once someone has proven them out.
      - paganel 2y ago
        > The US is still very much at the forefront of this stuff Doesn't look like it, because the some of the biggest US tech companies now active (including Meta and Alphabet) couldn't come up with what this much-smaller Chinese company has. Which begs the question, what is that companies like Meta, Alphabet and the like do with the (already) hundreds of billions of dollars that they invested in this space?
        vmbm 2y ago
        Best guess is that they were all caught up in the arms race to try and make a better model, at whatever the cost. And if you work in this space you were probably getting thrown fistfuls of money to join in on it. I read somewhere on reddit that anyone trying to push for efficiency at these places was getting ignored or pushed aside. DeepSeek had an incentive to focus on efficiency because of the chip embargo. So I don't think this is necessarily a knock on US AI capabilities. It is just that the incentives were different and when stock prices are going to the moon regardless of how much capex was getting spent, it was easy for everyone to just go along with it. With that said, I think all of these companies are capable of learning from this and implementing these efficiency improvement. And I think the arms race is still on. The goal is to achieve super human level of intelligence, and they have a ways to go to get there. It is possible that these new efficiency improvements might even help them take the next step as they can now do a lot more with a lot less.
kvgr 2y ago
Not sure if its as hot as it looks. Valuation of those private AIs is stupid. This looks like Chinese propaganda. Say we dont need that hardware, but it looks like they need it. https://wccftech.com/chinese-ai-lab-deepseek-has-50000-nvidia-h100-ai-gpus-says-ai-ceo/ https://wccftech.com/chinese-ai-lab-deepseek-has-50000-nvidi...
impossiblefork 2y ago
It's a very strange result. I believe that NVIDIA is overvalued, but if DeepSeek really is as great as has been said, then it'll be even greater when scaled up to OpenAI sizes, and when you get more out you have more reason to pay, so this should if it pans out lead to more demand for GPUs-- basically Jevon's paradox.
- franktankbank 2y ago
  The architecture doesn't keep yielding better results, Jevon's paradox doesn't apply.
  - impossiblefork 2y ago
    But surely it can be scaled up, or is this compression thing something making the approach good only for small models (I haven't read the Deepseek papers (can't allocate time to it))?
  - joak 2y ago
    Anyhow, if you can deliver more with less, this is huge good news for AI industry. After some readjustment we can expect AI companies to start using the new method to deliver more. Science fiction might happen sooner than expected. Buy the dip.
- lmm 2y ago
  If the top-tier premium GPUs aren't the difference-maker they were thought to be then that will hurt NVIDIA's margins, even if they make some of it up on volume.
- alecco 2y ago
  The limit is high quality data, not compute.
  - belter 2y ago
    > The limit is high quality data If, as some companies claim, these models truly possess emergent reasoning, their ability to handle imperfect data should serve as a proof of that capability.
qwytw 2y ago
> overtakes ChatGPT That's arguable, though. I mean it's much cheaper and reasonably competitive which is almost the same but IMHO DeepSeek seems to get stuck in random loops and hallucinates more frequently than o1.
mgh2 2y ago
Let's see how the open vs close ecosystem wins in AI

by nabla9 2y ago

gostsamo 2y ago

Consider that the chinese might be misrepresenting their costs. A newsletter was implying that they might do it to undermine the sanctions justifications. Agree that the AI bubble should pop though and the earlier, the better.

nabla9 2y ago
Their model is open and they published paper describing it https://arxiv.org/pdf/2412.19437 https://arxiv.org/pdf/2412.19437 The can't be far off, or it would be noticed. Even if they are heavily government subsidized for energy and hardware, I don see how the cost of training in the US would be more than double.
- imdsm 2y ago
  Let's wait for reproduction first
  - jredwards 2y ago
    Posted elsewhere: https://xyzlabs.substack.com/p/berkeley-researchers-replicate-deepseek https://xyzlabs.substack.com/p/berkeley-researchers-replicat...
- KeplerBoy 2y ago
  People on the internet can lie. Especially when such a lie could cause the nasdaq to dip multiple percent points. Not saying they are lying, but there incentives.
  - BoredPositron 2y ago
    Huggingface is reproducing it live on their blog…
  - pizza 2y ago
    Much easier to identify the incentives of the people who just lost a lot of money who were betting on the idea that it was their money that was going to make artificial intelligence intelligent. Everyone’s already begun trying this recipe in-house. Either it works with much less compute, or it doesn’t. For instance, HKUST just did an experiment where small weak base models trained with DeepSeek’s method beat stronger small base models being trained with much more costly RL methods. Already this seems like it is enough to upend the low end models niche market, things like haiku and 4o-mini. Be really skeptical why the people who should be making tons of money by realizing actually it was all a mirage and that they can now get the real stuff for even cheaper, would spend so much effort shouting about this, in order to undercut their own profitability..
- logicchains 2y ago
  They express their cost in terms of GPU hours, then convert that to USD based on market GPU rental rates, so it's not affected by subsidies. It's possible however they lied about GPU hours, but if that was the case an expert should be able to show they lied by working out how many flops are needed to train based on the amount of tokens they say they used vs the flops of the GPUs they say they used.
  - rfoo 2y ago
    Total training FLOPs can be deduced from model architecture (which they can't hide since they released weights) and how many tokens they trained on. With total training FLOPs and GPU hours you can calculate MFU. And the MFU of their deepseek-v3 train is around 40%, which sounds right. Both Google and Meta reported higher MFU. So the GPU hours should be correct. The only thing they could have lied is on how many tokens they trained the model on. DeepSeek reported 14T which is also similar to what Meta did so nothing crazy here. tl;dr all numbers check up and the winnings come from the model architecture innovations they made.
- bythreads 2y ago
  Doesnt that say they based it on llama?, sooooo not really a bottoms up training - since the cost of llama is 100% surely not part of their quote.
  - diggan 2y ago
    I did a quick search for "llama" and didn't find anywhere they outright state they just fine-tuned some llama weights. Is it possible that they based their model architecture on the llama model architecture? Rather than just fine-tuned already training llama weights? In that case, they'd still have to do "bottoms up" training.

esperent 2y ago

> DeepSeek spends $6 million in old H800 hardware to develop open source model that overtakes ChatGPT. DeepSeek claims that's what they spent. They're under a trade embargo, and if they had access to any more than that it would have been obtained illegally. They might be telling the truth, but let's wait until someone else replicates it before we fully accept it.

belter 2y ago
They have a lot of H100: https://www.reddit.com/r/NVDA_Stock/comments/1iadc0s/evidence_that_h100_nvidia_gpus_are_in_china_dated/ https://www.reddit.com/r/NVDA_Stock/comments/1iadc0s/evidenc...
- msoad 2y ago
  four GPUs are very convincing indeed! :D
  - belter 2y ago
    You missed the black unwrapped boxes in the background....
  - bayindirh 2y ago
    That's 8 (not 4), on a NVIDIA platform board to start with. You can't buy them as "GPU"s and integrate them to your system. NVIDIA sells your the platform (GPUs + platform board which includes switches and all the support infra), and you integrate that behemoth of a board to your server, as a single unit. So that open server and the wrapped ones at the back are more telling than it looks.
senko 2y ago
There are already some (limited) reproductions that suggest they're not completely lying (ie that there are indeed perf benefits).
sekai 2y ago
[flagged]

nolok 2y ago

They were massively overhyped though, it feels more like a correction (and a partial one at that) than a fall.

benterix 2y ago
Of course they are overhyped but in spite of this Altman is always asking for more money. And we know financially they are just burning money. So when someone finally brings a cheap but good model for the masses, this is where money should go. (This will also help all small AI startups.)

negamax 2y ago

The bigger correction will be in tech stocks that are overly exposed to datacenter investments to accommodate for ever rising AI demands. MSFT, AMZN, META they are all exposed

myth_drannon 2y ago
MSFT is down 7%
- voidfunc 2y ago
  It's kind of silly. It's not like MSFT and the other hyper-scalers dont need the capacity build out for other reasons too. This should be an easy pivot if DeepSeek turns out to be as good as promised.

SebFender 2y ago

Taking out the popcorn early today.

scrlk 2y ago

Not a good day for those who decided to hold 3x Nvidia ETPs - down 40% earlier.

chad1n 2y ago

This is not exactly right, they said they spent $6M on training V3, there aren't numbers out there related to the training of R1, I can feel it will be cheaper than o1, but it's hard to tell how much cheaper. I can guess that overall deepseek spent way less than openai to release the model, because I have the feeling that the R&D part was cheaper too, but we don't have the numbers yet. Anyway, we can assume that deepseek and Alibaba will try to get the most out of their current GPUs however.

thelastgallon 2y ago

[flagged]

GuB-42 2y ago
The issue here is not that DeepSeek exists as a competitor to GPT, Claude, Gemini,... The issue is that DeepSeek have shown that you don't need that much raw computing power to run an AI, which means that companies including OpenAI may focus more on efficiency than on throwing more GPUs at the problem, which is not good news for those in the business of making GPUs. At least according to the market.
- paganel 2y ago
  One of the questions about this is that of the US’s human capital, i.e. does the US (still) have enough capable tech people in order to make that happen?
  - mike_hearn 2y ago
    Lol, yes. The US is still very much at the forefront of this stuff. DeepSeek have presented some neat optimizations, but there have been many such papers and optimizations get implemented quickly once someone has proven them out.
    - paganel 2y ago
      > The US is still very much at the forefront of this stuff Doesn't look like it, because the some of the biggest US tech companies now active (including Meta and Alphabet) couldn't come up with what this much-smaller Chinese company has. Which begs the question, what is that companies like Meta, Alphabet and the like do with the (already) hundreds of billions of dollars that they invested in this space?
      - vmbm 2y ago
        Best guess is that they were all caught up in the arms race to try and make a better model, at whatever the cost. And if you work in this space you were probably getting thrown fistfuls of money to join in on it. I read somewhere on reddit that anyone trying to push for efficiency at these places was getting ignored or pushed aside. DeepSeek had an incentive to focus on efficiency because of the chip embargo. So I don't think this is necessarily a knock on US AI capabilities. It is just that the incentives were different and when stock prices are going to the moon regardless of how much capex was getting spent, it was easy for everyone to just go along with it. With that said, I think all of these companies are capable of learning from this and implementing these efficiency improvement. And I think the arms race is still on. The goal is to achieve super human level of intelligence, and they have a ways to go to get there. It is possible that these new efficiency improvements might even help them take the next step as they can now do a lot more with a lot less.

kvgr 2y ago

Not sure if its as hot as it looks. Valuation of those private AIs is stupid. This looks like Chinese propaganda. Say we dont need that hardware, but it looks like they need it. https://wccftech.com/chinese-ai-lab-deepseek-has-50000-nvidia-h100-ai-gpus-says-ai-ceo/ https://wccftech.com/chinese-ai-lab-deepseek-has-50000-nvidi...

impossiblefork 2y ago

It's a very strange result. I believe that NVIDIA is overvalued, but if DeepSeek really is as great as has been said, then it'll be even greater when scaled up to OpenAI sizes, and when you get more out you have more reason to pay, so this should if it pans out lead to more demand for GPUs-- basically Jevon's paradox.

franktankbank 2y ago
The architecture doesn't keep yielding better results, Jevon's paradox doesn't apply.
- impossiblefork 2y ago
  But surely it can be scaled up, or is this compression thing something making the approach good only for small models (I haven't read the Deepseek papers (can't allocate time to it))?
- joak 2y ago
  Anyhow, if you can deliver more with less, this is huge good news for AI industry. After some readjustment we can expect AI companies to start using the new method to deliver more. Science fiction might happen sooner than expected. Buy the dip.
lmm 2y ago
If the top-tier premium GPUs aren't the difference-maker they were thought to be then that will hurt NVIDIA's margins, even if they make some of it up on volume.
alecco 2y ago
The limit is high quality data, not compute.
- belter 2y ago
  > The limit is high quality data If, as some companies claim, these models truly possess emergent reasoning, their ability to handle imperfect data should serve as a proof of that capability.

qwytw 2y ago

> overtakes ChatGPT That's arguable, though. I mean it's much cheaper and reasonably competitive which is almost the same but IMHO DeepSeek seems to get stuck in random loops and hallucinates more frequently than o1.

mgh2 2y ago

Let's see how the open vs close ecosystem wins in AI