10 ms·

DeepSeek could represent Nvidia CEO Jensen Huang's worst nightmare

68 points by pera 2y ago

Refusing23 2y ago
DeepSeek could represent a much larger amount of costumers to nvidia since now people can get an AI while spending less ftfy
- logical_proof 2y ago
  Did you mean consumers? If you're going to ftfy to someone you should use spell check.
alephnerd 2y ago
There was no market reaction when Deepseek r1 was released. There was no market reaction when Deepseek v3 was released. There was a massive market reaction when a Deepseek app was released. There is too much damn irrationality in the AI and AI adjacent market right now, and pop articles like this are only making it worse. Jevons Paradox absolutely holds for model development, and Deepseek should (and is - all my cybersecurity startup peers are investigating the feasibility of building their own models now) be viewed as an opportunity for smaller or medium stage companies to build their own competitive domain specific models, while not having to pay what is essentially protection money to OpenAI or Anthropic. The only people at risk with the democratization of model development are the investors in foundational model companies --------- Also, I hope Deepseek FINALLY reprioritizes distributed systems education in CS. There are too many "ML Engineers" who do not know the basics of OS or Hardware Engineering (eg. Cannot optimize Mellanox hardware, really dig into workload optimization, etc) and are thus burning resources inefficiently. Most MLEs I meet now just import a package and glue code together WITHOUT also understanding the performance capabilities that their compute has. Most universities in the US either don't require OS or CompArch classes for CS majors (eg. Harvard) or dumb them down significantly (eg. Not going deep into Linux kernel implementation, scheduling, etc). This is why the entire cybersecurity industry has largely shifted to Israel and India, and the skillset overlaps significantly with MLOps and MLEng (if you know how to implement spinlocks in C you can easily be taught CUDA) <"Old Man yelling at clouds" rant over>
- high_na_euv 2y ago
  Or people in charge cannot properly evaluate things :) It is very possible, when things require advanced technical knowledge and experience ><"Old Man yelling at clouds" rant over> They are too busy deploying ruby apps and juggling jsons over https to mess with ugly cpp, segfaults, kernel details, hardware intrinsics and semiconductors
  - alephnerd 2y ago
    At the scale that the selloffs happened, that meant there were some shifts by institutional investors. These are the kinds of managers that would have technical people on retainer to consult (eg. Guidepoint). > They are too busy deploying ruby apps and juggling jsons over https Ruby is "legacy" as well nowadays. Most younger devs I meet tend to really only understand JS and Python, and that too while heavily relying on outside packages or dependencies. Not a bad thing per say, but if a VC like me has deeper knowledge about OS internals or Mellanox tuning than some of the (American) MLEs in companies they've done due diligence on, something's very wrong with the talent pipeline.
  - highcountess 2y ago
    Humans are generally not very good at properly evaluating things in general. They are exceptionally good at believing themselves to be very good at properly evaluating things though. It is why empires collapse from miscalculations about war by their experts, it is the foundational mental model underlying communism that simply cannot work but keeps being attempted, and it is how countries can be destroyed in every which way while the experts maintain that everything is just fine. Never underestimate the powers of the ego, the destroyer of worlds. I mean they don’t know what they are doing, but we have advanced technical knowledge and experience, so we should clearly be in charge of all things, including those beyond our narrow scope of advanced technical knowledge and experience.
    - dagss 2y ago
      To be fair, capitalism doesn't really work either, you get a very unstable systems with cycles (which people try to stabilize in ad hoc ways), you optimize for the wrong things long term (right now producing much more goods than we need, without factoring in devastating costs that climate change will bring in a few decades, and destruction of nature and eradication of species) and when things crash many people are hurt. And it practically needs wars and destruction to reset wealth distribution every 50 to 100 years. No communism doesn't work either. I wish there was more alternatives. And some of the usual examples of "socialism doesn't work" might have worked if US hadn't interfered. E.g. in Chile, no system would have survived US actively supporting Pinochet. We will never know how socialism would have played out there if US (and Soviet too) left it alone.
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
zerof1l 2y ago
Could someone explain as to why DeepSeek is bad for Nvidia? The demand for Nvidia GPUs should now go up. Now anyone can run a GPT-like model by themselves. It's a prime time for businesses to start investing and setting up on-prem infra for that. I know some have been avoiding ChatGPT due to legal concerns and sensitive data.
- amazingamazing 2y ago
  Given some demand, D - more efficiency means less supply is needed to meet demand D, thus, bad for Nvidia.
  - aftbit 2y ago
    That assumes demand remains constant. Maybe lowering the overall cost to train a model will mean that more people want to train models, thus raising the demand.
    - amazingamazing 2y ago
      Maybe, maybe not. Personally I find it unlikely. There are not a huge amount of folks needing to train their own model.
      - baq 2y ago
        There are a lot of folks lining up to use a smart model. This takes tokens. I’m not convinced nvidia is blown up by this news at all. The trade has become less crowded, that is true.
        amazingamazing 2y ago
        if things become too efficient then you can use commodity compute and don't need GPUs at all. I'm not sure why you would think breakthroughs in efficiency would be good for nvidia. eventually a regular mac or PC will be able to do what you need a H100 to do now. that won't be good for nvidia.
stogot 2y ago
Did they say which chips they used? And how many? I haven’t seen this in any reporting
- lysace 2y ago
  The DeepSeek V3 paper says they used 2.8 million Nvidia H800 hours in a cluster of 2048 of them.
maupin 2y ago
> “We didn’t think China was ever going to compete in AI,” said Dion Hinchcliffe, vice president of the CIO practice at Futurum Research last week. What? Oh, come on now.
mtkd 2y ago
Wait until some surprise new silicon suddenly appears
j_not_j 2y ago
I don't for a minute believe Deepseek v3 was built with a $6M rental. Their paper (arxiv 2412:1947) explains they used 2048 H800s. A computer cluster based on 2048 GPUs would have cost around $400M about two years ago when they built it. (Give or take, feel free to post corrections.) The point is they got it done cheaper than OpenAI/Google/Meta/... etc. But not cheaply. I believe the markets are overreacting. Time to buy (tinfa).
- aisio 2y ago
  2048 GPUs cost $400m? pretty sure the GPUs don't cost 200k each?
  - bottlelion 2y ago
    And a pile of GPUs doesn't really do much good without servers, racks, networking, power, cooling, and a building to house it all in.
    - moralestapia 2y ago
      And a pile of "servers, racks, networking, power, cooling, and a building to house it all in" doesn't cost 10-20x more than the GPUs. If that were the case, Cooler Master would be the trillion dollar company, lmao.
      - dessimus 2y ago
        If Cooler Master could charge enterprise computing prices for their consumer level gear, they probably would be.
        moralestapia 2y ago
        Something like an H100 is definitely a feat of engineering, though. Nothing prevents Cooler Master from releasing a line of GPUs equally performant and, while at it, even cheaper. But when we measure reality, after the wave function of fentanyl and good intentions collapses ... oh yeah, turns out only nVidia is making those chips, whoops ...
NotYourLawyer 2y ago
NVDA could go to zero and he’d still be a multi billionaire. His worst nightmare isn’t very bad.
feverzsj 2y ago
Nvidia AI Chip smuggling is a huge business in china. DeepSeek company on its own smuggled 50k h100.
- TechDebtDevin 2y ago
  Its not smuggling if Nvidia vendors openly sell it to them :) I'm pretty sure most of Russia's military equipment is filled with Texas Instrument MCs. These export controls don't work.
- deleted 2y ago
  [deleted]
- dannyfreeman 2y ago
  Source: trust me
childintime 2y ago
Time to give our family it's own shared networked AI station, accessible from any of our devices. Then I'll trust it with all of our personal, medical and legal docs. Seems to me the market is expanding, not contracting.
chvid 2y ago
It is the end of neural language model scaling as described here: https://openai.com/index/scaling-laws-for-neural-language-models/ https://openai.com/index/scaling-laws-for-neural-language-mo... And a return to more normal world where progress comes from refinement of algorithms and approach rather than brute force compute. Deepseek is just the poster child.
- K0balt 2y ago
  If we look at the processing requirements in nature, I think that the main trend in AI going forward is going to be doing more with less, not doing less with more, as the current scaling is going. Thermodynamic neural networks may also basically turn everything on its ear, especially if we figure out how to scale them like NAND flash. If anything, I would estimate that this is a space-race type effort to “win” the AI “wars”. In the short term, it might work. In the long term, it’s probably going to result in a massive glut in accelerated data center capacity. The trend of technology is towards converging with or doing better than natural processes, not doing it 100000x less efficiently. I don’t think AI will be an exception. If we look at what is -theoretically- possible using thermodynamic wells, with current model architectures, for instance, we could (theoretically) make a network that applies 1t parameters in something like 1cm2. It would use about 20watts, back of the napkin, and be able to generate a few thousand T/S. Operational thermodynamic wells have already been demonstrated en silica. There are scaling challenges, cooling requirements, etc but AFAIK no theoretical roadblocks to scaling. Obviously, the theoretical doesn’t translate to results, but it does correlate strongly with the trend. So the real question is, what can we build that can only be done if there are hundreds of millions of NVIDIA GPUs sitting around idle in a few years? Or alternatively, if those systems are depreciated and available on secondary markets? What does that look like?
- wongarsu 2y ago
  Also evidenced by 3B and 7B models becoming ever more capable, gaining qualities that were previously only achieved by models a magnitude larger. Bigger models are more capable, but smaller models can be iterated on faster. For a couple months now most of the impressive achievements have been in increasingly smaller and cheaper models. Deepseek just has the perfect storm of impressive results, accessibility and international rivalry that made it go viral
seydor 2y ago
I find it hard to believe that serious investors thought that vertical progress in ai had peaked, and it would be only horizontal expansion from now on. We know that it takes decades for this to happen in every industry. It seems more like , investors were looking for a significant event to pop the obvious bubble if the last year
- Terretta 2y ago
  Which investors, specifically, were looking for a significant event to pop the obvious bubble? Investors who were actually invested, or onlookers with FOMO who hadn't invested yet? While I think the "AI" companies invested in are overpriced*, I think the market as a whole hasn't even begun to price in the changes in store across non-tech sectors and industries even if LLMs were to advance no further than now, with all additional progress at the prompting and agentic levels. * Even here, Jevons paradox applies. So what's being called "priced for perfection" may actually be priced for business as usual with multiplied demand.
colinnordin 2y ago
R1 and O1 points towards a world where training models will be a small bucket of the overall compute. That doesn't mean the total amount of AI compute will stop accelerating, just that interconnected mega clusters is not the only or most efficient way to run a majority of future workloads. That should be negative news for the company that is currently the only one that is capable of making chips for these clusters, and positive news for the players that can run inference on a single chip, as they will be able to grab more parts of the compute pie.
nashashmi 2y ago
Was the US sanctions on chip selling to China worth it? Yesssss. It was. It still is. (But I frown on this action.) China is good on dumb stupid manufacturing. It does this with so much enthusiasm HNers are left scratching their heads why the excitement. The possibility of China manufacturing the next computing powerhouse was too high. Higher than it could ever be in the US. The threat to US leadership was that the fastest supercomputer for AI in the world could be in China (the first sputnik moment per Barack Obama). So the chip embargo was to prevent that from happening. So China did not spend/waste $$ & time building hardware. They spent it on software. Normally the Chinese version is cheap enough and good enough. But this time it was cheap enough and surprisingly terrific enough. Props to Liang the founder for having that game face on.
numpad0 2y ago
It can't get sillier than this. All that DeepSeek did were showing that NVIDIA products hold more value than believed, and that US dominance more illusory than thought. It has zero negative bearing on eventual path to AGI. And that's somehow supposed to be end of NVIDIA? Hello?
cpldcpu 2y ago
All this media frenzy around DS V1 makes me feel sick to my stomach. It increased the noise in the AI space by orders of magnitude. Every media outlet is bombarding you with a relentless torrent of half-true information, exaggered interpretation of single facts and speculation. Sometimes I wonder whether this is amplified by a state actor? Also curious, how Minimax-01, which is also an excellent model with impressive improvements, went by completely unnocited. The only good thing is that this certainly put an end to OpenAIs price gauging - pretty sure that $200/month individual plan is not the limit of their imagination.
shnp 2y ago
Is deepseek’s architecture not scalable anymore? I can see in a couple months some company with access to GPUs create a much larger and better performing model based on the deepseek architecture.
iambateman 2y ago
Does DeepSeek make it more possible to use AMD hardware? What chips are possible with DS that weren’t possible before?

68 points by pera 2y ago

DeepSeek could represent a much larger amount of costumers to nvidia since now people can get an AI while spending less ftfy

logical_proof 2y ago
Did you mean consumers? If you're going to ftfy to someone you should use spell check.

alephnerd 2y ago

There was no market reaction when Deepseek r1 was released. There was no market reaction when Deepseek v3 was released. There was a massive market reaction when a Deepseek app was released. There is too much damn irrationality in the AI and AI adjacent market right now, and pop articles like this are only making it worse. Jevons Paradox absolutely holds for model development, and Deepseek should (and is - all my cybersecurity startup peers are investigating the feasibility of building their own models now) be viewed as an opportunity for smaller or medium stage companies to build their own competitive domain specific models, while not having to pay what is essentially protection money to OpenAI or Anthropic. The only people at risk with the democratization of model development are the investors in foundational model companies --------- Also, I hope Deepseek FINALLY reprioritizes distributed systems education in CS. There are too many "ML Engineers" who do not know the basics of OS or Hardware Engineering (eg. Cannot optimize Mellanox hardware, really dig into workload optimization, etc) and are thus burning resources inefficiently. Most MLEs I meet now just import a package and glue code together WITHOUT also understanding the performance capabilities that their compute has. Most universities in the US either don't require OS or CompArch classes for CS majors (eg. Harvard) or dumb them down significantly (eg. Not going deep into Linux kernel implementation, scheduling, etc). This is why the entire cybersecurity industry has largely shifted to Israel and India, and the skillset overlaps significantly with MLOps and MLEng (if you know how to implement spinlocks in C you can easily be taught CUDA) <"Old Man yelling at clouds" rant over>

high_na_euv 2y ago
Or people in charge cannot properly evaluate things :) It is very possible, when things require advanced technical knowledge and experience ><"Old Man yelling at clouds" rant over> They are too busy deploying ruby apps and juggling jsons over https to mess with ugly cpp, segfaults, kernel details, hardware intrinsics and semiconductors
- alephnerd 2y ago
  At the scale that the selloffs happened, that meant there were some shifts by institutional investors. These are the kinds of managers that would have technical people on retainer to consult (eg. Guidepoint). > They are too busy deploying ruby apps and juggling jsons over https Ruby is "legacy" as well nowadays. Most younger devs I meet tend to really only understand JS and Python, and that too while heavily relying on outside packages or dependencies. Not a bad thing per say, but if a VC like me has deeper knowledge about OS internals or Mellanox tuning than some of the (American) MLEs in companies they've done due diligence on, something's very wrong with the talent pipeline.
- highcountess 2y ago
  Humans are generally not very good at properly evaluating things in general. They are exceptionally good at believing themselves to be very good at properly evaluating things though. It is why empires collapse from miscalculations about war by their experts, it is the foundational mental model underlying communism that simply cannot work but keeps being attempted, and it is how countries can be destroyed in every which way while the experts maintain that everything is just fine. Never underestimate the powers of the ego, the destroyer of worlds. I mean they don’t know what they are doing, but we have advanced technical knowledge and experience, so we should clearly be in charge of all things, including those beyond our narrow scope of advanced technical knowledge and experience.
  - dagss 2y ago
    To be fair, capitalism doesn't really work either, you get a very unstable systems with cycles (which people try to stabilize in ad hoc ways), you optimize for the wrong things long term (right now producing much more goods than we need, without factoring in devastating costs that climate change will bring in a few decades, and destruction of nature and eradication of species) and when things crash many people are hurt. And it practically needs wars and destruction to reset wealth distribution every 50 to 100 years. No communism doesn't work either. I wish there was more alternatives. And some of the usual examples of "socialism doesn't work" might have worked if US hadn't interfered. E.g. in Chile, no system would have survived US actively supporting Pinochet. We will never know how socialism would have played out there if US (and Soviet too) left it alone.

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

zerof1l 2y ago

Could someone explain as to why DeepSeek is bad for Nvidia? The demand for Nvidia GPUs should now go up. Now anyone can run a GPT-like model by themselves. It's a prime time for businesses to start investing and setting up on-prem infra for that. I know some have been avoiding ChatGPT due to legal concerns and sensitive data.

amazingamazing 2y ago
Given some demand, D - more efficiency means less supply is needed to meet demand D, thus, bad for Nvidia.
- aftbit 2y ago
  That assumes demand remains constant. Maybe lowering the overall cost to train a model will mean that more people want to train models, thus raising the demand.
  - amazingamazing 2y ago
    Maybe, maybe not. Personally I find it unlikely. There are not a huge amount of folks needing to train their own model.
    - baq 2y ago
      There are a lot of folks lining up to use a smart model. This takes tokens. I’m not convinced nvidia is blown up by this news at all. The trade has become less crowded, that is true.
      - amazingamazing 2y ago
        if things become too efficient then you can use commodity compute and don't need GPUs at all. I'm not sure why you would think breakthroughs in efficiency would be good for nvidia. eventually a regular mac or PC will be able to do what you need a H100 to do now. that won't be good for nvidia.

stogot 2y ago

Did they say which chips they used? And how many? I haven’t seen this in any reporting

lysace 2y ago
The DeepSeek V3 paper says they used 2.8 million Nvidia H800 hours in a cluster of 2048 of them.

maupin 2y ago

> “We didn’t think China was ever going to compete in AI,” said Dion Hinchcliffe, vice president of the CIO practice at Futurum Research last week. What? Oh, come on now.

mtkd 2y ago

Wait until some surprise new silicon suddenly appears

j_not_j 2y ago

I don't for a minute believe Deepseek v3 was built with a $6M rental. Their paper (arxiv 2412:1947) explains they used 2048 H800s. A computer cluster based on 2048 GPUs would have cost around $400M about two years ago when they built it. (Give or take, feel free to post corrections.) The point is they got it done cheaper than OpenAI/Google/Meta/... etc. But not cheaply. I believe the markets are overreacting. Time to buy (tinfa).

aisio 2y ago
2048 GPUs cost $400m? pretty sure the GPUs don't cost 200k each?
- bottlelion 2y ago
  And a pile of GPUs doesn't really do much good without servers, racks, networking, power, cooling, and a building to house it all in.
  - moralestapia 2y ago
    And a pile of "servers, racks, networking, power, cooling, and a building to house it all in" doesn't cost 10-20x more than the GPUs. If that were the case, Cooler Master would be the trillion dollar company, lmao.
    - dessimus 2y ago
      If Cooler Master could charge enterprise computing prices for their consumer level gear, they probably would be.
      - moralestapia 2y ago
        Something like an H100 is definitely a feat of engineering, though. Nothing prevents Cooler Master from releasing a line of GPUs equally performant and, while at it, even cheaper. But when we measure reality, after the wave function of fentanyl and good intentions collapses ... oh yeah, turns out only nVidia is making those chips, whoops ...

NotYourLawyer 2y ago

NVDA could go to zero and he’d still be a multi billionaire. His worst nightmare isn’t very bad.

feverzsj 2y ago

Nvidia AI Chip smuggling is a huge business in china. DeepSeek company on its own smuggled 50k h100.

TechDebtDevin 2y ago
Its not smuggling if Nvidia vendors openly sell it to them :) I'm pretty sure most of Russia's military equipment is filled with Texas Instrument MCs. These export controls don't work.
deleted 2y ago
[deleted]
dannyfreeman 2y ago
Source: trust me

childintime 2y ago

Time to give our family it's own shared networked AI station, accessible from any of our devices. Then I'll trust it with all of our personal, medical and legal docs. Seems to me the market is expanding, not contracting.

chvid 2y ago

It is the end of neural language model scaling as described here: https://openai.com/index/scaling-laws-for-neural-language-models/ https://openai.com/index/scaling-laws-for-neural-language-mo... And a return to more normal world where progress comes from refinement of algorithms and approach rather than brute force compute. Deepseek is just the poster child.

K0balt 2y ago
If we look at the processing requirements in nature, I think that the main trend in AI going forward is going to be doing more with less, not doing less with more, as the current scaling is going. Thermodynamic neural networks may also basically turn everything on its ear, especially if we figure out how to scale them like NAND flash. If anything, I would estimate that this is a space-race type effort to “win” the AI “wars”. In the short term, it might work. In the long term, it’s probably going to result in a massive glut in accelerated data center capacity. The trend of technology is towards converging with or doing better than natural processes, not doing it 100000x less efficiently. I don’t think AI will be an exception. If we look at what is -theoretically- possible using thermodynamic wells, with current model architectures, for instance, we could (theoretically) make a network that applies 1t parameters in something like 1cm2. It would use about 20watts, back of the napkin, and be able to generate a few thousand T/S. Operational thermodynamic wells have already been demonstrated en silica. There are scaling challenges, cooling requirements, etc but AFAIK no theoretical roadblocks to scaling. Obviously, the theoretical doesn’t translate to results, but it does correlate strongly with the trend. So the real question is, what can we build that can only be done if there are hundreds of millions of NVIDIA GPUs sitting around idle in a few years? Or alternatively, if those systems are depreciated and available on secondary markets? What does that look like?
wongarsu 2y ago
Also evidenced by 3B and 7B models becoming ever more capable, gaining qualities that were previously only achieved by models a magnitude larger. Bigger models are more capable, but smaller models can be iterated on faster. For a couple months now most of the impressive achievements have been in increasingly smaller and cheaper models. Deepseek just has the perfect storm of impressive results, accessibility and international rivalry that made it go viral

seydor 2y ago

I find it hard to believe that serious investors thought that vertical progress in ai had peaked, and it would be only horizontal expansion from now on. We know that it takes decades for this to happen in every industry. It seems more like , investors were looking for a significant event to pop the obvious bubble if the last year

Terretta 2y ago
Which investors, specifically, were looking for a significant event to pop the obvious bubble? Investors who were actually invested, or onlookers with FOMO who hadn't invested yet? While I think the "AI" companies invested in are overpriced*, I think the market as a whole hasn't even begun to price in the changes in store across non-tech sectors and industries even if LLMs were to advance no further than now, with all additional progress at the prompting and agentic levels. * Even here, Jevons paradox applies. So what's being called "priced for perfection" may actually be priced for business as usual with multiplied demand.

colinnordin 2y ago

R1 and O1 points towards a world where training models will be a small bucket of the overall compute. That doesn't mean the total amount of AI compute will stop accelerating, just that interconnected mega clusters is not the only or most efficient way to run a majority of future workloads. That should be negative news for the company that is currently the only one that is capable of making chips for these clusters, and positive news for the players that can run inference on a single chip, as they will be able to grab more parts of the compute pie.

nashashmi 2y ago

Was the US sanctions on chip selling to China worth it? Yesssss. It was. It still is. (But I frown on this action.) China is good on dumb stupid manufacturing. It does this with so much enthusiasm HNers are left scratching their heads why the excitement. The possibility of China manufacturing the next computing powerhouse was too high. Higher than it could ever be in the US. The threat to US leadership was that the fastest supercomputer for AI in the world could be in China (the first sputnik moment per Barack Obama). So the chip embargo was to prevent that from happening. So China did not spend/waste $$ & time building hardware. They spent it on software. Normally the Chinese version is cheap enough and good enough. But this time it was cheap enough and surprisingly terrific enough. Props to Liang the founder for having that game face on.

numpad0 2y ago

It can't get sillier than this. All that DeepSeek did were showing that NVIDIA products hold more value than believed, and that US dominance more illusory than thought. It has zero negative bearing on eventual path to AGI. And that's somehow supposed to be end of NVIDIA? Hello?

cpldcpu 2y ago

All this media frenzy around DS V1 makes me feel sick to my stomach. It increased the noise in the AI space by orders of magnitude. Every media outlet is bombarding you with a relentless torrent of half-true information, exaggered interpretation of single facts and speculation. Sometimes I wonder whether this is amplified by a state actor? Also curious, how Minimax-01, which is also an excellent model with impressive improvements, went by completely unnocited. The only good thing is that this certainly put an end to OpenAIs price gauging - pretty sure that $200/month individual plan is not the limit of their imagination.

shnp 2y ago

Is deepseek’s architecture not scalable anymore? I can see in a couple months some company with access to GPUs create a much larger and better performing model based on the deepseek architecture.

iambateman 2y ago

Does DeepSeek make it more possible to use AMD hardware? What chips are possible with DS that weren’t possible before?