6 ms·

Could someone explain as to why DeepSeek is bad for Nvidia? The demand for Nvidia GPUs should now go up. Now anyone can run a GPT-like model by themselves. It'

• amazingamazing 2y ago
  Given some demand, D - more efficiency means less supply is needed to meet demand D, thus, bad for Nvidia.
  • aftbit 2y ago
    That assumes demand remains constant. Maybe lowering the overall cost to train a model will mean that more people want to train models, thus raising the demand.
    • amazingamazing 2y ago
      Maybe, maybe not. Personally I find it unlikely. There are not a huge amount of folks needing to train their own model.
      • baq 2y ago
        There are a lot of folks lining up to use a smart model. This takes tokens. I’m not convinced nvidia is blown up by this news at all. The trade has become less crowded, that is true.
        • amazingamazing 2y ago
          if things become too efficient then you can use commodity compute and don't need GPUs at all. I'm not sure why you would think breakthroughs in efficiency would be good for nvidia. eventually a regular mac or PC will be able to do what you need a H100 to do now. that won't be good for nvidia.
          • baq 2y ago
            My bet is there's no such thing as 'too efficient' in this space. If you can get a very good model on a small device, it's going to be totally amazing on a huge GPU.
            • amazingamazing 2y ago
              I bet you're wrong - there are already massive diminishing returns in the best models from 2024 vs 2023. This idea that you can just through more compute and it scales with performance is fiction. You do get more performance with more compute, but it doesn't scale, and it's a waste of money, as shown with deepseek. this conversation reminds me of people when the PS2 came out saying that by 2010 games would look literally better than real life, because they thought graphics quality would exponentially improve...
              • baq 2y ago
                I would agree except I think it’s 1997. I’ve gotten DeepSeek to mostly solve a not very complex problem in a somewhat obscure domain (home assistant automations) in 214 seconds of its own ‘thinking’ and if you can get this two orders of magnitude lower this unlocks completely new use cases ie. demand.
• thefounder 2y ago
  If I can write an email from my small and cheap phone why would I buy that big mainframe? The chips to do the job become cheap. The high end chips will become a niche again for research and mil. stuff.
  • IshKebab 2y ago
    Because the big mainframe will write a much better email?
    • thefounder 2y ago
      It looks like that’s not the case anymore. That’s what this is all about.
      • senko 2y ago
        It still is. The DeepSeek R1 is 670B parameters, requiring more than 300GB of video ram (or unified memory), even if using 4-bit quantized models.
        • amazingamazing 2y ago
          How much video RAM was needed for similar performance a year ago?
          • senko 2y ago
            Roughly the same, as our collective goalposts have shifted, and are still shifting. What was amazing output last year is today's slop. NV is in no immediate danger; medium/long term, to anyone knowing what "75% gross margin" means it's no secret that there will be serious threats. You don't need DeepSeek for that particular realization.
            • amazingamazing 2y ago
              Which open source model last January gave you similar results with the same amount of video ram?
• throwawaymaths 2y ago
  i believe the article is making the case that the market has already priced in the expectation that training (but especially training which is what NVIDIA is useful for) and inference will be continually more expensive and deepseek implies that the hardware required to train and infer really good models has a lower ceiling.
  • easygenes 2y ago
    Mostly it just means that we’ve got a new tool we can leverage which is a positive feedback for improving models, and will ultimately increase both train and inference demand and shorten timescales for deeper capabilities.
• Ekaros 2y ago
  Question really is does the demand go up at current prices of units? Thus do current profit levels keep. If the training was too expensive previously is it now cheap enough or does it need to be even cheaper? As if it weren't too expensive everyone who were going to invest did. And if it is still too expensive, well prices have to go lower or more efficiency has to be found. I am not entirely sure if there is huge amount of unmet demand for training.
• WhyNotHugo 2y ago
  The US has prohibited nvidia from exporting to China, so DeepSeek (the company) won’t be a customer, and others wanting to self-host their models have the freedom to pick any hardware vendor.
  • Cthulhu_ 2y ago
    This thing underlines how nvidia is in trouble; it proves (unless Deepseek used servers outside of china) that banning nvidia from china does not give them any kind of advantage anymore.
• InkCanon 2y ago
  People here quoting Jevons paradox are grossly misapplying it. Jevons paradox will predict an increase in usage of compute for AI. Not Nvidia specifically. The specific insane valuation of Nvidia comes from an unending stream of hundreds in billions in big tech capex, constantly buying cutting edge GPUs, because on SV scaling laws are practically religious commandments. It remains to be seen how this affects Nvidia, but it is not a case of Nvidia like many posts here imply, business wise.
  • kandesbunzler 2y ago
    These are all NVDA owners coping, they dont want rational thoughts. The hit in market cap is absolutely deserved, especially now with the threat of tarrifs.
    • SR2Z 2y ago
      NVDA has an exceptionally reasonable P/E ratio and is a great pick on fundamentals alone. The tariffs are not going to make other graphics cards + their software 2x better.
      • kandesbunzler 2y ago
        pure cope. Hyperscalers already have a ton of gpus who just got 90% more efficient. They will absolutely reduce spend especially once the tariffs hit lol
        • numpad0 2y ago
          from non-investor perspective it's a long awaited sign of eventual profitability in the money pit and 10x sales potential. Not trying to debunk you, to me it's just proof as to how unrelated tech and market worlds are
          • kandesbunzler 2y ago
            except that deepseek r1 isnt a gigantic leap in terms of actual intelligence. These models still have questionable use, I have no doubt that more people will want to use them now but its not going to compensate for how much more efficient they are now. Not even close.
• ogrisel 2y ago
  I don't understand why it's bad for Nvidia either. The fact that DeepSeek-R1 is so much better than DeepSeek-V3 at various important tasks means that Chain-of-though / thinking-before-answering models are better. But they are also more compute intensive at inference time than their instruction non-thinking counterparts. So even if the DeepSeek-V3 pretraining + GRPO COT post-training procedure was cheaper than anticipated to reach o1 grade performance, inference is still costly, even if you use a distilled model.
  • bildung 2y ago
    Deepseek offers API pricing directly on their website, so it's pretty easy to compare inference costs indirectly: It's $60.00 vs. $2.19 for 1M output tokens. Openai is 27x as expensive.
• ivan_gammel 2y ago
  Nvidia was and still is overpriced. Yes, AI is now on the mainframe stage. Soon it will go to consumer devices with a billion of potential users and the training and the required hardware for it will be very different: much less power consumption, continuous training etc, just like our or animal brain. Consumers did buy computers for $1000 in 1990s. They may buy a robot for $10000 tomorrow. The benchmark for the market cap in this segment are car manufacturers or Intel in 1990s.
• samsartor 2y ago
  I think it's also generally understood that Nvidia owns the training space, but not the inference space. They have a lot more competition there and the margins are smaller. More people running AI models is still good for them, but a drop in the bucket compared to the money they were making from training clusters being built out. And I think everyone just realized they can probably make do with their existing clusters.
• DannyBee 2y ago
  Inference was already cheapish - businesses already can run gpt-like models by themselves. NVIDIA sales are not driven by inference spending, but training spend. While sure, maybe now people spend more on inference, NVIDIA was not an investment based money on selling GPUs for inference, because it requires so much fewer resources. Also, inference is easier to get disrupted in just because of how it works. All told, this makes it a much less likely play. Finally, the assumption that it goes up assumes people want on-prem infrastructure for this (in your case). Maybe true, maybe not. Overall, going from being a "sure" thing selling 200m worth of clusters to random companies hand over fist to not necessarily being able to do that is definitely "bad'. There are other common claims about why it will increase demand, and they rely on different assumptions (that you won't hit a good enough point quickly, etc)
  • ecocentrik 2y ago
    Is anyone canceling their orders? Has anyone announced that the efficiency savings from Deepseek's innovations mean we will have superhuman AGI by the end of the year using existing hardware? Or that existing hardware fully covers all of the expected training demand for the next year? Or will all of the efficiency savings get immediately absorbed by the demand for better performance and feed the demand for inference?
    • DannyBee 2y ago
      They don't exactly release this data, as you know, which is why it's so volatile - nobody has any real idea, just best guesses, and lots of different smart people have lots of different opinions.. But i agree in practice that over time nvidia's path now depends heavily on the answer to "Or will all of the efficiency savings get immediately absorbed by the demand for better performance and feed the demand for inference?" Before, their next 5-10 years did not really depend on meaningful efficiency savings existing and getting them absorbed - nobody expected meaningful efficiency savings. Now it does depend on that
      • ecocentrik 2y ago
        There have been continuous efficiency savings innovations for both training and inference over the last few years. Discovering and absorbing those gains is a normal part of the development process. It occurs in every industry and is facilitated in computer science by mechanisms like open source and published papers. Those savings don't usually make national news because they aren't the target. They aren't "man on the moon" moments, they are "our cannonball flies 96% as far as yours using 30% less powder" moments. This is still a race for tools that essentially brute force their way to greater utility. Until we hit a utility quotient or hit a wall, the race doesn't really end. The quickest and easiest path to advantage in the race is still more compute.
• nashashmi 2y ago
  It is not bad for Nvidia. It is bad for hype. Nvda position was good for being a company selling chips with lots of power. But now that not much power is needed, reliance on nvidia is less. And anyone with simpler processors has capabilities to sell AI chips. Nvidia will still sell many chips. It just won’t be the only one capable of selling them. The hype is gone. The moat is gone. The excitement and enthusiasm from investors is gone. This is called a correction.
  • baal80spam 2y ago
    > The moat is gone. CUDA begs to differ.
• elorant 2y ago
  What happens with excess hardware that’s been already bought? It could take years for all of that to be absorbed by smaller companies. And secondly, if this opens the era of self-training models then why go for a 671B one and not a smaller that’s fine-tuned to a company’s specific needs and can then run on consumer GPUs. At 70B with 8-bit quantization you’re good with just three 3090s.
  • pixl97 2y ago
    Deepseek isn't AGI yet and really still has a long way to go. Na, we have a long way to go with models, especially when you start adding different modes. We'll still need a metric shitload of compute for a long time.
• headcanon 2y ago
  Agreed, I believe AI/LLMs will create induced demand for compute long-term, even if we get more efficient models (long-term meaning >6mo). Markets don't look ahead for more than a quarter or two though, so it makes sense there would be a correction based on the news.
• feverzsj 2y ago
  The market is already losing patience on "profitable AI". Deepseek only improves cost with worse performance, while even chatgpt is still far from profitable.
• numpad0 2y ago
  Yeah, to me it looks like a lot of investors were into AI for skewed agendas and 1984 style fantasies and panicking after told it's not about that.
• chrsw 2y ago
  I still don't get why this is bad for NVIDIA either. If anything, it brings more people into their ecosystem. NVIDIA is the AI hardware/platform company right now and people will deploy AI for more tasks if it becomes cheaper to do so. But there's a more subtle point here which I don't see a lot of people talking about, maybe because they know more about this than me. Why wouldn't frontier model developers take DeepSeek R1 techniques and make their models even better or even larger? Or another way to ask: Are the DeepSeek R1 innovations only for making models cheaper (and slightly worse) or can the algorithms developed by the DeepSeek team be scaled up to make more powerful frontier models? Leading edge model developers don't just care about cost, they also want to release models with maximum capabilities. And as we've seen over the past few years, they're willing to pay almost anything to achieve this. I think most AI researchers know there are still many things to explore in this space so disruptive innovations shouldn't be seen as a bubble burst but as more opportunity. NVIDIA is in an extremely strong position right now. Even if someone has a major breakthrough on the hardware design side that dramatically lowers the cost of compute for AI workloads (which is highly unlikely), NVIDIA will just create their own implementation that will outperform the original since they have a stranglehold on an entire stack-up of technology: circuits, drivers, libraries and software.
  • dgreensp 2y ago
    You’re absolutely right. If there are corrections, it’s only because people have gone a little crazy. People sometimes get enamored with the idea of one country or company dominating in a winner-takes-all situation. OpenAI has been trying to spin narratives in which the only rational move is for everyone to invest all of their money in OpenAI, immediately. I don’t think the Nvidia CEO is losing any sleep. He doesn’t have a planet-sized ego, like some. They are still selling pickaxes during a gold rush, as the saying goes. And the stock is still up 100% in the past year. CEOs don’t control the stock price (much as some try), anyway, the market does. Also, the market is insane. You also can’t control the competition, or what new innovations come along. You just have to run a good business, which they are doing. Innovation is good for all players. Reality-checks are good. Nothing here is unexpected. There are lots of smart researchers in the world. Software innovations that make better use of hardware are expected. People just like drama. Like if Tim does a better job at a skateboard trick that has always been Tom’s thing, people want to be that kid who is the one to say, “Oh, snap!!” and won’t stop talking about it at school, because they were there. And how it’s so mind-blowing and previously inconceivable.
• nijuashi 2y ago
  Absolutely agree on this. 1. DeepSeek just inspired a LOT of startups to develop their own as you no longer need to be a tech-giant to compete on training. 2. Companies with sensitive info will now buy their own GPUs to run their models locally as the range of application increased (as it did with Llama3) 3. As with 2, new services that were prohibitive with ChatGPT API will spring up. It’s difficult to reliably rent GPUs for services even with enough money, so people will buy more GPUs to host it themselves. I understand if new CPU/GPU can outperform Nvidia and DeepSeek was developed using another GPU, but this is not the case. Lower requirement for higher performance historically never reduced the need for computational capacity. There is very poor reasoning for this move other than purely “technical” (trading-wise) reasons.