4 ms·

Having read the paper from deepseek I seriously don't get how anyone can think "Oh now is the time to sell nvidia". Both at the tactical level and strategic le

Having read the paper from deepseek I seriously don't get how anyone can think "Oh now is the time to sell nvidia". Both at the tactical level and strategic level that paper means we will see _more_ nvidia gpus fly off the shelves. If anything it's showing why the h800 is a decent card and everyone in China should buy one (dozen thousands).

bilbo0s 2y ago
I had analyzed the paper as well, and gave some quick thoughts on a few of the low level things that stood out and would affect NVDA here: https://news.ycombinator.com/item?id=42856652 https://news.ycombinator.com/item?id=42856652 I have to agree. I'm not sure the architectural and engineering changes here don't put NVDA further ahead. Some stuff all gpu makers can do. Like quantize on transfer. But NVDA is so far ahead in other areas and DeepSeek leans heavily on that industry leading tech. At least at the low level. But I'm betting the average investor doesn't have the level of facility with ML to properly analyze the information that's come out. Heck, I'm betting most of the people working in AI are really just TF or PyTorch monkeys. The real AI experts have probably been hovered up by MS or FB or AMZN or OpenAI or what have you. So it feels like we could get lucky here and the AI related market could drop like a rock taking the good with the bad. Which I think would be awesome for most people in a position to wait for it to happen.
- llm_trw 2y ago
  I can't wait for the 2000 dot com crash for AI. What gets me about the deepseek work is that it's neither novel or theoretically difficult. I've been using a similar synthetic data pipeline to generate examples for sota bert embedding models. That low-hanging fruits like this are left unexplored by meta or alphabet - and if we're to be believed lead to crisis meetings for a week straight - makes one wonder wtf the engineers there are doing?
  - spamizbad 2y ago
    This is just a baseless conspiracy theory that I have, but I do wonder if they intentionally avoided certain avenues of research because it could erode the “moat” major tech firms have by dramatically reducing the capital costs required for training and inference. If you focus your research around work that mandates high-end hardware at scale you can lock out tons of potential incumbents
    - llm_trw 2y ago
      There is nothing in the deepseek paper that suggests you can't use the order of magnitude in hardware costs you saved to just train models that are ten times as large.
      - spamizbad 2y ago
        But there are thresholds of commercial viability in all of this. DeepSeek's technology gets you over that line with less sophisticated hardware. There's already some pretty impressive work being done with folks using just a pair of M2 Ultras with r1 in a "home lab" context that goes way beyond what you could previously do with llama.
        llm_trw 2y ago
        Yes, but the r1 model is shiny and new. I've been using it locally for a day and I'm already finding odd spots. One thing that I've seen it do a number of times is get the correct answer in the thinking tokens, then ignore it, generate thousands more tokens until it reaches it context length then give a wrong answer. There are things, like doc strings for functions, that llama models give better results for.
      - YurgenJurgensen 2y ago
        …and even if you can’t, you could train models that are ten times more specific, or update them ten times more frequently.
gmm1990 2y ago
majority of Nvidia's profits are from huge companies particularly Microsoft and META not individuals buying graphics cards. Look at their current video gaming graphics sales and even if these ai consumer purchases were 10x what there previous graphics sales were then the current valuation wouldn't make sense. This is all ignoring the fact that if the h800 was good enough amd's top line cards should be too.
- llm_trw 2y ago
  >This is all ignoring the fact that if the h800 was good enough amd's top line cards should be too. You can't pay me to use AMD cards. I have work to do. I don't have time to write drivers for cards that no one supports. The point of the deepseek r1 paper is that the more synthetic data you generate by over trained models the better the resulting model. That means more GPUs. There is no reason why you can't train a dozen models like their nameless logic model in other domains and then use them for synthetic data augmentation.
  - aurareturn 2y ago
    The better the foundation model, the better the distilled thinking model. So yes, an increase need for ever better thinking models.
pmarreck 2y ago
Perception, not facts, primarily drive markets
- mrbungie 2y ago
  A supposed future event, invoked by a unconstrained interpretation of Jevons Paradox, is not a fact.
ijidak 2y ago
But, I thought the CUDA moat is primarily needed for training. On the inference side, why does one need Nvidia GPUs over Intel and AMD GPUs?
- belter 2y ago
  Tensor Cores accelerate inference...
- aurareturn 2y ago
  Probably because companies buy Nvidia GPUs training and then use them for inference. Dual use. Not to mention there’s little real world evidence that AMD is better at inference speed not cost/inference over Nvidia.

bilbo0s 2y ago

I had analyzed the paper as well, and gave some quick thoughts on a few of the low level things that stood out and would affect NVDA here: https://news.ycombinator.com/item?id=42856652 https://news.ycombinator.com/item?id=42856652 I have to agree. I'm not sure the architectural and engineering changes here don't put NVDA further ahead. Some stuff all gpu makers can do. Like quantize on transfer. But NVDA is so far ahead in other areas and DeepSeek leans heavily on that industry leading tech. At least at the low level. But I'm betting the average investor doesn't have the level of facility with ML to properly analyze the information that's come out. Heck, I'm betting most of the people working in AI are really just TF or PyTorch monkeys. The real AI experts have probably been hovered up by MS or FB or AMZN or OpenAI or what have you. So it feels like we could get lucky here and the AI related market could drop like a rock taking the good with the bad. Which I think would be awesome for most people in a position to wait for it to happen.

llm_trw 2y ago
I can't wait for the 2000 dot com crash for AI. What gets me about the deepseek work is that it's neither novel or theoretically difficult. I've been using a similar synthetic data pipeline to generate examples for sota bert embedding models. That low-hanging fruits like this are left unexplored by meta or alphabet - and if we're to be believed lead to crisis meetings for a week straight - makes one wonder wtf the engineers there are doing?
- spamizbad 2y ago
  This is just a baseless conspiracy theory that I have, but I do wonder if they intentionally avoided certain avenues of research because it could erode the “moat” major tech firms have by dramatically reducing the capital costs required for training and inference. If you focus your research around work that mandates high-end hardware at scale you can lock out tons of potential incumbents
  - llm_trw 2y ago
    There is nothing in the deepseek paper that suggests you can't use the order of magnitude in hardware costs you saved to just train models that are ten times as large.
    - spamizbad 2y ago
      But there are thresholds of commercial viability in all of this. DeepSeek's technology gets you over that line with less sophisticated hardware. There's already some pretty impressive work being done with folks using just a pair of M2 Ultras with r1 in a "home lab" context that goes way beyond what you could previously do with llama.
      - llm_trw 2y ago
        Yes, but the r1 model is shiny and new. I've been using it locally for a day and I'm already finding odd spots. One thing that I've seen it do a number of times is get the correct answer in the thinking tokens, then ignore it, generate thousands more tokens until it reaches it context length then give a wrong answer. There are things, like doc strings for functions, that llama models give better results for.
    - YurgenJurgensen 2y ago
      …and even if you can’t, you could train models that are ten times more specific, or update them ten times more frequently.

gmm1990 2y ago

majority of Nvidia's profits are from huge companies particularly Microsoft and META not individuals buying graphics cards. Look at their current video gaming graphics sales and even if these ai consumer purchases were 10x what there previous graphics sales were then the current valuation wouldn't make sense. This is all ignoring the fact that if the h800 was good enough amd's top line cards should be too.

llm_trw 2y ago
>This is all ignoring the fact that if the h800 was good enough amd's top line cards should be too. You can't pay me to use AMD cards. I have work to do. I don't have time to write drivers for cards that no one supports. The point of the deepseek r1 paper is that the more synthetic data you generate by over trained models the better the resulting model. That means more GPUs. There is no reason why you can't train a dozen models like their nameless logic model in other domains and then use them for synthetic data augmentation.
- aurareturn 2y ago
  The better the foundation model, the better the distilled thinking model. So yes, an increase need for ever better thinking models.

pmarreck 2y ago

Perception, not facts, primarily drive markets

mrbungie 2y ago
A supposed future event, invoked by a unconstrained interpretation of Jevons Paradox, is not a fact.

ijidak 2y ago

But, I thought the CUDA moat is primarily needed for training. On the inference side, why does one need Nvidia GPUs over Intel and AMD GPUs?

belter 2y ago
Tensor Cores accelerate inference...
aurareturn 2y ago
Probably because companies buy Nvidia GPUs training and then use them for inference. Dual use. Not to mention there’s little real world evidence that AMD is better at inference speed not cost/inference over Nvidia.