6 ms·

As someone who is out of the loop, what’s the verdict on R1? Was anyone able to reproduce the results yet? Is the claim that it only took $5M to train generally

• huijzer 2y ago
  > Is the claim that it only took $5M to train generally accepted? Based on Nvidia being down 18% yesterday I would say the claim is generally accepted.
  • deskamess 2y ago
    > Nvidia being down 18% The only part of DeepSeek-R1 I do not like. I hope it's over, but I am not holding my breath.
    • coffeebeqn 2y ago
      Nvidia is now up only 1906% over 5 years. What a disgrace
      • samvher 2y ago
        It crashed all the way back to June 2024 levels, eons of progress wiped out
  • tarruda 2y ago
    It is still unconfirmed since no one outside of deepseek reproduced it. If confirmed, Nvidia could go down even more
    • Wheaties466 2y ago
      based on information and background they thoroughly gave when releasing their research its pretty easy to put together that it did take them significantly less resources to train this model. only having specific parameters available at a time instead of activating everything all at once is pretty ingenious. that and they just happened to be undergoing a large scale "cyber attack"
    • Majromax 2y ago
      I'm not sure I see the bear argument for NVidia here. Huge AI models certainly drive NVidia sales, but huge AI models are also widely thought to be untrainable and nearly un-runnable save for large datacenters. To me, this is ripe for an application of the Jevons paradox. If architectural improvements make similar models cheaper, I would expect to see more of them trained and deployed, not fewer, ultimately increasing the market for GPU-like hardware.
  • willsmith72 2y ago
    Because the markets are rational, all-knowing, and have never been wrong?
    • huijzer 2y ago
      That was not the question.
      • mcv 2y ago
        It is if you're using market movements as evidence of anything factual. If markets aren't rational, you can't use them that way.
        • whimsicalism 2y ago
          do you only take advice/learn from all-knowing people?
          • mcv 2y ago
            Do you know any? But here's my advice: drop the fallacious arguments and try something more honest.
            • whimsicalism 2y ago
              my argument isn’t fallacious - it is logical: we can learn/use evidence from something without presuming it is all knowing. you are putting words in others mouths that they did not say
              • mcv 2y ago
                I'm sorry, I thought you introduced the "all-knowing" out of nowhere, but this was indeed mentioned by willsmith72. I'd missed that. Still, his implied assertion that markets that markets can often behave irrationally, and can't be used as evidence of technical matters, seems pretty valid to me. But I suppose you could see it as a sign that something is at least temporarily "generally accepted" among investors. That doesn't mean it's generally accepted among AI researchers, though. Although I thought it was $6M rather than $5M, and that that was only the last step, and not the total investment. What does seem to be generally accepted among investors that this isn't good news for NVidia's profits, but that still doesn't mean that all the specific facts are generally accepted.
  • infecto 2y ago
    While Deepseek was an instigator in the price movements I would not say its accepted.
  • afavour 2y ago
    I don’t see them as related. The market moves when there is money to be made. It’s only tangentially related to any kind of general sentiment. “I don’t believe this, but I know others will, so I’m selling”
• Kye 2y ago
  Huggingface is working on reproducing it: https://github.com/huggingface/open-r1 https://github.com/huggingface/open-r1
• deleted 2y ago
  [deleted]
• infecto 2y ago
  I think the jury is out. With folks trying to replicate the process we will see if the low budget is true or not. I am still on the fence, there was comments from Scale CEO that they have a huge number of H100s they used. On the market side I think regardless if this was true or not, this gave people the opportunity to sell what is perhaps overinflated valuations.
• dinosaurdynasty 2y ago
  That's likely only the marginal cost of training this model, and doesn't include a lot of other costs, like the datacenters and GPUs themselves which they already had and also the staff. If they aren't lying because they have hardware they're not supposed to have, which is also a possibility.
  • whimsicalism 2y ago
    these claims are getting more wrong every time i see them, weird game of telephone going around tech circles. the cost absolutely includes the cost of GPUs and data centers, they quoted a standard price for renting h800 which has all of this built in. but yes, as very explicitly noted in the paper, it does not include cost of test iterations
• whimsicalism 2y ago
  r1 probably cost way less to train, $5m is the alleged price tag for dsv3
• ryao 2y ago
  They claimed that it only took $5 million of GPUs to train Deepseek v3, which was the base model. They did not claim that the total costs were $5 million. They omitted the costs of additional hardware, electricity, personnel, training dataset acquisition, etcetera. They likely spent tens of times more on this at a minimum. That said, what they did with $5 million of GPUs is impressive. Reportedly, they resorted to using PTX assembly to make it possible: https://www.tomshardware.com/tech-industry/artificial-intelligence/deepseeks-ai-breakthrough-bypasses-industry-standard-cuda-uses-assembly-like-ptx-programming-instead https://www.tomshardware.com/tech-industry/artificial-intell...