7 ms·

“Pure garage-energy” is a great phrase. Most interested to see their inference stack, hope that’s one of the 5. I think most people are running R1 on a single

“Pure garage-energy” is a great phrase. Most interested to see their inference stack, hope that’s one of the 5. I think most people are running R1 on a single H200 node but Deepseek had much lower RAM per GPU for their inference and so had some cluster based MoE deployment.

sva_ 2y ago
I don't think the RAM size of the H800 was nerfed (80GB), but rather the memory bandwidth between gpus. But yeah, would be interesting to see how they optimized for that.
- NitpickLawyer 2y ago
  Correct. There are 3 main ways to "gimp" high end GPUs meant for training - "cores", "on-chip memory speed" and "interconnects". IIUC the H800 had the first 2 unchanged but halved the interconnect speeds. H20 is the next iteration of the "sanctions" that I believe also limited the "cores" but left the on-chip memory intact, or slightly higher (from the new generation).
golly_ned 2y ago
“Pure garage-energy” with 10,000 A100s, apparently. I’d love to have a garage like that.
- blackeyeblitzar 2y ago
  From https://semianalysis.com/2025/01/31/deepseek-debates/ https://semianalysis.com/2025/01/31/deepseek-debates/ > We believe DeepSeek has access to around 10,000 of these H800s and about 10,000 H100s. Furthermore they have orders for many more H20’s, with Nvidia having produced over 1 million of the China specific GPU in the last 9 months.
  - tw1984 2y ago
    [flagged]
    - brookst 2y ago
      Appeals to authority are so totally unconvincing. What claims from the semianalysis article do you think are false? And based on what evidence?
      - tw1984 2y ago
        [flagged]
        blackeyeblitzar 2y ago
        Semianalysis is a trusted and reputable source that has deep knowledge of the semiconductor industry and supply chains. So yes, they know. > love your joke! > Get a real life please. > Too bad that you didn't learn this back in school. Read the site guidelines, as you’re repeatedly resorting to personal attacks in your comments on discussions around DeepSeek or China. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
        tempusalaria 2y ago
        SemiAnalysis has made up many things. They claim that a small Chinese hedge fund could acquire $1bln in GPUs, with no state support, including many sanctioned chips, then trained a model optimized for a far smaller server compute size, and that they have a source at this very small fund who is willing to admit to export violations. A 40bln param active model is exactly the size you would expect from a server of the size they claim. What’s more likely - that semianalysis made it up like they have a bunch of other things, or that all the above is true?
        rfoo 2y ago
        They had their A100s back in 2021 to early 2022, well before any GPU sanction kicked in. For a few months H800 wasn't sanctioned and that's when they bought them.
  - 1oooqooq 2y ago
    that report is lazy. they assume all GPUs owned (openly reported) by the parent company (a hedge fund which claims to use those GPUs to generate trades) were used by the invested company. that's as dumb as saying coca cola have acccess to all offices of Berkshire Hathaway. likewise, all comments praising deepseek history are also misleading as the company barely exists for a year. everything is opaque marketing being repeated. just drop the off topic bla bla bla and focus on the facts and code in front of you. thanks for coming to my ted talk.
  - golly_ned 2y ago
    The paper in the repo says: “ For DL training, we deployed the Fire-Flyer 2 with 10,000 PCIe A100 GPUs“
mmoskal 2y ago
Their tech report says one inference deployment is around 400 GPUs...
- fspeech 2y ago
  You need that to optimize load balancing. Unfortunately that gain is not available to small or individual deployment.
kgh1337 2y ago
[flagged]
- tokioyoyo 2y ago
  You do realize that... it's literally their entire company right? It's pretty damn cool that they're including everyone.
  - KronisLV 2y ago
    That's around 200 names. It's cool what you can achieve with the right people!
- spaceguillotine 2y ago
  [flagged]
  - dang 2y ago
    Please don't do this here. It's against the site guidelines and just makes the threads worse. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- ramon156 2y ago
  You know you're doing well as a company when someone uses bots to boycott

sva_ 2y ago

I don't think the RAM size of the H800 was nerfed (80GB), but rather the memory bandwidth between gpus. But yeah, would be interesting to see how they optimized for that.

NitpickLawyer 2y ago
Correct. There are 3 main ways to "gimp" high end GPUs meant for training - "cores", "on-chip memory speed" and "interconnects". IIUC the H800 had the first 2 unchanged but halved the interconnect speeds. H20 is the next iteration of the "sanctions" that I believe also limited the "cores" but left the on-chip memory intact, or slightly higher (from the new generation).

golly_ned 2y ago

“Pure garage-energy” with 10,000 A100s, apparently. I’d love to have a garage like that.

blackeyeblitzar 2y ago
From https://semianalysis.com/2025/01/31/deepseek-debates/ https://semianalysis.com/2025/01/31/deepseek-debates/ > We believe DeepSeek has access to around 10,000 of these H800s and about 10,000 H100s. Furthermore they have orders for many more H20’s, with Nvidia having produced over 1 million of the China specific GPU in the last 9 months.
- tw1984 2y ago
  [flagged]
  - brookst 2y ago
    Appeals to authority are so totally unconvincing. What claims from the semianalysis article do you think are false? And based on what evidence?
    - tw1984 2y ago
      [flagged]
      - blackeyeblitzar 2y ago
        Semianalysis is a trusted and reputable source that has deep knowledge of the semiconductor industry and supply chains. So yes, they know. > love your joke! > Get a real life please. > Too bad that you didn't learn this back in school. Read the site guidelines, as you’re repeatedly resorting to personal attacks in your comments on discussions around DeepSeek or China. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
        tempusalaria 2y ago
        SemiAnalysis has made up many things. They claim that a small Chinese hedge fund could acquire $1bln in GPUs, with no state support, including many sanctioned chips, then trained a model optimized for a far smaller server compute size, and that they have a source at this very small fund who is willing to admit to export violations. A 40bln param active model is exactly the size you would expect from a server of the size they claim. What’s more likely - that semianalysis made it up like they have a bunch of other things, or that all the above is true?
        rfoo 2y ago
        They had their A100s back in 2021 to early 2022, well before any GPU sanction kicked in. For a few months H800 wasn't sanctioned and that's when they bought them.
- 1oooqooq 2y ago
  that report is lazy. they assume all GPUs owned (openly reported) by the parent company (a hedge fund which claims to use those GPUs to generate trades) were used by the invested company. that's as dumb as saying coca cola have acccess to all offices of Berkshire Hathaway. likewise, all comments praising deepseek history are also misleading as the company barely exists for a year. everything is opaque marketing being repeated. just drop the off topic bla bla bla and focus on the facts and code in front of you. thanks for coming to my ted talk.
- golly_ned 2y ago
  The paper in the repo says: “ For DL training, we deployed the Fire-Flyer 2 with 10,000 PCIe A100 GPUs“

mmoskal 2y ago

Their tech report says one inference deployment is around 400 GPUs...

fspeech 2y ago
You need that to optimize load balancing. Unfortunately that gain is not available to small or individual deployment.

kgh1337 2y ago

[flagged]

tokioyoyo 2y ago
You do realize that... it's literally their entire company right? It's pretty damn cool that they're including everyone.
- KronisLV 2y ago
  That's around 200 names. It's cool what you can achieve with the right people!
spaceguillotine 2y ago
[flagged]
- dang 2y ago
  Please don't do this here. It's against the site guidelines and just makes the threads worse. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
ramon156 2y ago
You know you're doing well as a company when someone uses bots to boycott