9 ms·

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived va

The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better.

gumperbumper 2y ago
The US economy is predicated on the perception that AI requires a lot of GPUs? That seems like a stretch.
refulgentis 2y ago
I've been confused over this. I've seen a $5.5M # for training, and commensurate commentary along the lines of what you said, but it elides the cost of the base model AFAICT.
- m_a_g 2y ago
  With $5.5M, you can buy around 150 H100s. Experts correct me if I’m wrong but it’s practically impossible to train a model like that with that measly amount. So I doubt that figure includes all the cost of training.
  - logicchains 2y ago
    The cost, as expressed in the DeepSeek V3 paper, was expressed in terms of training hours based on the market rate per hour if they'd rented the 2k GPUs they used.
  - etc-hosts 2y ago
    It's even more. You also need to fund power and maintain infrastructure to run the GPUs. You need to build fast networks between the GPUs for RDMA. Ethernet is going to be too slow. Infiniband is unreliable and expensive.
    - FridgeSeal 2y ago
      You’ll also need sufficient storage, and fast IO to keep them fed with data. You also need to keep the later generation cards from burning themselves out because they draw so much. Oh also, depending on when your data centre was built, you may also need them to upgrade their power and cooling capabilities because the new cards draw _so much_.
  - nejsjsjsbsb 2y ago
    Is it a fine tune effectively?
    - refulgentis 2y ago
      No, it's a full model. It's just...most concisely, it doesn't include the actual costs. Claude gave me a good analogy, been struggling for hours: its like only accounting for the gas grill bill when pricing your meals as a restaurant owner The thing is, that elides a lot, and you could argue it out and theoratically no one would be wrong. But $5.5 million elides so much info as to be silly. ex. they used 2048 H100 GPUs for 2 months. That's $72 million. And we're still not even approaching the real bill for the infrastructure. And for every success, there's another N that failed, 2 would be an absurdly conservative estimate. People are reading the # and thinking it says something about American AI lab efficiency, rather, it says something about how fast it is to copy when you can scaffold by training on another model's outputs. That's not a bad thing, or at least, a unique phenomena. That's why it's hard talking about this IMHO
- logicchains 2y ago
  $5.5 million is the cost of training the base model, DeepSeek V3. I haven't seen numbers for how much extra the reinforcement learning that turned it into R1 cost.
  - refulgentis 2y ago
    Ahhh, ty ty.
logicchains 2y ago
>I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better. DeepSeek's R1 also blew all the other China LLM teams out of the water, in spite of their larger training budgets and greater hardware resources (e.g. Alibaba). I suspect it's because its creators' background in a trading firm made them more willing to take calculated risks and incorporate all the innovations that made R1 such a success, rather than just copying what other teams are doing with minimal innovation.
jvanderbot 2y ago
How likely is this? Just a cursory probing of deepseek yields all kinds of censoring of topics. Isn't it just as likely Chinese sponsors of this have incentivized and sponsored an undercutting of prices so that a more favorable LLM is preferred on the market? Think about it, this is something they are willing to do with other industries. And, if LLMs are going to be engineering accelerators as the world believes, then it wouldn't do to have your software assistants be built with a history book they didn't write. Better to dramatically subsidize your own domestic one then undercut your way to dominance. It just so happens deepseek is the best one, but whichever was the best Chinese sponsored LLM would be the one we're supposed to use.
- refulgentis 2y ago
  You raise an interesting point, and both of your points seem well-founded and have wide cache. However, I strongly believe both points are in error. - OP elides costs of anything at all outside renting GPUs, and they purchased them, paid GPT-4 to generate training data, etc. etc. - Non-Qwen models they trained are happy to talk about ex. Tiananmen
- logicchains 2y ago
  >Isn't it just as likely Chinese sponsors of this have incentivized and sponsored an undercutting of prices so that a more favorable LLM is preferred on the market? Since the model is open weights, it's easy to estimate the cost of serving it. If the cost was significantly higher than DeepSeek charges on their API, we'd expect other LLM hosting providers to charge significantly more for DeepSeek (since they aren't subsidised, so need to cover their costs), but that isn't the case. This isn't possible with OpenAI because we don't know the size or architecture of their models. Regarding censorship, most of it is done at the API level, not the model level, so running locally (or with another hosting provider) is much less expensive.
- siltcakes 2y ago
  I trust China a lot more than Meta and my own early tests do indeed show that Deepseek is far less censored than Llama.
tayo42 2y ago
More effecient use of hardware just increases productivity. No more people/teams can interate faster and in parralel
thelastparadise 2y ago
But do we know that the same techniques won't scale if trained in the huge clusters?
pdntspa 2y ago
From what I've read, DeepSeek is a "side project" at a Chinese quant fund. They had the GPU capacity to spare.
- browningstreet 2y ago
  I’ve read that too, and if true, and their strongest skill and output resides elsewhere, that would point to other interesting… impacts.
leetharris 2y ago
CEO of Scale said Deepseek is lying and actually has a 50k GPU cluster. He said they lied in the paper because technically they aren't supposed to have them due to export laws. I feel like this is very likely. They obvious did some great breakthroughs, but I doubt they were able to train on so much less hardware.
- pdntspa 2y ago
  I would think the CEO of an American AI company has every reason to neg and downplay foreign competition... And since it's a businessperson they're going to make it sound as cute and innocuous as possible
  - stale2002 2y ago
    Or, more likely, there wasn't a magic innovation that nobody else thought of, that reduced costs by orders of magnitude. When deciding between mostly like scenarios, it is more likely that the company lied than they found some industry changing magic innovation.
  - pjfin123 2y ago
    It's hard to tell if they're telling the truth about the number of GPUs they have. They open sourced the model and the inference is much more efficient than the best American models so it's not implausible that the training was also much more efficient.
  - leetharris 2y ago
    If we're going to play that card, couldn't we also use the "Chinese CEO has every reason to lie and say they did something 100x more efficient than the Americans" card? I'm not even saying they did it maliciously, but maybe just to avoid scrutiny on GPUs they aren't technically supposed to have? I'm thinking out loud, not accusing anyone of anything.
    - mrbungie 2y ago
      Then the question becomes, who sold the GPUs to them? They are supposedly scarse and every player in the field is trying to get ahold as many as they can, before anyone else in fact. Something makes little sense in the accusations here.
Leary 2y ago
or maybe the US economy will do even better because more people will be able to use AI at a low cost. OpenAI will be also be able to serve o3 at a lower cost if Deepseek had some marginal breakthrough OpenAI did not already think of.
- 7thpower 2y ago
  I think this is the most productive mindset. All of the costs thus far are sunk, the only move forward is to learn and adjust. This is a net win for nearly everyone. The world needs more tokens and we are learning that we can create higher quality tokens with fewer resources than before. Finger pointing is a very short term strategy.
rikafurude21 2y ago
Why do americans think china is like a hivemind controlled by an omnisicient Xi, making strategic moves to undermine them? Is it really that unlikely that a lab of genius engineers found a way to improve efficiency 10x?
- wumeow 2y ago
  Because that’s the way China presents itself and that’s the way China boosters talk about China.
- deleted 2y ago
  [deleted]
- mritchie712 2y ago
  think about how big the prize is, how many people are working on it and how much has been invested (and targeted to be invested, see stargate). And they somehow yolo it for next to nothing? yes, it seems unlikely they did it exactly they way they're claiming they did. At the very least, they likely spent more than they claim or used existing AI API's in way that's against the terms.
- logicchains 2y ago
  > Is it really that unlikely that a lab of genius engineers found a way to improve efficiency 10x They literally published all their methodology. It's nothing groundbreaking, just western labs seem slow to adopt new research. Mixture of experts, key-value cache compression, multi-token prediction, 2/3 of these weren't invented by DeepSeek. They did invent a new hardware-aware distributed training approach for mixture-of-experts training that helped a lot, but there's nothing super genius about it, western labs just never even tried to adjust their model to fit the hardware available.
  - blackeyeblitzar 2y ago
    But those approaches alone wouldn’t yield the improvements claimed. How did they train the foundational model upon which they applied RL, distillations, etc? That part is unclear and I don’t think anything they’ve released anything that explains the low cost. It’s also curious why some people are seeing responses where it thinks it is an OpenAI model. I can’t find the post but someone had shared a link to X with that in one of the other HN discussions.
robertclaus 2y ago
Doesn't this just mean throwing a gazillion GPUs at the new architecture and defining a new SOTA?
eightysixfour 2y ago
I don't believe that the model was trained on so few GPUs, personally, but it also doesn't matter IMO. I don't think SOTA models are moats, they seem to be more like guiding lights that others can quickly follow. The volume of research on different approaches says we're still in the early days, and it is highly likely we continue to get surprises with models and systems that make sudden, giant leaps. Many "haters" seem to be predicting that there will be model collapse as we run out of data that isn't "slop," but I think they've got it backwards. We're in the flywheel phase now, each SOTA model makes future models better, and others catch up faster.
- eightysixfour 2y ago
  I take back my comment. It seem plausible that they took their model and made it reason for the costs based on this: https://hkust-nlp.notion.site/simplerl-reason https://hkust-nlp.notion.site/simplerl-reason
blackeyeblitzar 2y ago
It’s not just the economy that is vulnerable, but global geopolitics. It’s definitely worrying to see this type of technology in the hands of an authoritarian dictatorship, especially considering the evidence of censorship. See this article for a collected set of prompts and responses from DeepSeek highlighting the propaganda: https://medium.com/the-generator/deepseek-hidden-china-political-bias-5d838bbf3ef9 https://medium.com/the-generator/deepseek-hidden-china-polit... But also the claimed cost is suspicious. I know people have seen DeepSeek claim in some responses that it is one of the OpenAI models, so I wonder if they somehow trained using the outputs of other models, if that’s even possible (is there such a technique?). Maybe that’s how the claimed cost is so low that it doesn’t make mathematical sense?
- rightbyte 2y ago
  I am certainly reliefed there is no super power lock in for this stuff. In theory I could run this one at home too without giving my data or money to Sam Altman.
- buyucu 2y ago
  have you tried asking chatgpt something even slightly controversial? chatgpt censors much more than deepseek does. also deepseek is open-weights. there is nothing preventing you from doing a finetune that removes the censorship. they did that with llama2 back in the day.
  - blackeyeblitzar 2y ago
    > chatgpt censors much more than deepseek does This is an outrageous claim with no evidence, as if there was any equivalence between government enforced propaganda and anything else. Look at the system prompts for DeepSeek and it’s even more clear. Also: fine tuning is not relevant when what is deployed at scale brainwashes the masses through false and misleading responses.
    - buyucu 2y ago
      [flagged]
deleted 2y ago
[deleted]
ak_111 2y ago
Would you say they were more vulnerable if the PRC kept it secret so as not to disclose their edge in AI while continuing to build on it?
tomjen3 2y ago
We will know soon enough if this replicates since Huggingface is working on replicating it. To know that this would work requires insanely deep technical knowledge about state of the art computing, and the top leadership of the PRC does not have that.
- handzhiev 2y ago
  Researchers from TikTok claim they already replicated it https://x.com/sivil_taram/status/1883184784492666947?t=NzFZjU4tSHmiK3z_jcRFXw&s=19 https://x.com/sivil_taram/status/1883184784492666947?t=NzFZj...
  - astrange 2y ago
    IIRC they clarified that they'd already been doing something similar, and just called it a replication since it's now second to publish.
ecocentrik 2y ago
I don't think we were wrong to look at this as a commodity problem and ask how many widgets we need. Most people will still get their access to this technology through cloud services and nothing in this paper changes the calculations for inference compute demand. I still expect inference compute demand to be massive and distilled models aren't going to cut it for most agentic use cases.
pfisherman 2y ago
> The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I do not quite follow. GPU compute is mostly spent in inference, as training is a one time cost. And these chain of thought style models work by scaling up inference time compute, no? So proliferation of these types of models would portend in increase in demand for GPUs?
- miohtama 2y ago
  As far as I understand the model needs way less active parameters, reducing GPU cost in inference.
  - Alive-in-2025 2y ago
    If you don't need so many gpu calcs regardless of how you get there, maybe nvidia loses money from less demand (or stock price), or there are more wasted power companies in the middle of no where (extremely likely), and maybe these dozen doofus almost trillion dollar ai companies also out on a few 100 billion of spending. So it's not the end of the world. Look at the efficiency of databases from the mid 1970s to now. We have figured out so many optimizations and efficiencies and better compression and so forth. We are just figuring out what parts of these systems are needed.
    - mrbungie 2y ago
      Hyperscalers need to justify their current GPU investments with pay2go and provisioned throughput LLM usage revenue. If models get more efficient too quickly and therefore GPUs less loaded by end users, short of a strong example of Jevon's paradox they might not reach their revenue targets for the next years. They bought them at "you need a lot of these" prices, but now there is the possibility they are going to rent them at "I dont need this so much" rates.
cedws 2y ago
Good. This gigantic hype cycle needs a reality check. And if it turns out Deepseek is hiding GPUs, good for them for doing what they need to do to get ahead.
buyucu 2y ago
Seeing what china is doing to the car market, I give it 5 years for China to do to the AI/GPU market to do the same. This will be good. Nvidia/OpenAI monopoly is bad for everyone. More competition will be welcome.
- mrbungie 2y ago
  That is not going to happen without currently embargo'ed litography tech. They'd be already making more powerful GPUs if they could right now.
  - buyucu 2y ago
    they seem to be doing fine so far. every day we wake up to more success stories from china's AI/semiconductory industry.
    - mrbungie 2y ago
      I only know about Moore Threads GPUs. Last time I took a look at their consumer offerings (e.g. MTT S80 - S90), they were at GTX1650-1660 or around the latest AMD APU performance levels.
    - astrange 2y ago
      That's at a lower standard. If they can't do EUV they can't catch up, and they can't do EUV.
      - buyucu 2y ago
        what makes you think they can't do it? they had no trouble doing everything else. they'll be fine: https://www.msn.com/en-us/news/technology/huawei-smic-to-build-5nm-chip-without-euv-machines/ar-BB1muckE https://www.msn.com/en-us/news/technology/huawei-smic-to-bui...
        astrange 2y ago
        Because they're using DUV machines someone else sold them. They didn't build it themselves.
flaque 2y ago
This only makes sense if you think scaling laws won't hold. If someone gets something to work with 1k h100s that should have taken 100k h100s, that means the group with the 100k is about to have a much, much better model.
aprilthird2021 2y ago
> If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. Correct me if I'm wrong, but couldn't you take the optimization and tricks for training, inference, etc. from this model and apply to the Big Corps' huge AI data centers and get an even better model? I'll preface this by saying, better and better models may not actually unlock the economic value they are hoping for. It might be a thing where the last 10% takes 90% of the effort so to speak
deleted 2y ago
[deleted]

gumperbumper 2y ago

The US economy is predicated on the perception that AI requires a lot of GPUs? That seems like a stretch.

refulgentis 2y ago

I've been confused over this. I've seen a $5.5M # for training, and commensurate commentary along the lines of what you said, but it elides the cost of the base model AFAICT.

m_a_g 2y ago
With $5.5M, you can buy around 150 H100s. Experts correct me if I’m wrong but it’s practically impossible to train a model like that with that measly amount. So I doubt that figure includes all the cost of training.
- logicchains 2y ago
  The cost, as expressed in the DeepSeek V3 paper, was expressed in terms of training hours based on the market rate per hour if they'd rented the 2k GPUs they used.
- etc-hosts 2y ago
  It's even more. You also need to fund power and maintain infrastructure to run the GPUs. You need to build fast networks between the GPUs for RDMA. Ethernet is going to be too slow. Infiniband is unreliable and expensive.
  - FridgeSeal 2y ago
    You’ll also need sufficient storage, and fast IO to keep them fed with data. You also need to keep the later generation cards from burning themselves out because they draw so much. Oh also, depending on when your data centre was built, you may also need them to upgrade their power and cooling capabilities because the new cards draw _so much_.
- nejsjsjsbsb 2y ago
  Is it a fine tune effectively?
  - refulgentis 2y ago
    No, it's a full model. It's just...most concisely, it doesn't include the actual costs. Claude gave me a good analogy, been struggling for hours: its like only accounting for the gas grill bill when pricing your meals as a restaurant owner The thing is, that elides a lot, and you could argue it out and theoratically no one would be wrong. But $5.5 million elides so much info as to be silly. ex. they used 2048 H100 GPUs for 2 months. That's $72 million. And we're still not even approaching the real bill for the infrastructure. And for every success, there's another N that failed, 2 would be an absurdly conservative estimate. People are reading the # and thinking it says something about American AI lab efficiency, rather, it says something about how fast it is to copy when you can scaffold by training on another model's outputs. That's not a bad thing, or at least, a unique phenomena. That's why it's hard talking about this IMHO
logicchains 2y ago
$5.5 million is the cost of training the base model, DeepSeek V3. I haven't seen numbers for how much extra the reinforcement learning that turned it into R1 cost.
- refulgentis 2y ago
  Ahhh, ty ty.

logicchains 2y ago

>I wonder if this was a deliberate move by PRC or really our own fault in falling for the fallacy that more is always better. DeepSeek's R1 also blew all the other China LLM teams out of the water, in spite of their larger training budgets and greater hardware resources (e.g. Alibaba). I suspect it's because its creators' background in a trading firm made them more willing to take calculated risks and incorporate all the innovations that made R1 such a success, rather than just copying what other teams are doing with minimal innovation.

jvanderbot 2y ago

How likely is this? Just a cursory probing of deepseek yields all kinds of censoring of topics. Isn't it just as likely Chinese sponsors of this have incentivized and sponsored an undercutting of prices so that a more favorable LLM is preferred on the market? Think about it, this is something they are willing to do with other industries. And, if LLMs are going to be engineering accelerators as the world believes, then it wouldn't do to have your software assistants be built with a history book they didn't write. Better to dramatically subsidize your own domestic one then undercut your way to dominance. It just so happens deepseek is the best one, but whichever was the best Chinese sponsored LLM would be the one we're supposed to use.

refulgentis 2y ago
You raise an interesting point, and both of your points seem well-founded and have wide cache. However, I strongly believe both points are in error. - OP elides costs of anything at all outside renting GPUs, and they purchased them, paid GPT-4 to generate training data, etc. etc. - Non-Qwen models they trained are happy to talk about ex. Tiananmen
logicchains 2y ago
>Isn't it just as likely Chinese sponsors of this have incentivized and sponsored an undercutting of prices so that a more favorable LLM is preferred on the market? Since the model is open weights, it's easy to estimate the cost of serving it. If the cost was significantly higher than DeepSeek charges on their API, we'd expect other LLM hosting providers to charge significantly more for DeepSeek (since they aren't subsidised, so need to cover their costs), but that isn't the case. This isn't possible with OpenAI because we don't know the size or architecture of their models. Regarding censorship, most of it is done at the API level, not the model level, so running locally (or with another hosting provider) is much less expensive.
siltcakes 2y ago
I trust China a lot more than Meta and my own early tests do indeed show that Deepseek is far less censored than Llama.

tayo42 2y ago

More effecient use of hardware just increases productivity. No more people/teams can interate faster and in parralel

thelastparadise 2y ago

But do we know that the same techniques won't scale if trained in the huge clusters?

pdntspa 2y ago

From what I've read, DeepSeek is a "side project" at a Chinese quant fund. They had the GPU capacity to spare.

browningstreet 2y ago
I’ve read that too, and if true, and their strongest skill and output resides elsewhere, that would point to other interesting… impacts.

leetharris 2y ago

CEO of Scale said Deepseek is lying and actually has a 50k GPU cluster. He said they lied in the paper because technically they aren't supposed to have them due to export laws. I feel like this is very likely. They obvious did some great breakthroughs, but I doubt they were able to train on so much less hardware.

pdntspa 2y ago
I would think the CEO of an American AI company has every reason to neg and downplay foreign competition... And since it's a businessperson they're going to make it sound as cute and innocuous as possible
- stale2002 2y ago
  Or, more likely, there wasn't a magic innovation that nobody else thought of, that reduced costs by orders of magnitude. When deciding between mostly like scenarios, it is more likely that the company lied than they found some industry changing magic innovation.
- pjfin123 2y ago
  It's hard to tell if they're telling the truth about the number of GPUs they have. They open sourced the model and the inference is much more efficient than the best American models so it's not implausible that the training was also much more efficient.
- leetharris 2y ago
  If we're going to play that card, couldn't we also use the "Chinese CEO has every reason to lie and say they did something 100x more efficient than the Americans" card? I'm not even saying they did it maliciously, but maybe just to avoid scrutiny on GPUs they aren't technically supposed to have? I'm thinking out loud, not accusing anyone of anything.
  - mrbungie 2y ago
    Then the question becomes, who sold the GPUs to them? They are supposedly scarse and every player in the field is trying to get ahold as many as they can, before anyone else in fact. Something makes little sense in the accusations here.

Leary 2y ago

or maybe the US economy will do even better because more people will be able to use AI at a low cost. OpenAI will be also be able to serve o3 at a lower cost if Deepseek had some marginal breakthrough OpenAI did not already think of.

7thpower 2y ago
I think this is the most productive mindset. All of the costs thus far are sunk, the only move forward is to learn and adjust. This is a net win for nearly everyone. The world needs more tokens and we are learning that we can create higher quality tokens with fewer resources than before. Finger pointing is a very short term strategy.

rikafurude21 2y ago

Why do americans think china is like a hivemind controlled by an omnisicient Xi, making strategic moves to undermine them? Is it really that unlikely that a lab of genius engineers found a way to improve efficiency 10x?

wumeow 2y ago
Because that’s the way China presents itself and that’s the way China boosters talk about China.
deleted 2y ago
[deleted]
mritchie712 2y ago
think about how big the prize is, how many people are working on it and how much has been invested (and targeted to be invested, see stargate). And they somehow yolo it for next to nothing? yes, it seems unlikely they did it exactly they way they're claiming they did. At the very least, they likely spent more than they claim or used existing AI API's in way that's against the terms.
logicchains 2y ago
> Is it really that unlikely that a lab of genius engineers found a way to improve efficiency 10x They literally published all their methodology. It's nothing groundbreaking, just western labs seem slow to adopt new research. Mixture of experts, key-value cache compression, multi-token prediction, 2/3 of these weren't invented by DeepSeek. They did invent a new hardware-aware distributed training approach for mixture-of-experts training that helped a lot, but there's nothing super genius about it, western labs just never even tried to adjust their model to fit the hardware available.
- blackeyeblitzar 2y ago
  But those approaches alone wouldn’t yield the improvements claimed. How did they train the foundational model upon which they applied RL, distillations, etc? That part is unclear and I don’t think anything they’ve released anything that explains the low cost. It’s also curious why some people are seeing responses where it thinks it is an OpenAI model. I can’t find the post but someone had shared a link to X with that in one of the other HN discussions.

robertclaus 2y ago

Doesn't this just mean throwing a gazillion GPUs at the new architecture and defining a new SOTA?

eightysixfour 2y ago

I don't believe that the model was trained on so few GPUs, personally, but it also doesn't matter IMO. I don't think SOTA models are moats, they seem to be more like guiding lights that others can quickly follow. The volume of research on different approaches says we're still in the early days, and it is highly likely we continue to get surprises with models and systems that make sudden, giant leaps. Many "haters" seem to be predicting that there will be model collapse as we run out of data that isn't "slop," but I think they've got it backwards. We're in the flywheel phase now, each SOTA model makes future models better, and others catch up faster.

eightysixfour 2y ago
I take back my comment. It seem plausible that they took their model and made it reason for the costs based on this: https://hkust-nlp.notion.site/simplerl-reason https://hkust-nlp.notion.site/simplerl-reason

blackeyeblitzar 2y ago

It’s not just the economy that is vulnerable, but global geopolitics. It’s definitely worrying to see this type of technology in the hands of an authoritarian dictatorship, especially considering the evidence of censorship. See this article for a collected set of prompts and responses from DeepSeek highlighting the propaganda: https://medium.com/the-generator/deepseek-hidden-china-political-bias-5d838bbf3ef9 https://medium.com/the-generator/deepseek-hidden-china-polit... But also the claimed cost is suspicious. I know people have seen DeepSeek claim in some responses that it is one of the OpenAI models, so I wonder if they somehow trained using the outputs of other models, if that’s even possible (is there such a technique?). Maybe that’s how the claimed cost is so low that it doesn’t make mathematical sense?

rightbyte 2y ago
I am certainly reliefed there is no super power lock in for this stuff. In theory I could run this one at home too without giving my data or money to Sam Altman.
buyucu 2y ago
have you tried asking chatgpt something even slightly controversial? chatgpt censors much more than deepseek does. also deepseek is open-weights. there is nothing preventing you from doing a finetune that removes the censorship. they did that with llama2 back in the day.
- blackeyeblitzar 2y ago
  > chatgpt censors much more than deepseek does This is an outrageous claim with no evidence, as if there was any equivalence between government enforced propaganda and anything else. Look at the system prompts for DeepSeek and it’s even more clear. Also: fine tuning is not relevant when what is deployed at scale brainwashes the masses through false and misleading responses.
  - buyucu 2y ago
    [flagged]

deleted 2y ago

[deleted]

ak_111 2y ago

Would you say they were more vulnerable if the PRC kept it secret so as not to disclose their edge in AI while continuing to build on it?

tomjen3 2y ago

We will know soon enough if this replicates since Huggingface is working on replicating it. To know that this would work requires insanely deep technical knowledge about state of the art computing, and the top leadership of the PRC does not have that.

handzhiev 2y ago
Researchers from TikTok claim they already replicated it https://x.com/sivil_taram/status/1883184784492666947?t=NzFZjU4tSHmiK3z_jcRFXw&s=19 https://x.com/sivil_taram/status/1883184784492666947?t=NzFZj...
- astrange 2y ago
  IIRC they clarified that they'd already been doing something similar, and just called it a replication since it's now second to publish.

ecocentrik 2y ago

I don't think we were wrong to look at this as a commodity problem and ask how many widgets we need. Most people will still get their access to this technology through cloud services and nothing in this paper changes the calculations for inference compute demand. I still expect inference compute demand to be massive and distilled models aren't going to cut it for most agentic use cases.

pfisherman 2y ago

> The US Economy is pretty vulnerable here. If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. I do not quite follow. GPU compute is mostly spent in inference, as training is a one time cost. And these chain of thought style models work by scaling up inference time compute, no? So proliferation of these types of models would portend in increase in demand for GPUs?

miohtama 2y ago
As far as I understand the model needs way less active parameters, reducing GPU cost in inference.
- Alive-in-2025 2y ago
  If you don't need so many gpu calcs regardless of how you get there, maybe nvidia loses money from less demand (or stock price), or there are more wasted power companies in the middle of no where (extremely likely), and maybe these dozen doofus almost trillion dollar ai companies also out on a few 100 billion of spending. So it's not the end of the world. Look at the efficiency of databases from the mid 1970s to now. We have figured out so many optimizations and efficiencies and better compression and so forth. We are just figuring out what parts of these systems are needed.
  - mrbungie 2y ago
    Hyperscalers need to justify their current GPU investments with pay2go and provisioned throughput LLM usage revenue. If models get more efficient too quickly and therefore GPUs less loaded by end users, short of a strong example of Jevon's paradox they might not reach their revenue targets for the next years. They bought them at "you need a lot of these" prices, but now there is the possibility they are going to rent them at "I dont need this so much" rates.

cedws 2y ago

Good. This gigantic hype cycle needs a reality check. And if it turns out Deepseek is hiding GPUs, good for them for doing what they need to do to get ahead.

buyucu 2y ago

Seeing what china is doing to the car market, I give it 5 years for China to do to the AI/GPU market to do the same. This will be good. Nvidia/OpenAI monopoly is bad for everyone. More competition will be welcome.

mrbungie 2y ago
That is not going to happen without currently embargo'ed litography tech. They'd be already making more powerful GPUs if they could right now.
- buyucu 2y ago
  they seem to be doing fine so far. every day we wake up to more success stories from china's AI/semiconductory industry.
  - mrbungie 2y ago
    I only know about Moore Threads GPUs. Last time I took a look at their consumer offerings (e.g. MTT S80 - S90), they were at GTX1650-1660 or around the latest AMD APU performance levels.
  - astrange 2y ago
    That's at a lower standard. If they can't do EUV they can't catch up, and they can't do EUV.
    - buyucu 2y ago
      what makes you think they can't do it? they had no trouble doing everything else. they'll be fine: https://www.msn.com/en-us/news/technology/huawei-smic-to-build-5nm-chip-without-euv-machines/ar-BB1muckE https://www.msn.com/en-us/news/technology/huawei-smic-to-bui...
      - astrange 2y ago
        Because they're using DUV machines someone else sold them. They didn't build it themselves.

flaque 2y ago

This only makes sense if you think scaling laws won't hold. If someone gets something to work with 1k h100s that should have taken 100k h100s, that means the group with the 100k is about to have a much, much better model.

aprilthird2021 2y ago

> If it turns out that you, in fact, don't need a gazillion GPUs to build SOTA models it destroys a lot of perceived value. Correct me if I'm wrong, but couldn't you take the optimization and tricks for training, inference, etc. from this model and apply to the Big Corps' huge AI data centers and get an even better model? I'll preface this by saying, better and better models may not actually unlock the economic value they are hoping for. It might be a thing where the last 10% takes 90% of the effort so to speak

deleted 2y ago

[deleted]