7 ms·

I thought this was common knowledge. DeepSeek’s Wikipedia entry says that they trained all their models on Nvidia chips procured before the U.S. embargo to Chi

I thought this was common knowledge. DeepSeek’s Wikipedia entry says that they trained all their models on Nvidia chips procured before the U.S. embargo to China on them. It wouldn’t surprise me if they continued acquiring them through, well, less than legal means. I also read somewhere (not Wikipedia) that they trained on ChatGPT, Claude, and Gemini queries, basically feeding in the output of competitor’s LLMs as training data. Kinda surprised they didn’t run into model collapse problems, but they stole their training data from other people who stole their training data from data collections that arguably stole them from content creators. It’s bandits all the way down, so adding a little smuggling to that doesn’t surprise me.

lysace 11mo ago
Singapore is where it happens.
echelon 11mo ago
> Kinda surprised they didn’t run into model collapse problems, This is just model distillation. Anyone with the expertise to build a model from scratch (which DeepSeek certainly can) can do this in a careful manner. > but they stole their training data from other people who stole their training data from data collections that arguably stole them from content creators. Bingo. I have no problem with pirates pirating other pirates. Screw OpenAI and Anthropic closed source models built from public data. The law should be that weights trained from non-owned sources should be public domain, or that any copyright holder can sue them and demand model takedown. Google and Meta are probably the only two AI companies that have a right to license massive amounts of training data from social media and user file uploads given that their ToSes grant them these rights. But even Meta is pirating stuff. Even if OpenAI and Anthropic continue pirating training data and keeping the results closed, China's open source strategy will win out in the end. It erodes the crust of value that is carefully guarded by the American giants. Everyone else will be integrating open models and hacking them apart, splicing them in new ways.
- rkagerer 11mo ago
  Google and Meta are probably the only two companies that have a right to license their training data For the sake of someone unfamiliar... Why is that? Did they pay teams of monkeys to generate their own, novel training data? Or gain explicit, opt-in permission from users who entrust them with their files/content?
  - echelon 11mo ago
    > For the sake of someone unfamiliar... Why is that? I edited my comment, but basically they both own massive social media properties (YouTube, Instagram, Facebook) or file upload sites (Google Drive, Google Photos, Gmail) and their ToSes grant them these rights. You accept these terms when you use their services. That's not great, but we are getting free services. It's in the terms. It's a whole lot better than just scraping without permission, compensation, acknowledgement, or even notice. To be clear, I have no problem with these models being built. But if they "steal" the data, the resultant model shouldn't be owned by anyone. It should be public domain and not allowed to be kept as a trade secret. And it's funny that Anthropic is trying to depress our wages by training on our code. Again - I'm fine with that - I want to work faster, and I like these models and their capabilities. But Anthropic shouldn't be able to own the models they train off of us exclusively since they didn't license or buy our data. They provided us with nothing at all.
    - rkagerer 11mo ago
      You accept these terms when you use their services. I certainly didn't*. I'd love to see litigation testing just how solid those insidious opt-in-by-default schemes are as a basis for "ownership". If they had users explicitly opt-in with a "Yes, go ahead and train on my stuff and by the way I assert that I have all the rights to grant you the same", I'd have no problem with that, and they'd have a much stronger claim. (*Before others inevitably disagree: I do opt-out of this stuff aggressively, and further send notice to companies from time to time that I don't agree to certain objectionable clauses of their ToS and they're welcome to close my account).
MangoToupe 11mo ago
> less than legal means. This is an absurd concept when it comes to international trade. Even intellectual property is mostly meaningless outside a state. Of course people will evade sanctions; what is the us going to do, invade singapore or malaysia?
- tlb 11mo ago
  Just because something is hard to enforce doesn't mean it's absurd. Embargoes aren't impossible to enforce against the foreign importer. If a foreign entity is found to have placed orders with false documents, they can be sanctioned, which can be enforced against any of their international operations. It makes it hard for them to do future business in global markets. I would not recommend violating US sanctions no matter where you are.
  - MangoToupe 11mo ago
    > Just because something is hard to enforce doesn't mean it's absurd. Expecting to strangle world markets with intellectual property as your moat is absurd. You can only fight honest competition with dishonest means for so long, and intellectual property is one of the dirtiest tricks in the book.
- PunchyHamster 11mo ago
  The companies it "stole" from broke the law in their own country while acquiring the training data; frankly sanction avoidance is lesser and arguably not even their (it's the people in US that smuggle them, nothing breaking china law afaik) crime
- themafia 11mo ago
  > This is an absurd concept when it comes to international trade. In this case it's just wrong. I don't know what people think "e-waste" recycling actually is or what happens to their "unrepairable" units after they rid themselves of them. > Even intellectual property is mostly meaningless outside a state. Interestingly the Dollar is most definitely meaningful outside of our state. I think the assumption becomes, that if this is true, then using it's power to enforce trade sanctions isn't that big a stretch. > Of course people will evade sanctions What's less clear if they should expect their government to actively help them in this evasion or not. I think the Chinese citizens are in unique international territory here. > what is the us going to do, invade singapore or malaysia? Deny our exports to them. This will cost the political donor class a lot of profits. So this is why it doesn't get done. None of this is a fait accompli. This is the result of years of intentional corruption of the core systems involved.
ComputerGuru 11mo ago
> I also read somewhere (not Wikipedia) that they trained on ChatGPT, Claude, and Gemini queries, basically feeding in the output of competitor’s LLMs as training data All the labs permitting synthetic data do that.
- rllearneratwork 11mo ago
  pretty sure it is against ToS for ChatGPT, Claude and Gemini.
  - rhines 11mo ago
    ToS didn't stop the companies that built those models and it won't stop the companies that bootstrap off them. Until an AI company eats a multi billion dollar lawsuit for unlawful data use they will continue to operate this way.
    - bluefirebrand 11mo ago
      > Until an AI company eats a multi billion dollar lawsuit for unlawful data use they will continue to operate this way If only. That's my dream, massive copyright lawsuits against all of these AI players and maybe the courts can do something good for a change, put an end to all of this AI bullshit
    - nandomrumber 11mo ago
      Didn’t Anthropic already eat a $1.5 billion lawsuit?
  - codedokode 11mo ago
    Did OpenAI observe any ToS when scraping content from the Internet? Sorry, but you cannot complain about stealing the stolen. OpenAI cannot even have the copyright on ChatGPT output, because it's a tool.
  - vkou 11mo ago
    It's against my ToS for ChatGPT, Claude, and Gemini to train their models on my writing, but they've all done that. They are free to pound sand.
ComputerGuru 11mo ago
> It’s bandits all the way down, so adding a little smuggling to that doesn’t surprise me. Implying it’s *morally* wrong for a Chinese company to bypass US sanctions is hilarious. You really say that with a straight face when even the president admits this is only protectionism?
- qoez 11mo ago
  [flagged]
  - Spooky23 11mo ago
    Is it morally OK as long as they pay their 25% vig to POTUS? The concept of morality in this context is absurd.
  - deleted 11mo ago
    [deleted]
  - behringer 11mo ago
    I like how you're careful to use the "west" because if you had said US your argument would have been laughable.
    - forinti 11mo ago
      Most European countries did awful things during their colonial years, some late into the XX century. Some still do shady things in Africa right now. And they all welcome money siphoned off peripheral countries by crooks. Nobody can claim moral superiority.
  - platevoltage 11mo ago
    Give me 1 reason why the USA, the (for now) leader of "the west", is morally superior to China?
    - nish__ 11mo ago
      [flagged]
KumaBear 11mo ago
Lets take the weapons embargos placed on Israel by our allies. In the NDAA must pass bill we set funds aside to procure those weapons and sell them to Israel. We don't really care about these things we have selective enforcement.
htrp 11mo ago
Everyone trains on queries from other models, it's called distillation
- LogicFailsMe 11mo ago
  Well, given AI content cannot be copyrighted, haters can hate hate hate hate hate...
  - halJordan 11mo ago
    That's a little disingenuous. If i buy a printer and use the ink in the cartridge to reverse engineer a beautiful red, have i stolen something from the printer manufacturer? Especially if they lose business because they no longer have what distinguished them? Clean room design is not new (or illegal), but it's always been a form of stealing
epolanski 11mo ago
Lol @ quoting 3 companies that broke any possible copyright law as victims.
coliveira 11mo ago
They don't need to break any laws for this. Where do you think are the customers for data centers in the Middle East? Chinese companies do everything legally, paying for access to data centers that got the chips directly from the US.
atleastoptimal 11mo ago
training on data isn’t stealing the data, in the same way learning from a textbook doesn’t mean youre stealing from it
sh34r 11mo ago
There’s no honor among thieves. You don’t get to cry about Chinese “bandits” when Anthropic just had to pay $1 billion to settle a massive copyright infringement lawsuit. All of these models were created through the mass-scale theft of humanity’s intellectual property, personal data, and dignity. Open always beats closed. Drain the moats. Starve the ClosedAI beast.
SpaceManNabs 11mo ago
i find it weird that this comment got so much pushback. i don't think it was portraying deepseek as any more morally wrong than anyone else, or castigating anyone as morally wrong. but maybe i gave it a gracious reading.
nsoonhui 11mo ago
I think the fact that DeepSeek trains on competitor queries (i.e., distillation) — along with using banned Nvidia chips — helps explain how it can achieve such low training costs (USD 6 million vs. billions) while delivering only slightly worse performance than its American counterparts. It also undermines the narrative that DeepSeek or China is posing a serious challenge to the U.S. lead in AI. The gap may be closing, but the initial reactions now seem knee-jerk. That the discussion has being hijacked and shifted to moral superiority is really unfortunate, because that was never the point in the first place.
- whimsicalism 11mo ago
  These models never cost billions to train and I doubt the final training run for models like GPT-4 cost more than 8 figures. 6 million is definitely cheaper and I would attribute that to distillation.
InkCanon 11mo ago
It's not even stealing. They paid OpenAI for the tokens. It violates the OpenAI TOS, which specifically forbids using it's outputs for training competing models (which is very ironic)
WhyNotHugo 11mo ago
> It wouldn’t surprise me if they continued acquiring them through, well, less than legal means. Strictly speaking, it's not illegal for them to acquire it, it's illegal for an exporter in the US to sell (even if transitively) to them.
Aqua0 11mo ago
"The TBD group is using several third-party models as part of the training process for Avocado, distilling from rival models including Google’s Gemma, OpenAI’s gpt-oss and Qwen, a model from the Chinese tech giant Alibaba Group Holding Ltd., the people said." LOL. Distillation doesn't count as plagiarism, or you should call Meta out on it. They're distilling the Chinese model. Ref: https://www.moneycontrol.com/news/business/inside-meta-s-pivot-from-open-source-to-money-making-ai-model-13720231.html https://www.moneycontrol.com/news/business/inside-meta-s-piv...
bfeynman 11mo ago
| Kinda surprised they didn’t run into model collapse problems Not sure why you would expect this, all the models started doing this as its much more cost effective to get data for post training don't you remember the first grok release where many times it started replies "as a model trained by openai..."

lysace 11mo ago

Singapore is where it happens.

echelon 11mo ago

> Kinda surprised they didn’t run into model collapse problems, This is just model distillation. Anyone with the expertise to build a model from scratch (which DeepSeek certainly can) can do this in a careful manner. > but they stole their training data from other people who stole their training data from data collections that arguably stole them from content creators. Bingo. I have no problem with pirates pirating other pirates. Screw OpenAI and Anthropic closed source models built from public data. The law should be that weights trained from non-owned sources should be public domain, or that any copyright holder can sue them and demand model takedown. Google and Meta are probably the only two AI companies that have a right to license massive amounts of training data from social media and user file uploads given that their ToSes grant them these rights. But even Meta is pirating stuff. Even if OpenAI and Anthropic continue pirating training data and keeping the results closed, China's open source strategy will win out in the end. It erodes the crust of value that is carefully guarded by the American giants. Everyone else will be integrating open models and hacking them apart, splicing them in new ways.

rkagerer 11mo ago
Google and Meta are probably the only two companies that have a right to license their training data For the sake of someone unfamiliar... Why is that? Did they pay teams of monkeys to generate their own, novel training data? Or gain explicit, opt-in permission from users who entrust them with their files/content?
- echelon 11mo ago
  > For the sake of someone unfamiliar... Why is that? I edited my comment, but basically they both own massive social media properties (YouTube, Instagram, Facebook) or file upload sites (Google Drive, Google Photos, Gmail) and their ToSes grant them these rights. You accept these terms when you use their services. That's not great, but we are getting free services. It's in the terms. It's a whole lot better than just scraping without permission, compensation, acknowledgement, or even notice. To be clear, I have no problem with these models being built. But if they "steal" the data, the resultant model shouldn't be owned by anyone. It should be public domain and not allowed to be kept as a trade secret. And it's funny that Anthropic is trying to depress our wages by training on our code. Again - I'm fine with that - I want to work faster, and I like these models and their capabilities. But Anthropic shouldn't be able to own the models they train off of us exclusively since they didn't license or buy our data. They provided us with nothing at all.
  - rkagerer 11mo ago
    You accept these terms when you use their services. I certainly didn't*. I'd love to see litigation testing just how solid those insidious opt-in-by-default schemes are as a basis for "ownership". If they had users explicitly opt-in with a "Yes, go ahead and train on my stuff and by the way I assert that I have all the rights to grant you the same", I'd have no problem with that, and they'd have a much stronger claim. (*Before others inevitably disagree: I do opt-out of this stuff aggressively, and further send notice to companies from time to time that I don't agree to certain objectionable clauses of their ToS and they're welcome to close my account).

MangoToupe 11mo ago

> less than legal means. This is an absurd concept when it comes to international trade. Even intellectual property is mostly meaningless outside a state. Of course people will evade sanctions; what is the us going to do, invade singapore or malaysia?

tlb 11mo ago
Just because something is hard to enforce doesn't mean it's absurd. Embargoes aren't impossible to enforce against the foreign importer. If a foreign entity is found to have placed orders with false documents, they can be sanctioned, which can be enforced against any of their international operations. It makes it hard for them to do future business in global markets. I would not recommend violating US sanctions no matter where you are.
- MangoToupe 11mo ago
  > Just because something is hard to enforce doesn't mean it's absurd. Expecting to strangle world markets with intellectual property as your moat is absurd. You can only fight honest competition with dishonest means for so long, and intellectual property is one of the dirtiest tricks in the book.
PunchyHamster 11mo ago
The companies it "stole" from broke the law in their own country while acquiring the training data; frankly sanction avoidance is lesser and arguably not even their (it's the people in US that smuggle them, nothing breaking china law afaik) crime
themafia 11mo ago
> This is an absurd concept when it comes to international trade. In this case it's just wrong. I don't know what people think "e-waste" recycling actually is or what happens to their "unrepairable" units after they rid themselves of them. > Even intellectual property is mostly meaningless outside a state. Interestingly the Dollar is most definitely meaningful outside of our state. I think the assumption becomes, that if this is true, then using it's power to enforce trade sanctions isn't that big a stretch. > Of course people will evade sanctions What's less clear if they should expect their government to actively help them in this evasion or not. I think the Chinese citizens are in unique international territory here. > what is the us going to do, invade singapore or malaysia? Deny our exports to them. This will cost the political donor class a lot of profits. So this is why it doesn't get done. None of this is a fait accompli. This is the result of years of intentional corruption of the core systems involved.

ComputerGuru 11mo ago

> I also read somewhere (not Wikipedia) that they trained on ChatGPT, Claude, and Gemini queries, basically feeding in the output of competitor’s LLMs as training data All the labs permitting synthetic data do that.

rllearneratwork 11mo ago
pretty sure it is against ToS for ChatGPT, Claude and Gemini.
- rhines 11mo ago
  ToS didn't stop the companies that built those models and it won't stop the companies that bootstrap off them. Until an AI company eats a multi billion dollar lawsuit for unlawful data use they will continue to operate this way.
  - bluefirebrand 11mo ago
    > Until an AI company eats a multi billion dollar lawsuit for unlawful data use they will continue to operate this way If only. That's my dream, massive copyright lawsuits against all of these AI players and maybe the courts can do something good for a change, put an end to all of this AI bullshit
  - nandomrumber 11mo ago
    Didn’t Anthropic already eat a $1.5 billion lawsuit?
- codedokode 11mo ago
  Did OpenAI observe any ToS when scraping content from the Internet? Sorry, but you cannot complain about stealing the stolen. OpenAI cannot even have the copyright on ChatGPT output, because it's a tool.
- vkou 11mo ago
  It's against my ToS for ChatGPT, Claude, and Gemini to train their models on my writing, but they've all done that. They are free to pound sand.

ComputerGuru 11mo ago

> It’s bandits all the way down, so adding a little smuggling to that doesn’t surprise me. Implying it’s *morally* wrong for a Chinese company to bypass US sanctions is hilarious. You really say that with a straight face when even the president admits this is only protectionism?

qoez 11mo ago
[flagged]
- Spooky23 11mo ago
  Is it morally OK as long as they pay their 25% vig to POTUS? The concept of morality in this context is absurd.
- deleted 11mo ago
  [deleted]
- behringer 11mo ago
  I like how you're careful to use the "west" because if you had said US your argument would have been laughable.
  - forinti 11mo ago
    Most European countries did awful things during their colonial years, some late into the XX century. Some still do shady things in Africa right now. And they all welcome money siphoned off peripheral countries by crooks. Nobody can claim moral superiority.
- platevoltage 11mo ago
  Give me 1 reason why the USA, the (for now) leader of "the west", is morally superior to China?
  - nish__ 11mo ago
    [flagged]

KumaBear 11mo ago

Lets take the weapons embargos placed on Israel by our allies. In the NDAA must pass bill we set funds aside to procure those weapons and sell them to Israel. We don't really care about these things we have selective enforcement.

htrp 11mo ago

Everyone trains on queries from other models, it's called distillation

LogicFailsMe 11mo ago
Well, given AI content cannot be copyrighted, haters can hate hate hate hate hate...
- halJordan 11mo ago
  That's a little disingenuous. If i buy a printer and use the ink in the cartridge to reverse engineer a beautiful red, have i stolen something from the printer manufacturer? Especially if they lose business because they no longer have what distinguished them? Clean room design is not new (or illegal), but it's always been a form of stealing

epolanski 11mo ago

Lol @ quoting 3 companies that broke any possible copyright law as victims.

coliveira 11mo ago

They don't need to break any laws for this. Where do you think are the customers for data centers in the Middle East? Chinese companies do everything legally, paying for access to data centers that got the chips directly from the US.

atleastoptimal 11mo ago

training on data isn’t stealing the data, in the same way learning from a textbook doesn’t mean youre stealing from it

sh34r 11mo ago

There’s no honor among thieves. You don’t get to cry about Chinese “bandits” when Anthropic just had to pay $1 billion to settle a massive copyright infringement lawsuit. All of these models were created through the mass-scale theft of humanity’s intellectual property, personal data, and dignity. Open always beats closed. Drain the moats. Starve the ClosedAI beast.

SpaceManNabs 11mo ago

i find it weird that this comment got so much pushback. i don't think it was portraying deepseek as any more morally wrong than anyone else, or castigating anyone as morally wrong. but maybe i gave it a gracious reading.

nsoonhui 11mo ago

I think the fact that DeepSeek trains on competitor queries (i.e., distillation) — along with using banned Nvidia chips — helps explain how it can achieve such low training costs (USD 6 million vs. billions) while delivering only slightly worse performance than its American counterparts. It also undermines the narrative that DeepSeek or China is posing a serious challenge to the U.S. lead in AI. The gap may be closing, but the initial reactions now seem knee-jerk. That the discussion has being hijacked and shifted to moral superiority is really unfortunate, because that was never the point in the first place.

whimsicalism 11mo ago
These models never cost billions to train and I doubt the final training run for models like GPT-4 cost more than 8 figures. 6 million is definitely cheaper and I would attribute that to distillation.

InkCanon 11mo ago

It's not even stealing. They paid OpenAI for the tokens. It violates the OpenAI TOS, which specifically forbids using it's outputs for training competing models (which is very ironic)

WhyNotHugo 11mo ago

> It wouldn’t surprise me if they continued acquiring them through, well, less than legal means. Strictly speaking, it's not illegal for them to acquire it, it's illegal for an exporter in the US to sell (even if transitively) to them.

Aqua0 11mo ago

"The TBD group is using several third-party models as part of the training process for Avocado, distilling from rival models including Google’s Gemma, OpenAI’s gpt-oss and Qwen, a model from the Chinese tech giant Alibaba Group Holding Ltd., the people said." LOL. Distillation doesn't count as plagiarism, or you should call Meta out on it. They're distilling the Chinese model. Ref: https://www.moneycontrol.com/news/business/inside-meta-s-pivot-from-open-source-to-money-making-ai-model-13720231.html https://www.moneycontrol.com/news/business/inside-meta-s-piv...

bfeynman 11mo ago

| Kinda surprised they didn’t run into model collapse problems Not sure why you would expect this, all the models started doing this as its much more cost effective to get data for post training don't you remember the first grok release where many times it started replies "as a model trained by openai..."