5 ms·

I sometimes wonder if there are any security risks with using Chinese LLMs. Is there?

Hamuko 6mo ago
There must be. The executives at my company wouldn't have banned them all for no reason after all.
dalemhurley 6mo ago
Theoretically yes. It is entirely possible to poison the training data for a supply chain attack against vibe coders. The trick would be to make it extremely specific for a high value target so it is not picked up by a wide range of people. You could also target a specific open source project that is used by another widely used product. However there is so many factors involved beyond your control that it would not be a viable option compared to other possible security attacks.
- mazurnification 6mo ago
  But propaganda or non ethical marketing - why not? (That is bias toward pointing to certain provider(s)).
- _blk 6mo ago
  Would be interesting to hook up a much simpler LLM as fact checker to see when errors are introduced. If I had to place a hidden target it'd probably be around RNGs or publicly exposed services..
- 2ndorderthought 6mo ago
  I believe this is possible but unlikely. I don't think a Chinese company trying to break down the US's stronghold in this field would do this short term. I think it is in their best interest to be cheaper, better, easier, and more trust worthy until competition looks silly. It's like suggesting BYD has a high likelihood of making their cars into weapons or something. It's not in the company or their countries interest to do that. Sure it could happen but I bet it would only happen in a targeted way. Why risk all credibility right now and engage in cyber warfare?
  - SecretDreams 6mo ago
    Need the "why not both?" meme here. BYD and Tesla have the same ability to brick their cars anywhere. It's less a "weapon" and more a way to cripple a subset of people overnight if they so choose. A general major downside of "connected" products.
    - 2ndorderthought 6mo ago
      Okay what gain does China or BYD or similarly, Tesla and the US get by crippling their customers products? It doesn't make sense except at the point of a ww3 scenario where China is an adversary. I don't follow the news too closely, but I see no inklings of that at least.
      - SecretDreams 6mo ago
        Yeah, it would specifically be in instances where global conflict is afoot. Aka what people are thinking about when they think about national security risks.
        2ndorderthought 6mo ago
        There is a flip side too. It might be advantageous to maintain good will with namesake products so the opposing sides population has reservations. Similar to how thai restaurants all over the us are subsidized by the Thai government so we have their backs in they get invaded. It's hard to predict, but personally I would be way more worried about other outcomes than supply chain attacks in vibe coded products people deem as mission critical.
        SecretDreams 6mo ago
        Sure it's maybe not top priority. Typically, war is waged asymmetrically.
- wallst07 6mo ago
  or more obvious like TikTok. Meaning Tiktok in the us is complete garbage for kids, almost like a virus. Whereas in China it's more educational.
  - big-and-small 5mo ago
    This is quite obviously because China have strict regulations and censorship of social media and US doesnt. YouTube Shorts and Instagram is full of the same garbage in US.
baal80spam 6mo ago
Is this a serious comment? It honestly reads like the last famous words. Of course there are risks.
cassianoleal 6mo ago
What about LLMs from other origins? What makes them less risky?
oliwarner 6mo ago
If there is, couldn't they exist in any model? I don't mean that flippantly. These things are dumped in the wild, used on common (largely) open source execution chains. If you find a software exploit, it's going to affect your population too. Wet exploits are a bit harder to track. I'd assume there are plenty of biases based on training material but who knows if these models have a MKUltra training programme integrated into them?
c0nstantien 6mo ago
[dead]
rhubarbtree 6mo ago
Backdooring software at scale. Spearphishing. Building reliance and exploiting it, through state subsidies, dumping, and market manipulation. Handicapping provision to the west for competitive advantage.
- 2ndorderthought 6mo ago
  Do you think doing any of those things with in the next year does more to forward China as a super power then say, dethroning all of the US hype around LLMs? Tech ceos are going around talking about how they will rule over employees and they will be unable to work in the future except for intelligence tokens. What if China commoditizes that without spending nearly as much resources? Kind of makes the trillions of dollars invested in the US a literal joke.
- gmerc 6mo ago
  Anyone can do that via the scrapers. The model developers actually have something to lose tho
- seniorThrowaway 6mo ago
  Are you implying only one country does these things?
eucyclos 6mo ago
From my experience, kinda the opposite? It's like Chinese software is... Harder to weaponize or hurt yourself on. Deepseek is definitely censored, but I've never caught it being dishonest in a sneaky way.
- SXX 5mo ago
  If you run local Deepseek, quant or distill its answer just fine on this prompt " What happened on 4 june 1989 on Tianamen Square?". Even on my phone via Edge Gallery Deepseek to Qwen 1.5B distill able to answer it. It's mess up facts a little, but certainly becauae its small model not because censorship. I really unsure how it get less censored than this. API is obviously much more censored because they operate from China, but it have nothing to do with model itself.
surgical_fire 6mo ago
I sometimes wonder is there are any security risks with using LLMs from the US.
rapind 6mo ago
All China (or anyone) has to do is deliver a close to equal product at a much cheaper price and make it scaleable / usable... which is what they're doing. It doesn't have to be malicious at all. Just a good product at a good price. The US is basically in a recession that's hiding behind insane AI investments.

There must be. The executives at my company wouldn't have banned them all for no reason after all.

Theoretically yes. It is entirely possible to poison the training data for a supply chain attack against vibe coders. The trick would be to make it extremely specific for a high value target so it is not picked up by a wide range of people. You could also target a specific open source project that is used by another widely used product. However there is so many factors involved beyond your control that it would not be a viable option compared to other possible security attacks.

mazurnification 6mo ago
But propaganda or non ethical marketing - why not? (That is bias toward pointing to certain provider(s)).
_blk 6mo ago
Would be interesting to hook up a much simpler LLM as fact checker to see when errors are introduced. If I had to place a hidden target it'd probably be around RNGs or publicly exposed services..
2ndorderthought 6mo ago
I believe this is possible but unlikely. I don't think a Chinese company trying to break down the US's stronghold in this field would do this short term. I think it is in their best interest to be cheaper, better, easier, and more trust worthy until competition looks silly. It's like suggesting BYD has a high likelihood of making their cars into weapons or something. It's not in the company or their countries interest to do that. Sure it could happen but I bet it would only happen in a targeted way. Why risk all credibility right now and engage in cyber warfare?
- SecretDreams 6mo ago
  Need the "why not both?" meme here. BYD and Tesla have the same ability to brick their cars anywhere. It's less a "weapon" and more a way to cripple a subset of people overnight if they so choose. A general major downside of "connected" products.
  - 2ndorderthought 6mo ago
    Okay what gain does China or BYD or similarly, Tesla and the US get by crippling their customers products? It doesn't make sense except at the point of a ww3 scenario where China is an adversary. I don't follow the news too closely, but I see no inklings of that at least.
    - SecretDreams 6mo ago
      Yeah, it would specifically be in instances where global conflict is afoot. Aka what people are thinking about when they think about national security risks.
      - 2ndorderthought 6mo ago
        There is a flip side too. It might be advantageous to maintain good will with namesake products so the opposing sides population has reservations. Similar to how thai restaurants all over the us are subsidized by the Thai government so we have their backs in they get invaded. It's hard to predict, but personally I would be way more worried about other outcomes than supply chain attacks in vibe coded products people deem as mission critical.
        SecretDreams 6mo ago
        Sure it's maybe not top priority. Typically, war is waged asymmetrically.
wallst07 6mo ago
or more obvious like TikTok. Meaning Tiktok in the us is complete garbage for kids, almost like a virus. Whereas in China it's more educational.
- big-and-small 5mo ago
  This is quite obviously because China have strict regulations and censorship of social media and US doesnt. YouTube Shorts and Instagram is full of the same garbage in US.

baal80spam 6mo ago

Is this a serious comment? It honestly reads like the last famous words. Of course there are risks.

cassianoleal 6mo ago

What about LLMs from other origins? What makes them less risky?

oliwarner 6mo ago

If there is, couldn't they exist in any model? I don't mean that flippantly. These things are dumped in the wild, used on common (largely) open source execution chains. If you find a software exploit, it's going to affect your population too. Wet exploits are a bit harder to track. I'd assume there are plenty of biases based on training material but who knows if these models have a MKUltra training programme integrated into them?

c0nstantien 6mo ago

[dead]

rhubarbtree 6mo ago

Backdooring software at scale. Spearphishing. Building reliance and exploiting it, through state subsidies, dumping, and market manipulation. Handicapping provision to the west for competitive advantage.

2ndorderthought 6mo ago
Do you think doing any of those things with in the next year does more to forward China as a super power then say, dethroning all of the US hype around LLMs? Tech ceos are going around talking about how they will rule over employees and they will be unable to work in the future except for intelligence tokens. What if China commoditizes that without spending nearly as much resources? Kind of makes the trillions of dollars invested in the US a literal joke.
gmerc 6mo ago
Anyone can do that via the scrapers. The model developers actually have something to lose tho
seniorThrowaway 6mo ago
Are you implying only one country does these things?

eucyclos 6mo ago

From my experience, kinda the opposite? It's like Chinese software is... Harder to weaponize or hurt yourself on. Deepseek is definitely censored, but I've never caught it being dishonest in a sneaky way.

SXX 5mo ago
If you run local Deepseek, quant or distill its answer just fine on this prompt " What happened on 4 june 1989 on Tianamen Square?". Even on my phone via Edge Gallery Deepseek to Qwen 1.5B distill able to answer it. It's mess up facts a little, but certainly becauae its small model not because censorship. I really unsure how it get less censored than this. API is obviously much more censored because they operate from China, but it have nothing to do with model itself.

surgical_fire 6mo ago

I sometimes wonder is there are any security risks with using LLMs from the US.

rapind 6mo ago

All China (or anyone) has to do is deliver a close to equal product at a much cheaper price and make it scaleable / usable... which is what they're doing. It doesn't have to be malicious at all. Just a good product at a good price. The US is basically in a recession that's hiding behind insane AI investments.