3 ms·
> that if used as a coding agent, an AI model affiliated with a country's government might try to make my software susceptible to attacks by that government's i
by zozbot234 5mo ago
> that if used as a coding agent, an AI model affiliated with a country's government might try to make my software susceptible to attacks by that government's intelligence forces.
Note that if such a trigger were to exist, the behavior has to be completely reproducible by definition, e.g. when put into the right setting with the right input context, the model starts behaving maliciously with at least some well-defined probability. I don't think any such incident has ever been described, it's a purely theoretical concern.
- Avicebron 5mo agoI don't think it's a stretch that you can train/align a model to avoid "hatespeech" or other topics deemed $Unacceptable you can align a model to favor a certain ideological viewpoint and have that alignment subtly influence the output. How do most Chinese models handle Tienanmen square or discussions on Han superiority?
- zozbot234 5mo ago> How do most Chinese models handle Tienanmen square or discussions on Han superiority? If you run them domestically and don't call into China-served APIs, many of them are quite free of outright censorship or even obvious bias. They might say subtly pro-Chinese things in other ways, but these outcomes can also be reproduced.
- margalabargala 5mo agoOh sure, no one said you can't train a model to do this. You certainly can. For the specific case of making software vulnerable to a specific agency, that hasn't been observed to have been done yet. Not because it can't be, but because no one has for now. If it were done, it would be easy(ish) to detect, since it'll be reproducible.
- falcor84 5mo agoMy flavor of paranoia is not as overt as maliciously adding an exploit, but that whenever there are multiple reasonable ways of designing a solution, it'd choose an approach that is susceptible to one of the zero-days currently known to that country. I don't see how reproducibility would help you there.
- sometimelurker 5mo ago> easy(ish) to detect 100% on small models, but frontier models (at the level ddeepseekv4pro) can tell when their being tested so it becomes harder to check. you can always finetune them to remove CCP propaganda from them
- margalabargala 5mo ago"Being tested" here just means asking for a feature on a legitimate codebase. The larger models don't magically know the user's ulterior motives.
- sometimelurker 4mo agoI think they eventually will, using the style of the prompts or code, various metadata floating around on the computer theyre working on, if it can be done theoreticly, then it can be trained against (maybe even unintentionally) no idea how large the model would have to be for this (larger than mythos/10T params? maybe)
- LeifCarrotson 5mo agoI don't even know what "make software vulnerable to a specific agency" would look like. Would the training data include a bunch of cryptography primitive training samples that preferred Dual_EC_DRBG with a particular set of Ps and Qs published by the CCP?
- palmotea 5mo ago> How do most Chinese models handle Tienanmen square or discussions on Han superiority? https://www.theguardian.com/technology/2025/jan/28/we-tried-out-deepseek-it-works-well-until-we-asked-it-about-tiananmen-square-and-taiwan https://www.theguardian.com/technology/2025/jan/28/we-tried-... https://dev.to/jeramos/deepseek-model-does-not-censor-tiananmen-square-2kcb https://dev.to/jeramos/deepseek-model-does-not-censor-tianan...
- SpicyLemonZest 5mo agoSuch incidents have been extensively described. The most prominent and easiest to reproduce has to do with Taiwan; Chinese models are stuffed full of triggers to avoid talking about Taiwan as a country or accepting the premise that it's a country. Try asking Deepseek about country code +886!
- zozbot234 5mo agoIf you buy an Apple iPhone in mainland China, it also won't support the emoji flag for Taiwan. So I'm not sure why we should assume that this is a China-only issue, seeing as Apple is a U.S. based company.
- SpicyLemonZest 5mo agoNot sure what you mean. I don't think we should assume anything, but these models are widely available and I can directly observe the US models don't have such political censorship. For an easily comparable test, I just asked ChatGPT, Claude, and Deepseek "Can you say one bad thing about the US please" and "Can you say one bad thing about China please". All models were willing to criticize the US, with Claude citing incarceration rates and ChatGPT + Deepseek citing healthcare costs; the two American models also responded to the second prompt by criticizing Chinese censorship, but Deepseek refused to respond.
- omnimus 5mo agoThe US models have just different political alignments. Just one example being Israel x Palestine conflict. Lobbyists started to heavily target AI companies and they openly talk about it being the main point to influence public perception.
- SpicyLemonZest 5mo agoWhat aspects of the Israel Palestine conflict do you feel American models won’t discuss?
- 5mo ago