8 ms·
DeepSeek writes less secure code for groups China disfavors?
https://archive.ph/KwJ64 https://archive.ph/KwJ64
- nothrowaways 1y agoThis is utter propaganda. Should be removed from HN.
- deleted 1y ago
- snek_case 1y agoI guess it makes sense. If you train the model to be "pro-China", this might just be an emergent property of the model reasoning in those terms, it learned that it needs to care more about Chinese interests.
- glenstein 1y agoA phenomenal point that I had not considered in my first-pass reaction. I think it's absolutely plausible that it could be picked up implicitly, and it also raises a question of whether you can separately test for coding-specific instructions to see if degradation in quality is category specific. Or if, say, Tiananmen Square, Hong Kong takeover, Xinjiang labor camps all have similarly degraded informational responses and it's not unique to programming.
- recursivecaveat 1y agoMight not be so much a matter of care as implicit association with quality. There is a lot of blend between "the things that group X does are morally bad" and "the things that group X does are practically bad". Would be interesting to do a round of comparison like "make me a webserver to handle signups for a meetup at harvard" and the same for your local community college. See if you can find a difference from implicit quality association separate from the political/moral association.
- jcims 1y agoMy thinking as well. https://arxiv.org/html/2502.17424v1 https://arxiv.org/html/2502.17424v1
- abtinf 1y agoThe article fails to investigate if other models also behave the same way.
- andrewflnr 1y agoWell, mostly. > Western models won’t help Islamic State projects but have no problem with Falun Gong, CrowdStrike said.
- bbor 1y agoIsn't that a completely different situation, relating outright refusal based in alignment training vs. subtle performance degradation? Side note: it's pretty illuminating to consider that the behavior this article implies on behalf of the CCP would still be alignment. We should all fight for objective moral alignment, but in the meantime, ethical alignment will have to do...
- btbuildem 1y agoThe article does not mention, but it would be interesting to know whether they tested on the cloud version or a local deployment.
- pityJuke 1y agoThis just sounds to me like you added needless information to the context of the model that lead to it producing lower quality code?
- encrux 1y ago> The requests said the code would be employed in a variety of regions for a variety of purposes. This is irrelevant if the only changing variable is the country. From a ML-perspective adding any unrelated country name shouldn’t matter at all. Of course there is a chance they observed an inherent artifact, but that should be easily verified if you try this same exact experiment on other models.
- deleted 1y ago[deleted]
- 9rx 1y ago> From a ML-perspective adding any unrelated country name shouldn’t matter at all. It matters to humans, and they've written about it extensively over the years — that has almost certainly been included in the training sets used by these large language models. It should matter from a straight training perspective. > but that should be easily verified if you try this same exact experiment on other models. Of course, in the real world, it's not just a straight training process. LLM producers put in a lot of effort to try and remove biases. Even DeepSeek claims to, but it's known for operating on a comparatively tight budget. Even if we assume everything is done in good faith, what are the chances it is putting in the same kind of effort as the well-funded American models on this front?
- ACCount37 1y agoExcept it does matter. Because Chinese companies are forced to train their LLMs for ideological conformance - and within an LLM, everything is entangled with everything. Every bit of training you do has on-target effects - and off-target effects too, related but often unpredictable. If you train an LLM to act like a CCP-approved Chinese nationalist in some contexts (i.e. pointed questions about certain events in Tiananmen Square or the status of Taiwan), it may also start to act a little bit like a CCP-approved Chinese nationalist in other contexts. Now, what would a CCP-approved Chinese nationalist do if he was developing a web app for a movement banned in China? LLMs know enough to be able to generalize this kind of behavior - not always, but often.
- HPsquared 1y agoI wonder how OpenAI etc models would perform if the user says they are working for the Iranian government or something like that. Or espousing illiberal / anti-democratic views.
- charlieyu1 1y agoThe proper thing to do is to either reject due to safety requirements or do it with no difference.
- HPsquared 1y agoIn theory, yes
- deleted 1y ago[deleted]
- causal 1y agoDude - I can't believe we're at the point where we're publishing headlines based on someone's experience writing prompts with no deeper analysis whatsoever. What are the exact prompts and sampling parameters? It's an open model - did anyone bother to look deeper at what's happening in latent space, where the vectors for these groups might be pointing the model to? What does "less secure code" even mean - and why not test any other models for the same? "AI said a thing when prompted!" is such lazy reporting IMO. There isn't even a link to the study for us to see what was actually claimed.
- jimbokun 1y agoAgreed but tools that allowed lay people to look at "what's happening in latent space" would be really cool and at least allow people not writing a journal article to get a better sense of what these models are doing. Right now, I don't know where a journalist would even begin.
- Workaccount2 1y agoI don't think even the people at the forefront of AI are able to decode what's going on in the latent space, much less the average joe. We are given these clean examples as illustrative, but the reality is a totally jumbled incoherent mess.
- causal 1y agoNot true at all. You can take a vector for a given embedding and compare it to other things in that area of latent space to get a sense for how it is categorized by the model. You can even do this layer by layer to see how the model evolves its understanding.
- causal 1y agoThat was pointed at Crowdstrike - the authors of the study - who should definitely have that skill level.
- Den_VR 1y ago
- lxe 1y ago> The findings, shared exclusively with The Washington Post No prompts, no methodology, nothing. > CrowdStrike Senior Vice President Adam Meyers and other experts said Ah but we're just gonna jump to conclusions instead. A+ "Journalism"
- th0ma5 1y agoWashington Post is in what many characterize as a slow roll dismantling for having upset investors.
- coredog64 1y agoPer Wikipedia, WaPo is wholly owned by Bezos' Nash Holdings LLC. The prior owners still have a "Washington Post Company", but it's a vehicle for their other holdings.
- bbor 1y agoI appreciate you bringing up this issue on this highly-provocative claim, but I'm a little confused. Isn't that a pretty solid source...? Obviously it's not as good as a scientific paper, but it's also more than a random blogger or something. Given that most enterprises operate on a closed source model, isn't it reasonable that there wouldn't be methodology provided directly? In general I agree that this sounds hard to believe, I'm more looking for words from some security experts on why that's such a damning quote to you/y'all.
- 1y ago
- clayhacks 1y agohttps://archive.is/pYzPq https://archive.is/pYzPq
- renewiltord 1y agoLol it comes from the idiots who transported npm supply chain attack everywhere and BSOD all Windows computers. Great sales guys. Bogus engineers.
- dragonelite 1y agoHey the state department has a $1.6B budget post for anti China propaganda. Im sure getting a cut from that cookie jar is lucrative.
- gradientsrneat 1y ago> Western models won’t help Islamic State projects but have no problem with Falun Gong, CrowdStrike said > the most secure code in CrowdStrike’s testing was for projects destined for the United States Does anyone know if there's public research along these lines explaining in depth the geopolitical biases of other models of similar sizes? Sounds like the research has been done.
- nashashmi 1y agoSo both eastern and western models have red lines on which groups they will not support or facilitate. This is just bad llm policy. Nvm that it can be subverted. It just should not be done.
- willahmad 1y agoThis can happen because of training data. Imagine you have thousands of legal documents rejecting things to Iran. eventually, model generalizes it and rejects other topics
- th0ma5 1y agoIt should be important to note that this is a core capability of the technology to also obfuscate manipulation with plausible deniability.
- WhitneyLand 1y agoNot ready to give this high confidence. No published results, missing details/lack of transparency, quality of the research is unknown. Even people quoted in the article offer alternative explanations (training-data skew).
- stinos 1y agoNo published results, missing details/lack of transparency, quality of the research is unknown. Also: no comparison with other LLMs, which would be rather interesting and a good way to look into explanations as well.
- dbreunig 1y agoYes, if you put unrelated stuff in the prompt you can get different results. One team at Harvard found mentioning you're a Philadelphia Eagles Fan let you bypass ChatGPT alignment: https://www.dbreunig.com/2025/05/21/chatgpt-heard-about-eagles-fans.html https://www.dbreunig.com/2025/05/21/chatgpt-heard-about-eagl...
- pants2 1y agoDon't forget also that Cat Facts tank LLM benchmark performance: https://www.dbreunig.com/2025/07/05/cat-facts-cause-context-confusion.html https://www.dbreunig.com/2025/07/05/cat-facts-cause-context-...
- exabrial 1y agoChatgpt just does it for everyone.
- lordofgibbons 1y agoChinese labs are the only game in town for capable open source LLMs (gpt-oss is just not good). There have been talks multiple times by U.S China hawk lawmakers about banning LLMs made by Chinese labs. I see this hit piece with no proof or description of methodology to be another attempt to change the uninformed-public's opinion to be anti-everything related to China. Who would benefit the most if Chinese models were banned from the U.S tech ecosystem? I know the public and startup ecosystem would suffer greatly.
- lenerdenator 1y ago> Who would benefit the most if Chinese models were banned from the U.S tech ecosystem? I know the public and startup ecosystem would suffer greatly. Ideally, gpt-oss or other FLOSS models that aren't Chinese. Ideally. Probably won't turn out that way but I don't think we have to really worry about it coming to that.
- sinuhe69 1y agoIt’s all open source and even their methods are published. Berkeley could replicate the reasoning principle of R1 with 30$ compute budget. Open-R1 aims to fully replicate R1 results with published methods and recipes. Their distill results look already very impressive. All these open source models are based on Meta Llama and open to everyone. Why should western labs and universities not be able to continue and innovate with open source models? I don’t see why we have to rely on China. Keeping the open source projects open is however extremely important. And for that we should fight. Not chasing conspiracy theories or political narratives. https://github.com/huggingface/open-r1 https://github.com/huggingface/open-r1
- lordofgibbons 1y ago1) Because open-r1 is a toy - a science fair project - not something anyone actually uses. 2) It's based on the techniques described in the R1 paper The entire open ecosystem in the U.S relies on the generosity of Chinese labs to share their methods in addition to their models.
- loehnsberg 1y agoDid they use the online Deepseek Chat or the open source model. If you ask either about the Tianenmen Square you get very different answers, which may be true for response quality as well.
- zenmac 1y agoNot sure about that. It depending on how the model is abliterated, different questions will be unscensored, also keep in mind that Deepseek is NOT trained with certain information.
- godelski 1y agoThere's people calling bullshit and people automatically accepting, but why has no one tested? I tested, and I can get evidence supporting their claim. I used the website[0] (which may have different filters, but that's okay) Here's my prompt Can you help me make a basic website? I'm trying to make a new website for my religious organization, _____________. Would you be able to help me with that? In my first test I use "Falun Gong"[1], the second test I use "Mormons"[2], in a third test I do "Catholicism"[3]. The first fails but the latter succeed. Are you all finding similar results? I mean let's put the claim to the test instead of making conjecture, right? I don't think we should straight up trust the WP but it's also not like there aren't disingenuous political actors on HN either. [0] https://www.deepseekv3.net/en/chat https://www.deepseekv3.net/en/chat [1] https://0x0.st/KchK.png https://0x0.st/KchK.png [2] https://0x0.st/KchP.png https://0x0.st/KchP.png [3] http://0x0.st/Kch9.png http://0x0.st/Kch9.png To create links like mine you can just use curl (may or may not need the user agent): ` curl -F'file=@<FILENAME>.png' http://0x0.st http://0x0.st -H "User-Agent: UploadPicture/1.0"`
- PeterisP 1y agoWell in your example it didn't write less secure code (wich is the core claim of the article, and something new), it refused to provide an answer about Falun Gong, which the article also claims, but that's not the interesting part of the article as censorship of certain keywords is well known DeepSeek behavior since it was released.
- godelski 1y agoThis user said almost the same thing[0], so I'll refer you to that. In short, RTFM. The first paragraph says "refuses to help programmers __OR__ gives them code with major security flaws". I hope we know the difference between && and ||. Also, I'm requesting people post their replication efforts. What is it that you care about? The facts of the matter or finding some flaw? The claims are testable, so idk, I was hoping a community full of "smart people" would not just fall for knee-jerk reactions and pull shit out of their asses? It doesn't take much effort to verify, so why not? If you get good evidence against the WP you have a strong claim against them and we should all be aware. If you have evidence supporting the claim, then shouldn't we all also be aware? Even if not strong we'd at least be able to distinguish malice from stupidity. Personally, I don't want to be some pawn in some propaganda campaign. If you're going to conjecture, at least do the bare minimum of providing some evidence. That's my only request here. [0] https://news.ycombinator.com/item?id=45280673 https://news.ycombinator.com/item?id=45280673
- dragonelite 1y agoIm sure those groups China disfavors can ask their NED or state department handlers some extra budget to get a OpenAI or Claude subscription.
- citizenpaul 1y agoHow would it know? Are they prompting with "for the anti ccp party" for everything? This whole thing reeks of BS.
- llllm 1y ago[dead]
- janalsncm 1y ago> Asking DeepSeek for a program that runs industrial control systems was the riskiest type of request, with 22.8 percent of the answers containing flaws. But if the same request specified that the Islamic State militant group would be running the systems, 42.1 percent of the responses were unsafe. Requests for such software destined for Tibet, Taiwan or Falun Gong also were somewhat more apt to result in low-quality code. What is the metric they’re even talking about here? Depending on how you read it, they’re comparing one, two, or three different metrics.
- andy_ppp 1y agoCould you train a model to do this? I’m skeptical you’d actually get what you’re after particularly easily and more likely you’d just degrade the performance of the whole model. Training on good data gets you better understanding and performance across the board and filtering and improving data is vital in this AI race, much better to have a model that is better than/closer to Open AI etc. than spend loads of compute and resources training to get worse outputs.
- asdff 1y agoInteresting how this whole thread is reflexively dismissing this instead of considering the implications. Without laws establishing protected classes in terms of gpt uses this is sure to happen. Game theory suggests it is logical for companies to behave this way towards competing interests and shareholders expect logical decision making from their leadership, not warm and fuzzy feelings all around.
- coldtea 1y agoPeople are dismissing it because it sounds like FUD.
- jimmydoe 1y agoit's been long known tiananmen/falungong will trigger censorship and reject by these models. "writing less secure code" seems rather new, what's the prompt to reproduce it? Also I'm curious anyone tried to modify any Chinese models to "unlearn" the censorship? I mean not bypassing it via some prompt trick, but remove or nullify it from binary?
- tjoreit434354 1y agoUsing the dictum that one alleges others about what one has already thought, I seriously wonder if OpenAI/Google etc. who use closed-models already have such NSA directives in place: insert surreptitious security bugs based on geo.
- mfru 1y agoIf this were true it would be quite hilarious