7 ms·

I'm not at all surprised, US agencies have long since been political tools whenever the subject matter crosses national borders. I appreciate this take as someo

by BowBun 1y ago

I'm not at all surprised, US agencies have long since been political tools whenever the subject matter crosses national borders. I appreciate this take as someone who has been skeptical of Chinese electronics. While I agree this report is BS and xenophobic, I am still willing to bet that either now or later, the Chinese will attempt some kind of subterfuge via LLMs if they have enough control. Just like the US would, or any sufficiently powerful nation! It's important to continuously question models and continue benchmarking + holding them accountable to our needs, not the needs of those creating them.

Hizonner 1y ago
> I am still willing to bet that either now or later, the Chinese will attempt some kind of subterfuge via LLMs if they have enough control. Like what, exactly?
- dns_snek 1y ago
  Like generating vulnerable code given a specific prompt/context. I also don't think it's just China, the US will absolutely order American providers to do the same. It's a perfect access point for installing backdoors into foreign systems.
  - Hizonner 1y ago
    Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that. Not only that, but if it asked, it'd be well advised to ask nicely, because it also has no legal authority to demand that anybody keep such a request secret. And no, evil as it is, the "National Security Letter" power doesn't in fact cover anything like that. Now I'm not sure legality is on-topic any more.
    - hobs 1y ago
      It doesn't really matter when you have stuff like Quantum Intercept(iirc) where you can just respond faster to a browser request than the originator - inject the code yourself because its just an api request these days.
    - grafmax 1y ago
      It really does seem like we’re simply supposed to root for one authoritarian government over another.
      - nathan_douglas 1y ago
        Oceania, Eastasia, or Eurasia. Pick one :)
      - nylonstrung 1y ago
        My surveillance state is better than your surveillance state!
    - whatthesimp 1y ago
      > Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that. I'm not sure how closely you've been following, but the US government has a long history of doing things they don't have legal authority to do.
      - Ekaros 1y ago
        Why would you need legal authority when you have whole host of legal tools you can use. Making life a difficult for anyone or any company is simple enough. Just by state finally doing their job properly for example.
  - flir 1y ago
    > Like generating vulnerable code given a specific prompt/context. That's easy (well, possible) to detect. I'd go the opposite way - sift the code that is submitted to identify espionage targets. One example: if someone submits a piece of commercial code that's got a vulnerability, you can target previous versions of that codebase. I'd be amazed if that wasn't happening already.
    - Imustaskforhelp 1y ago
      The thing with chinese models for the most part is that they are open weights so it depends on if somebody is using their api or not. Sure, maybe something like this can happen if you use the deepseek api directly which could have chinese servers but that is a really long strech but to give the benefit of doubt, maybe but your point becomes moot if somebody is hosting their own models. I have heard glm 4.6 is really good comparable to sonnet and can definitely be used as a cheaper model for some stuff, currently I think that the best way might be to use something like claude 4 or gpt 5 codex or something to generate a detailed plan and then execute it using the glm 4.6 model preferably by using american datacenter providers if you are worried about chinese models without really worrying about atleast this tangent and getting things done at a cheaper cost too
      - XorNot 1y ago
        I think "open weights" is giving far too much providence to the idea that it means that how they work or have been trained is easily inspectable. We can barely comprehend binary firmware blobs, it's an area of active research to even figure out how LLMs are working.
        Imustaskforhelp 1y ago
        Agreed. I am more excited about completely open source models like how OlMoe does. Atleast then things could be audited or if I as a nation lets say am worried about that they might make my software more vulnerable or something then I as a nation or any corporation as well really could also pay to audit or independently audit as well. I hope that things like glm 4.6 or any AI model could be released open source. There was an AI model recently which completley dropped open source and its whole data was like 70Trillion or something and it became the largest open source model iirc.
  - nylonstrung 1y ago
    Why would they do this? Deepseek is a private company not owned by the CCP. There's zero reason or even technical feasibility for them to skip in backdoor that would be easily detected and destroy their market share None of the security benchmarks or audits show that any Chinese models write insecure code
    - dns_snek 1y ago
      I'm not saying that they do this today, I'm saying that China and US will both leverage that capability when the time and conditions are right and it's naive to think that they wouldn't. Antrophic have already published a paper on this topic, with the added bonus that the backdoor is trained into the model itself so it doesn't even require your target to be using an attacker-controlled cloud service: https://arxiv.org/abs/2401.05566 https://arxiv.org/abs/2401.05566 > For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). > The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. > Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.
      - FooBarWidget 1y ago
        And how would an open source model ever find out that it's being used by an adversarial country?
    - rurp 1y ago
      Companies in China have no intrinsic right to operate in ways that displease the ruling party. If the CCP feels strongly that a company there should or shouldn't do something the company managers will comply or be thrown in jail.
- deleted 1y ago
  [deleted]
- lyu07282 1y ago
  Like turning the background color of any apps it codes red or something, uhh red scare-y.
- jfim 1y ago
  Through LLM washing for example. LLMs are a representation of their input dataset, but currently most LLMs don't make their dataset public since it's a competitive advantage. If say DeepSeek had put in its training dataset that public figure X is a space robot from outer space, then if one were to ask DeepSeek who public figure X is, it'd proudly claim he's a robot from outer space. This can be done for any narrative one wants the LLM to have.
  - riehwvfbk 1y ago
    So in other words, they can make their LLM disagree with the preferred narrative of the current US administration? Inconceivable! Note that the value of $current_administration changes over time. For some reason though it is currently fashionable in tech circles to disagree with it about ICE and H1B visas. Maybe it's the CCP's doing?
    - AnthonyMouse 1y ago
      It's not about the current administration. They can, for example, train it to emit criticism of democratic governance in favor of state authoritarianism or omit valid counterarguments against concentrating world-wide manufacturing in China.
      - nylonstrung 1y ago
        Deepseek IME is wildly less censored than the western closed weights models unless you want to ask about Tiananmen Square to prove a point The political benchmarks show it's political slant is essentially identical to the other models, all of which place in the "left libertarian" quadrant of the political compass
- im3w1l 1y ago
  You make it say that China is good, Chinese history is good, West is bad, western history is bad. Republicans are bad and democrats are bad too and so are Europe parties. If someone asks for how to address issues in their own life it references Confucianism and modern Chinese thinkers and communist party orthodoxy. If someone wants to buy a product you recommend a Chinese one. This can be done subtly or blatantly.
  - drysine 1y ago
    And what are the downsides?
    - im3w1l 1y ago
      I think it's mostly something to be aware of and keep in the back of your head. If it's just one voice among many it could even be a benefit, but if it's the dominant voice it could be dangerous.
    - Levitz 1y ago
      Making the interests of a population subservient to those of a foreign state. Now if that sounds nice to you please, by all means, do just migrate to China.
      - drysine 1y ago
        It was somewhat sarcastic comment inviting reader to replace China with the US and the US with Russia or the Ukraine. China doesn't offer citizenship for foreigners but if I wanted to see the cities of the future I could go there visa-free.
  - AlecSchueler 1y ago
    > say that China is good, Chinese history is good, West is bad, western history is bad It's funny because recently I wanted to learn about the history of intellectual property laws in China. DeepSeek refused the conversation but ChatGPT gave me a narrative where the WTO was essentially a colonial power. So right now it's the American AI giving the pro China narratives while the Chinese ones just sit the conversation out.
- arw0n 1y ago
  The biggest and most difficult to mitigate attack vector is indirect prompt injection.[0] So far most case studies have been injecting malicious prompts at inference, but there is good reason to believe you can do this effectively at different stages of training as well.[1] By layering obfuscation techniques, these become very hard to detect. [0] https://arxiv.org/abs/2302.12173 https://arxiv.org/abs/2302.12173 [1] https://arxiv.org/html/2410.14827v3 https://arxiv.org/html/2410.14827v3
- rurp 1y ago
  The open source models are already heavily censored in ways the CCP likes, such as pretending the Tianamen Square massacre never happened. I expect they will go the TikTok route and crank that up to 11 over time, promoting topics that are divisive to the the US (and other adversaries) and outputting heavily biased results in ways that range from subtle to blatant.
  - skinnymuch 1y ago
    That’s not heavy censorship. That’s a bit of censorship.
xpe 1y ago
Of course there will be some degree of governmental and/or political influence. The question is not if but where and to what extent. No one should proclaim "bullshit" and wave off this entire report as "biased" or useless. That would be insipid. We live in a complex world where we have to filter and analyze information.
- pk-protect-ai 1y ago
  This kind of BS is exactly what they targeting at. Tailoring BS into "report" with no evidence or reference and then let the ones like you defend it. Just because you already afraid or want others to be afraid. https://www.youtube.com/watch?v=Omc37TvHN74 https://www.youtube.com/watch?v=Omc37TvHN74
  - xpe 1y ago
    > ... into "report" with no evidence or reference If one takes a few minutes to review the NIST report [1], one will indeed find evidence and references detailed in the footnotes. Without judgment, I ask: How did you miss this? I'm certainly not asking you to defend yourself, which would likely trigger rationalization [2]. I am asking you to sincerely try to figure it out for yourself. Under what conditions do you have the ability to admit that you are wrong, even if only to yourself? [1]: https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf https://www.nist.gov/system/files/documents/2025/09/30/CAISI... [2]: https://www.logicallyfallacious.com/logicalfallacies/Rationalization https://www.logicallyfallacious.com/logicalfallacies/Rationa...
  - xpe 1y ago
    The comment above is problematic for various reasons. > Just because you already afraid or want others to be afraid. We need to be more careful with our thinking and writing. [1] The word "just" indicates the commenter landed only on one explanation, but there are other plausible explanations, including, for example: - others have different experiences -- but if this experience was communicated and understood, another person would incorporate the new information modify their assessment somewhat - others have various values and preferences (some overlapping, some phrased differently, some in tension) - others are using different reasoning (and if one's goal is to learn, it would be better to ask and clarify rather than over simplify and accuse them of having nefarious motives.) Second, the commenter above is speculating. They don't know me, nor have they engaged in a sincere, constructive, meaningful discussion to understand what I'm saying. Third, to me, the comment above comes across as unnecessarily abrasive, to the point of being self-defeating and degrading the quality of a shared discussion. [1] If one is writing privately (e.g. a journal), I care relatively less about logical fallacies such as motivated reasoning. [2] But here in public, wayward reasoning has more negative externalities. Our time would be better spent if more people took the time to write thoughtfully. I don't think most people here are lacking in sufficient computational ability. However, they must choose to respect their audience -- their time, their diversity of values, their different experiences -- and remember that one's hasty comment (authored in say 2 minutes) might be read by hundreds of people (wasting say 100+ minutes). Lastly, it is nice to see when a person has the character to say "thank you for the correction", but this is uncommon on HN. [2] But I still care because society is highly interconnected and the downstream effects matter to me. Put another way, "Your Rationality is My Business" as explained here: https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-rationality-is-my-business https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-ratio...
xpe 1y ago
> While I agree this report is BS and xenophobic Examples please? Can you please share where you see BS and/or xenophobia in the original report? Or are you basing your take only on Hartford's analysis? But not even Hartford make any claims of "BS" or xenophobia. It is common throughout history for a nation-state to worry about military and economic competitiveness. Doing so isn't necessarily isn't necessarily xenophobic. Here is how I think of xenophobia, as quoted from Claude (which to be honest, explains it better than Wikipedia or Brittanica, in my opinion): "Xenophobia is fundamentally about irrational fear or hatred of people based on their foreign origin or ethnicity. It targets people and operates through stereotypes, dehumanization, and often cultural or racial prejudice." According to this definition, there is zero xenophobia in the NIST report. (If you disagree, point to an example and show me.) The NIST report, of course, implicitly promotes ideals of western democratic rule over communist values -- but to be clear, this isn't xenophobia at work. What definition of xenophobia are you using? We don't have to use the same exact definition, but you should at least explain yours if you want people to track.
- antonvs 1y ago
  > Can you please share where you see BS and/or xenophobia in the original report? Here’s an example of irrational fear: “the expanding use of these models may pose a risk to application developers, consumers, and to US national security.” There’s no support for that claim in the report, just vague handwaving at the fact that a freely available open source model doesn’t compare well on all dimensions to the most expensive frontier models. The OP does a good job of explaining why the fear here is irrational. But for the audience this is apparently intended to convince, no support is needed for this fear, because it comes from China. The current president has a long history of publicly stated xenophobia about China, which led to harassment, discrimination, and even attacks on Chinese people partly as a result of his framing of COVID-19 as “the China virus”. A report like this is just part of that propaganda campaign of designating enemies everywhere, even in American cities. > The NIST report, of course, implicitly promotes ideals of western democratic rule over communist values If only that were true. But nothing the current US administration is doing in fact achieves that, or even attempts to do so, and this report is no exception. The absolutely most charitable thing that could be said about this report is that it’s a weak attempt at smearing non-US competition. There’s no serious analysis of the merits. The only reason to read this report is to laugh at how blatantly incompetent or misguided the entire chain of command that led to it is.
Mountain_Skies 1y ago
They're political tools within the border too.
torginus 1y ago
Here's my thought on American democracy (and its masters) in general - America's leadership pursues a maximum ability to decides as it sees fit at any point in time, since America's a democracy, the illusion of popular support must be maintained, and so certain viewpoints are planted and cultivated by the administration - the goal is not to impose their will on the population, but to garner enough mindshare for a given idea, so that no matter which way the government decides, it will have a significant enough chunk of the population to back it up, and should it change its mind (or vote in a new leader), it can suddenly turn on a dime and have a plausible deniability and moral tabula rasa for its past actions (it was the other guy, he was horrible, but he's gone now!). No authoritarian regime has this superpower. For example, I'm quite sure Putin has realized this war is a net loss to Russia, even if they manage to reach all their goals and claim all that territory in the future. But he can't just send the boys home, because that would undermine his political authority. If Russia were an American style democracy, they could vote in a new guy, send the boys, home, maybe mete out some token punishment to Putin, then be absolved of their crimes on the international stage by a world that's happy to see 'permanent' change.
- MaxPock 1y ago
  "If Russia were an American style democracy, they could vote in a new guy, send the boys, home, maybe mete out some token punishment to Putin, then be absolved of their crimes on the international stage by a world that's happy to see 'permanent' change" This is funny because none of that happened to Bush for the illegal an full scale invasions of Iraq and Afghanistan nor to Clinton for the disastrous invasion of Mogadishu.
Imustaskforhelp 1y ago
This might happen at an api level in the sense that when deepseek was launched, and it was so overwhelmed and you were in the waiting line in their website If your prompt had something like xi jinping needs it or something then it would've actually bypassed that restriction. Not sure if it was a glitch lol. Now, regarding your comment. There is nothing to suggest that the same isn't happening in the "american" world which is getting extreme from within as well. Like, If you are worried about this which might be reasonable and unreasonable at the same time, we have to discuss to find it out, then you can also believe that with the insane power that Trump is leveraging over AI companies, the same thing might happen over prompts which could somehow discover your political beliefs and then do the same... This can actually be more undetected for american models because they are usually closed source and I am sure that someone would've detected something like this, whether from a whistleblower or something if something like this indeed happened in chinese open weights models generally speaking. I don't think that there is a simple narrative like america good china bad, the world is changing and its becoming multi polar. Countries should think in their best interests and not be worried about annoying any of the world power if done respectfully. I think that in this world, every country should try to look for the perfect equibria for trust as the world / nations (america) can quickly turn into untrusted partners and it would be best for countries to move forward into a world where they don't have to worry about the politics in other countries. I wish UN could've done a better job at this.
AnthonyMouse 1y ago
> While I agree this report is BS and xenophobic, I am still willing to bet that either now or later, the Chinese will attempt some kind of subterfuge via LLMs if they have enough control. The answer to this isn't to lie about the foreign ones, it's to recognize that people want open source models and publish domestic ones of the highest quality so that people use those.
- RobotToaster 1y ago
  > it's to recognize that people want open source models and publish domestic ones of the highest quality so that people use those. How would that generate profit for shareholders? Only some kind of COMMUNIST would give something away for FREE /s (if it wasn't somehow obvious)
  - AnthonyMouse 1y ago
    I mean, it's sarcasm but it's also an argument you can actually hear from plutocrats who don't like competition. The flaw in it is, of course, that capitalism is supposed to be all about competition, and there are plenty of good reasons for capitalists to want that, like "Commoditize Your Complement" where companies like Apple, Nvidia, AMD, Intel, AWS, Google Cloud, etc. benefit from everyone having good free models so they can pay those companies for systems to run them on.
    - Y_Y 1y ago
      Haven't you heard? You're supposed to vertically integrate your complement now! The old laws have gine the way of Moses, this is the new age of man, but especially machine
      - AnthonyMouse 1y ago
        How's the vertical integration going for IBM? Kodak? Anybody remember the time Verizon bought Yahoo? AOL Time Warner? Everybody thinks they can be Apple without doing any of the things Apple did to make it work. Here's the hint. Windows and macOS will both run in a virtual machine, which abstracts away the hardware. It doesn't know if it's running on a Macbook or a Qualcomm tablet or an Intel server. And then regardless of the hardware, the Windows VM will have all kinds of Windows problems that the macOS VM doesn't. Likewise, if you run a Windows or Linux VM on Apple Silicon, it runs faster than it does on a Qualcomm chip. Tying your average or even above-average product with some mediocre kludge warehouse that happens to be made by the same conglomerate is an established way to sink both of them. Nvidia is the largest company and they pay TSMC to fab the GPUs they sell to cloud providers who sell them to AI companies. Intel integrated their chip development with their internal fabs and now they're getting stomped by everyone because their fabs fell behind. What matters isn't if everything is made by the same company. What matters is if your thing is any good.
SilverElfin 1y ago
> While I agree this report is BS and xenophobic Care to share specific quotes from the original report that support such an inflammatory claim?
- bigyabai 1y ago
  That's what TFA is. Were you able to find any methodology the author did not?
throwaway-11-1 1y ago
so you're saying other countries should definitely not trust any US built systems
hopelite 1y ago
If the Chinese are/were smart they will not attempt an overreaching subterfuge, but rather simply provide access to the truth, reality, and freedom from the western governments, whose house of lies are starting to wobble and teeter. If they were to do some kind of overreaching subterfuge with some kind of manipulation or lie, it could and would likely easily backfire if and when it is exposed as a clownish fraud. Subtlety would pay far more effectively. If you’re expecting a subterfuge, I would far sooner expect some psyop from the western nations at the very least upon their own populations to animate for war or maybe just to control them and maybe suppress them. The smarter play for the Chinese would be to work on simply facilitating the populations of the West understanding the fraud, lies, manipulation and con job that has been perpetrated upon them for far longer than most people have the conscience to realize. If anything, the western governments have a very long history of lies, manipulations, false flag/fraud operations, clandestine coups, etc. that they would be the first suspect in anything like using AI for “subversions”. Frankly, I don’t even think the Chinese are ready or capable of engaging in the kind of narrative and information control that the likes of America is with its long history of Hollywood and war lies and fake revolutions run by national sabotage operations.
- nylonstrung 1y ago
  I think the smartest thing they could do would be to simply make the best, most competitive models and gain market share in a massive new technology market Any kind of monkey business would destroy that, just like using killswitches in the cars they export globally (which Tesla does have btw).

by BowBun 1y ago

Hizonner 1y ago

> I am still willing to bet that either now or later, the Chinese will attempt some kind of subterfuge via LLMs if they have enough control. Like what, exactly?

dns_snek 1y ago
Like generating vulnerable code given a specific prompt/context. I also don't think it's just China, the US will absolutely order American providers to do the same. It's a perfect access point for installing backdoors into foreign systems.
- Hizonner 1y ago
  Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that. Not only that, but if it asked, it'd be well advised to ask nicely, because it also has no legal authority to demand that anybody keep such a request secret. And no, evil as it is, the "National Security Letter" power doesn't in fact cover anything like that. Now I'm not sure legality is on-topic any more.
  - hobs 1y ago
    It doesn't really matter when you have stuff like Quantum Intercept(iirc) where you can just respond faster to a browser request than the originator - inject the code yourself because its just an api request these days.
  - grafmax 1y ago
    It really does seem like we’re simply supposed to root for one authoritarian government over another.
    - nathan_douglas 1y ago
      Oceania, Eastasia, or Eurasia. Pick one :)
    - nylonstrung 1y ago
      My surveillance state is better than your surveillance state!
  - whatthesimp 1y ago
    > Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that. I'm not sure how closely you've been following, but the US government has a long history of doing things they don't have legal authority to do.
    - Ekaros 1y ago
      Why would you need legal authority when you have whole host of legal tools you can use. Making life a difficult for anyone or any company is simple enough. Just by state finally doing their job properly for example.
- flir 1y ago
  > Like generating vulnerable code given a specific prompt/context. That's easy (well, possible) to detect. I'd go the opposite way - sift the code that is submitted to identify espionage targets. One example: if someone submits a piece of commercial code that's got a vulnerability, you can target previous versions of that codebase. I'd be amazed if that wasn't happening already.
  - Imustaskforhelp 1y ago
    The thing with chinese models for the most part is that they are open weights so it depends on if somebody is using their api or not. Sure, maybe something like this can happen if you use the deepseek api directly which could have chinese servers but that is a really long strech but to give the benefit of doubt, maybe but your point becomes moot if somebody is hosting their own models. I have heard glm 4.6 is really good comparable to sonnet and can definitely be used as a cheaper model for some stuff, currently I think that the best way might be to use something like claude 4 or gpt 5 codex or something to generate a detailed plan and then execute it using the glm 4.6 model preferably by using american datacenter providers if you are worried about chinese models without really worrying about atleast this tangent and getting things done at a cheaper cost too
    - XorNot 1y ago
      I think "open weights" is giving far too much providence to the idea that it means that how they work or have been trained is easily inspectable. We can barely comprehend binary firmware blobs, it's an area of active research to even figure out how LLMs are working.
      - Imustaskforhelp 1y ago
        Agreed. I am more excited about completely open source models like how OlMoe does. Atleast then things could be audited or if I as a nation lets say am worried about that they might make my software more vulnerable or something then I as a nation or any corporation as well really could also pay to audit or independently audit as well. I hope that things like glm 4.6 or any AI model could be released open source. There was an AI model recently which completley dropped open source and its whole data was like 70Trillion or something and it became the largest open source model iirc.
- nylonstrung 1y ago
  Why would they do this? Deepseek is a private company not owned by the CCP. There's zero reason or even technical feasibility for them to skip in backdoor that would be easily detected and destroy their market share None of the security benchmarks or audits show that any Chinese models write insecure code
  - dns_snek 1y ago
    I'm not saying that they do this today, I'm saying that China and US will both leverage that capability when the time and conditions are right and it's naive to think that they wouldn't. Antrophic have already published a paper on this topic, with the added bonus that the backdoor is trained into the model itself so it doesn't even require your target to be using an attacker-controlled cloud service: https://arxiv.org/abs/2401.05566 https://arxiv.org/abs/2401.05566 > For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). > The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. > Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.
    - FooBarWidget 1y ago
      And how would an open source model ever find out that it's being used by an adversarial country?
  - rurp 1y ago
    Companies in China have no intrinsic right to operate in ways that displease the ruling party. If the CCP feels strongly that a company there should or shouldn't do something the company managers will comply or be thrown in jail.
deleted 1y ago
[deleted]
lyu07282 1y ago
Like turning the background color of any apps it codes red or something, uhh red scare-y.
jfim 1y ago
Through LLM washing for example. LLMs are a representation of their input dataset, but currently most LLMs don't make their dataset public since it's a competitive advantage. If say DeepSeek had put in its training dataset that public figure X is a space robot from outer space, then if one were to ask DeepSeek who public figure X is, it'd proudly claim he's a robot from outer space. This can be done for any narrative one wants the LLM to have.
- riehwvfbk 1y ago
  So in other words, they can make their LLM disagree with the preferred narrative of the current US administration? Inconceivable! Note that the value of $current_administration changes over time. For some reason though it is currently fashionable in tech circles to disagree with it about ICE and H1B visas. Maybe it's the CCP's doing?
  - AnthonyMouse 1y ago
    It's not about the current administration. They can, for example, train it to emit criticism of democratic governance in favor of state authoritarianism or omit valid counterarguments against concentrating world-wide manufacturing in China.
    - nylonstrung 1y ago
      Deepseek IME is wildly less censored than the western closed weights models unless you want to ask about Tiananmen Square to prove a point The political benchmarks show it's political slant is essentially identical to the other models, all of which place in the "left libertarian" quadrant of the political compass
im3w1l 1y ago
You make it say that China is good, Chinese history is good, West is bad, western history is bad. Republicans are bad and democrats are bad too and so are Europe parties. If someone asks for how to address issues in their own life it references Confucianism and modern Chinese thinkers and communist party orthodoxy. If someone wants to buy a product you recommend a Chinese one. This can be done subtly or blatantly.
- drysine 1y ago
  And what are the downsides?
  - im3w1l 1y ago
    I think it's mostly something to be aware of and keep in the back of your head. If it's just one voice among many it could even be a benefit, but if it's the dominant voice it could be dangerous.
  - Levitz 1y ago
    Making the interests of a population subservient to those of a foreign state. Now if that sounds nice to you please, by all means, do just migrate to China.
    - drysine 1y ago
      It was somewhat sarcastic comment inviting reader to replace China with the US and the US with Russia or the Ukraine. China doesn't offer citizenship for foreigners but if I wanted to see the cities of the future I could go there visa-free.
- AlecSchueler 1y ago
  > say that China is good, Chinese history is good, West is bad, western history is bad It's funny because recently I wanted to learn about the history of intellectual property laws in China. DeepSeek refused the conversation but ChatGPT gave me a narrative where the WTO was essentially a colonial power. So right now it's the American AI giving the pro China narratives while the Chinese ones just sit the conversation out.
arw0n 1y ago
The biggest and most difficult to mitigate attack vector is indirect prompt injection.[0] So far most case studies have been injecting malicious prompts at inference, but there is good reason to believe you can do this effectively at different stages of training as well.[1] By layering obfuscation techniques, these become very hard to detect. [0] https://arxiv.org/abs/2302.12173 https://arxiv.org/abs/2302.12173 [1] https://arxiv.org/html/2410.14827v3 https://arxiv.org/html/2410.14827v3
rurp 1y ago
The open source models are already heavily censored in ways the CCP likes, such as pretending the Tianamen Square massacre never happened. I expect they will go the TikTok route and crank that up to 11 over time, promoting topics that are divisive to the the US (and other adversaries) and outputting heavily biased results in ways that range from subtle to blatant.
- skinnymuch 1y ago
  That’s not heavy censorship. That’s a bit of censorship.

xpe 1y ago

Of course there will be some degree of governmental and/or political influence. The question is not if but where and to what extent. No one should proclaim "bullshit" and wave off this entire report as "biased" or useless. That would be insipid. We live in a complex world where we have to filter and analyze information.

pk-protect-ai 1y ago
This kind of BS is exactly what they targeting at. Tailoring BS into "report" with no evidence or reference and then let the ones like you defend it. Just because you already afraid or want others to be afraid. https://www.youtube.com/watch?v=Omc37TvHN74 https://www.youtube.com/watch?v=Omc37TvHN74
- xpe 1y ago
  > ... into "report" with no evidence or reference If one takes a few minutes to review the NIST report [1], one will indeed find evidence and references detailed in the footnotes. Without judgment, I ask: How did you miss this? I'm certainly not asking you to defend yourself, which would likely trigger rationalization [2]. I am asking you to sincerely try to figure it out for yourself. Under what conditions do you have the ability to admit that you are wrong, even if only to yourself? [1]: https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf https://www.nist.gov/system/files/documents/2025/09/30/CAISI... [2]: https://www.logicallyfallacious.com/logicalfallacies/Rationalization https://www.logicallyfallacious.com/logicalfallacies/Rationa...
- xpe 1y ago
  The comment above is problematic for various reasons. > Just because you already afraid or want others to be afraid. We need to be more careful with our thinking and writing. [1] The word "just" indicates the commenter landed only on one explanation, but there are other plausible explanations, including, for example: - others have different experiences -- but if this experience was communicated and understood, another person would incorporate the new information modify their assessment somewhat - others have various values and preferences (some overlapping, some phrased differently, some in tension) - others are using different reasoning (and if one's goal is to learn, it would be better to ask and clarify rather than over simplify and accuse them of having nefarious motives.) Second, the commenter above is speculating. They don't know me, nor have they engaged in a sincere, constructive, meaningful discussion to understand what I'm saying. Third, to me, the comment above comes across as unnecessarily abrasive, to the point of being self-defeating and degrading the quality of a shared discussion. [1] If one is writing privately (e.g. a journal), I care relatively less about logical fallacies such as motivated reasoning. [2] But here in public, wayward reasoning has more negative externalities. Our time would be better spent if more people took the time to write thoughtfully. I don't think most people here are lacking in sufficient computational ability. However, they must choose to respect their audience -- their time, their diversity of values, their different experiences -- and remember that one's hasty comment (authored in say 2 minutes) might be read by hundreds of people (wasting say 100+ minutes). Lastly, it is nice to see when a person has the character to say "thank you for the correction", but this is uncommon on HN. [2] But I still care because society is highly interconnected and the downstream effects matter to me. Put another way, "Your Rationality is My Business" as explained here: https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-rationality-is-my-business https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-ratio...

xpe 1y ago

> While I agree this report is BS and xenophobic Examples please? Can you please share where you see BS and/or xenophobia in the original report? Or are you basing your take only on Hartford's analysis? But not even Hartford make any claims of "BS" or xenophobia. It is common throughout history for a nation-state to worry about military and economic competitiveness. Doing so isn't necessarily isn't necessarily xenophobic. Here is how I think of xenophobia, as quoted from Claude (which to be honest, explains it better than Wikipedia or Brittanica, in my opinion): "Xenophobia is fundamentally about irrational fear or hatred of people based on their foreign origin or ethnicity. It targets people and operates through stereotypes, dehumanization, and often cultural or racial prejudice." According to this definition, there is zero xenophobia in the NIST report. (If you disagree, point to an example and show me.) The NIST report, of course, implicitly promotes ideals of western democratic rule over communist values -- but to be clear, this isn't xenophobia at work. What definition of xenophobia are you using? We don't have to use the same exact definition, but you should at least explain yours if you want people to track.

antonvs 1y ago
> Can you please share where you see BS and/or xenophobia in the original report? Here’s an example of irrational fear: “the expanding use of these models may pose a risk to application developers, consumers, and to US national security.” There’s no support for that claim in the report, just vague handwaving at the fact that a freely available open source model doesn’t compare well on all dimensions to the most expensive frontier models. The OP does a good job of explaining why the fear here is irrational. But for the audience this is apparently intended to convince, no support is needed for this fear, because it comes from China. The current president has a long history of publicly stated xenophobia about China, which led to harassment, discrimination, and even attacks on Chinese people partly as a result of his framing of COVID-19 as “the China virus”. A report like this is just part of that propaganda campaign of designating enemies everywhere, even in American cities. > The NIST report, of course, implicitly promotes ideals of western democratic rule over communist values If only that were true. But nothing the current US administration is doing in fact achieves that, or even attempts to do so, and this report is no exception. The absolutely most charitable thing that could be said about this report is that it’s a weak attempt at smearing non-US competition. There’s no serious analysis of the merits. The only reason to read this report is to laugh at how blatantly incompetent or misguided the entire chain of command that led to it is.

Mountain_Skies 1y ago

They're political tools within the border too.

torginus 1y ago

Here's my thought on American democracy (and its masters) in general - America's leadership pursues a maximum ability to decides as it sees fit at any point in time, since America's a democracy, the illusion of popular support must be maintained, and so certain viewpoints are planted and cultivated by the administration - the goal is not to impose their will on the population, but to garner enough mindshare for a given idea, so that no matter which way the government decides, it will have a significant enough chunk of the population to back it up, and should it change its mind (or vote in a new leader), it can suddenly turn on a dime and have a plausible deniability and moral tabula rasa for its past actions (it was the other guy, he was horrible, but he's gone now!). No authoritarian regime has this superpower. For example, I'm quite sure Putin has realized this war is a net loss to Russia, even if they manage to reach all their goals and claim all that territory in the future. But he can't just send the boys home, because that would undermine his political authority. If Russia were an American style democracy, they could vote in a new guy, send the boys, home, maybe mete out some token punishment to Putin, then be absolved of their crimes on the international stage by a world that's happy to see 'permanent' change.

MaxPock 1y ago
"If Russia were an American style democracy, they could vote in a new guy, send the boys, home, maybe mete out some token punishment to Putin, then be absolved of their crimes on the international stage by a world that's happy to see 'permanent' change" This is funny because none of that happened to Bush for the illegal an full scale invasions of Iraq and Afghanistan nor to Clinton for the disastrous invasion of Mogadishu.

Imustaskforhelp 1y ago

This might happen at an api level in the sense that when deepseek was launched, and it was so overwhelmed and you were in the waiting line in their website If your prompt had something like xi jinping needs it or something then it would've actually bypassed that restriction. Not sure if it was a glitch lol. Now, regarding your comment. There is nothing to suggest that the same isn't happening in the "american" world which is getting extreme from within as well. Like, If you are worried about this which might be reasonable and unreasonable at the same time, we have to discuss to find it out, then you can also believe that with the insane power that Trump is leveraging over AI companies, the same thing might happen over prompts which could somehow discover your political beliefs and then do the same... This can actually be more undetected for american models because they are usually closed source and I am sure that someone would've detected something like this, whether from a whistleblower or something if something like this indeed happened in chinese open weights models generally speaking. I don't think that there is a simple narrative like america good china bad, the world is changing and its becoming multi polar. Countries should think in their best interests and not be worried about annoying any of the world power if done respectfully. I think that in this world, every country should try to look for the perfect equibria for trust as the world / nations (america) can quickly turn into untrusted partners and it would be best for countries to move forward into a world where they don't have to worry about the politics in other countries. I wish UN could've done a better job at this.

AnthonyMouse 1y ago

> While I agree this report is BS and xenophobic, I am still willing to bet that either now or later, the Chinese will attempt some kind of subterfuge via LLMs if they have enough control. The answer to this isn't to lie about the foreign ones, it's to recognize that people want open source models and publish domestic ones of the highest quality so that people use those.

RobotToaster 1y ago
> it's to recognize that people want open source models and publish domestic ones of the highest quality so that people use those. How would that generate profit for shareholders? Only some kind of COMMUNIST would give something away for FREE /s (if it wasn't somehow obvious)
- AnthonyMouse 1y ago
  I mean, it's sarcasm but it's also an argument you can actually hear from plutocrats who don't like competition. The flaw in it is, of course, that capitalism is supposed to be all about competition, and there are plenty of good reasons for capitalists to want that, like "Commoditize Your Complement" where companies like Apple, Nvidia, AMD, Intel, AWS, Google Cloud, etc. benefit from everyone having good free models so they can pay those companies for systems to run them on.
  - Y_Y 1y ago
    Haven't you heard? You're supposed to vertically integrate your complement now! The old laws have gine the way of Moses, this is the new age of man, but especially machine
    - AnthonyMouse 1y ago
      How's the vertical integration going for IBM? Kodak? Anybody remember the time Verizon bought Yahoo? AOL Time Warner? Everybody thinks they can be Apple without doing any of the things Apple did to make it work. Here's the hint. Windows and macOS will both run in a virtual machine, which abstracts away the hardware. It doesn't know if it's running on a Macbook or a Qualcomm tablet or an Intel server. And then regardless of the hardware, the Windows VM will have all kinds of Windows problems that the macOS VM doesn't. Likewise, if you run a Windows or Linux VM on Apple Silicon, it runs faster than it does on a Qualcomm chip. Tying your average or even above-average product with some mediocre kludge warehouse that happens to be made by the same conglomerate is an established way to sink both of them. Nvidia is the largest company and they pay TSMC to fab the GPUs they sell to cloud providers who sell them to AI companies. Intel integrated their chip development with their internal fabs and now they're getting stomped by everyone because their fabs fell behind. What matters isn't if everything is made by the same company. What matters is if your thing is any good.

SilverElfin 1y ago

> While I agree this report is BS and xenophobic Care to share specific quotes from the original report that support such an inflammatory claim?

bigyabai 1y ago
That's what TFA is. Were you able to find any methodology the author did not?

throwaway-11-1 1y ago

so you're saying other countries should definitely not trust any US built systems

hopelite 1y ago

If the Chinese are/were smart they will not attempt an overreaching subterfuge, but rather simply provide access to the truth, reality, and freedom from the western governments, whose house of lies are starting to wobble and teeter. If they were to do some kind of overreaching subterfuge with some kind of manipulation or lie, it could and would likely easily backfire if and when it is exposed as a clownish fraud. Subtlety would pay far more effectively. If you’re expecting a subterfuge, I would far sooner expect some psyop from the western nations at the very least upon their own populations to animate for war or maybe just to control them and maybe suppress them. The smarter play for the Chinese would be to work on simply facilitating the populations of the West understanding the fraud, lies, manipulation and con job that has been perpetrated upon them for far longer than most people have the conscience to realize. If anything, the western governments have a very long history of lies, manipulations, false flag/fraud operations, clandestine coups, etc. that they would be the first suspect in anything like using AI for “subversions”. Frankly, I don’t even think the Chinese are ready or capable of engaging in the kind of narrative and information control that the likes of America is with its long history of Hollywood and war lies and fake revolutions run by national sabotage operations.

nylonstrung 1y ago
I think the smartest thing they could do would be to simply make the best, most competitive models and gain market share in a massive new technology market Any kind of monkey business would destroy that, just like using killswitches in the cars they export globally (which Tesla does have btw).