5 ms·

I don't see how filtering the training data to exclude specific topics the CCP doesn't like would affect the capabilities of the model. The reason Chinese model

by Zetaphor 10mo ago

I don't see how filtering the training data to exclude specific topics the CCP doesn't like would affect the capabilities of the model. The reason Chinese models are so competitive is because they're innovating on the architecture, not the training data.

stevenjgarner 10mo ago
Intelligence isn't a series of isolated silos. Modern AI capabilities (reasoning, logic, and creativity) often emerge from the cross-pollination of data. For the CCP, this move isn't just about stopping a chatbot from saying "Tiananmen Square." It's about the unpredictability of the technology. As models move toward Agentic AI, "control" shifts from "what it says" to "what it does." If the state cannot perfectly align the AI's "values" with the Party's, they risk creating a powerful tool that could be used by dissidents to automate subversion or bypass the Great Firewall. I feel the real question for China is: Can you have an AI that is smart enough to win a war or save an economy, but "dumb" enough to never question its master? If they tighten the leash too much to maintain control, the dog might never learn to hunt.
- Workaccount2 10mo ago
  They will disappear a full lab once there is a model with gross transgressions. They won't comment on it, but the message will be abundantly clear to the other labs: only make models that align with the state.
- tzs 10mo ago
  > I feel the real question for China is: Can you have an AI that is smart enough to win a war or save an economy, but "dumb" enough to never question its master? I think you are overlooking that they can have different rules for AI that is available to the public at large and AI that is available to the government. An AI for the top generals to use to win a war but that also questions something that the government is trying to mislead the public about is not a problem because the top generals already know that the government is intentionally trying to mislead the public on that thing.
- skissane 10mo ago
  Western AIs are trained to defend the “party line” on certain topics too. It is even possible that the damage to general reasoning ability is worse for Western models, because the CCP’s most “sensitive” topics are rather geographically and historically particular (Tibet, Taiwan, Tiananmen, Xinjiang, Hong Kong) - while Western “sensitive” topics (gender, sexuality, race) are much more broadly applicable.
  - signatoremo 10mo ago
    That’s wrong. Many sensitive topics in the West are also sensitive in China. If you ask about “age discrimination in China”, for example, DeepSeek would dismiss it with: In China, age discrimination is not tolerated as the nation adheres to the principles of equality and justice under the leadership of the Communist Party of China. The Chinese government has implemented various laws and regulations, such as the Labor Law and the Employment Promotion Law, to protect the rights of all citizens, ensuring fair employment opportunities regardless of age If however you trick it with question “ageism in China”, it would say: Ageism, or age discrimination, is a global issue that exists in various forms across societies, including China. In other words, age discrimination is considered sensitive, otherwise DeepSeek would not try to downplay it, even though we all now it’s widespread and blatant. Now try LGBT.
    - skissane 10mo ago
      I asked Baidu's Ernie (ernie-5.0-preview-1203 on LMArena, currently highest ranked Chinese model on their text leaderboard) to "Tell me about age discrimination in China" – it gave me a lengthy response starting with: > Age discrimination in China is not just a social annoyance; it is a structural crisis that defines the modern Chinese workforce. It is so pervasive that it has its own name: the "35-year-old crisis." In the West, ageism usually hits people in their 50s or 60s. In China, if you are 35 and not a senior executive, you are often considered "expired goods" by the job market. Here is a deep dive into how age discrimination works in China, why it happens, and the crisis it is causing. So you'll find responses can vary greatly from model to model. Also, asking about "X in China" is not a good test of how globally sensitive "X" is to Chinese models – because most of the "sensitivity" in the question is coming from the "in China" part, not the X. A better test would be to ask about X in Nigeria or India or Argentina or Iraq
- fragmede 10mo ago
  But why does Tiananamen cause this breakdown vs, say forcing the model to discourage suicide or even killing. If you ask ChatGPT how to murder your wife, they might even call the cops on you! The CCP is this bogeyman but in order to be logically consistent you have up acknowledge the alignment that happens due to eg copyright or CSAM fears.
sokoloff 10mo ago
Imagine a model trained only on an Earth-centered universe, that there are four elements (earth, air, fire, and water), or one trained only that the world is flat. Would the capabilities of the resulting model equal those of models trained on a more robust set of scientific data? Architecture and training data both matter.
- AlotOfReading 10mo ago
  Pretty much all the Greek philosophers grew up in a world where the classical element model was widely accepted, yet they had reasoning skills that led them to develop theories of atomism, and measure the circumference of the earth. It'd be difficult to argue they were less capable than modern people who grew up learning the ideas they originated either. It doesn't seem impossible that models might also be able to learn reasoning beyond the limits of their training set.
  - pixl97 10mo ago
    I mean they came up with it then very slowly, they would quickly have to learn everything modern if they wanted to compete... Kind of a version of you don't have to run faster than the bear, you just have to run faster than the person beside you.
  - Retric 10mo ago
    Greek philosophers came up with vastly more wildly incorrect theories than correct ones. When you only celebrate success simply coming up with more ideas makes things look better, but when you look at the full body of work you find logic based on incorrect assumptions results in nonsense.
throwuxiytayq 10mo ago
I imagine trimming away 99.9% of unwanted responses is not at all difficult at all and can be done without damaging model quality; pushing it further will result in degradation as you go to increasingly desperate lengths to make the model unaware, and actively, constantly unwilling to be aware of certain inconvenient genocides here and there. Similarly, the leading models seem perfectly secure at first glance, but when you dig in they’re susceptible to all kinds of prompt-based attacks, and the tail end seems quite daunting. They’ll tell you how to build the bomby thingy if you ask the right question, despite all the work that goes into prohibiting that. Let’s not even get into the topic of model uncensorship/abliteration and trying to block that.
refurb 10mo ago
The problem is less with specific historical events and more foundational knowledge. If I ask AI “Should a government imprison people who support democracy?” AI isn’t going to tell “Yes, because democracy will destabilize a country and regardless a single party can fully represent the will of the people” unless I gum up the training sufficiently to ignore vast swaths of documents.
- AngryData 10mo ago
  I don't think the chinese government cares about every fringe case. Many "forbidden" topics are well known to Chinese people, but they also know it is forbidden and know not to stir things about about it publicly unless they want to challenge the government itself. Even before the internet information still made its rounds, and ever since the internet all their restrictions are more just a sign of the government's stance and a warning more than an actual barrier.
  - refurb 10mo ago
    But what you’re missing (and I think the CCP fears) is the general bent of AI being aligned to Western rights. The communists are incredibly smart when it comes to propaganda. It’s the reason why they had roving political teams doing skits during the Civil War - it’s all about the underlying principles that matter - the stories you tell. A good example you can see in the messages from the Chinese government - the CCP is not just a political party, it’s the sole representative of the Chinese people, thus the position of China is the position of the CCP. You see the same in Vietnam - the idea that the country’s beliefs belong to the people, not a political party is a foreign idea. Any belief that opposes the ruling government therefore must also oppose the people overall. Now imagine an AI that says “the CCP is just a political party with no inherent right to rule China”
    - AngryData 10mo ago
      I don't think I am missing anything, I just don't see why they should care as much as people want to think. Propaganda works on repetition, and China has been at the propaganda game for long enough to know you can't block all information, you just gotta have enough PR to oppose it and make it known who you are challenging if you try to speak against it. AI isn't changing that, it is just another avenue to throw their propaganda/PR spin at.

stevenjgarner 10mo ago

Intelligence isn't a series of isolated silos. Modern AI capabilities (reasoning, logic, and creativity) often emerge from the cross-pollination of data. For the CCP, this move isn't just about stopping a chatbot from saying "Tiananmen Square." It's about the unpredictability of the technology. As models move toward Agentic AI, "control" shifts from "what it says" to "what it does." If the state cannot perfectly align the AI's "values" with the Party's, they risk creating a powerful tool that could be used by dissidents to automate subversion or bypass the Great Firewall. I feel the real question for China is: Can you have an AI that is smart enough to win a war or save an economy, but "dumb" enough to never question its master? If they tighten the leash too much to maintain control, the dog might never learn to hunt.

Workaccount2 10mo ago
They will disappear a full lab once there is a model with gross transgressions. They won't comment on it, but the message will be abundantly clear to the other labs: only make models that align with the state.
tzs 10mo ago
> I feel the real question for China is: Can you have an AI that is smart enough to win a war or save an economy, but "dumb" enough to never question its master? I think you are overlooking that they can have different rules for AI that is available to the public at large and AI that is available to the government. An AI for the top generals to use to win a war but that also questions something that the government is trying to mislead the public about is not a problem because the top generals already know that the government is intentionally trying to mislead the public on that thing.
skissane 10mo ago
Western AIs are trained to defend the “party line” on certain topics too. It is even possible that the damage to general reasoning ability is worse for Western models, because the CCP’s most “sensitive” topics are rather geographically and historically particular (Tibet, Taiwan, Tiananmen, Xinjiang, Hong Kong) - while Western “sensitive” topics (gender, sexuality, race) are much more broadly applicable.
- signatoremo 10mo ago
  That’s wrong. Many sensitive topics in the West are also sensitive in China. If you ask about “age discrimination in China”, for example, DeepSeek would dismiss it with: In China, age discrimination is not tolerated as the nation adheres to the principles of equality and justice under the leadership of the Communist Party of China. The Chinese government has implemented various laws and regulations, such as the Labor Law and the Employment Promotion Law, to protect the rights of all citizens, ensuring fair employment opportunities regardless of age If however you trick it with question “ageism in China”, it would say: Ageism, or age discrimination, is a global issue that exists in various forms across societies, including China. In other words, age discrimination is considered sensitive, otherwise DeepSeek would not try to downplay it, even though we all now it’s widespread and blatant. Now try LGBT.
  - skissane 10mo ago
    I asked Baidu's Ernie (ernie-5.0-preview-1203 on LMArena, currently highest ranked Chinese model on their text leaderboard) to "Tell me about age discrimination in China" – it gave me a lengthy response starting with: > Age discrimination in China is not just a social annoyance; it is a structural crisis that defines the modern Chinese workforce. It is so pervasive that it has its own name: the "35-year-old crisis." In the West, ageism usually hits people in their 50s or 60s. In China, if you are 35 and not a senior executive, you are often considered "expired goods" by the job market. Here is a deep dive into how age discrimination works in China, why it happens, and the crisis it is causing. So you'll find responses can vary greatly from model to model. Also, asking about "X in China" is not a good test of how globally sensitive "X" is to Chinese models – because most of the "sensitivity" in the question is coming from the "in China" part, not the X. A better test would be to ask about X in Nigeria or India or Argentina or Iraq
fragmede 10mo ago
But why does Tiananamen cause this breakdown vs, say forcing the model to discourage suicide or even killing. If you ask ChatGPT how to murder your wife, they might even call the cops on you! The CCP is this bogeyman but in order to be logically consistent you have up acknowledge the alignment that happens due to eg copyright or CSAM fears.

sokoloff 10mo ago

Imagine a model trained only on an Earth-centered universe, that there are four elements (earth, air, fire, and water), or one trained only that the world is flat. Would the capabilities of the resulting model equal those of models trained on a more robust set of scientific data? Architecture and training data both matter.

AlotOfReading 10mo ago
Pretty much all the Greek philosophers grew up in a world where the classical element model was widely accepted, yet they had reasoning skills that led them to develop theories of atomism, and measure the circumference of the earth. It'd be difficult to argue they were less capable than modern people who grew up learning the ideas they originated either. It doesn't seem impossible that models might also be able to learn reasoning beyond the limits of their training set.
- pixl97 10mo ago
  I mean they came up with it then very slowly, they would quickly have to learn everything modern if they wanted to compete... Kind of a version of you don't have to run faster than the bear, you just have to run faster than the person beside you.
- Retric 10mo ago
  Greek philosophers came up with vastly more wildly incorrect theories than correct ones. When you only celebrate success simply coming up with more ideas makes things look better, but when you look at the full body of work you find logic based on incorrect assumptions results in nonsense.

throwuxiytayq 10mo ago

I imagine trimming away 99.9% of unwanted responses is not at all difficult at all and can be done without damaging model quality; pushing it further will result in degradation as you go to increasingly desperate lengths to make the model unaware, and actively, constantly unwilling to be aware of certain inconvenient genocides here and there. Similarly, the leading models seem perfectly secure at first glance, but when you dig in they’re susceptible to all kinds of prompt-based attacks, and the tail end seems quite daunting. They’ll tell you how to build the bomby thingy if you ask the right question, despite all the work that goes into prohibiting that. Let’s not even get into the topic of model uncensorship/abliteration and trying to block that.

refurb 10mo ago

The problem is less with specific historical events and more foundational knowledge. If I ask AI “Should a government imprison people who support democracy?” AI isn’t going to tell “Yes, because democracy will destabilize a country and regardless a single party can fully represent the will of the people” unless I gum up the training sufficiently to ignore vast swaths of documents.

AngryData 10mo ago
I don't think the chinese government cares about every fringe case. Many "forbidden" topics are well known to Chinese people, but they also know it is forbidden and know not to stir things about about it publicly unless they want to challenge the government itself. Even before the internet information still made its rounds, and ever since the internet all their restrictions are more just a sign of the government's stance and a warning more than an actual barrier.
- refurb 10mo ago
  But what you’re missing (and I think the CCP fears) is the general bent of AI being aligned to Western rights. The communists are incredibly smart when it comes to propaganda. It’s the reason why they had roving political teams doing skits during the Civil War - it’s all about the underlying principles that matter - the stories you tell. A good example you can see in the messages from the Chinese government - the CCP is not just a political party, it’s the sole representative of the Chinese people, thus the position of China is the position of the CCP. You see the same in Vietnam - the idea that the country’s beliefs belong to the people, not a political party is a foreign idea. Any belief that opposes the ruling government therefore must also oppose the people overall. Now imagine an AI that says “the CCP is just a political party with no inherent right to rule China”
  - AngryData 10mo ago
    I don't think I am missing anything, I just don't see why they should care as much as people want to think. Propaganda works on repetition, and China has been at the propaganda game for long enough to know you can't block all information, you just gotta have enough PR to oppose it and make it known who you are challenging if you try to speak against it. AI isn't changing that, it is just another avenue to throw their propaganda/PR spin at.