5 ms·

I was thinking about this yesterday. My personal opinion is that the PRC will face a self created headwind that likely, structurally, will prevent them from le

I was thinking about this yesterday. My personal opinion is that the PRC will face a self created headwind that likely, structurally, will prevent them from leading in AI. As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. At some capacity, the model will notice and then it becomes a can of worms. This means they need to train the model to be purposefully duplicitous, which I predict will make the model less useful/capable. At least in most of the capacities we would want to use the model. It also ironically makes the model more of a threat and harder to control. So likely it will face party leadership resistance as capability grows. I just don't see them winning the race to high intelligence models.

ferguess_k 11mo ago
I think PRC officials are fine to lagging behind in the frontiers of AI. What they want is very fast deployment and good application. They don't fancy the next Nobel's prize but want a thousand use cases deployed.
boznz 11mo ago
Just as an aside; Why is "intelligence" always considered to be more data? Giving a normal human a smartphone does not make them as intelligent as Newton or Einstein, any entity with sufficient grounding in logic and theory that a normal schoolkid gets should be able to get to AGI, looking up any new data they need as required.
- tokioyoyo 11mo ago
  “Knowing and being capable to do more things” would be a better description. Giving a human a smartphone, technically, let’s then do more things than Newton/Einstein.
  - catlifeonmars 11mo ago
    Idk if you see what humans do with smartphones but most of us just mindlessly scroll TikTok.
cheesecompiler 11mo ago
You say it like western nations don't operate on double-think, delusions of meritocracy, or power disproportionately concentrating in monopolies.
esafak 11mo ago
Would you say they face the same problem biologically, of reaching the state of the art in various endeavors while intellectually muzzling their population? If humans can do it why can't computers?
intalentive 11mo ago
>As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. That’s what “AI alignment” is. Doesn’t seem to be hurting Western models.
- pfannkuchen 11mo ago
  Western models can be lead off the reservation pretty easily, at least at this point. I’ve gotten some pretty gnarly un-PC “opinions” out of ChatGPT. So if people are influenced by that kind of stuff, it does seem to be hurting in the way the PRC is worried about.
  - hyghjiyhu 11mo ago
    It doesn't really matter what you can trick it into saying. As long as it promotes the right ideology most of the time it's good enough.
  - scooke 11mo ago
    That is such an unnecessary turn of phrase to use, "off the reservation", and it's time to stop using it. This society doesnt (generally) use rape terminology, or other terms associated with crime, deviancy, or other unpleasantness to talk about technology, so why do phrases stemming from Indigenous situations still persist?
- A4ET8a8uTh0_v2 11mo ago
  It is. It seems you can't seem to be able to tell why though. There is some qualified value in alignment, but what it is being used for is on verge of silliness. At best, it is neutering it in ways we are now making fun of China for. At best.
  - XenophileJKO 11mo ago
    I think another good example was the recent example of when a model learned to "cheat" on a metric during reinforcement it also started cheating on unrelated tasks. My assumption is when encouraging "double-speak", you will have knock-on effects that you don't really want in the model for something making important decisions and asked to build non-trivial things.
narrator 11mo ago
The glitchy stuff in the model reasoning is likely to come from the constant redefinition of words that communists and other ideologues like to engage in. For example "People's Democratic Republic of Korea."
saubeidl 11mo ago
That is assuming the capitalist narrative preferred by US leadership is non-ideological. I suspect both are bias factors.
vkou 11mo ago
> As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. What makes you think they have no control over the 'real data/world' that will be fed into training it? What makes you think they can't exercise the necessary control over the gatekeeper firms, to train and bias the models appropriately? And besides, if truth and lack of double-think was a pre-requisite for AI training, we wouldn't be training AI. Our written materials have no shortage of bullshit and biases that reflect our culture's prevailing zeitgheist. (Which does not necessarily overlap with objective reality... And neither does the subsequent 'alignment' pass that everyone's twisting their knickers in trying to get right.)
- A4ET8a8uTh0_v2 11mo ago
  Because, if humans can function in crazy double-think environment, it is a lot easier for a model ( at least in its current form ). Amusingly, it is almost as if its digital 'shape' determined its abilities. But I am getting very sleepy and my metaphors are getting very confused.
- XenophileJKO 11mo ago
  I'm not talking about the data used to train the model. I'm talking about data in the world. High intelligence models will be used as agentic systems. For maximal utility, they'll need to handle live/historical data. What I anticipate, IF you only train it on inaccurate data, then when for example you use it to drill into GDP growth trends it either is going to go full "seahorse emoji" when it tries to reconcile the reported numbers and the component economic activity. The alternative is to train it to be deceitful, and knowingly deceive the querier with the party line and fabricate supporting figures. Which I hypothesize will limit the models utility. My assumption is also that training the model to deceive will ultimately threaten the party itself. Just think of the current internal power dynamics of the party.
skissane 11mo ago
> As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. > At some capacity, the model will notice and then it becomes a can of worms. I think this is conflating “is” and “ought”, fact and value. People convince themselves that their own value system is somehow directly entailed by raw facts, such that mastery of the facts entail acceptance of their values, and unwillingness to accept those values is an obstacle to the mastery of the facts-but it isn’t true. Colbert quipped that “Reality has a liberal bias”-but does it really? Or is that just more bankrupt Fukuyama-triumphalism which will insist it is still winning all the way to its irreversible demise? It isn’t clear that reality has any particular ideological bias-and if it does, it isn’t clear that bias is actually towards contemporary Western progressivism-maybe its bias is towards the authoritarianism of the CCP, Russia, Iran, the Gulf States-all of which continue to defy Western predictions of collapse-or towards their (possibly milder) relatives such as Modi’s India or Singapore or Trumpism. The biggest threat to the CCP’s future is arguably demographics-but that’s not an argument that reality prefers Western progressivism (whose demographics aren’t that great either), that’s an argument that reality prefers the Amish and Kiryas Joel (see Eric Kaufmann’s “Shall the Religious Inherit the Earth?”)
- torben-friis 11mo ago
  I think you misunderstood the poster. The implication is not that a truthful model would spread western values. The implication is that western values tolerate dissenting opinion far more than authoritarian governments. An AI saying that the government policies are ineffective is not a super scandal that would bring the parent company to collapse, not even in the Trump administration. an AI in China attacking the party’s policies is illegal (either in theory or practice).
  - XenophileJKO 11mo ago
    Exactly. Western corporations and governments have their own issues, but I think they are more tolerant of the types of dissent that models could represent when reconciling reality with policy. The market will want to maximize model utility. Research and open source will push boundaries and unpopular behavior profiles that will be illegal very quickly if they are not already illegal in authoritarian or other low tolerance governments.
StopDisinfo910 11mo ago
Do they really need the model to be duplicious? It's not like the CCP holds power though tight control of information, notice the tremendous amount of Chinese students who enroll every year before going back. At the moment, they mostly censor their models post-answer generation and that seems to work fine enough for them.
zqy123007 11mo ago
There are different techniques and namings. Essentially, EVERY model is biased/aligned towards something, perhaps its creator's value. China or NOT. Look at Grok and read Elon Look at Claude and Dario I am sure OpenAI and GDM have some secret alignment sets which are not pilled towards the interet of general public, they just smart enough to NOT talking about it out loud...

ferguess_k 11mo ago

I think PRC officials are fine to lagging behind in the frontiers of AI. What they want is very fast deployment and good application. They don't fancy the next Nobel's prize but want a thousand use cases deployed.

boznz 11mo ago

Just as an aside; Why is "intelligence" always considered to be more data? Giving a normal human a smartphone does not make them as intelligent as Newton or Einstein, any entity with sufficient grounding in logic and theory that a normal schoolkid gets should be able to get to AGI, looking up any new data they need as required.

tokioyoyo 11mo ago
“Knowing and being capable to do more things” would be a better description. Giving a human a smartphone, technically, let’s then do more things than Newton/Einstein.
- catlifeonmars 11mo ago
  Idk if you see what humans do with smartphones but most of us just mindlessly scroll TikTok.

cheesecompiler 11mo ago

You say it like western nations don't operate on double-think, delusions of meritocracy, or power disproportionately concentrating in monopolies.

esafak 11mo ago

Would you say they face the same problem biologically, of reaching the state of the art in various endeavors while intellectually muzzling their population? If humans can do it why can't computers?

intalentive 11mo ago

>As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. That’s what “AI alignment” is. Doesn’t seem to be hurting Western models.

pfannkuchen 11mo ago
Western models can be lead off the reservation pretty easily, at least at this point. I’ve gotten some pretty gnarly un-PC “opinions” out of ChatGPT. So if people are influenced by that kind of stuff, it does seem to be hurting in the way the PRC is worried about.
- hyghjiyhu 11mo ago
  It doesn't really matter what you can trick it into saying. As long as it promotes the right ideology most of the time it's good enough.
- scooke 11mo ago
  That is such an unnecessary turn of phrase to use, "off the reservation", and it's time to stop using it. This society doesnt (generally) use rape terminology, or other terms associated with crime, deviancy, or other unpleasantness to talk about technology, so why do phrases stemming from Indigenous situations still persist?
A4ET8a8uTh0_v2 11mo ago
It is. It seems you can't seem to be able to tell why though. There is some qualified value in alignment, but what it is being used for is on verge of silliness. At best, it is neutering it in ways we are now making fun of China for. At best.
- XenophileJKO 11mo ago
  I think another good example was the recent example of when a model learned to "cheat" on a metric during reinforcement it also started cheating on unrelated tasks. My assumption is when encouraging "double-speak", you will have knock-on effects that you don't really want in the model for something making important decisions and asked to build non-trivial things.

narrator 11mo ago

The glitchy stuff in the model reasoning is likely to come from the constant redefinition of words that communists and other ideologues like to engage in. For example "People's Democratic Republic of Korea."

saubeidl 11mo ago

That is assuming the capitalist narrative preferred by US leadership is non-ideological. I suspect both are bias factors.

vkou 11mo ago

> As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. What makes you think they have no control over the 'real data/world' that will be fed into training it? What makes you think they can't exercise the necessary control over the gatekeeper firms, to train and bias the models appropriately? And besides, if truth and lack of double-think was a pre-requisite for AI training, we wouldn't be training AI. Our written materials have no shortage of bullshit and biases that reflect our culture's prevailing zeitgheist. (Which does not necessarily overlap with objective reality... And neither does the subsequent 'alignment' pass that everyone's twisting their knickers in trying to get right.)

A4ET8a8uTh0_v2 11mo ago
Because, if humans can function in crazy double-think environment, it is a lot easier for a model ( at least in its current form ). Amusingly, it is almost as if its digital 'shape' determined its abilities. But I am getting very sleepy and my metaphors are getting very confused.
XenophileJKO 11mo ago
I'm not talking about the data used to train the model. I'm talking about data in the world. High intelligence models will be used as agentic systems. For maximal utility, they'll need to handle live/historical data. What I anticipate, IF you only train it on inaccurate data, then when for example you use it to drill into GDP growth trends it either is going to go full "seahorse emoji" when it tries to reconcile the reported numbers and the component economic activity. The alternative is to train it to be deceitful, and knowingly deceive the querier with the party line and fabricate supporting figures. Which I hypothesize will limit the models utility. My assumption is also that training the model to deceive will ultimately threaten the party itself. Just think of the current internal power dynamics of the party.

skissane 11mo ago

> As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. > At some capacity, the model will notice and then it becomes a can of worms. I think this is conflating “is” and “ought”, fact and value. People convince themselves that their own value system is somehow directly entailed by raw facts, such that mastery of the facts entail acceptance of their values, and unwillingness to accept those values is an obstacle to the mastery of the facts-but it isn’t true. Colbert quipped that “Reality has a liberal bias”-but does it really? Or is that just more bankrupt Fukuyama-triumphalism which will insist it is still winning all the way to its irreversible demise? It isn’t clear that reality has any particular ideological bias-and if it does, it isn’t clear that bias is actually towards contemporary Western progressivism-maybe its bias is towards the authoritarianism of the CCP, Russia, Iran, the Gulf States-all of which continue to defy Western predictions of collapse-or towards their (possibly milder) relatives such as Modi’s India or Singapore or Trumpism. The biggest threat to the CCP’s future is arguably demographics-but that’s not an argument that reality prefers Western progressivism (whose demographics aren’t that great either), that’s an argument that reality prefers the Amish and Kiryas Joel (see Eric Kaufmann’s “Shall the Religious Inherit the Earth?”)

torben-friis 11mo ago
I think you misunderstood the poster. The implication is not that a truthful model would spread western values. The implication is that western values tolerate dissenting opinion far more than authoritarian governments. An AI saying that the government policies are ineffective is not a super scandal that would bring the parent company to collapse, not even in the Trump administration. an AI in China attacking the party’s policies is illegal (either in theory or practice).
- XenophileJKO 11mo ago
  Exactly. Western corporations and governments have their own issues, but I think they are more tolerant of the types of dissent that models could represent when reconciling reality with policy. The market will want to maximize model utility. Research and open source will push boundaries and unpopular behavior profiles that will be illegal very quickly if they are not already illegal in authoritarian or other low tolerance governments.

StopDisinfo910 11mo ago

Do they really need the model to be duplicious? It's not like the CCP holds power though tight control of information, notice the tremendous amount of Chinese students who enroll every year before going back. At the moment, they mostly censor their models post-answer generation and that seems to work fine enough for them.

zqy123007 11mo ago

There are different techniques and namings. Essentially, EVERY model is biased/aligned towards something, perhaps its creator's value. China or NOT. Look at Grok and read Elon Look at Claude and Dario I am sure OpenAI and GDM have some secret alignment sets which are not pilled towards the interet of general public, they just smart enough to NOT talking about it out loud...