4 ms·

Given that this is a Chinese model, I’m genuinely curious if researchers have been evaluating risk that these models could be used for soft propaganda or simila

by yumraj 3y ago

Given that this is a Chinese model, I’m genuinely curious if researchers have been evaluating risk that these models could be used for soft propaganda or similar purpose? As others have reported, English and Chinese queries return different replies on topics that are not kosher in China. What’s the risk that such models could be used for nefarious purposes by providing propaganda/biased/incorrect/… responses that on a cursory glance seem factual.

ithkuil 3y ago
At the very least models will exhibit the bias present in the underlying training text and on top of that there will be a bias imposed by those wanting to correct the unwanted bias present in the underlying training text, possibly swinging the pendulum too far in the other side. I have the feeling you're asking something more specific, something more of a direct interference coming from politics and not just the natural "point of view" about various topics that are present in the chinese training corpora that is understandably different from western corpora. Do you have anything specific in mind about something that you expect the Chinese government to feed as propaganda that is not already widely being sculpted into the chinese text corpora available on the internet?
- yumraj 3y ago
  > I have the feeling you're asking something more specific... > Do you have anything specific in mind about something that you expect the Chinese government to feed as propaganda that is not already widely being sculpted into the chinese text corpora available on the internet? I don't have anything specific, and it doesn't have to be different from "chinese text corpora available on the internet", it's just that these models can become yet another channel of distribution for the "chinese text corpora available on the internet" especially if they are unknowingly/naively picked up and used as the foundation by others to build their offerings.
  - maxglute 3y ago
    PRC will definitely weaponize this for mass foreign propaganda, which up until now PRC has been thoroughly deficient in, despite all the reees of 50cents on western net. The reality pre-LLM is PRC propaganda on western social media platforms has been very limited in scale for the simple reason that they are not wasting valuable English fluency to shit post on western platforms enmass. Low 100s-1000s of accounts, most of which target diasphora in spambot efforts, frequently in Chinese. Now that LLM has made it cheap to spam passable English/foreign languages, I'd expect increased volumes of PRC propaganda on western social media where anonymous posting is asymmetrically easier. But then again, they don't need a PRC LLM for that, plenthy of US bad posting on western platforms from international audiences and US herself.
    - ithkuil 3y ago
      > they are not wasting valuable English fluency to shit post on western platforms enmass How's the economy of that different for the Russian campaigns? Do they have a larger pool of English fluency to draw from or is the urgency of the operation higher in their case?
      - maxglute 3y ago
        A PRC person with English fluency good enough to blend in with native English on western platform has much better job opporunities. Even in PRC, 50c posts are largely civil servants told to write a few perfunctory platitutdes on domestic platforms. The MO is to overwhelm with spam not engage where effort:return is low. Like even Ministry of Foreign Affairs and most of thinktanks that also publish in English can rarely find people to write "casual" English. You'd have to write 1000s of "50c" comments for 1 hour of English tutoring gig. The economics of it doesn't make sense pre LLM.
        ithkuil 3y ago
        Does it mean that in Russia of 10 years ago a person with the same english language skills would not be able to find a better job or does it mean that the troll farms pay more? Or is it just patriotism? (I genuinely would like to learn more about this topic)
TaylorAlexander 3y ago
It’s a fair question, but one we should be asking about all models, perhaps especially our own. It’s of course easier to see the propaganda of foreign cultures, and this should be investigated, but let’s not let ourselves believe that a model is more likely to contain propaganda because it is Chinese. It will just contain propaganda that is easier for us to see as propaganda. Noam Chomsky and Edward Herman wrote extensively about propaganda in democratic societies in their 1988 book Manufacturing Consent. A nice introductory excerpt is here, and the first two or three paragraphs are enough to begin to see the argument: https://chomsky.info/consent01/ https://chomsky.info/consent01/ Put as briefly as possible: propaganda in totalitarian societies is simpler. They just use force to remove people who say the wrong things, and state media to broadcast the “right things”. In democratic societies, institutional power still wants to protect itself, and this is achieved through more complex means, but it is nonetheless still rather effective.
- yumraj 3y ago
  > It’s a fair question, but one we should be asking about all models, perhaps especially our own. It’s of course easier to see the propaganda of foreign cultures, and this should be investigated, but let’s not let ourselves believe that a model is more likely to contain propaganda because it is Chinese. It will just contain propaganda that is easier for us to see as propaganda. Yes, but in this particular case I'm coming from a viewpoint where I view China as a hostile power. So, at the moment, my worry is about that. In future, if US slips into authoritarianism, which TBH it might depending on the outcome of the next election, what you note would become a very real problem. So, putting it differently and more neutrally, is there any research being done on evaluating a political, and other, bias in a model or is it just being all put in the bucket of hallucination?
  - TaylorAlexander 3y ago
    > if US slips into authoritarianism The point of Chomsky’s work in this case is to show that authoritarianism does not make propaganda more or less likely, it just changes the means by which propaganda is created and reinforced. Chinese propaganda is easier to identify as a foreigner, but the propaganda of your home country has a much more significant effect on your life. The nature of living with pervasive propaganda is that it is hard to see or consider how your life would be different without the propaganda, and that’s what makes it so dangerous. > is there any research being done on evaluating a political, and other, bias in a model or is it just being all put in the bucket of hallucination? It’s a better question, and again one that we should ask regardless of the model’s origins.

by yumraj 3y ago

ithkuil 3y ago

At the very least models will exhibit the bias present in the underlying training text and on top of that there will be a bias imposed by those wanting to correct the unwanted bias present in the underlying training text, possibly swinging the pendulum too far in the other side. I have the feeling you're asking something more specific, something more of a direct interference coming from politics and not just the natural "point of view" about various topics that are present in the chinese training corpora that is understandably different from western corpora. Do you have anything specific in mind about something that you expect the Chinese government to feed as propaganda that is not already widely being sculpted into the chinese text corpora available on the internet?

yumraj 3y ago
> I have the feeling you're asking something more specific... > Do you have anything specific in mind about something that you expect the Chinese government to feed as propaganda that is not already widely being sculpted into the chinese text corpora available on the internet? I don't have anything specific, and it doesn't have to be different from "chinese text corpora available on the internet", it's just that these models can become yet another channel of distribution for the "chinese text corpora available on the internet" especially if they are unknowingly/naively picked up and used as the foundation by others to build their offerings.
- maxglute 3y ago
  PRC will definitely weaponize this for mass foreign propaganda, which up until now PRC has been thoroughly deficient in, despite all the reees of 50cents on western net. The reality pre-LLM is PRC propaganda on western social media platforms has been very limited in scale for the simple reason that they are not wasting valuable English fluency to shit post on western platforms enmass. Low 100s-1000s of accounts, most of which target diasphora in spambot efforts, frequently in Chinese. Now that LLM has made it cheap to spam passable English/foreign languages, I'd expect increased volumes of PRC propaganda on western social media where anonymous posting is asymmetrically easier. But then again, they don't need a PRC LLM for that, plenthy of US bad posting on western platforms from international audiences and US herself.
  - ithkuil 3y ago
    > they are not wasting valuable English fluency to shit post on western platforms enmass How's the economy of that different for the Russian campaigns? Do they have a larger pool of English fluency to draw from or is the urgency of the operation higher in their case?
    - maxglute 3y ago
      A PRC person with English fluency good enough to blend in with native English on western platform has much better job opporunities. Even in PRC, 50c posts are largely civil servants told to write a few perfunctory platitutdes on domestic platforms. The MO is to overwhelm with spam not engage where effort:return is low. Like even Ministry of Foreign Affairs and most of thinktanks that also publish in English can rarely find people to write "casual" English. You'd have to write 1000s of "50c" comments for 1 hour of English tutoring gig. The economics of it doesn't make sense pre LLM.
      - ithkuil 3y ago
        Does it mean that in Russia of 10 years ago a person with the same english language skills would not be able to find a better job or does it mean that the troll farms pay more? Or is it just patriotism? (I genuinely would like to learn more about this topic)

TaylorAlexander 3y ago

It’s a fair question, but one we should be asking about all models, perhaps especially our own. It’s of course easier to see the propaganda of foreign cultures, and this should be investigated, but let’s not let ourselves believe that a model is more likely to contain propaganda because it is Chinese. It will just contain propaganda that is easier for us to see as propaganda. Noam Chomsky and Edward Herman wrote extensively about propaganda in democratic societies in their 1988 book Manufacturing Consent. A nice introductory excerpt is here, and the first two or three paragraphs are enough to begin to see the argument: https://chomsky.info/consent01/ https://chomsky.info/consent01/ Put as briefly as possible: propaganda in totalitarian societies is simpler. They just use force to remove people who say the wrong things, and state media to broadcast the “right things”. In democratic societies, institutional power still wants to protect itself, and this is achieved through more complex means, but it is nonetheless still rather effective.

yumraj 3y ago
> It’s a fair question, but one we should be asking about all models, perhaps especially our own. It’s of course easier to see the propaganda of foreign cultures, and this should be investigated, but let’s not let ourselves believe that a model is more likely to contain propaganda because it is Chinese. It will just contain propaganda that is easier for us to see as propaganda. Yes, but in this particular case I'm coming from a viewpoint where I view China as a hostile power. So, at the moment, my worry is about that. In future, if US slips into authoritarianism, which TBH it might depending on the outcome of the next election, what you note would become a very real problem. So, putting it differently and more neutrally, is there any research being done on evaluating a political, and other, bias in a model or is it just being all put in the bucket of hallucination?
- TaylorAlexander 3y ago
  > if US slips into authoritarianism The point of Chomsky’s work in this case is to show that authoritarianism does not make propaganda more or less likely, it just changes the means by which propaganda is created and reinforced. Chinese propaganda is easier to identify as a foreigner, but the propaganda of your home country has a much more significant effect on your life. The nature of living with pervasive propaganda is that it is hard to see or consider how your life would be different without the propaganda, and that’s what makes it so dangerous. > is there any research being done on evaluating a political, and other, bias in a model or is it just being all put in the bucket of hallucination? It’s a better question, and again one that we should ask regardless of the model’s origins.