6 ms·

AI moderation is no match for hate speech in Ethiopian languages

21 points by impish9208 3y ago

alephnerd 3y ago
This is a problem with in-person moderation as well. As internet penetration rises, more and more regional language communities are forming, but the economics of moderation are such that it's difficult to hire moderators to handle hundreds of different languages. For example, with the Rakhine crisis, how many Arakan speaking moderators can you realistically find and hire? And this is just one example of hundreds.
- Shaanie 3y ago
  Supposedly 1-2 million people speak Arakan, so I doubt it's very difficult to find enough moderators. It's likely not worth the effort to do so unless regulation forces you to do it, however.
  - alephnerd 3y ago
    It is when 1. Moderation jobs tend to require a 4 year degree, which is hard to find in a region with a 75% literacy rate 2. Requires incorporation in that country. Try opening a FB office or contractor in Myanmar during an active Civil War 3. (Speculative) Hard social faultlines exist. In Rakhine, society was communalized into Buddhist ethnic Arakan - Muslim ethnic Rohingya and Bengali - Hindu Bengali and Arakan. All ethnic and religious groups spoke Arakan, but all were diametrically opposed to each other due to the communalized society that developed. This is just one example from the Rohingya Genocide, but there are sadly dozens all over the developing world. For example, when dealing with the Tigray war like in the article above, you'd need to moderate for the 100+ languages spoke in Ethiopia, Eritrea, Somalia, South Sudan, and Kenya (all regions with ethnic fault lines that extend into Ethiopia). This is a problem that can only realistically be solved by AI (as it's an intent polarity problem and a Machine Translation problem), but these are less sourced languages with limited corpora or knowledge. All of this is known btw. This is why Google, Amazon, and Meta hire hundreds of linguists in India and Chinese companies like Tencent, Baidu, and Ant hired similarly within China to work on MT and Polarity in South Asia and China respectively, but a similar effort hasn't occurred in Africa and most of ASEAN yet.
everdrive 3y ago
AI is going to be very disappointing if it’s always censored in a way which ensures it’s always in line with mainstream views.
TheLoafOfBread 3y ago
Because it is possible to offend a person without using a single offensive word. And because AI does not comprehend what is written in the sentence, it will let it pass.
- Enginerrrd 3y ago
  >And because AI does not comprehend what is written in the sentence, it will let it pass. This is demonstrably false. GPT4 can easily expound upon, rephrase, simplify, clarify and correct things it has said when prompted to or provided new information. That cannot be done without some form of conceptual understanding of what was written. The issue here, which is both obvious and covered in the article, is that they aren't using AI trained on the target languages. They are translating (imperfectly) to English first and then trying to moderate that. But, frankly, that is a difficult task for a human. Idioms, slang, coded speech (analogous to whinnie the poo=Xi) are all going to slip through the cracks even with a human.
- deleted 3y ago
  [deleted]
veave 3y ago
Good, speech should be as free as possible.
- Devasta 3y ago
  FB is implicated in facilitating genocide in Myanmar due to its failing to moderate speech on its platforms.
  - veave 3y ago
    That's not Facebook's fault. It also isn't the fault of those who post stuff. It's the fault of the ones who kill people.
    - noduerme 3y ago
      Inciting violence does make it partially the fault of those who post stuff. Moreover, they have no "right" to use a platform to spread incitement. Giving them use of a platform makes Facebook complicit. Also, how do you not know the people who post stuff are the same people who kill people?
      - hgs3 3y ago
        > Giving them use of a platform makes Facebook complicit. I don't understand this perspective. Johannes Gutenberg, inventor of the printing press, shouldn't be responsible for what folks print. Neither should Sir Tim Berners-Lee, inventor of the world wide web, be responsible for what people use the web for. Humans needs to accept personal reasonability and not blame their tools.
        Devasta 3y ago
        Inventing the printer wouldn't make them responsible. Distributing things on their behalf would, as FB did.
        3y ago
akomtu 3y ago
Note how they spin AI-powered thoughtpolice of entire people as a desirable thing.
mutant_glofish 3y ago
> And what, exactly, is "hate speech?" This sounds like a phrase out of 1984. — Paul Graham [1] [1]: http://www.paulgraham.com/say.html http://www.paulgraham.com/say.html
SpicyLemonZest 3y ago
> “ Social media companies are arguing that the most scalable solution is also the best solution. That’s a very convenient outcome,” said Nicholas. “What’s going to work best is going to work best for each different language … and we don’t even see social media companies even considering that.” I think this is a misreading of their position. I don't think any social media company would say it's bad or unhelpful to research new strategies which might enable better moderation of Amharic content. But if we reject the idea that there is a solution that works acceptably for languages in general, and insist on social media companies auditing every individual language, that raises obvious and uncomfortable questions about support for smaller languages. Should a social media company which hasn't yet been able to verify the health of its Swazi moderation tools prohibit you from posting in Swazi?

21 points by impish9208 3y ago

This is a problem with in-person moderation as well. As internet penetration rises, more and more regional language communities are forming, but the economics of moderation are such that it's difficult to hire moderators to handle hundreds of different languages. For example, with the Rakhine crisis, how many Arakan speaking moderators can you realistically find and hire? And this is just one example of hundreds.

Shaanie 3y ago
Supposedly 1-2 million people speak Arakan, so I doubt it's very difficult to find enough moderators. It's likely not worth the effort to do so unless regulation forces you to do it, however.
- alephnerd 3y ago
  It is when 1. Moderation jobs tend to require a 4 year degree, which is hard to find in a region with a 75% literacy rate 2. Requires incorporation in that country. Try opening a FB office or contractor in Myanmar during an active Civil War 3. (Speculative) Hard social faultlines exist. In Rakhine, society was communalized into Buddhist ethnic Arakan - Muslim ethnic Rohingya and Bengali - Hindu Bengali and Arakan. All ethnic and religious groups spoke Arakan, but all were diametrically opposed to each other due to the communalized society that developed. This is just one example from the Rohingya Genocide, but there are sadly dozens all over the developing world. For example, when dealing with the Tigray war like in the article above, you'd need to moderate for the 100+ languages spoke in Ethiopia, Eritrea, Somalia, South Sudan, and Kenya (all regions with ethnic fault lines that extend into Ethiopia). This is a problem that can only realistically be solved by AI (as it's an intent polarity problem and a Machine Translation problem), but these are less sourced languages with limited corpora or knowledge. All of this is known btw. This is why Google, Amazon, and Meta hire hundreds of linguists in India and Chinese companies like Tencent, Baidu, and Ant hired similarly within China to work on MT and Polarity in South Asia and China respectively, but a similar effort hasn't occurred in Africa and most of ASEAN yet.

everdrive 3y ago

AI is going to be very disappointing if it’s always censored in a way which ensures it’s always in line with mainstream views.

TheLoafOfBread 3y ago

Because it is possible to offend a person without using a single offensive word. And because AI does not comprehend what is written in the sentence, it will let it pass.

Enginerrrd 3y ago
>And because AI does not comprehend what is written in the sentence, it will let it pass. This is demonstrably false. GPT4 can easily expound upon, rephrase, simplify, clarify and correct things it has said when prompted to or provided new information. That cannot be done without some form of conceptual understanding of what was written. The issue here, which is both obvious and covered in the article, is that they aren't using AI trained on the target languages. They are translating (imperfectly) to English first and then trying to moderate that. But, frankly, that is a difficult task for a human. Idioms, slang, coded speech (analogous to whinnie the poo=Xi) are all going to slip through the cracks even with a human.
deleted 3y ago
[deleted]

veave 3y ago

Good, speech should be as free as possible.

Devasta 3y ago
FB is implicated in facilitating genocide in Myanmar due to its failing to moderate speech on its platforms.
- veave 3y ago
  That's not Facebook's fault. It also isn't the fault of those who post stuff. It's the fault of the ones who kill people.
  - noduerme 3y ago
    Inciting violence does make it partially the fault of those who post stuff. Moreover, they have no "right" to use a platform to spread incitement. Giving them use of a platform makes Facebook complicit. Also, how do you not know the people who post stuff are the same people who kill people?
    - hgs3 3y ago
      > Giving them use of a platform makes Facebook complicit. I don't understand this perspective. Johannes Gutenberg, inventor of the printing press, shouldn't be responsible for what folks print. Neither should Sir Tim Berners-Lee, inventor of the world wide web, be responsible for what people use the web for. Humans needs to accept personal reasonability and not blame their tools.
      - Devasta 3y ago
        Inventing the printer wouldn't make them responsible. Distributing things on their behalf would, as FB did.
        3y ago

akomtu 3y ago

Note how they spin AI-powered thoughtpolice of entire people as a desirable thing.

mutant_glofish 3y ago

> And what, exactly, is "hate speech?" This sounds like a phrase out of 1984. — Paul Graham [1] [1]: http://www.paulgraham.com/say.html http://www.paulgraham.com/say.html

SpicyLemonZest 3y ago

> “ Social media companies are arguing that the most scalable solution is also the best solution. That’s a very convenient outcome,” said Nicholas. “What’s going to work best is going to work best for each different language … and we don’t even see social media companies even considering that.” I think this is a misreading of their position. I don't think any social media company would say it's bad or unhelpful to research new strategies which might enable better moderation of Amharic content. But if we reject the idea that there is a solution that works acceptably for languages in general, and insist on social media companies auditing every individual language, that raises obvious and uncomfortable questions about support for smaller languages. Should a social media company which hasn't yet been able to verify the health of its Swazi moderation tools prohibit you from posting in Swazi?