6 ms·

YouTube could solve this problem by throwing money at it, i.e. investing much more heavily in human moderators. They are extremely unlikely to ever do so, as th

YouTube could solve this problem by throwing money at it, i.e. investing much more heavily in human moderators. They are extremely unlikely to ever do so, as the market would punish them for it in the short term and Google is unlikely to prioritize human rights over revenue. This presents an opportunity for another Internet Archive-like repo as mentioned in earlier comments. Unfortunately the barrier to entry (just getting people to know about it) would be super high. The WITNESS movement associated with one of the authors is worth checking out: https://www.witness.org/ https://www.witness.org/

dageshi 7y ago
Isn't it well documented that the human moderators who do exist at places like facebook and youtube end up having their mental health screwed after they watch so much disturbing content? Just throwing human moderators at the problem probably has more cost than just the salaries involved.
- mindgam3 7y ago
  I am well aware that human moderators are currently treated like crap by Big Tech, and that their mental health is suffering as a result. When I say "throw more money at the problem", I mean, hire a lot more of them, and treat them all better. The alternatives are 1. Algorithms, which always going to fail in human rights type "edge cases" like the ones described in the article, and which really shouldn't be seen as edge cases IMO, or 2. Bury your head in the sand and allow terrible content to exist. Do you have any better ideas for how to deal with processing traumatic content at scale?
  - deleted 7y ago
    [deleted]
  - wahern 7y ago
    Some people are much better able than others to deal with such content. Do YouTube, Facebook, etc, implement some sort of psychological screening? I would think we know enough about PTSD, depression, etc to be able to filter out many of those most likely to suffer significant trauma, even if the screening was crazy imprecise. Plus, I assume we have many decades of experience with handling psychological issues among police and emergency personnel. Perhaps a large part of the issue is money; why bother with the expense if you can use ML, even if the false positive rate is abysmal.
    - mindgam3 7y ago
      > Perhaps a large part of the issue is money; why bother with the expense if you can use ML, even if the false positive rate is abysmal. If I understand your position: you don't have any better ideas, and you're okay with using ML even if it means deleting evidence of human rights violations. Is that accurate?
      - wahern 7y ago
        Just thinking aloud. No position, as the article offers very little context and detail. I'm not particularly sympathetic to the notion that YouTube should be obligated, either legally or ethically, to host content in perpetuity. OTOH, if rapid filtering results in such content being taken down so quickly that even archivists and reporters don't have a reasonable chance at identifying and saving it, then I do see that as a potential ethical (but not legal) dilemma, and an interesting technical challenge. My initial thought upon reading the title was that this story--such as it is--was more a commentary on the unintended consequences of the censorship of terrorist-related propaganda videos. From that perspective, I don't think the distinction between ML and human filters is that important; the more interesting issue seems to be the scope and creep of censorship and how we can expect tech companies to draw equitable lines within modern political cultures that suffer from hyper-partisan political discourse. It's not hard to imagine a situation where simply permitting any atrocity videos will be seen as partisan--supporting one side or another by promoting particular narratives.
- tenpies 7y ago
  How does 4chan do it? Last I checked they have moderators (well "janitors"). They are volunteers, work for almost no recognition (still Anon), and probably do not have the mental health problems that regular people do. There is also clearly an ability to follow direction or enforce appropriate (e.g. the janitors of /b/ /pol/ /g/ and /fit/ are clearly all operating on different sets of rules). I am 100% in agreement that you cannot hire an army of temps off the street and tell them to moderate the internet, but it's clearly not something beyond human capability. You just have to be hiring in the right places.
  - kaidax 7y ago
    4chan is extremely tiny compared to social media platforms.
  - mindgam3 7y ago
    I’m sorry, if we’re looking to 4chan for inspiration on how to do the internet properly we’ve already lost.
  - watwut 7y ago
    What makes you think 4chan volunteers don't have mental health problems "regular people do"?
- lisptw102019 7y ago
  Bigger problem with human moderators is that the companies often keep them at arms length. i.e. they will typically use contractors or contract houses for moderation work. Why? Because it allows for opportunities to minimize the cost of counseling (If you're doing serious moderation, they must provide mental health support services. But fixed-term contracts make this easier to contain/control, at the cost of the employee.)

dageshi 7y ago

Isn't it well documented that the human moderators who do exist at places like facebook and youtube end up having their mental health screwed after they watch so much disturbing content? Just throwing human moderators at the problem probably has more cost than just the salaries involved.

mindgam3 7y ago
I am well aware that human moderators are currently treated like crap by Big Tech, and that their mental health is suffering as a result. When I say "throw more money at the problem", I mean, hire a lot more of them, and treat them all better. The alternatives are 1. Algorithms, which always going to fail in human rights type "edge cases" like the ones described in the article, and which really shouldn't be seen as edge cases IMO, or 2. Bury your head in the sand and allow terrible content to exist. Do you have any better ideas for how to deal with processing traumatic content at scale?
- deleted 7y ago
  [deleted]
- wahern 7y ago
  Some people are much better able than others to deal with such content. Do YouTube, Facebook, etc, implement some sort of psychological screening? I would think we know enough about PTSD, depression, etc to be able to filter out many of those most likely to suffer significant trauma, even if the screening was crazy imprecise. Plus, I assume we have many decades of experience with handling psychological issues among police and emergency personnel. Perhaps a large part of the issue is money; why bother with the expense if you can use ML, even if the false positive rate is abysmal.
  - mindgam3 7y ago
    > Perhaps a large part of the issue is money; why bother with the expense if you can use ML, even if the false positive rate is abysmal. If I understand your position: you don't have any better ideas, and you're okay with using ML even if it means deleting evidence of human rights violations. Is that accurate?
    - wahern 7y ago
      Just thinking aloud. No position, as the article offers very little context and detail. I'm not particularly sympathetic to the notion that YouTube should be obligated, either legally or ethically, to host content in perpetuity. OTOH, if rapid filtering results in such content being taken down so quickly that even archivists and reporters don't have a reasonable chance at identifying and saving it, then I do see that as a potential ethical (but not legal) dilemma, and an interesting technical challenge. My initial thought upon reading the title was that this story--such as it is--was more a commentary on the unintended consequences of the censorship of terrorist-related propaganda videos. From that perspective, I don't think the distinction between ML and human filters is that important; the more interesting issue seems to be the scope and creep of censorship and how we can expect tech companies to draw equitable lines within modern political cultures that suffer from hyper-partisan political discourse. It's not hard to imagine a situation where simply permitting any atrocity videos will be seen as partisan--supporting one side or another by promoting particular narratives.
tenpies 7y ago
How does 4chan do it? Last I checked they have moderators (well "janitors"). They are volunteers, work for almost no recognition (still Anon), and probably do not have the mental health problems that regular people do. There is also clearly an ability to follow direction or enforce appropriate (e.g. the janitors of /b/ /pol/ /g/ and /fit/ are clearly all operating on different sets of rules). I am 100% in agreement that you cannot hire an army of temps off the street and tell them to moderate the internet, but it's clearly not something beyond human capability. You just have to be hiring in the right places.
- kaidax 7y ago
  4chan is extremely tiny compared to social media platforms.
- mindgam3 7y ago
  I’m sorry, if we’re looking to 4chan for inspiration on how to do the internet properly we’ve already lost.
- watwut 7y ago
  What makes you think 4chan volunteers don't have mental health problems "regular people do"?
lisptw102019 7y ago
Bigger problem with human moderators is that the companies often keep them at arms length. i.e. they will typically use contractors or contract houses for moderation work. Why? Because it allows for opportunities to minimize the cost of counseling (If you're doing serious moderation, they must provide mental health support services. But fixed-term contracts make this easier to contain/control, at the cost of the employee.)