4 ms·

That was good article and I think something really needs to be done to make content moderation more humane. This quote from the article “It gets to a point wh

That was good article and I think something really needs to be done to make content moderation more humane. This quote from the article “It gets to a point where you can eat your lunch while watching a video of someone dying. … But at the end of the day, you still have to be human.” reminded me of a similar article published by WIRED six years ago [1]. Eight years after the fact, Jake Swearingen can still recall the video that made him quit. He was 24 years old and between jobs in the Bay Area when he got a gig as a moderator for a then-new startup called VideoEgg. Three days in, a video of an apparent beheading came across his queue. “Oh fuck! I've got a beheading!” he blurted out. A slightly older colleague in a black hoodie casually turned around in his chair. “Oh,” he said, “which one?” At that moment Swearingen decided he did not want to become a connoisseur of beheading videos. “I didn't want to look back and say I became so blasé to watching people have these really horrible things happen to them that I'm ironic or jokey about it.” [1] https://www.wired.com/2014/10/content-moderation/ https://www.wired.com/2014/10/content-moderation/

kevinskii 6y ago
It would be great if there were a way to make content moderation more humane, but perhaps this is like saying that it would be great if we could make hospital emergency room work more humane. It is unavoidably traumatic, and ER staff are known to detach and use dark humor as a coping mechanism. Content moderation is a similarly noble and difficult profession.
- agentdrtran 6y ago
  ER staff are paid a lot more, are sometimes even in a union, and also get the benefit of helping people and seeing the effect of their help.
  - Spooky23 6y ago
    Ever hang out with an ER staffer? Most of their workload is bullshit. People with colds and sore throats, depressing people using ER as primary care, assholes using 911 to score Medicaid cab vouchers. And at any time, any number of people can show up with any kind of personal tragedies from stokes to various traumas. My sister quit after a 13 year old bled out from a gsw, and she walked out of the room and got kicked in the head by an prisoner who had been stabbed after he bit the ears off of three other prisoners, and broke the arm of a guard.
    - kevinskii 6y ago
      Bite one inmate's ear...shame on you. Bite a 2nd inmate's ear...shame on him. Bite a 3rd inmate's ear...that's fricking badass.
      - dang 6y ago
        Please don't do this here.
    - unicodepepper 6y ago
      I would imagine that cases should follow a bell curve - sure, the exceptionally good cases where you feel you saved someone's life are rare, but so are the terrible cases that haunt you at night; and most would fall somewhere in the middle. That said, I would also wager that the curve can be skewed towards one side or the other depending on the location, timing of your shift, or any other factors.
  - briandear 6y ago
    Having worked as a psych tech and later as an EMT in a major public hospital in Houston, you’d be surprised about how not-well paid many allied health professons are. And the janitors get to clean up the blood and hear the exact same screams or being on the receiving end of drug fueled rage attacks — and they aren’t getting paid much. If you sign up to work in what is effectively a battlefield hospital, you are exposed to the vile underbelly of society on a daily basis. And how does a union matter? It doesn’t shield you from getting attacked or vomited in by violent drunks or attacked by hysterical family members. Let’s say a content moderator got paid $200k per year, that doesn’t change the images they see. This isn’t about money as much as it’s about the requirements and preparation required for the job.
- santigr 6y ago
  I think the diffrence is that ER personal are prepare for that kind of situations and they more or less know what to expect.
- ogurechny 6y ago
  I think that you are wrong in general, but that parallel is right on point. The root of the problem, which is rarely mentioned in these discussions, is that many people enjoying comfortable living conditions believe themselves to be “pure-blooded” and unable to deal with “unclean” things because of the possible “traumas”. (What is an ideal person free of “traumas”, anyway? A vegetable stoned on drugs?) You can easily see here the resemblance of a old time countess complaining about her “nerves” while her serfs do all the dirty work. Is there a natural law that says a poor grunt is bound to clean the junk from what you see on the website? Do it yourself. The same happens with medical, law enforcement, and other professions. They are considered “special” jobs for “special” people (which is not true, the one who defines it like that is “special”). Then, any worries and doubts are neutralized by convenient “but it's their profession, they chose it, they are paid for it, etc.” This creates a stupid rift between “us” and “them”, and affects how both sides reason about themselves. The resulting imbalance is routinely abused. Social networks are competing in presenting a rose-colored walled fantasy to their human cattle to increase yield — everyone can see they use “The Matrix” as a business plan. This creates a lot of shitty jobs where people waste their emotional capacity to make others pleased, not unlike in prostitution. On the other hand, media is hooking the softened consumer with the sensationalist outrage bait about maniacs, wars, corruption, presumed orgies, etc. To makes things worse, the limits to all of that are set not by some moral standards or concern about someone's well-being, as it is often implied, but by advertisement contracts. The bureaucratic machine decides what is “suitable” for the visitors. For example, this makes most of the talk around “Free The Nipple” moot: there is no undercover agents of patriarchy to battle, it's advertisers who bring money and judge what is unwanted “sexual content” (some might say it's a perfect example of capitalist oppression — cue “L'Internationale”). Even this discussion is full of overreacting people. No, the video on your screen is not as “horrible, horrible” as experiencing something firsthand, nor the pain of the tortured can be compared to the pain of the random witnesses on the internet (if we don't count their own wild fantasies). If it happens, then it's no good to pretend it's not. Who knows, it may even make you think. All in all, humans don't show a lot of originality in what they produce, so the number of things you can expect to see is quite limited. I can only conclude by reiterating that knowledge is power, ignorance is bliss, and there's a lot of people who'd like to benefit from others' ignorance in many ways.
- vanusa 6y ago
  It would be great if there were a way to make content moderation more humane, Oh I'm sure FB could think of a few ways. A decent compensation structure might be a good place to start.
  - thephyber 6y ago
    > I'm sure FB could think of I still maintain expecting companies to do this on their own is a category error. Companies think moderation is a cost center. Even if the moderators do the best possible job, they aren't adding value, only removing liabilities. The company is the wrong entity we should be asking for solutions to this problem. They are going to minimize costs and shift liabilities to 3rd parties (which will in turn hire independent contractors and play shell games in case they are sued). We have to demand adversarial outside actors (government, labor unions, lawyers, non-profits, mental health professionals, etc) to start shaping the direction of the solution.
mirimir 6y ago
> ... make content moderation more humane. Modest proposal: Find people who enjoy horrific content, and have them rate it. Then black-hole whatever they rate as most extreme. I was is a dark place after 9/11, both from the trauma of that, and some personal issues. So I spent considerable time on Rotten, Something Awful, and worse. Based on that experience, many would likely do content moderation for free.
- mirimir 6y ago
  It'd be very cool if someone who thinks that this is a horrible idea (and there must be some) would explain why. Just sayin'.
- throwawayway9 6y ago
  That sounds like a great idea, not only to just black hole the content, but to have people who are less prone to have adverse side effects from watching such material be moderators. Thanks for sharing that part about being in that kind of place, I have a feeling many could relate.
- peroporque 6y ago
  Maybe not enjoy, but there are certainly people who are much less affected, or not affected at all, by it. These media stories always focus on those who picked the wrong job for their caracter, and then make a big drama of it. I mean, there are people cutting open other humans daily. We call them surgeons. I would puke right there and could never do that. Different people have different abilities. Simple.
- vanusa 6y ago
  Find people who enjoy horrific content, Yup - these are exactly the kind of people you want to rub shoulders, and stock your company's ranks with.
  - mirimir 6y ago
    Yes, I get it. Those are not nice people. Few of us would want to be friends with them. Or even work in the same building. So have them work from home, or for a standalone subcontractor. I also get the downside of employing them. But people need to look at that stuff, even if software does most of the work. So it might as well be people who can tolerate it, and perhaps even enjoy it.
artificial 6y ago
Sounds like /b/ during the gore era.

kevinskii 6y ago

It would be great if there were a way to make content moderation more humane, but perhaps this is like saying that it would be great if we could make hospital emergency room work more humane. It is unavoidably traumatic, and ER staff are known to detach and use dark humor as a coping mechanism. Content moderation is a similarly noble and difficult profession.

agentdrtran 6y ago
ER staff are paid a lot more, are sometimes even in a union, and also get the benefit of helping people and seeing the effect of their help.
- Spooky23 6y ago
  Ever hang out with an ER staffer? Most of their workload is bullshit. People with colds and sore throats, depressing people using ER as primary care, assholes using 911 to score Medicaid cab vouchers. And at any time, any number of people can show up with any kind of personal tragedies from stokes to various traumas. My sister quit after a 13 year old bled out from a gsw, and she walked out of the room and got kicked in the head by an prisoner who had been stabbed after he bit the ears off of three other prisoners, and broke the arm of a guard.
  - kevinskii 6y ago
    Bite one inmate's ear...shame on you. Bite a 2nd inmate's ear...shame on him. Bite a 3rd inmate's ear...that's fricking badass.
    - dang 6y ago
      Please don't do this here.
  - unicodepepper 6y ago
    I would imagine that cases should follow a bell curve - sure, the exceptionally good cases where you feel you saved someone's life are rare, but so are the terrible cases that haunt you at night; and most would fall somewhere in the middle. That said, I would also wager that the curve can be skewed towards one side or the other depending on the location, timing of your shift, or any other factors.
- briandear 6y ago
  Having worked as a psych tech and later as an EMT in a major public hospital in Houston, you’d be surprised about how not-well paid many allied health professons are. And the janitors get to clean up the blood and hear the exact same screams or being on the receiving end of drug fueled rage attacks — and they aren’t getting paid much. If you sign up to work in what is effectively a battlefield hospital, you are exposed to the vile underbelly of society on a daily basis. And how does a union matter? It doesn’t shield you from getting attacked or vomited in by violent drunks or attacked by hysterical family members. Let’s say a content moderator got paid $200k per year, that doesn’t change the images they see. This isn’t about money as much as it’s about the requirements and preparation required for the job.
santigr 6y ago
I think the diffrence is that ER personal are prepare for that kind of situations and they more or less know what to expect.
ogurechny 6y ago
I think that you are wrong in general, but that parallel is right on point. The root of the problem, which is rarely mentioned in these discussions, is that many people enjoying comfortable living conditions believe themselves to be “pure-blooded” and unable to deal with “unclean” things because of the possible “traumas”. (What is an ideal person free of “traumas”, anyway? A vegetable stoned on drugs?) You can easily see here the resemblance of a old time countess complaining about her “nerves” while her serfs do all the dirty work. Is there a natural law that says a poor grunt is bound to clean the junk from what you see on the website? Do it yourself. The same happens with medical, law enforcement, and other professions. They are considered “special” jobs for “special” people (which is not true, the one who defines it like that is “special”). Then, any worries and doubts are neutralized by convenient “but it's their profession, they chose it, they are paid for it, etc.” This creates a stupid rift between “us” and “them”, and affects how both sides reason about themselves. The resulting imbalance is routinely abused. Social networks are competing in presenting a rose-colored walled fantasy to their human cattle to increase yield — everyone can see they use “The Matrix” as a business plan. This creates a lot of shitty jobs where people waste their emotional capacity to make others pleased, not unlike in prostitution. On the other hand, media is hooking the softened consumer with the sensationalist outrage bait about maniacs, wars, corruption, presumed orgies, etc. To makes things worse, the limits to all of that are set not by some moral standards or concern about someone's well-being, as it is often implied, but by advertisement contracts. The bureaucratic machine decides what is “suitable” for the visitors. For example, this makes most of the talk around “Free The Nipple” moot: there is no undercover agents of patriarchy to battle, it's advertisers who bring money and judge what is unwanted “sexual content” (some might say it's a perfect example of capitalist oppression — cue “L'Internationale”). Even this discussion is full of overreacting people. No, the video on your screen is not as “horrible, horrible” as experiencing something firsthand, nor the pain of the tortured can be compared to the pain of the random witnesses on the internet (if we don't count their own wild fantasies). If it happens, then it's no good to pretend it's not. Who knows, it may even make you think. All in all, humans don't show a lot of originality in what they produce, so the number of things you can expect to see is quite limited. I can only conclude by reiterating that knowledge is power, ignorance is bliss, and there's a lot of people who'd like to benefit from others' ignorance in many ways.
vanusa 6y ago
It would be great if there were a way to make content moderation more humane, Oh I'm sure FB could think of a few ways. A decent compensation structure might be a good place to start.
- thephyber 6y ago
  > I'm sure FB could think of I still maintain expecting companies to do this on their own is a category error. Companies think moderation is a cost center. Even if the moderators do the best possible job, they aren't adding value, only removing liabilities. The company is the wrong entity we should be asking for solutions to this problem. They are going to minimize costs and shift liabilities to 3rd parties (which will in turn hire independent contractors and play shell games in case they are sued). We have to demand adversarial outside actors (government, labor unions, lawyers, non-profits, mental health professionals, etc) to start shaping the direction of the solution.

mirimir 6y ago

> ... make content moderation more humane. Modest proposal: Find people who enjoy horrific content, and have them rate it. Then black-hole whatever they rate as most extreme. I was is a dark place after 9/11, both from the trauma of that, and some personal issues. So I spent considerable time on Rotten, Something Awful, and worse. Based on that experience, many would likely do content moderation for free.

mirimir 6y ago
It'd be very cool if someone who thinks that this is a horrible idea (and there must be some) would explain why. Just sayin'.
throwawayway9 6y ago
That sounds like a great idea, not only to just black hole the content, but to have people who are less prone to have adverse side effects from watching such material be moderators. Thanks for sharing that part about being in that kind of place, I have a feeling many could relate.
peroporque 6y ago
Maybe not enjoy, but there are certainly people who are much less affected, or not affected at all, by it. These media stories always focus on those who picked the wrong job for their caracter, and then make a big drama of it. I mean, there are people cutting open other humans daily. We call them surgeons. I would puke right there and could never do that. Different people have different abilities. Simple.
vanusa 6y ago
Find people who enjoy horrific content, Yup - these are exactly the kind of people you want to rub shoulders, and stock your company's ranks with.
- mirimir 6y ago
  Yes, I get it. Those are not nice people. Few of us would want to be friends with them. Or even work in the same building. So have them work from home, or for a standalone subcontractor. I also get the downside of employing them. But people need to look at that stuff, even if software does most of the work. So it might as well be people who can tolerate it, and perhaps even enjoy it.

artificial 6y ago

Sounds like /b/ during the gore era.