10 ms·

My Awful Life as the Internet's Janitor

117 points by lnguyen 9y ago

stock_toaster 9y ago
That job sounds horrifying -- surprised it doesn't come with some kind of a minimum therapy hours requirement. Aside: After reading that article, henceforth every time Google drops a product or weirdly changes focus/direction, I am going to say "Google is shaking the bear again".
bgammon 9y ago
After reading this, I'm just wondering about two possibilities for the development of Internet culture: Increased policing -- people fear repercussions for posting disturbing content. The darkest corners live on. Increased exposure -- people share the psychological burden of knowing disturbing content exists, and develop meaningful discussions and coping mechanisms.
- miiwq 9y ago
  Talk to an actual law enforcement professional. They see shit too. This isn't a new problem. Their processes and techniques are open. The number of cops we need to maintain order is budget item that responsible experienced qualified people debate about publicly every year. What's new is silicon valleys private police force of the internet is all secret.
  - matt4077 9y ago
    It's actually a pretty big problem–there's pretty clear evidence that policemen serving in majority-black neighbourhoods tend to drift towards prejudice from workday to workday, only to somewhat recover on off-days. That's obviously a problem for the communities, but it's actually noticeable to the affected person as well, which really isn't enjoyable either. What helps is to have people work as police officers, and paramedic or firefighter in turn. That way, you get to experience the good side of humanity and don't lose faith. But of course that makes training somewhat more complicated.
    - novia 9y ago
      Do you have a source for paragraph 1? It definitely jibes with what I've observed in real life, but it's always nice to have a source.
- bostik 9y ago
  I see a third one. Increased tolerance -- people become so thoroughly jaded that the lines between shocking, disturbing and sick will go from uncertain, to blurred, and eventually insignificant. In fact, we've seen this development already. It began even before the public became aware of the internet, and we're all becoming more benumbed. Disassociation can't be a healthy coping mechanism.
- intended 9y ago
  This isn't a new problem, so based on how often and easily it gets pushed out of sight I'd calmly say we are going for a third option Pay the least we can, to ensure that this goes away. From a company perspective, this is a cost of doing business, its not a "Profit Center". It gets treated and measured by the same metrics as credit card fraud. As long as the number is X low, then we need Y number of people to handle it. X is variant on size of site, so Y is the only thing we can control, so lets keep it as low as possible. Don't assume tolerance and open-ness on the new Network. Do note - Increased exposure, without guidance, means that people will become ANGRIER and MORE intolerant. This is the stuff that gets "weaponized" and you see one community yell about it by saying "SEE! This is Happening and no one is talking about it".
- Spooky23 9y ago
  Another possibility is that it slowly devolves into prodigy or aol, and this content starts to vanish as the need to make money appears.
deleted 9y ago
[deleted]
miiwq 9y ago
Here's the weird thing about"policing the internet" narrative...youtube, twitter, facebook, reddit etc nobody in these companies talks about how the shadier parts of their networks are the main draw. We hear more about the great firewall of China and transparency needs of our police departments than how these companies are policing the net. If China needs hundreds of thousands of people to monitor it's content why should we believe YouTube and Facebook don't? I mean the content is so diverse the networks are global. How is some 20 year old kid going to know what vid is going to trigger a riot in Nigeria, or what tweet is promoting cannibalism in Uruguay. These companies have just been hiding behind the free speech folk with their heads buried in the sand about the long term effects of all this content. If we can figure out ways to get our govts to tell us know how many policemen we need and the process it takes to become one, there is no reason Silicon Valley should be doing this vital policing in secret.
- matt4077 9y ago
  The article isn't actually about any form of censorship. They weren't (primarily) policing search results, but the ad sense network. There's just no theory of free speech that would require google to run ads on white supremacist websites. After all it's the advertisers that don't want it, and they don't want it because the backlash would end up hurting them, on balance. All that means that this sort of content simply doesn't have any value as a vehicle for mainstream advertisement. It's actually quite democratic in a way, because the value originates with the broad public. If everyone starts only buying cars advertised on strangulation porn sites tomorrow, the money would follow. Until then, I'm quite happy at current arrangement: There's a few Silos "policed" to different levels of sensibility, then there's the almost free-for-all web, and governments largely don't play a role. The big advantage of private companies doing this (even if intransparent), is that there are several of these companies, and that none of them have an army. P. S/:Please let me know where these shadier parts of Facebook and Youtube are, because I don't think there's much there, beyond some vintage softporn.
- DanBC 9y ago
  > nobody in these companies talks about how the shadier parts of their networks are the main draw The vast majority of people are not going to Youtube, Twitter, or Facebook to watch dog killing videos.
deleted 9y ago
[deleted]
nudpiedo 9y ago
I thought this kind of job should be highly paid because of the evident risks on mental health... when your physical health is on risk your contract reflects that with additional money, benefits, therapy and earlier retirement... (not just in Europe right?) I guess what would happen if this person becomes permanently burnout or suicides because of this long exposure; my understanding is that the employer is responsible for that unless the contract has some compensation and mitigation for the effects on the worker's health...
- ktta 9y ago
  People's payment is never related to the mental or physical effects. A friend of mine pointed it out that it is about how replaceable a person is.
  - erikbye 9y ago
    Which is why Facebook's hourly rate for their moderators is $1-15.
  - nudpiedo 9y ago
    I guess you mean in USA, right? Because my experience in Germany and Spain was different.
    - ktta 9y ago
      Yes. I can only talk about what I know, so can you explain how you think Germany and Spain are different? Are people there getting paid based on the risk they take instead of how replaceable they are?
- beagle3 9y ago
  Pay is directly decided by supply & demand. Indirectly, that causes some riskier jobs to be more highly paid (no supply of people willing to take the risk). But in general, risk is not a direct factor in pay.
  - 9y ago
tofflos 9y ago
There is usually a group within the police who has to review this type of content. I couldn't find a reference to it but I'm fairly certain that the Swedish police places a hard limit, expressed in months, for how much time a person is allowed to do this type of work. Perhaps Google should consider doing the same?
PappaPatat 9y ago
Compartmentalization. Learn it. I've always made sure my team got trained and mentally supervised to prevent this exact problem (we're in... hmm call it "CERT for hire". We see shit), it makes no sense to me that Google seems not to do this. Might be because you're "just a contractor" or because they get away with burning people by the law in that particular part of the world.
- chrisallenlane 9y ago
  Isn't compartmentalization easier said than done, though? Is "just don't let it bother you" really useful advice, or even possible?
cannonpr 9y ago
Moderators and professionals in this area, like police officers, are often the least psychologically prepared to cope with this, they also often receive the least support in quality and time to do so. Psychiatrists receive a lot more training to cope, and are monitored by other psychiatrists them selves to ensure that they get the help they need. Yet there are too few psychiatrists and psychologists for this job and they are too expensive, so we just burn out regular individuals.
nzjrs 9y ago
It's worth remembering this is why Google's AI will always be better than yours - annotation annotation annotation. Only they can afford to throw this much money at a problem
wodenokoto 9y ago
It seems a lot of commenters missed that the time at Google described in the article was 10 years ago. I'm not saying things are done any better in terms of psychological environment, but things run differently now. > Google still felt like a scrappy company when I started working there in 2007, but it basically already ruled the world.
chrisallenlane 9y ago
> I watched a video of a man stomping on a dog's back, breaking it, and listening to the dog's high-pitched whines as it died. I watched all of it. I wonder if the job truly requires these contractors to watch these videos in their entirety. If so - why? Regarding the above, surely you'd only need to watch up until the moment the man broke the dog's back to determine that the video was not "advertiser-friendly".
- eraserhead12 9y ago
  ugh that shouldnt disgust you at all. Cows and pigs are in for much worse when they are slaughtered to feed you, unless you're a vegan. Your outrage simply implies you've detached yourself from the gruesome violence you, and our community at large, are directly responsible for. Animal rights are bullshit until we're all vegan
  - justbaker 9y ago
    ??? I'm never going to be a vegan :)
  - brainfire 9y ago
    There's a difference between a quick slaughter for food and torturing something to death for fun.
  - deleted 9y ago
    [deleted]
- shakna 9y ago
  In Google's case, maybe not. When I worked with the police, yes. You needed every scrap of horrible information and context. Usually because it's much worse than you think initially. (Background voices tell you someone was forced to watch, etc).
eraserhead12 9y ago
OK SOMEONE PLEASE TELL ME HOW THIS IS DISTURBING. He just killed a bear. Most Americans are personally responsible for the gruesome deaths of atleast idk a 1000 cows, more chickens pigs ducks and so on. Unless you're a vegan you can't claim to be any more moral than the asshole who made the video
- eraserhead12 9y ago
  Not being physically involved in the slaughter doesnt absolve you of the crime.
  - eraserhead12 9y ago
    I agree. So this can only be called 'disturbing to watch'. They're not any more guilty than any of us, murderous meat eaters. I'm asking to be spared the sanctimony ffs
    - justbaker 9y ago
      Then leave FFS. Being a vegan is a choice. Just as reading the article is. Just as the article states, his work was completed voluntarily.
- novia 9y ago
  With the animals we consume, since they are killed out of our sight, we can pretend that they are treated with respect. I'm pretty sure everyone would be disturbed by a video of a couple having sex on a freshly killed cow, which is afterwards torn apart by many hungry dogs.
- justbaker 9y ago
  So they kill the bear.. then they fuck they bear. Then a pack of dogs tears the bear apart. That sound any different?
- allengeorge 9y ago
  The person (people) in question killed for pleasure and then proceeded to degrade the killed animal. In my mind it's the combination of these two factors that tips the scale to disturbing.
jaclaz 9y ago
Here some professionals talk about the issues (more focused on Child Pornography but the kind of trauma/wear is similar): https://www.forensicfocus.com/Forums/viewtopic/t=2329/ https://www.forensicfocus.com/Forums/viewtopic/t=2329/
nbanks 9y ago
This reminds me of an issue of Dabiq magazine I read (ISIS propaganda). It was surprising that most of the arguments for their actions are cogent presuming all the ahadith (ie. oral traditions) that were considered authentic a millennium ago should still be practiced today. However their arguments for prophecies were stretched, with claims that the US is the Roman Empire and such. The photos were stunning. Each one looked as if it ought to be a stock Windows wallpaper. And then I saw the photo of a beheading that I wish I could unsee. Now I'm quite happy to follow events using secondary sources, and I don't envy anyone who has the task of reading this shit.

117 points by lnguyen 9y ago

That job sounds horrifying -- surprised it doesn't come with some kind of a minimum therapy hours requirement. Aside: After reading that article, henceforth every time Google drops a product or weirdly changes focus/direction, I am going to say "Google is shaking the bear again".

bgammon 9y ago

After reading this, I'm just wondering about two possibilities for the development of Internet culture: Increased policing -- people fear repercussions for posting disturbing content. The darkest corners live on. Increased exposure -- people share the psychological burden of knowing disturbing content exists, and develop meaningful discussions and coping mechanisms.

miiwq 9y ago
Talk to an actual law enforcement professional. They see shit too. This isn't a new problem. Their processes and techniques are open. The number of cops we need to maintain order is budget item that responsible experienced qualified people debate about publicly every year. What's new is silicon valleys private police force of the internet is all secret.
- matt4077 9y ago
  It's actually a pretty big problem–there's pretty clear evidence that policemen serving in majority-black neighbourhoods tend to drift towards prejudice from workday to workday, only to somewhat recover on off-days. That's obviously a problem for the communities, but it's actually noticeable to the affected person as well, which really isn't enjoyable either. What helps is to have people work as police officers, and paramedic or firefighter in turn. That way, you get to experience the good side of humanity and don't lose faith. But of course that makes training somewhat more complicated.
  - novia 9y ago
    Do you have a source for paragraph 1? It definitely jibes with what I've observed in real life, but it's always nice to have a source.
bostik 9y ago
I see a third one. Increased tolerance -- people become so thoroughly jaded that the lines between shocking, disturbing and sick will go from uncertain, to blurred, and eventually insignificant. In fact, we've seen this development already. It began even before the public became aware of the internet, and we're all becoming more benumbed. Disassociation can't be a healthy coping mechanism.
intended 9y ago
This isn't a new problem, so based on how often and easily it gets pushed out of sight I'd calmly say we are going for a third option Pay the least we can, to ensure that this goes away. From a company perspective, this is a cost of doing business, its not a "Profit Center". It gets treated and measured by the same metrics as credit card fraud. As long as the number is X low, then we need Y number of people to handle it. X is variant on size of site, so Y is the only thing we can control, so lets keep it as low as possible. Don't assume tolerance and open-ness on the new Network. Do note - Increased exposure, without guidance, means that people will become ANGRIER and MORE intolerant. This is the stuff that gets "weaponized" and you see one community yell about it by saying "SEE! This is Happening and no one is talking about it".
Spooky23 9y ago
Another possibility is that it slowly devolves into prodigy or aol, and this content starts to vanish as the need to make money appears.

deleted 9y ago

[deleted]

miiwq 9y ago

Here's the weird thing about"policing the internet" narrative...youtube, twitter, facebook, reddit etc nobody in these companies talks about how the shadier parts of their networks are the main draw. We hear more about the great firewall of China and transparency needs of our police departments than how these companies are policing the net. If China needs hundreds of thousands of people to monitor it's content why should we believe YouTube and Facebook don't? I mean the content is so diverse the networks are global. How is some 20 year old kid going to know what vid is going to trigger a riot in Nigeria, or what tweet is promoting cannibalism in Uruguay. These companies have just been hiding behind the free speech folk with their heads buried in the sand about the long term effects of all this content. If we can figure out ways to get our govts to tell us know how many policemen we need and the process it takes to become one, there is no reason Silicon Valley should be doing this vital policing in secret.

matt4077 9y ago
The article isn't actually about any form of censorship. They weren't (primarily) policing search results, but the ad sense network. There's just no theory of free speech that would require google to run ads on white supremacist websites. After all it's the advertisers that don't want it, and they don't want it because the backlash would end up hurting them, on balance. All that means that this sort of content simply doesn't have any value as a vehicle for mainstream advertisement. It's actually quite democratic in a way, because the value originates with the broad public. If everyone starts only buying cars advertised on strangulation porn sites tomorrow, the money would follow. Until then, I'm quite happy at current arrangement: There's a few Silos "policed" to different levels of sensibility, then there's the almost free-for-all web, and governments largely don't play a role. The big advantage of private companies doing this (even if intransparent), is that there are several of these companies, and that none of them have an army. P. S/:Please let me know where these shadier parts of Facebook and Youtube are, because I don't think there's much there, beyond some vintage softporn.
DanBC 9y ago
> nobody in these companies talks about how the shadier parts of their networks are the main draw The vast majority of people are not going to Youtube, Twitter, or Facebook to watch dog killing videos.

deleted 9y ago

[deleted]

nudpiedo 9y ago

I thought this kind of job should be highly paid because of the evident risks on mental health... when your physical health is on risk your contract reflects that with additional money, benefits, therapy and earlier retirement... (not just in Europe right?) I guess what would happen if this person becomes permanently burnout or suicides because of this long exposure; my understanding is that the employer is responsible for that unless the contract has some compensation and mitigation for the effects on the worker's health...

ktta 9y ago
People's payment is never related to the mental or physical effects. A friend of mine pointed it out that it is about how replaceable a person is.
- erikbye 9y ago
  Which is why Facebook's hourly rate for their moderators is $1-15.
- nudpiedo 9y ago
  I guess you mean in USA, right? Because my experience in Germany and Spain was different.
  - ktta 9y ago
    Yes. I can only talk about what I know, so can you explain how you think Germany and Spain are different? Are people there getting paid based on the risk they take instead of how replaceable they are?
beagle3 9y ago
Pay is directly decided by supply & demand. Indirectly, that causes some riskier jobs to be more highly paid (no supply of people willing to take the risk). But in general, risk is not a direct factor in pay.
- 9y ago

tofflos 9y ago

There is usually a group within the police who has to review this type of content. I couldn't find a reference to it but I'm fairly certain that the Swedish police places a hard limit, expressed in months, for how much time a person is allowed to do this type of work. Perhaps Google should consider doing the same?

PappaPatat 9y ago

Compartmentalization. Learn it. I've always made sure my team got trained and mentally supervised to prevent this exact problem (we're in... hmm call it "CERT for hire". We see shit), it makes no sense to me that Google seems not to do this. Might be because you're "just a contractor" or because they get away with burning people by the law in that particular part of the world.

chrisallenlane 9y ago
Isn't compartmentalization easier said than done, though? Is "just don't let it bother you" really useful advice, or even possible?

cannonpr 9y ago

Moderators and professionals in this area, like police officers, are often the least psychologically prepared to cope with this, they also often receive the least support in quality and time to do so. Psychiatrists receive a lot more training to cope, and are monitored by other psychiatrists them selves to ensure that they get the help they need. Yet there are too few psychiatrists and psychologists for this job and they are too expensive, so we just burn out regular individuals.

nzjrs 9y ago

It's worth remembering this is why Google's AI will always be better than yours - annotation annotation annotation. Only they can afford to throw this much money at a problem

wodenokoto 9y ago

It seems a lot of commenters missed that the time at Google described in the article was 10 years ago. I'm not saying things are done any better in terms of psychological environment, but things run differently now. > Google still felt like a scrappy company when I started working there in 2007, but it basically already ruled the world.

chrisallenlane 9y ago

> I watched a video of a man stomping on a dog's back, breaking it, and listening to the dog's high-pitched whines as it died. I watched all of it. I wonder if the job truly requires these contractors to watch these videos in their entirety. If so - why? Regarding the above, surely you'd only need to watch up until the moment the man broke the dog's back to determine that the video was not "advertiser-friendly".

eraserhead12 9y ago
ugh that shouldnt disgust you at all. Cows and pigs are in for much worse when they are slaughtered to feed you, unless you're a vegan. Your outrage simply implies you've detached yourself from the gruesome violence you, and our community at large, are directly responsible for. Animal rights are bullshit until we're all vegan
- justbaker 9y ago
  ??? I'm never going to be a vegan :)
- brainfire 9y ago
  There's a difference between a quick slaughter for food and torturing something to death for fun.
- deleted 9y ago
  [deleted]
shakna 9y ago
In Google's case, maybe not. When I worked with the police, yes. You needed every scrap of horrible information and context. Usually because it's much worse than you think initially. (Background voices tell you someone was forced to watch, etc).

eraserhead12 9y ago

OK SOMEONE PLEASE TELL ME HOW THIS IS DISTURBING. He just killed a bear. Most Americans are personally responsible for the gruesome deaths of atleast idk a 1000 cows, more chickens pigs ducks and so on. Unless you're a vegan you can't claim to be any more moral than the asshole who made the video

eraserhead12 9y ago
Not being physically involved in the slaughter doesnt absolve you of the crime.
- eraserhead12 9y ago
  I agree. So this can only be called 'disturbing to watch'. They're not any more guilty than any of us, murderous meat eaters. I'm asking to be spared the sanctimony ffs
  - justbaker 9y ago
    Then leave FFS. Being a vegan is a choice. Just as reading the article is. Just as the article states, his work was completed voluntarily.
novia 9y ago
With the animals we consume, since they are killed out of our sight, we can pretend that they are treated with respect. I'm pretty sure everyone would be disturbed by a video of a couple having sex on a freshly killed cow, which is afterwards torn apart by many hungry dogs.
justbaker 9y ago
So they kill the bear.. then they fuck they bear. Then a pack of dogs tears the bear apart. That sound any different?
allengeorge 9y ago
The person (people) in question killed for pleasure and then proceeded to degrade the killed animal. In my mind it's the combination of these two factors that tips the scale to disturbing.

jaclaz 9y ago

Here some professionals talk about the issues (more focused on Child Pornography but the kind of trauma/wear is similar): https://www.forensicfocus.com/Forums/viewtopic/t=2329/ https://www.forensicfocus.com/Forums/viewtopic/t=2329/

nbanks 9y ago

This reminds me of an issue of Dabiq magazine I read (ISIS propaganda). It was surprising that most of the arguments for their actions are cogent presuming all the ahadith (ie. oral traditions) that were considered authentic a millennium ago should still be practiced today. However their arguments for prophecies were stretched, with claims that the US is the Roman Empire and such. The photos were stunning. Each one looked as if it ought to be a stock Windows wallpaper. And then I saw the photo of a beheading that I wish I could unsee. Now I'm quite happy to follow events using secondary sources, and I don't envy anyone who has the task of reading this shit.