6 ms·

I hope the take-away that comes out of the leaked documents is this: community moderation is very, very hard. It's not just a simple cut-and-dry "free speech vs

I hope the take-away that comes out of the leaked documents is this: community moderation is very, very hard. It's not just a simple cut-and-dry "free speech vs censorship" issue. There's an enormous amount of nuance involved to sustain a community that won't just devolve into a toxic waste dump that's poison for 99% of the people there. In my experience working with startups, many of them don't treat their content moderation team well, or it tends to be an afterthought. In reality, this can be the hardest and most important job at a company. PTSD is a real problem with these teams, and pay is often sub-par. And when they make mistakes (or when they don't but people still get mad), they tend to get the bulk of the criticism from it. And then at the end, they usually aren't allowed to talk publicly about any of the work they do. Please take your content moderation department seriously. If you have one, read through the crap Facebook's team has to deal with on a daily basis, and then go over and hug them.

coldtea 9y ago
>There's an enormous amount of nuance involved to sustain a community that won't just devolve into a toxic waste dump that's poison for 99% of the people there. Is it? I always admired the much-scoffed YouTube as the free-est place to talk on the internet. Where a vegan gay liberal from NY can discuss on the same video with a neo-con Mongolian and a 15-year old hipster from Sweden, an Israeli Jew, an Arab, and a black South African truck driver, and even agree and share experiences, and at worse you just get a few insults thrown and that's it.
- kyledrake 9y ago
  I agree with you, but this isn't just about morons lobbing dumb insults at each other in all caps with bad spelling. I'm not even comfortable describing in a comment some of the horrible things that content moderators have had to deal with. Dig through the documents (particularly the child abuse doc) and you'll get an idea. Almost all of these policies were likely created from real-life examples, because people do things that are disturbing beyond imagination. They certainly didn't pre-plan it when they were hacking together Facebook in a dorm. I can only imagine how much worse it is for the team over at Youtube. The first thing I would do as CEO over there is hug them, give them a raise, double their vacation time, and make sure they have very good access to mental health services in case they need it. I'm not saying they're treated poorly right now, but again, it would be one of the first things I looked at.
- praneshp 9y ago
  They can all write down what they feel/think. There is no discussion, IMO.
- hashhar 9y ago
  I find Reddit a lot better for that kind of conversation (excepting certain subs like /r/the_donald, /r/resist, /r/worldnews, /r/politics and certain others). /r/AskReddit is one of the best places to have such free and open conversations and the best thing is that you get to set the topic for the conversation instead of some arbitrary video remotely related to what you want to talk about.
  - coldtea 9y ago
    >and the best thing is that you get to set the topic for the conversation instead of some arbitrary video remotely related to what you want to talk about. That's a good point (allowing for deeper discussion), but in YouTube I also like how the image/video transcends borders and politics etc in a way that targeted discussion doesn't. (You can be anything, black, KKK, hipster barrista, 40yo soccer mon, Jew, Arab, donald-fan, etc, but still like/dislike the same cat clip, or old bluegrass song, or belly dancing or whatever).
- matt4077 9y ago
  That's probably a good example of why "freedom" isn't the right way to measure a community's value. Because the entirety of all youtube comments' value can only be measured in what worse things they authors were prevented from doing while commenting there. Freedom of speech is great, and important. That doesn't mean that there's any value in just anybody exploring the boundaries of it.
  - qb45 9y ago
    > Because the entirety of all youtube comments' value can only be measured in what worse things they authors were prevented from doing while commenting there. This says more about your sampling bias than youtube itself. And the fact you are posting this says more about the magnitude of your political bias (its exact direction being yet tbd) than anything you read. I'm not sure if you realize how many nuts really say what they believe and believe what they say.
    - coldtea 9y ago
      >This says more about your sampling bias than youtube itself. And the fact you are posting this says more about the magnitude of your political bias (its exact direction being yet tbd) than anything you read. And both of the above say more about your willingness to characterize the parent than about the issue under discussion. >I'm not sure if you realize how many nuts really say what they believe and believe what they say. Doesn't seem to matter, as long as they don't also ACT on what they believe/say. Which is the case for most people (both positive, e.g. they are racist but they wont act on it, and negative, e.g. they say their are this or that good thing, but they're all talk about it).
diebkdnd 9y ago
Complete crap. Go look up the Alexis ranking of 4chan, certainly more popular then the pious bigots who comment here. Moderation isn't hard, it's only hard for those who need to justify the silencing of opinions they don't like.
wiz21c 9y ago
Listen. I used the phone since 40 years. With the phone I had conversations with people that could have, well, annoyed others to say the least. I've said super bad things with my phone. Was I moderated ? No. Was it a problem ? No. Moderating is just here to make sure that FaceBook can still operate. That's just a problem for FaceBook to protect its image. Let's imagine that there is no moderation. What will happen ? OK, some people will have argument, there will be insult, blood, etc. And, well, depending on a tipping point, we may realize that oooooops, letting people talk in the open is not so good. FB would be blamed and hen disappear. So be it. Now, another thing to do is : prosecute those who say unacceptable things. With prosecution, you have the justice system that will handle the case. That'll be slower but there will be discussions and, hopefully better laws. FaceBook is becoming the Judge Dredd of free speech. It decides what's criminal and what's not and it applies his own power to handle the case directly.
- mc32 9y ago
  I hear you but... This also happens in sports. Someone says something in an interview, in a social network post, in a private conversation which gets published... and what happens? Depending on offense the team may, ignore, fine, suspend or fire the person in question, outside the legal framework. That is, someone may say something insensitive or can be accused (not yet proven) to have done something unbecoming... teams often feel pressure to act extrajudicially. It's just how it is where money and the public sphere of opinion intersect.
  - wiz21c 9y ago
    You're right. But the ratio between number of people employed by FaceBook versus number of customer managed by FB is much greater, so I guess it makes me a tad more nervous :-)
- skybrian 9y ago
  Why do you think getting the police involved all the time would be better? They only have limited capacity; do you think they really want to look at every flagged post? Moderators do refer the most serious cases to the police. But there's a lot of crap that falls short of that, so milder penalties work better.
sillysaurus3 9y ago
To rephrase a flagged comment: 4chan is a pretty good counterexample. It's not necessarily true that a community destroys itself if you have no moderation. This community would; 4chan hasn't. Why?
- Sir_Cmpwn 9y ago
  Do you actually use 4chan? There's more than /b/ and /pol/. Most of the boards have a topic and off-topic posts are removed by janitors.
  - sillysaurus3 9y ago
    Here's a pretty typical /a/ post: http://i.imgur.com/zAoDHlT.png http://i.imgur.com/zAoDHlT.png It's true that off-topic posts are removed, but I'm mainly curious why this kind of "say whatever you want" attitude is so pervasive on 4chan yet doesn't kill the site. On Reddit, for example, comments like that would be moderated via downvoting, which physically hides the comment unless the user clicks on it. That's still a form of moderation.
    - Sir_Cmpwn 9y ago
      Eh, /a/ is pretty trashy too. The thing about 4chan is that the users of the trashier boards have grown used to it. Reddit users feel entitled to a more moderated space. A lot of 4chan posts are also highly satirical, there's very few serious posts, and the posts thare are serioues are mocked for being serious. This mostly applies to the trashy boards, though. The majority of 4chan's boards are not like the subset you may be familiar with.
    - ue_ 9y ago
      But people like that material; is it against the rules? If not, what's the problem? You may think it's trashy, but apparently it's rather popular and doesn't seem to be trolling, at least not to me.
    - qb45 9y ago
      > I'm mainly curious why this kind of "say whatever you want" attitude is so pervasive on 4chan yet doesn't kill the site. On Reddit, for example, comments like that would be moderated via downvoting, which physically hides the comment unless the user clicks on it. That's still a form of moderation. I don't hang out on chans but sometimes end up reading them for various reasons. I think the constant noise and random insults actually are moderation: they keep away people prone to butthurt, who due to their stubbornness and obnoxity are absolutely the worst fun killers and threat to any discussion. Normally people call each other retards as a matter of course and don't give much fuck, but if you try to argue emotionally then you are flamed to the death. To me it was quite eye opening to read a bit of Encyclopedia Dramatica once. Many times I saw it throw shit all over some people or idea just to click on the "see also: some ostensibly opposite idea" link at the bottom and see them throw shit all over that one too. That's a very simple and elegant solution to tribalism if you ask me, HN totally could take inspiration from them because the mods keep complaining about tribalism here :)
nextlevelwizard 9y ago
>PTSD is a real problem with these teams Yeah, that's not even close. They might get stressed, but it's not like seeing your best friend(s) getting blown to pieces every time you close your eyes.
tripzilch 9y ago
> a community that won't just devolve into a toxic waste dump that's poison for 99% of the people there Have you seen Facebook? That's not what it's optimizing for. Profit and eyeballs. Don't delude yourself. Toxic? Poison? Sure thing, as long as they can sell your attention span, Facebook don't care. Honestly. You know what it's doing. Doing to everybody. In as far as this team is building or sustaining a community, it's a tiny sandcastle on a beach facing the automated tidal wave of corporate exploitation of our dopamine/addiction response. Sustaining a community of junkies is very, very hard. It's not just a simple cut-and-dry "freedom to do whatever to your body" vs "personal health". There's an enormous amount of nuance involved to sustain a crack house that won't just devolve into a toxic waste dump that's poison for 99% of the people there. In my experience working at crack houses, many of them don't treat their bouncers well, or it tends to be an afterthought. In reality, this can be the hardest and most important job. PTSD is a real problem with these teams, and pay is often sub-par. And when they make mistakes (or when they don't but people still get mad), they tend to get the bulk of the criticism from it. And then at the end, they usually aren't allowed to talk publicly about any of the work they do. (the analogy holds about as far as until you try to hug them--which is when they'll punch your lights out. but they do mean well, or something)

coldtea 9y ago

>There's an enormous amount of nuance involved to sustain a community that won't just devolve into a toxic waste dump that's poison for 99% of the people there. Is it? I always admired the much-scoffed YouTube as the free-est place to talk on the internet. Where a vegan gay liberal from NY can discuss on the same video with a neo-con Mongolian and a 15-year old hipster from Sweden, an Israeli Jew, an Arab, and a black South African truck driver, and even agree and share experiences, and at worse you just get a few insults thrown and that's it.

kyledrake 9y ago
I agree with you, but this isn't just about morons lobbing dumb insults at each other in all caps with bad spelling. I'm not even comfortable describing in a comment some of the horrible things that content moderators have had to deal with. Dig through the documents (particularly the child abuse doc) and you'll get an idea. Almost all of these policies were likely created from real-life examples, because people do things that are disturbing beyond imagination. They certainly didn't pre-plan it when they were hacking together Facebook in a dorm. I can only imagine how much worse it is for the team over at Youtube. The first thing I would do as CEO over there is hug them, give them a raise, double their vacation time, and make sure they have very good access to mental health services in case they need it. I'm not saying they're treated poorly right now, but again, it would be one of the first things I looked at.
praneshp 9y ago
They can all write down what they feel/think. There is no discussion, IMO.
hashhar 9y ago
I find Reddit a lot better for that kind of conversation (excepting certain subs like /r/the_donald, /r/resist, /r/worldnews, /r/politics and certain others). /r/AskReddit is one of the best places to have such free and open conversations and the best thing is that you get to set the topic for the conversation instead of some arbitrary video remotely related to what you want to talk about.
- coldtea 9y ago
  >and the best thing is that you get to set the topic for the conversation instead of some arbitrary video remotely related to what you want to talk about. That's a good point (allowing for deeper discussion), but in YouTube I also like how the image/video transcends borders and politics etc in a way that targeted discussion doesn't. (You can be anything, black, KKK, hipster barrista, 40yo soccer mon, Jew, Arab, donald-fan, etc, but still like/dislike the same cat clip, or old bluegrass song, or belly dancing or whatever).
matt4077 9y ago
That's probably a good example of why "freedom" isn't the right way to measure a community's value. Because the entirety of all youtube comments' value can only be measured in what worse things they authors were prevented from doing while commenting there. Freedom of speech is great, and important. That doesn't mean that there's any value in just anybody exploring the boundaries of it.
- qb45 9y ago
  > Because the entirety of all youtube comments' value can only be measured in what worse things they authors were prevented from doing while commenting there. This says more about your sampling bias than youtube itself. And the fact you are posting this says more about the magnitude of your political bias (its exact direction being yet tbd) than anything you read. I'm not sure if you realize how many nuts really say what they believe and believe what they say.
  - coldtea 9y ago
    >This says more about your sampling bias than youtube itself. And the fact you are posting this says more about the magnitude of your political bias (its exact direction being yet tbd) than anything you read. And both of the above say more about your willingness to characterize the parent than about the issue under discussion. >I'm not sure if you realize how many nuts really say what they believe and believe what they say. Doesn't seem to matter, as long as they don't also ACT on what they believe/say. Which is the case for most people (both positive, e.g. they are racist but they wont act on it, and negative, e.g. they say their are this or that good thing, but they're all talk about it).

diebkdnd 9y ago

Complete crap. Go look up the Alexis ranking of 4chan, certainly more popular then the pious bigots who comment here. Moderation isn't hard, it's only hard for those who need to justify the silencing of opinions they don't like.

wiz21c 9y ago

Listen. I used the phone since 40 years. With the phone I had conversations with people that could have, well, annoyed others to say the least. I've said super bad things with my phone. Was I moderated ? No. Was it a problem ? No. Moderating is just here to make sure that FaceBook can still operate. That's just a problem for FaceBook to protect its image. Let's imagine that there is no moderation. What will happen ? OK, some people will have argument, there will be insult, blood, etc. And, well, depending on a tipping point, we may realize that oooooops, letting people talk in the open is not so good. FB would be blamed and hen disappear. So be it. Now, another thing to do is : prosecute those who say unacceptable things. With prosecution, you have the justice system that will handle the case. That'll be slower but there will be discussions and, hopefully better laws. FaceBook is becoming the Judge Dredd of free speech. It decides what's criminal and what's not and it applies his own power to handle the case directly.

mc32 9y ago
I hear you but... This also happens in sports. Someone says something in an interview, in a social network post, in a private conversation which gets published... and what happens? Depending on offense the team may, ignore, fine, suspend or fire the person in question, outside the legal framework. That is, someone may say something insensitive or can be accused (not yet proven) to have done something unbecoming... teams often feel pressure to act extrajudicially. It's just how it is where money and the public sphere of opinion intersect.
- wiz21c 9y ago
  You're right. But the ratio between number of people employed by FaceBook versus number of customer managed by FB is much greater, so I guess it makes me a tad more nervous :-)
skybrian 9y ago
Why do you think getting the police involved all the time would be better? They only have limited capacity; do you think they really want to look at every flagged post? Moderators do refer the most serious cases to the police. But there's a lot of crap that falls short of that, so milder penalties work better.

sillysaurus3 9y ago

To rephrase a flagged comment: 4chan is a pretty good counterexample. It's not necessarily true that a community destroys itself if you have no moderation. This community would; 4chan hasn't. Why?

Sir_Cmpwn 9y ago
Do you actually use 4chan? There's more than /b/ and /pol/. Most of the boards have a topic and off-topic posts are removed by janitors.
- sillysaurus3 9y ago
  Here's a pretty typical /a/ post: http://i.imgur.com/zAoDHlT.png http://i.imgur.com/zAoDHlT.png It's true that off-topic posts are removed, but I'm mainly curious why this kind of "say whatever you want" attitude is so pervasive on 4chan yet doesn't kill the site. On Reddit, for example, comments like that would be moderated via downvoting, which physically hides the comment unless the user clicks on it. That's still a form of moderation.
  - Sir_Cmpwn 9y ago
    Eh, /a/ is pretty trashy too. The thing about 4chan is that the users of the trashier boards have grown used to it. Reddit users feel entitled to a more moderated space. A lot of 4chan posts are also highly satirical, there's very few serious posts, and the posts thare are serioues are mocked for being serious. This mostly applies to the trashy boards, though. The majority of 4chan's boards are not like the subset you may be familiar with.
  - ue_ 9y ago
    But people like that material; is it against the rules? If not, what's the problem? You may think it's trashy, but apparently it's rather popular and doesn't seem to be trolling, at least not to me.
  - qb45 9y ago
    > I'm mainly curious why this kind of "say whatever you want" attitude is so pervasive on 4chan yet doesn't kill the site. On Reddit, for example, comments like that would be moderated via downvoting, which physically hides the comment unless the user clicks on it. That's still a form of moderation. I don't hang out on chans but sometimes end up reading them for various reasons. I think the constant noise and random insults actually are moderation: they keep away people prone to butthurt, who due to their stubbornness and obnoxity are absolutely the worst fun killers and threat to any discussion. Normally people call each other retards as a matter of course and don't give much fuck, but if you try to argue emotionally then you are flamed to the death. To me it was quite eye opening to read a bit of Encyclopedia Dramatica once. Many times I saw it throw shit all over some people or idea just to click on the "see also: some ostensibly opposite idea" link at the bottom and see them throw shit all over that one too. That's a very simple and elegant solution to tribalism if you ask me, HN totally could take inspiration from them because the mods keep complaining about tribalism here :)

nextlevelwizard 9y ago

>PTSD is a real problem with these teams Yeah, that's not even close. They might get stressed, but it's not like seeing your best friend(s) getting blown to pieces every time you close your eyes.

tripzilch 9y ago

> a community that won't just devolve into a toxic waste dump that's poison for 99% of the people there Have you seen Facebook? That's not what it's optimizing for. Profit and eyeballs. Don't delude yourself. Toxic? Poison? Sure thing, as long as they can sell your attention span, Facebook don't care. Honestly. You know what it's doing. Doing to everybody. In as far as this team is building or sustaining a community, it's a tiny sandcastle on a beach facing the automated tidal wave of corporate exploitation of our dopamine/addiction response. Sustaining a community of junkies is very, very hard. It's not just a simple cut-and-dry "freedom to do whatever to your body" vs "personal health". There's an enormous amount of nuance involved to sustain a crack house that won't just devolve into a toxic waste dump that's poison for 99% of the people there. In my experience working at crack houses, many of them don't treat their bouncers well, or it tends to be an afterthought. In reality, this can be the hardest and most important job. PTSD is a real problem with these teams, and pay is often sub-par. And when they make mistakes (or when they don't but people still get mad), they tend to get the bulk of the criticism from it. And then at the end, they usually aren't allowed to talk publicly about any of the work they do. (the analogy holds about as far as until you try to hug them--which is when they'll punch your lights out. but they do mean well, or something)