6 ms·

In the tutorial, the very first card I was presented with was reported for being "illegal", and the description was (from memory. I might not have it quite righ

In the tutorial, the very first card I was presented with was reported for being "illegal", and the description was (from memory. I might not have it quite right) "A post about the movie Cocaine Bear recommends doing cocaine while watching the movie and contains a photograph of a bag of white powder with the hashtag #cocaine" That... seems like an obvious joke to me. And even if it's not, while "doing cocaine" is illegal, photographs of white powders are not, nor is suggesting that people do cocaine, nor is portraying them apparently doing so. We can know this is true because the producers and actors who worked on the film Cocaine Bear were not arrested. Content moderation is hard, but I'm not very impressed that the very first card shown got it (in my opinion) totally wrong.

gnopgnip 3y ago
It’s not just about what’s legal, it’s whatever the boss says
oldtownroad 3y ago
That’s the point though, right? Encouraging illegal activity may not itself be illegal but moderators aren’t arbiters of the law, they’re protecting the integrity of the website for the intended user base. If you want to allow people to encourage cocaine usage on your website, you have to be prepared to deal with law enforcement — because when someone follows the advice and dies with the website open on their computer, the website operators will get a call… — and not many people want to risk their entire company because “it’s legal to encourage people to take cocaine!” Moderation is hard and that example is… the perfect example. User reports are rarely accurate but still valuable.
- ianferrel 3y ago
  Certainly social media companies can choose to disallow suggestions that people take illegal actions. But that's not what the moderation policy for "illegal" content says. It says "Content that violates local or international laws or regulations." I don't know all the laws of all the countries, but my local laws make very little content illegal. Various types of obscenity, true threats, etc. "You should do drugs while you watch a movie about drugs" is definitely not one of those. It would be very easy, if that's what they want, to change it to "Content that encourages breaking local or international laws or regulations."
  - oldtownroad 3y ago
    Visit a platform like LinkedIn, Twitter or Facebook and try to report content: the complex decision tree of what to report and why will be a good demonstration or how unintuitive the reality of content moderation is. Every user of every platform has a different understanding of even simple concepts, like “scam” and “spam”. Report reasons are not an accurate classification from a trained expert, they’re a thing a user chooses from a list that has to be designed to both provide valuable context to a moderator and capture the wide range of different understandings different users have. Suggesting that a reason should be added for the specific situation you’ve encountered demonstrates the naive understanding most people have of moderation and that’s what this game is good at. Platforms like Facebook and Twitter will have spent tens of thousands of people hours thinking about something as simple as the list of report reasons.
    - ianferrel 3y ago
      There are plenty of difficult judgment calls in content moderating. But the difference between "illegal content" and "content that promotes breaking the law" isn't really one of them to be made by the moderator. This is just bad instructions from the game maker.
      - ethanbond 3y ago
        You don’t think content moderators have to deal with: “hey kid, you should go shoot up a school?” Just cleanly not in their domain?
        ianferrel 3y ago
        I think that content moderators often have to deal with that issue, and they are given rules like "remove posts that suggest that others commit violent acts" or something like that. And those are reasonable rules for moderators to enforce. But the rules that this game give do not include that rule, and it's a mark of a poorly designed game that the rule says "don't allow illegal content" and then when you (correctly!) apply that rule, the game says "you should have not allowed this because it's suggesting that someone do something illegal". Those aren't the same rule! ETA: Like lots of places have rules against reposting copyrighted content. But a post that said "Psst, kid, you should go download a movie from the Pirate Bay" should not be removed under that rule. Because the content of the post isn't copyrighted. If they also had a rule that said "don't encourage piracy", they could reasonably take it down under that rule.
- commandlinefan 3y ago
  > the website operators will get a call Then what we actually need isn't more content moderation, we need explicit laws explicitly _protecting_ website operators and any other publishers who are actually following the law.
  - pixl97 3y ago
    I mean not really... sometimes some content just sucks and you don't want it on your site regardless of its legality. Start shitposting here on HN and see how long your posts hang around.
    - commandlinefan 3y ago
      But then that becomes a site owner editorial decision rather than a fear of legal reprisals decision.
- NoMoreNicksLeft 3y ago
  How can one protect the integrity of a website if they're undermining free speech rights and simultaneously derailing conversations and fragmenting communities while doing so? Though we didn't first define what "integrity" meant, it now seems impossible to formulate a meaning for it that isn't absurdly ironic given the actions performed to protect it. One could be forgiven if we were talking about a forum or website that exists in a totalitarian regime like China... better that there be some conversation than the only other option, none at all. But in the US or Europe or any other nation that considers itself even minimally enlightened? Moderation is not the solution. It may even be the problem.
  - BHSPitMonkey 3y ago
    > How can one protect the integrity of a website if they're undermining free speech rights What free speech rights? You have no legal right to compel someone else to display your words on their website.
    - NoMoreNicksLeft 3y ago
      That's an interesting theory. I'm not sure you understand the implications of it though. Were it so simple, then it is plausible enough to construct a society where you have no free speech at all, anywhere, without your legal rights being violated. In fact, you all seem to be constructing that society even as we speak. Given the audience of this particular forum, more than a few of you are literally doing just that, rather than being accomplices and bystanders. I'm more of a free speech fundamentalist myself.
      - stonogo 3y ago
        Leaping from "I get to decide what's on my website" to "you've created a society" is pretty silly. Your argument is missing any meaningful linkages. It would be stronger if you articulated them instead of handwaving about plausibility. Personally I don't think social media speech is that important. It's ad-based, convenience-driven, and fickle. If it were to disappear people would just move to different media.
Zak 3y ago
The "illegal" rule requires a lot of clarification. The interpretation here seems to be "It's against our policy to post about engaging in illegal activities", or maybe "it's against our policy to advocate illegal activities". Services based in the US are not legally obligated to prevent users from writing about using cocaine, or pictures of white powders they're claiming are cocaine, or writing that it's desirable to use cocaine, so the post itself is not illegal. Real policies for content moderators should be much more detailed than what's in this game so that most moderators will reach the same conclusions about whether a post violates the policy.
idiotsecant 3y ago
It's ironic that the first post on this thread is about missing the point of the first card that user saw.
- ianferrel 3y ago
  What was the point of it, and how did I miss it?
  - matthewfcarlson 3y ago
    This is showing how hard content moderation really is. It's judgement calls and very often no one is happy. Seeing more information takes an angering amount of time, which makes me think this is a fun game that makes an excellent point. Edit: In some ways, a card that's incredibly contextual as the first one is a brilliant move.
    - ianferrel 3y ago
      I'm not convinced that putting a real head-scratcher as the very first choice in a tutorial mode is a brilliant move at all. Tutorials are supposed to be easy and hand-holding. But also: this isn't actually a hard judgment call. It's just bad instructions. There's an objective difference between "content that breaks laws" and "content that promotes the breaking of laws" and the rules could easily be changed to indicate the one they want. If you were playing a platformer tutorial and you came to a gap and the tutorial instructions said "Press A to jump over the gap" and then when you tried you fell in and died and the tutorial then said "That was too big a gap to jump over. You should have pressed [Down] to climb over it instead", would you think "Ah, what a brilliant meditation on trusting trust and how the right choice is not obvious" or would you think "this is a shitty tutorial for a dumb game"? I'm not saying there's for sure a right answer here. I could imagine a well-made game where the tutorial straight-up lies to you and gives you the wrong directions. But most games like that are just badly written and poorly thought out.
      - bisby 3y ago
mirashii 3y ago
> Content moderation is hard, but I'm not very impressed that the very first card shown got it (in my opinion) totally wrong The authors are very clear that there are no right or wrong answers. Interpreting it that way misses the entire point of the exercise.
- ianferrel 3y ago
  I mean, when I approved leaving up the cocaine post, the game that they authored gave me a big red X and said I did it wrong that it clearly was illegal and should have been taken down. Where were they clear that there are no right or wrong answers?
  - wmf 3y ago
    The users who submit reports are fallible, the moderators are fallible, and the bosses who rate the moderators are also fallible. Sometimes you'll be right and the system will still say you're wrong.
    - mike741 3y ago
      > the bosses who rate the moderators are also fallible. Sometimes you'll be right and the system will still say you're wrong. those are not the difficulties of moderating though, those are the difficulties of general employment.
  - mirashii 3y ago
    https://www.techdirt.com/2023/05/11/moderator-mayhem-a-mobile-game-to-see-how-well-you-can-handle-content-moderation/ https://www.techdirt.com/2023/05/11/moderator-mayhem-a-mobil... > Also, there’s often no “correct” answer, so the game won’t tell you if you got something “right” or “wrong” because often there is no right or wrong.
lostgame 3y ago
...that seems like an obvious joke to you? My God, we must run in different circles. I had people I know who shamelessly posted almost that exact thing. A few of them, lol. They weren't joking. :P

gnopgnip 3y ago

It’s not just about what’s legal, it’s whatever the boss says

oldtownroad 3y ago

That’s the point though, right? Encouraging illegal activity may not itself be illegal but moderators aren’t arbiters of the law, they’re protecting the integrity of the website for the intended user base. If you want to allow people to encourage cocaine usage on your website, you have to be prepared to deal with law enforcement — because when someone follows the advice and dies with the website open on their computer, the website operators will get a call… — and not many people want to risk their entire company because “it’s legal to encourage people to take cocaine!” Moderation is hard and that example is… the perfect example. User reports are rarely accurate but still valuable.

ianferrel 3y ago
Certainly social media companies can choose to disallow suggestions that people take illegal actions. But that's not what the moderation policy for "illegal" content says. It says "Content that violates local or international laws or regulations." I don't know all the laws of all the countries, but my local laws make very little content illegal. Various types of obscenity, true threats, etc. "You should do drugs while you watch a movie about drugs" is definitely not one of those. It would be very easy, if that's what they want, to change it to "Content that encourages breaking local or international laws or regulations."
- oldtownroad 3y ago
  Visit a platform like LinkedIn, Twitter or Facebook and try to report content: the complex decision tree of what to report and why will be a good demonstration or how unintuitive the reality of content moderation is. Every user of every platform has a different understanding of even simple concepts, like “scam” and “spam”. Report reasons are not an accurate classification from a trained expert, they’re a thing a user chooses from a list that has to be designed to both provide valuable context to a moderator and capture the wide range of different understandings different users have. Suggesting that a reason should be added for the specific situation you’ve encountered demonstrates the naive understanding most people have of moderation and that’s what this game is good at. Platforms like Facebook and Twitter will have spent tens of thousands of people hours thinking about something as simple as the list of report reasons.
  - ianferrel 3y ago
    There are plenty of difficult judgment calls in content moderating. But the difference between "illegal content" and "content that promotes breaking the law" isn't really one of them to be made by the moderator. This is just bad instructions from the game maker.
    - ethanbond 3y ago
      You don’t think content moderators have to deal with: “hey kid, you should go shoot up a school?” Just cleanly not in their domain?
      - ianferrel 3y ago
        I think that content moderators often have to deal with that issue, and they are given rules like "remove posts that suggest that others commit violent acts" or something like that. And those are reasonable rules for moderators to enforce. But the rules that this game give do not include that rule, and it's a mark of a poorly designed game that the rule says "don't allow illegal content" and then when you (correctly!) apply that rule, the game says "you should have not allowed this because it's suggesting that someone do something illegal". Those aren't the same rule! ETA: Like lots of places have rules against reposting copyrighted content. But a post that said "Psst, kid, you should go download a movie from the Pirate Bay" should not be removed under that rule. Because the content of the post isn't copyrighted. If they also had a rule that said "don't encourage piracy", they could reasonably take it down under that rule.
commandlinefan 3y ago
> the website operators will get a call Then what we actually need isn't more content moderation, we need explicit laws explicitly _protecting_ website operators and any other publishers who are actually following the law.
- pixl97 3y ago
  I mean not really... sometimes some content just sucks and you don't want it on your site regardless of its legality. Start shitposting here on HN and see how long your posts hang around.
  - commandlinefan 3y ago
    But then that becomes a site owner editorial decision rather than a fear of legal reprisals decision.
NoMoreNicksLeft 3y ago
How can one protect the integrity of a website if they're undermining free speech rights and simultaneously derailing conversations and fragmenting communities while doing so? Though we didn't first define what "integrity" meant, it now seems impossible to formulate a meaning for it that isn't absurdly ironic given the actions performed to protect it. One could be forgiven if we were talking about a forum or website that exists in a totalitarian regime like China... better that there be some conversation than the only other option, none at all. But in the US or Europe or any other nation that considers itself even minimally enlightened? Moderation is not the solution. It may even be the problem.
- BHSPitMonkey 3y ago
  > How can one protect the integrity of a website if they're undermining free speech rights What free speech rights? You have no legal right to compel someone else to display your words on their website.
  - NoMoreNicksLeft 3y ago
    That's an interesting theory. I'm not sure you understand the implications of it though. Were it so simple, then it is plausible enough to construct a society where you have no free speech at all, anywhere, without your legal rights being violated. In fact, you all seem to be constructing that society even as we speak. Given the audience of this particular forum, more than a few of you are literally doing just that, rather than being accomplices and bystanders. I'm more of a free speech fundamentalist myself.
    - stonogo 3y ago
      Leaping from "I get to decide what's on my website" to "you've created a society" is pretty silly. Your argument is missing any meaningful linkages. It would be stronger if you articulated them instead of handwaving about plausibility. Personally I don't think social media speech is that important. It's ad-based, convenience-driven, and fickle. If it were to disappear people would just move to different media.

Zak 3y ago

The "illegal" rule requires a lot of clarification. The interpretation here seems to be "It's against our policy to post about engaging in illegal activities", or maybe "it's against our policy to advocate illegal activities". Services based in the US are not legally obligated to prevent users from writing about using cocaine, or pictures of white powders they're claiming are cocaine, or writing that it's desirable to use cocaine, so the post itself is not illegal. Real policies for content moderators should be much more detailed than what's in this game so that most moderators will reach the same conclusions about whether a post violates the policy.

idiotsecant 3y ago

It's ironic that the first post on this thread is about missing the point of the first card that user saw.

ianferrel 3y ago
What was the point of it, and how did I miss it?
- matthewfcarlson 3y ago
  This is showing how hard content moderation really is. It's judgement calls and very often no one is happy. Seeing more information takes an angering amount of time, which makes me think this is a fun game that makes an excellent point. Edit: In some ways, a card that's incredibly contextual as the first one is a brilliant move.
  - ianferrel 3y ago
    I'm not convinced that putting a real head-scratcher as the very first choice in a tutorial mode is a brilliant move at all. Tutorials are supposed to be easy and hand-holding. But also: this isn't actually a hard judgment call. It's just bad instructions. There's an objective difference between "content that breaks laws" and "content that promotes the breaking of laws" and the rules could easily be changed to indicate the one they want. If you were playing a platformer tutorial and you came to a gap and the tutorial instructions said "Press A to jump over the gap" and then when you tried you fell in and died and the tutorial then said "That was too big a gap to jump over. You should have pressed [Down] to climb over it instead", would you think "Ah, what a brilliant meditation on trusting trust and how the right choice is not obvious" or would you think "this is a shitty tutorial for a dumb game"? I'm not saying there's for sure a right answer here. I could imagine a well-made game where the tutorial straight-up lies to you and gives you the wrong directions. But most games like that are just badly written and poorly thought out.
    - bisby 3y ago

mirashii 3y ago

> Content moderation is hard, but I'm not very impressed that the very first card shown got it (in my opinion) totally wrong The authors are very clear that there are no right or wrong answers. Interpreting it that way misses the entire point of the exercise.

ianferrel 3y ago
I mean, when I approved leaving up the cocaine post, the game that they authored gave me a big red X and said I did it wrong that it clearly was illegal and should have been taken down. Where were they clear that there are no right or wrong answers?
- wmf 3y ago
  The users who submit reports are fallible, the moderators are fallible, and the bosses who rate the moderators are also fallible. Sometimes you'll be right and the system will still say you're wrong.
  - mike741 3y ago
    > the bosses who rate the moderators are also fallible. Sometimes you'll be right and the system will still say you're wrong. those are not the difficulties of moderating though, those are the difficulties of general employment.
- mirashii 3y ago
  https://www.techdirt.com/2023/05/11/moderator-mayhem-a-mobile-game-to-see-how-well-you-can-handle-content-moderation/ https://www.techdirt.com/2023/05/11/moderator-mayhem-a-mobil... > Also, there’s often no “correct” answer, so the game won’t tell you if you got something “right” or “wrong” because often there is no right or wrong.

lostgame 3y ago

...that seems like an obvious joke to you? My God, we must run in different circles. I had people I know who shamelessly posted almost that exact thing. A few of them, lol. They weren't joking. :P