11 ms·

One of the big problems with machine learning algorithms at this time is that they have a hard time figuring out intent. A lot of the terms used by the LGBTQ co

One of the big problems with machine learning algorithms at this time is that they have a hard time figuring out intent. A lot of the terms used by the LGBTQ community, have been used by homophobic people as slurs. Yes it is great that the community is reclaiming those words, but current ML tech has no way of distinguishing when those terms are being used by the community in a respectful manner from where they are used by homophobic people to promote hate. This is also the case with African Americans, and there is a similar problem of machine learning flagging African American content as hate speech. https://www.vox.com/recode/2019/8/15/20806384/social-media-hate-speech-bias-black-african-american-facebook-twitter https://www.vox.com/recode/2019/8/15/20806384/social-media-h... The problem may be that we cannot depend on machine learning algorithms to do an acceptable job of moderation, and that a lot more humans need to be hired as moderates. However, doing that really changes the economics of the platform

deleted 7y ago
[deleted]
mushufasa 7y ago
What about crowdsourcing efforts? Why are user flags insufficient? Why can't there be a balance between full automation and human review, such as an appeal system or 'whitelist' program? Are the economics really the obstacle, or is it more of a corporate stance towards not adjudicating subjective controversies to avoid liability under the Communications Decency Act Section 230?
- lordCarbonFiber 7y ago
  The problem with crowd sourcing, especially in the LGBT space, is it's really vulnerable to brigading. Give particularly bored and hatful people a bat to shut down channels they don't like and things go south fast.
  - prometheus76 7y ago
    Yeah, it's a good thing those brigading actions never go the other way, though, right? That would be wrong and immature and just plain hypocritical...Sure glad it never happens!
    - lordCarbonFiber 7y ago
      Off topic, give that the tread is about an idea for protecting minority groups largely unserved by the algorithm. I fail to see how a "but both sides" proclamation leads to any progress (other than maintaining the status quo which is awful good for a select group of people).
      - 12thsie34 7y ago
        The status quo favors LGBT
- RcouF1uZ4gsC 7y ago
  >What about crowdsourcing efforts? Why are user flags insufficient? Because this will devolve into different groups flagging their opponents' videos. This will especially hurt the voices of minorities and other vulnerable populations.
  - mushufasa 7y ago
    What about attaching a 'reputation' score to the flagger? E.g. something like: - if there are sufficient flags, trigger a moderator review - flaggers who correctly flagged content get +1 score or -1 score if not that would make it harder for people to consistently flag content maliciously. Like age-old moderation systems such as calling 911 or police. If you abuse calling 911, you can get fined. I imagine this has been tried before, and am genuinely curious why this is worse than flawed ML.
    - defertoreptar 7y ago
      Yep, I believe there is a way for sure. YouTube can evaluate users by how likely it is that their flag results in long term removal of a video. If a user cries wolf too many times or flags videos later marked as not controversial at all, then that user probably should not have flag powers.
    - viraptor 7y ago
      Have a look at some of the organised voting/flagging campaigns originating from *chans. Those can range from just manual action to automated scripts + account takeovers. Also if you can both post comments and flag them from a different account - free karma.
    - techdragon 7y ago
      This concept reminds me of the “meta-moderation” that happened/happens in SlashDot. It’s good to have a second order effect to damp down excessive fluctuations in your first order mechanisms.
    - prometheus76 7y ago
      The problem with that plan is "correctly" flagged content is subjective. You've only added a layer of moderation to the original problem.
- grogenaut 7y ago
  Read up on the term "brigading"... It's where a bunch of people usually try and sour a ml approval system for nefarious purposes or to enforce their opinions. Happens a lot with lbgtqia and gender issues in gaming. This is a difficult issues made harder because your going against intelligent opponents.
big_chungus 7y ago
> doing this really changes the economics of the platform Not to mention that moderating this kind of stuff can seriously mess you up: https://www.theverge.com/2019/6/19/18681845/facebook-moderator-interviews-video-trauma-ptsd-cognizant-tampa https://www.theverge.com/2019/6/19/18681845/facebook-moderat... You end up looking at not just mean words but CP, seriously gory stuff, fringe sexual deviancy, and more. Do we really want to do this to moderators?
- jakelazaroff 7y ago
  If YouTube can't protect its users, and the content people are posting is too distressing even for moderators, maybe YouTube shouldn't exist?
  - defertoreptar 7y ago
    Great solution that is not at all impractical in any way.
  - perl4ever 7y ago
    The obvious solution is to restrict who can use YouTube in some way that anti-correlates with posting awful stuff. I don't know specifically, but there must be something.
    - goatinaboat 7y ago
      Require verified ID for every uploader. Charge a token amount say $1 for every upload. Only accept uploads from countries with law enforcement that handles bad content and make it clear that law enforcement will be knocking on your door if you upload anything bad. Extend this to commenters. This is just not a difficult problem, YouTube just doesn’t want to solve it
      - 12thsie34 7y ago
        Holy shit you're a vile authoritatian.
philpem 7y ago
You make a good point - even humans have problems with context. Classic example being that in some areas of the UK, close friends will greet each other with a joking insult. If you don't know they're friends or you don't have the verbal clues to work from (the 'I'm joking' intonation), it looks a lot like the start of a bar fight. If humans can't get 100% accuracy on this - how can we expect it from business rules, machine learning and so on? To pinch a line from ST:TNG, "Life is an exercise in exceptions".
deleted 7y ago
[deleted]
Balgair 7y ago
> However, doing that really changes the economics of the platform Getting profit-loss data on Youtube is very difficult. If they are profitable, it can't be by a large amount. So, if the plan is to have a LOT of real people just trying to monitor YT all day long it will really gut YT's economics, not just change it. Destin of Smarter Every Day on YT went and talked to FB and their content mod team a few months back [0]. He talked with Rob Leathern of FB's Cyber Security Department. Rob mentiond that they now use ~30k real people to monitor FB. Soon thereafter, The Verge released a scoop on the horrific conditions these 30k people must endure [1] in that job. YT would likely have to have a proportinate number of people in similar conditions as well, or likely scale up to make things less worse and then have to pay more people. YT simply cannot possibly afford to have enough people moderate YT. Likely, this version of YT will become something of a blip in history. All this investment will likely not pan out as the economics of video platforms like YT cannot work as they are today. The real question is what is the subscription amount that does make YT work the way we want it to work? [0] https://www.youtube.com/watch?v=FY_NtO7SIrY https://www.youtube.com/watch?v=FY_NtO7SIrY Also, watch Destin's body language with Nate Gleischer (~14:30 mark). It's so strange. Something wierd happened in that interview with Nate. [1] https://www.theverge.com/2019/6/19/18681845/facebook-moderator-interviews-video-trauma-ptsd-cognizant-tampa https://www.theverge.com/2019/6/19/18681845/facebook-moderat...

deleted 7y ago

[deleted]

mushufasa 7y ago

What about crowdsourcing efforts? Why are user flags insufficient? Why can't there be a balance between full automation and human review, such as an appeal system or 'whitelist' program? Are the economics really the obstacle, or is it more of a corporate stance towards not adjudicating subjective controversies to avoid liability under the Communications Decency Act Section 230?

lordCarbonFiber 7y ago
The problem with crowd sourcing, especially in the LGBT space, is it's really vulnerable to brigading. Give particularly bored and hatful people a bat to shut down channels they don't like and things go south fast.
- prometheus76 7y ago
  Yeah, it's a good thing those brigading actions never go the other way, though, right? That would be wrong and immature and just plain hypocritical...Sure glad it never happens!
  - lordCarbonFiber 7y ago
    Off topic, give that the tread is about an idea for protecting minority groups largely unserved by the algorithm. I fail to see how a "but both sides" proclamation leads to any progress (other than maintaining the status quo which is awful good for a select group of people).
    - 12thsie34 7y ago
      The status quo favors LGBT
RcouF1uZ4gsC 7y ago
>What about crowdsourcing efforts? Why are user flags insufficient? Because this will devolve into different groups flagging their opponents' videos. This will especially hurt the voices of minorities and other vulnerable populations.
- mushufasa 7y ago
  What about attaching a 'reputation' score to the flagger? E.g. something like: - if there are sufficient flags, trigger a moderator review - flaggers who correctly flagged content get +1 score or -1 score if not that would make it harder for people to consistently flag content maliciously. Like age-old moderation systems such as calling 911 or police. If you abuse calling 911, you can get fined. I imagine this has been tried before, and am genuinely curious why this is worse than flawed ML.
  - defertoreptar 7y ago
    Yep, I believe there is a way for sure. YouTube can evaluate users by how likely it is that their flag results in long term removal of a video. If a user cries wolf too many times or flags videos later marked as not controversial at all, then that user probably should not have flag powers.
  - viraptor 7y ago
    Have a look at some of the organised voting/flagging campaigns originating from *chans. Those can range from just manual action to automated scripts + account takeovers. Also if you can both post comments and flag them from a different account - free karma.
  - techdragon 7y ago
    This concept reminds me of the “meta-moderation” that happened/happens in SlashDot. It’s good to have a second order effect to damp down excessive fluctuations in your first order mechanisms.
  - prometheus76 7y ago
    The problem with that plan is "correctly" flagged content is subjective. You've only added a layer of moderation to the original problem.
grogenaut 7y ago
Read up on the term "brigading"... It's where a bunch of people usually try and sour a ml approval system for nefarious purposes or to enforce their opinions. Happens a lot with lbgtqia and gender issues in gaming. This is a difficult issues made harder because your going against intelligent opponents.

big_chungus 7y ago

> doing this really changes the economics of the platform Not to mention that moderating this kind of stuff can seriously mess you up: https://www.theverge.com/2019/6/19/18681845/facebook-moderator-interviews-video-trauma-ptsd-cognizant-tampa https://www.theverge.com/2019/6/19/18681845/facebook-moderat... You end up looking at not just mean words but CP, seriously gory stuff, fringe sexual deviancy, and more. Do we really want to do this to moderators?

jakelazaroff 7y ago
If YouTube can't protect its users, and the content people are posting is too distressing even for moderators, maybe YouTube shouldn't exist?
- defertoreptar 7y ago
  Great solution that is not at all impractical in any way.
- perl4ever 7y ago
  The obvious solution is to restrict who can use YouTube in some way that anti-correlates with posting awful stuff. I don't know specifically, but there must be something.
  - goatinaboat 7y ago
    Require verified ID for every uploader. Charge a token amount say $1 for every upload. Only accept uploads from countries with law enforcement that handles bad content and make it clear that law enforcement will be knocking on your door if you upload anything bad. Extend this to commenters. This is just not a difficult problem, YouTube just doesn’t want to solve it
    - 12thsie34 7y ago
      Holy shit you're a vile authoritatian.

philpem 7y ago

You make a good point - even humans have problems with context. Classic example being that in some areas of the UK, close friends will greet each other with a joking insult. If you don't know they're friends or you don't have the verbal clues to work from (the 'I'm joking' intonation), it looks a lot like the start of a bar fight. If humans can't get 100% accuracy on this - how can we expect it from business rules, machine learning and so on? To pinch a line from ST:TNG, "Life is an exercise in exceptions".

deleted 7y ago

[deleted]

Balgair 7y ago

> However, doing that really changes the economics of the platform Getting profit-loss data on Youtube is very difficult. If they are profitable, it can't be by a large amount. So, if the plan is to have a LOT of real people just trying to monitor YT all day long it will really gut YT's economics, not just change it. Destin of Smarter Every Day on YT went and talked to FB and their content mod team a few months back [0]. He talked with Rob Leathern of FB's Cyber Security Department. Rob mentiond that they now use ~30k real people to monitor FB. Soon thereafter, The Verge released a scoop on the horrific conditions these 30k people must endure [1] in that job. YT would likely have to have a proportinate number of people in similar conditions as well, or likely scale up to make things less worse and then have to pay more people. YT simply cannot possibly afford to have enough people moderate YT. Likely, this version of YT will become something of a blip in history. All this investment will likely not pan out as the economics of video platforms like YT cannot work as they are today. The real question is what is the subscription amount that does make YT work the way we want it to work? [0] https://www.youtube.com/watch?v=FY_NtO7SIrY https://www.youtube.com/watch?v=FY_NtO7SIrY Also, watch Destin's body language with Nate Gleischer (~14:30 mark). It's so strange. Something wierd happened in that interview with Nate. [1] https://www.theverge.com/2019/6/19/18681845/facebook-moderator-interviews-video-trauma-ptsd-cognizant-tampa https://www.theverge.com/2019/6/19/18681845/facebook-moderat...