5 ms·

> It's not a trivial problem to be able to > identify what is hate speech in a regional > dialect automatically This sentiment, which I read repeated

> It's not a trivial problem to be able to > identify what is hate speech in a regional > dialect automatically This sentiment, which I read repeatedly, really irritates me. Most of us program here. It's not that hard a problem! When social media execs claim it is, what they really mean is "it's a hard problem to solve for us, because we are unwilling to risk losing any percentage of account sign-ups ever, under any circumstances." The irony is that they are so timid about their precious sign-ups that they act against their own interest. For example, YouTube won't risk a short term dip in user retention metrics to scrub garbage from their comments... even though doing so would likely greatly increase sign-ups in the long term.

LoSboccacc 8y ago
natural language processing is like one of the hardest problems that's left to crack, don't be so dismissive about it i.e: "get rid of all black people" "get rid of all black people unemployment problems" "get rid of all black people unemployment problems sending them back to Africa" this can be expanded almost infinitely in every direction, and we're not even beginning with the false positive problems. "I hate black cars" so no, it's not just a claim that it's an hard problem.
- charlesism 8y ago
  > natural language processing is like one > of the hardest problems that's left to crack You don't need to "crack it" or come anywhere remotely near to a perfect solution. False positives are okay, providing you accept that some percent of your users won't tolerate the errors, and will leave your platform. And we're talking about social networks. So there's a wealth of additional information (ie: who you follow. previous behavior. who has reported you. account age) beyond plain text to augment this. And there's the setting of policy (eg: how come I can post within an hour of signing up on most social networks?). The point is, what we have now on most social networks is not geared toward long-term customer satisfaction or the public good. It's geared toward medium-term account retention at any cost - regardless of whether the accounts are bots, sock puppets, or trolls.
  - LoSboccacc 8y ago
    > False positives are okay Yeah, well, you know, that's just, like, your opinion, man edit: holy crap people get with the most outlandish or unsubsantiated claims around here like that one in quote but point at the logical issue that basically sap the whole line of reasoning and you're the bad guy.
pjc50 8y ago
It's a problem that can't be reliably solved by machines yet, so they end up having to hire moderators. Which is a pretty miserable job and also hard to standardise.
magnetic 8y ago
> This sentiment, which I read repeatedly, really irritates me. Most of us program here. It's not that hard a problem! What solution do you think would work?
- dankotron 8y ago
  Node REST API with MongoDB and React. Duh.
- charlesism 8y ago
  A combination of things. There's a wealth of information available if you're running a social network. Start by just banning text like "n-----" and "(((". Then give users a reputation, and do a page rank style analysis. So if someone has a good reputation, and they report another user, a moderator doesn't need to be involved. Take account age into consideration... you can't post publicly the instant you get your account. Be okay with some % of users quitting because of your false positives. It's not hard to think of ways to aggressively moderate a social network. It's just that some users (certainly the most toxic ones) will leave.
  - seanmcdirmid 8y ago
    I think you've just reinvented Chinese censorship. Congrats!
    - deleted 8y ago
      [deleted]
    - charlesism 8y ago
      Facebook isn't the state. In any event, I try not to think of problems that way. Like most things, state censorship is neither universally good nor bad. It depends on the state, and the censorship. The universe doesn't care that it's easier for humans to have hard and fast rules about things.
  - magnetic 8y ago
    Word banning is known to be a poor method. First, because the use of the word can be in a positive or negative sentence, and thus can either agree or disagree with your "censorship rule". You need a deeper NLP analysis to disambiguate. Second, because - if online fora with this kind of ban has taught us anything - it's pretty easy to work around it by using punctuation, replacement characters, or other style variations to write the word in a way that it won't be caught by the regex. Third, this method ends up banning legit words. There was a forum where I couldn't write "booby trap". I don't need to explain. WRT reputation, networks that allow "unlimited registrations" are easy targets for https://en.wikipedia.org/wiki/Sybil_attack https://en.wikipedia.org/wiki/Sybil_attack - One can use account age to mitigate of course, but given the amount of fake accounts that exist on the market (look at fiverr marketing offers) with long life and activity, I wouldn't expect age to be that big of a helper.

LoSboccacc 8y ago

natural language processing is like one of the hardest problems that's left to crack, don't be so dismissive about it i.e: "get rid of all black people" "get rid of all black people unemployment problems" "get rid of all black people unemployment problems sending them back to Africa" this can be expanded almost infinitely in every direction, and we're not even beginning with the false positive problems. "I hate black cars" so no, it's not just a claim that it's an hard problem.

charlesism 8y ago
> natural language processing is like one > of the hardest problems that's left to crack You don't need to "crack it" or come anywhere remotely near to a perfect solution. False positives are okay, providing you accept that some percent of your users won't tolerate the errors, and will leave your platform. And we're talking about social networks. So there's a wealth of additional information (ie: who you follow. previous behavior. who has reported you. account age) beyond plain text to augment this. And there's the setting of policy (eg: how come I can post within an hour of signing up on most social networks?). The point is, what we have now on most social networks is not geared toward long-term customer satisfaction or the public good. It's geared toward medium-term account retention at any cost - regardless of whether the accounts are bots, sock puppets, or trolls.
- LoSboccacc 8y ago
  > False positives are okay Yeah, well, you know, that's just, like, your opinion, man edit: holy crap people get with the most outlandish or unsubsantiated claims around here like that one in quote but point at the logical issue that basically sap the whole line of reasoning and you're the bad guy.

pjc50 8y ago

It's a problem that can't be reliably solved by machines yet, so they end up having to hire moderators. Which is a pretty miserable job and also hard to standardise.

magnetic 8y ago

> This sentiment, which I read repeatedly, really irritates me. Most of us program here. It's not that hard a problem! What solution do you think would work?

dankotron 8y ago
Node REST API with MongoDB and React. Duh.
charlesism 8y ago
A combination of things. There's a wealth of information available if you're running a social network. Start by just banning text like "n-----" and "(((". Then give users a reputation, and do a page rank style analysis. So if someone has a good reputation, and they report another user, a moderator doesn't need to be involved. Take account age into consideration... you can't post publicly the instant you get your account. Be okay with some % of users quitting because of your false positives. It's not hard to think of ways to aggressively moderate a social network. It's just that some users (certainly the most toxic ones) will leave.
- seanmcdirmid 8y ago
  I think you've just reinvented Chinese censorship. Congrats!
  - deleted 8y ago
    [deleted]
  - charlesism 8y ago
    Facebook isn't the state. In any event, I try not to think of problems that way. Like most things, state censorship is neither universally good nor bad. It depends on the state, and the censorship. The universe doesn't care that it's easier for humans to have hard and fast rules about things.
- magnetic 8y ago
  Word banning is known to be a poor method. First, because the use of the word can be in a positive or negative sentence, and thus can either agree or disagree with your "censorship rule". You need a deeper NLP analysis to disambiguate. Second, because - if online fora with this kind of ban has taught us anything - it's pretty easy to work around it by using punctuation, replacement characters, or other style variations to write the word in a way that it won't be caught by the regex. Third, this method ends up banning legit words. There was a forum where I couldn't write "booby trap". I don't need to explain. WRT reputation, networks that allow "unlimited registrations" are easy targets for https://en.wikipedia.org/wiki/Sybil_attack https://en.wikipedia.org/wiki/Sybil_attack - One can use account age to mitigate of course, but given the amount of fake accounts that exist on the market (look at fiverr marketing offers) with long life and activity, I wouldn't expect age to be that big of a helper.