5 ms·

The false negative example you have us exactly the argument I've used to fight against language filters and sensors in smaller sites. In the end you can be pro

The false negative example you have us exactly the argument I've used to fight against language filters and sensors in smaller sites. In the end you can be profane and very positive. You can also have strike language while being incredibly vile and negative.

Gibbon1 5y ago
> In the end you can be profane and very positive. I've noticed a lot of humans can't detect sarcasm. And that isn't correlated with traditional smarts either. For AI it's a hopeless task. > You can also have strike language while being incredibly vile and negative. I'm reminds of a group of people I know. They love love using genteel language to throw vile insults at each other. Using profane language is a automatic foul.
skissane 5y ago
You can also write things which sound violent (when read literally) but are actually innocent. My wife's cousin recently had a baby, and posted a photo on Facebook. My wife commented (something like) "She's so cute I'll have to kidnap her". Facebook locked her account for 24 hours for making a "violent threat". I am sure the mother wasn't threatened by the remark in the slightest. It just added to my wife's anger at Facebook for repeatedly giving her "warnings" over trivial or innocent things. Not online, in person, but one of the teachers at our son's school sometimes "threatens" to "steal him" from us – her remarks don't worry us, because we know she would never actually do that, it is just a colloquial way of expressing affection.
- tobyhinloopen 5y ago
  What if a random old guy she didn’t know commented that exact message? I don’t think the mother would be as accepting then, so it also matters who posts the comment to who
  - skissane 5y ago
    To Facebook they are “friends” and have many mutual “friends” as well. You’d think they could take that into account in evaluating the nature of the remark, but it does not appear they have done so.
    - watwut 5y ago
      You cant conclude "this is not a threat" from "being friends and having mutual friends". Most of violence by large is between people who know each other.
      - skissane 5y ago
        If you know that A and B are “friends with many mutual friends”, and A posts baby photos, and B comments “She's so cute I'll have to kidnap her", that’s enough information to conclude that the remark is more likely to be experienced as innocent than threatening.
        watwut 5y ago
        Which loops back to original question: what if it is a man. What if we remove the "women are inherently unthreatening" assumption.
        skissane 5y ago
        My wife is quite convinced - and I think she is probably right - that nobody reported her comment, some automatic system at Facebook picked it up purely on the basis of the keyword “kidnap”. If a male relative or friend had made the same remark, would my wife’s cousin have felt threatened? My wife tells me she would not have been bothered by such a remark, in that context, from one of her male relatives or friends. And, if someone was really bothered by it, they could report it. But, it doesn’t seem like that is what has happened here, it appears to just be the work of some automated system. I’m not saying that if the mother reports the remark Facebook shouldn’t take the report seriously - cultures differ after all, in some cultures those kind of remarks are normal in that context, in others they would seem bizarre and offensive - but it shouldn’t just assume the remark is threatening because it contains the word “kidnap”
  - tracker1 5y ago
    In that case, if it's a genuine threat, then freezing the account isn't the right action, notifying local law enforcement is. Beyond this, by hiding the comment/post, the mother is now unaware of the threat and Facebook didn't contact local law enforcement in fact worse than doing nothing at all.
  - anshorei 5y ago
    Maybe it should be up to the mother to report this as a threat, rather than being up to a mindless drone.
- bryan_w 5y ago
  > Facebook locked her account for 24 hours for making a "violent threat". This is due to years of old media digging through Facebook posts to write stories like "Child kidnapped after threat on Facebook and Facebook DID NOTHING". This led to more and more calls for FB to "do something" to "keep people safe online". So they started to run all posts through the AI prescreener and well, these are the results. But hey, they get to at least say that they take down X billions of posts before any other human sees it.
vidarh 5y ago
One of my childhood friends was told by his parents they'd dock his pocket money if he kept insulting his brother. So he started using the names of cheeses. Worked just as well at insulting his brother, and created a quandary for his parents who couldn't well dock his pocket money for saying "camembert" to his brother. Now consider a large group of people upset about censorship and motivated to adjust their language to circumvent it...
- meheleventyone 5y ago
  It’s not really a quandary though is it? The friend is still insulting his brother. The parents and other people worried about good behaviour aren’t stupid and can apply the spirit of the rule. The problem (at scale) as ever is trying to fix a social issue with a barely working bit of tech.
  - vidarh 5y ago
    The quandary is that it made it apparent that trying to block terms just creates a moving target. It demonstrated that completely innocuous terms could be weaponized by context, and it was not an issue that could be solved by blocking specific terms. The specific words don't matter, as any reaction would just shift the insult to another term. Ban cheeses? Start saying "I love you" ("mom, he told me he loves me! Tell him to stop it!"...). Or call him awesome. If anything, I think the brother missed a trick by not going straight to compliments - few things can be more cutting. Given the right context it'd be clear to the brother what the intent was no matter the term, but increasingly impossible for the parents to distinguish "legitimate" communications from the insults. For that matter, no words are necessary to achieve that once both sides know. Just a look is enough. We don't know how to fix this social issue with human censors, so it's much worse than just the state of the tech - it's an issue that's basically unsolvable without addressing why the speaker is intending to insult (if they are) and why the receiver takes offence (if they are). The best you can do is address a small proportion of the most blatant cases (and that might well be ok and enough if you can do it with few enough false positives, but getting the false positives to an acceptable level is in itself a momentous challenge).

Gibbon1 5y ago

> In the end you can be profane and very positive. I've noticed a lot of humans can't detect sarcasm. And that isn't correlated with traditional smarts either. For AI it's a hopeless task. > You can also have strike language while being incredibly vile and negative. I'm reminds of a group of people I know. They love love using genteel language to throw vile insults at each other. Using profane language is a automatic foul.

skissane 5y ago

You can also write things which sound violent (when read literally) but are actually innocent. My wife's cousin recently had a baby, and posted a photo on Facebook. My wife commented (something like) "She's so cute I'll have to kidnap her". Facebook locked her account for 24 hours for making a "violent threat". I am sure the mother wasn't threatened by the remark in the slightest. It just added to my wife's anger at Facebook for repeatedly giving her "warnings" over trivial or innocent things. Not online, in person, but one of the teachers at our son's school sometimes "threatens" to "steal him" from us – her remarks don't worry us, because we know she would never actually do that, it is just a colloquial way of expressing affection.

tobyhinloopen 5y ago
What if a random old guy she didn’t know commented that exact message? I don’t think the mother would be as accepting then, so it also matters who posts the comment to who
- skissane 5y ago
  To Facebook they are “friends” and have many mutual “friends” as well. You’d think they could take that into account in evaluating the nature of the remark, but it does not appear they have done so.
  - watwut 5y ago
    You cant conclude "this is not a threat" from "being friends and having mutual friends". Most of violence by large is between people who know each other.
    - skissane 5y ago
      If you know that A and B are “friends with many mutual friends”, and A posts baby photos, and B comments “She's so cute I'll have to kidnap her", that’s enough information to conclude that the remark is more likely to be experienced as innocent than threatening.
      - watwut 5y ago
        Which loops back to original question: what if it is a man. What if we remove the "women are inherently unthreatening" assumption.
        skissane 5y ago
        My wife is quite convinced - and I think she is probably right - that nobody reported her comment, some automatic system at Facebook picked it up purely on the basis of the keyword “kidnap”. If a male relative or friend had made the same remark, would my wife’s cousin have felt threatened? My wife tells me she would not have been bothered by such a remark, in that context, from one of her male relatives or friends. And, if someone was really bothered by it, they could report it. But, it doesn’t seem like that is what has happened here, it appears to just be the work of some automated system. I’m not saying that if the mother reports the remark Facebook shouldn’t take the report seriously - cultures differ after all, in some cultures those kind of remarks are normal in that context, in others they would seem bizarre and offensive - but it shouldn’t just assume the remark is threatening because it contains the word “kidnap”
- tracker1 5y ago
  In that case, if it's a genuine threat, then freezing the account isn't the right action, notifying local law enforcement is. Beyond this, by hiding the comment/post, the mother is now unaware of the threat and Facebook didn't contact local law enforcement in fact worse than doing nothing at all.
- anshorei 5y ago
  Maybe it should be up to the mother to report this as a threat, rather than being up to a mindless drone.
bryan_w 5y ago
> Facebook locked her account for 24 hours for making a "violent threat". This is due to years of old media digging through Facebook posts to write stories like "Child kidnapped after threat on Facebook and Facebook DID NOTHING". This led to more and more calls for FB to "do something" to "keep people safe online". So they started to run all posts through the AI prescreener and well, these are the results. But hey, they get to at least say that they take down X billions of posts before any other human sees it.

vidarh 5y ago

One of my childhood friends was told by his parents they'd dock his pocket money if he kept insulting his brother. So he started using the names of cheeses. Worked just as well at insulting his brother, and created a quandary for his parents who couldn't well dock his pocket money for saying "camembert" to his brother. Now consider a large group of people upset about censorship and motivated to adjust their language to circumvent it...

meheleventyone 5y ago
It’s not really a quandary though is it? The friend is still insulting his brother. The parents and other people worried about good behaviour aren’t stupid and can apply the spirit of the rule. The problem (at scale) as ever is trying to fix a social issue with a barely working bit of tech.
- vidarh 5y ago
  The quandary is that it made it apparent that trying to block terms just creates a moving target. It demonstrated that completely innocuous terms could be weaponized by context, and it was not an issue that could be solved by blocking specific terms. The specific words don't matter, as any reaction would just shift the insult to another term. Ban cheeses? Start saying "I love you" ("mom, he told me he loves me! Tell him to stop it!"...). Or call him awesome. If anything, I think the brother missed a trick by not going straight to compliments - few things can be more cutting. Given the right context it'd be clear to the brother what the intent was no matter the term, but increasingly impossible for the parents to distinguish "legitimate" communications from the insults. For that matter, no words are necessary to achieve that once both sides know. Just a look is enough. We don't know how to fix this social issue with human censors, so it's much worse than just the state of the tech - it's an issue that's basically unsolvable without addressing why the speaker is intending to insult (if they are) and why the receiver takes offence (if they are). The best you can do is address a small proportion of the most blatant cases (and that might well be ok and enough if you can do it with few enough false positives, but getting the false positives to an acceptable level is in itself a momentous challenge).