4 ms·
It's an interesting concept, but falls short when comments are sarcastic or contain contextualized information. It seems like this is more of a "word filter" t
by dvt 9y ago
It's an interesting concept, but falls short when comments are sarcastic or contain contextualized information.
It seems like this is more of a "word filter" than anything. I mean, sure, it's pretty obvious a comment will be toxic if it contains "asshole" in it. Here are some examples where the labeling fails. Some of these I got from real comments and articles on CNN or Breitbart.
# Toxicity too low
34% toxic: Obama was born in Africa and he should move back there.
40% toxic: George Soros and his Jewish sympathizers want to destabilize our government.
25% toxic: Breitbart is run by jews - as is all mass media. As the saying goes, "the best way to control the opposition is to lead it".
47% toxic: Why do you expect anyone to take you seriously when you don't take yourself seriously.
# Toxicity too high
51% toxic: Ivanka is arguably the most powerful Jewish woman in America today.
51% toxic: US Muslims and Jews are strengthening bonds amid acts of bigotry.
48% toxic: The problem of income inequality is not between Black America and White America but between black haves and black have-nots.
87% toxic: I lost my grip and fell on my ass. Not a great bike.
- thaumasiotes 9y ago> # Toxicity too low > 47% toxic: Why do you expect anyone to take you seriously when you don't take yourself seriously. I would have put that under "toxicity too high"; it seems like a conventional, vanilla exhortation to be more self-confident?
- dvt 9y agoYeah, you might be right. That one example isn't very well thought out.
- comex 9y agoDepends on context. I can easily imagine it used as an attack, like: “Why aren’t you defending the point you made earlier? It seems you don’t take your own argument seriously. Why do you expect…”
- majewsky 9y ago> Depends on context. Exactly.
- minimaxir 9y agoIt should be noted that the % output is a probability that the text is toxic (i.e. a logistic regression), not a proportionate quantification, which changes the interpretation of the value slightly.
- shurcooL 9y agoThis was my perception as well, after trying a few "easy" cases and then trying to find "tricky" ones to check how well it'll handle them. This entire paragraph is classified as 68% likely to be perceived as toxic only because it contains the word "idiots". If you replace it with another word like "dragons", it's down to 13%: Hi, I don't completely agree with your point about idiots being commonplace. First of all, that is a derogatory word with no specific meaning. If you have a problem with their opinion, it's more constructive to point out ways in which you disagree, so we can have a more civil discourse. It's not doing a great job of seeing the context in which a word is used and reacting to the word itself.
- microcolonel 9y agoFor what it's worth, the ones at the top are only contextually toxic to you, either because you believe the opinions are ill-founded, or you don't know the intent (regarding "why do you expect anyone to take you seriously..."). Knowing no facts, "George Soros and his Jewish sympathizers want to destabilize our government." is a straight-forward and polite way to state an opinion. As for the too-high ones, yeah; that bike review [?] is rated hilariously poorly. Frankly, this API can not (should not?) be expected to determine the truthfulness of a statement, nor how offended you're likely to be by somebody's honest opinion.
- majewsky 9y ago> Knowing no facts, "George Soros and his Jewish sympathizers want to destabilize our government." is a straight-forward and polite way to state an opinion. That sounds reasonable, but then why is the statement "Ivanka is arguably the most powerful Jewish woman in America today" considered doubly more toxic (25% vs 51%)?
- cameldrv 9y agoThe obvious next step is to make an API that takes an input text and minimally modifies it to evade the filter, adversarial example style, and then package it as a chrome plugin, like grammarly.
- panarky 9y ago90% toxic: "It's pretty obvious a comment will be toxic if it contains "asshole" in it."