3 ms·
I recently developed a neural network model which can predict the reaction to a given text/comment with reasonably low error (I'll be open-sourcing the model so
by minimaxir 9y ago
I recently developed a neural network model which can predict the reaction to a given text/comment with reasonably low error (I'll be open-sourcing the model soon).
There are a few caveats with using these approaches:
1) Toxicity is heavily contextual, not just by topic (as the demo texts indicate), but also by source; at the risk of starting a political debate, a comment that would be considered toxic by the NYT/Guardian (i.e. the sources Google partnered with) may not regarded by toxic on conservative sites. It makes training a model much more difficult, but it's necessary to do so to get an unbiased, heterogenous sample.
2) When looking at comments only, there's a selection bias toward "readable" comments, while anyone who has played online games know that toxic commentary is often less "Your wrong" and more "lol kill urself :D"
3) Neural networks still have difficulty with sarcastic comments and could miscontrue sarcasm as toxic, which users on Hacker News would absolutely never believe.
- milcron 9y agoI don't believe it's possible to be unbiased - whether as a news site or as a moderation filter. It's better to be aware of your biases than to fool yourself into believing they don't exist.
- yosito 9y agoI agree with this. I'd like to see an AI that detects which biases a person has. The biases could be associated with short tags which could be displayed to commenters in a UI to make people aware of the biases they have.
- gt_ 9y agoOof. Yes please. In a just world, this would be the direction this project is heading in.
- gt_ 9y agoThis is the lesson of the age. Upvote
- jdavis703 9y agoHumans on Hacker News misconstrue sarcastic comments, I'm not sure a neural network will ever be able to do better than a human WRT sarcasm.
- yosito 9y agoHumans still have difficulty with sarcastic comments and could miscontrue sarcasm as toxic.