3 ms·
I've built a very similar algorithm and website for a class I was taking a few years ago, however, we never considered monetizing it like this: https://github.c
by wespiser_2018 7y ago
I've built a very similar algorithm and website for a class I was taking a few years ago, however, we never considered monetizing it like this: https://github.com/adamwespiser/toxic-twitter https://github.com/adamwespiser/toxic-twitter
There is a large collection of "negative tweets" available in a public dataset, and training a simple neural network will give you a rough way to classify a tweet as either "toxic" or not with decently accuracy. However, there are problems.
- Basic algorithms are "context-free" and therefore don't get sarcasm or jest. There are more sophisticated approaches that can handle these semantic categories, but they require large datasets of tweets that do not publicly exist.
- The easiest thing the algorithm can learn is profanity, so for example, the model we built classified tweets in poor taste as "okay", if they were used with decent language.
- There is definitely a balanced dataset issue that makes training good classifiers harder, due to the overwhelming majority of tweets on twitter being positive, and it being hard to find unblocked toxic users.
At the time, we figured the best use of the website was to check your own twitter history, and although I find some of this user's tweet pretty offensive, I'm sad the day has come where people would actually need to use something like "toxic-twitter" to get a job! If anyone wants to fork this repo and get it up again, please reach out!