6 ms·
I can't believe nobody has mentioned naive Bayesian text classification yet. It sounds like it could work wonders for Twitter. I'm much more likely to be intere
by sketerpot 16y ago
I can't believe nobody has mentioned naive Bayesian text classification yet. It sounds like it could work wonders for Twitter. I'm much more likely to be interested in tweets with words like "hylomorphism" than tweets with words like "omglol", and a text classification algorithm could learn that if you trained it up some. It doesn't have to be perfect; it just has to improve the signal-to-noise ratio significantly.
- kmavm 16y agoI've looked into this a bit, albeit more in a spam-filtering context; tweets have very little text for naive Bayes to latch onto. 140 characters would be 20-30 words, tops. That is so few words that it is hard to move the prior very much, unless there are blockbuster words that almost always indicate a bad tweet; as the article suggested, "breakfast", "beer", etc.
- jshen 16y agoDont many classifiers pick the most interesting n words and only use those to decide? Where n is something like 15.
- JCThoughtscream 16y agoFilter by account, then, instead of by post? With a sample body of, say, the ten or twenty of an account's most recent posts.
- kmavm 16y agoThe article's whole premise was that tweet quality does not correlate well within an account; e.g., some marvelous twitter streams include breakfast tweets.
- sketerpot 16y agoAs I said, I'm fine with a filter that has poor accuracy, as long as it makes twitter marginally better.
- fizx 16y agoHere's a start. It's a simple ruby app that runs on heroku and proxies your twitter account, dropping uninteresting things. Extremely raw code. http://github.com/fizx/sometweets http://github.com/fizx/sometweets http://sometweets.heroku.com/ http://sometweets.heroku.com/
- avk 16y agoI'm currently solving this very problem with a much simpler algorithm and slowly letting folks into my beta over at http://slipstre.am/ http://slipstre.am/
- retube 16y agoI think stat/learning classifiers could work, even with the brevity of tweets. However, as the author says, one persons gold is anothers garbage. You'd need a platform/system that would allow for personal classifiers for each/every tweet consumer.