7 ms·
Create your own machine-learning-powered RSS reader
- toisanji 12y agothe signup link doesn't work
- mmenafra 12y agoAwesome!
- Noctyrnal 12y agoThis is great stuff! Super useful!
- doppenhe 12y agoThanks! Please feel free to send any feedback to diego at algorithmia dot com.
- kevindavis 12y agoReally interesting, has my mind spinning around the possibilities. Would love to be able to apply this stuff to filter my Feedly account - how hard would that be using Algorithmia?
- doppenhe 12y agoI am not super familiar with the the feedly api (or how it works) but integrating with Algorithmia should be really simple. There is a registration link at the bottom of the blog post and you can check out the docs at http://algorithmia.com/docs http://algorithmia.com/docs. feel free to drop me a note at diego at algorithmia dot com as well.
- kevindavis 12y agoJust checked out the docs, looks like they have an easy call to mark one/multiple articles as read - so could have something running periodically filtering out stuff you're not interested in
- minimaxir 12y agoWhat are the rate limits on the APIs? Especially since they're very computationally expensive.
- doppenhe 12y agocurrently no rate limits while in private beta.
- photorized 12y agoI recently launched a Twitter app designed to filter out the noise by measuring the velocity of all tweets from all people I follow. It only shows me 5 things at any given moment that are most likely "interesting". Check it out, it's free, and it only wants read-only auth permissions via Twitter: http://skim.io/ http://skim.io/ Still tweaking the algorithm.
- doppenhe 12y agocool. You should add the algorithm you use to our library ! :). I have some examples already implementing twitter4j.
- photorized 12y agoThat's an interesting idea, will check out your docs.
- trickjarrett 12y agoYears ago I did something kind of similar. I created my own Twitter client (never public and now defunct) but one of the features was that it would prevent one user from monopolizing my stream by hiding their posts after the three most recent. So say someone was live tweeting an event I would only see their three most recent, and after that it would display a collapsed bar to let me know it had hidden something. Made skimming much easier, I still miss this feature.
- photorized 12y agopmarca would hate that feature, it would interfere with his tweet storms. :) In all seriousness, noise is the biggest problem I have with twitter. When you follow more than 100 accounts, the timeline becomes pretty much useless. Your comment gives me another idea for my app though - we will make it so each of the Top 5 is from a different unique publisher... and if more than one of their comments are trending, we choose the 'fastest'.
- jshholland 12y agoA friend had an idea to make a Twitter client that just drops half of all tweets at random. I don't think he ever got round to implementing it, but I thought it was quite a cool idea.
- photorized 12y agoOne sentiment analysis, there seems to be a lot of false negatives - at least when parsing Techcrunch. For example, all these were tagged as 'negative': - Timely Turns Your Calendar Into A Time Tracker - Audi Tests Self-Driving Cars On Florida’s Roads - Twitter Acquires Password Security Startup Mitro, Open Sources Its Product
- doppenhe 12y agoWe used the Stanford NLP library with their training data set which is considered to be one of the better ones. I did notice false negatives as well but it can definitely be trained to be more accurate.
- photorized 12y agoI decided there's no such thing as a good NLP library. In my other apps, I usually use several libraries at once, then make them vote. :) Works out better than Stanford NLP. I do need to add some intelligence to skim.io though.
- doppenhe 12y agoYes 100% agreed. One of our goals with Algorithmia is to have all these libraries already there , preloaded, available and standardized around a similar api signature. This way you can do exactly what you described or quick A/B testing on them.
- hnriot 12y agoThis is called ensemble classification where you feed the outputs of multiple classifiers as features into an ensemble classifier that produces the final result. How are you using the Stanford NLP? That's all GPL? There are alternatives you could look at for sentiment analysis but short "documents" like those referenced will always produce poor results because there's just not enough signal to work with. The training models need to have vocabulary overlap with the documents (at least for word features); try TextBlob which uses a lexicon approach rather than a classifier, or try rolling your own with an off-the-shelf SVM and pull labeled training data from one of the many sources (or generate your own using Crowdflower.) Small documents (tweets/titles etc) pose unique challenges, especially when there's irony or sarcasm involved or implicit sentiment through pragmatic knowledge. For example knowing Sarah Palin and how she's regarded automatically gives a person a head start in determining the sentiment of a short document with her name. This kind of pragmatic knowledge is hard for classifiers to learn.
- deleted 12y ago[deleted]
- deleted 12y ago[deleted]