4 ms·
How we built Tagger News: machine learning on a tight schedule
- minimaxir 9y agoLink to original HN submission: https://news.ycombinator.com/item?id=14337275 https://news.ycombinator.com/item?id=14337275 It's worth noting for future reference that in terms of supervised learning of labels given a text document input, fasttext (https://github.com/facebookresearch/fastText https://github.com/facebookresearch/fastText) is leagues ahead of conventional approaches in both accuracy and training speed, and there is a Python interface (https://github.com/salestock/fastText.py https://github.com/salestock/fastText.py) for use with Django/Flask (unfortunately, recent fasttext changes have broken the interface for now).
- snissn 9y agoCan you suggest any unsupervised learning? I want to take a body of text associated with users and come up with keywords/topics with each user. Thanks! :)
- capybara 9y agoSome fairly widely-used techniques include LSI, LDA, and word2vec or doc2vec. There a lot of different techniques out there! I'm one of the creators of Tagger News, and we used LDA with python's Gensim package. Here's a good tutorial: https://radimrehurek.com/gensim/tut2.html https://radimrehurek.com/gensim/tut2.html
- minimaxir 9y agoNote that fasttext is the next generation of word2vec/doc2vec, and shares many of the same creators.
- projectorlochsa 9y agoHow does fasttext compare to vowpal wabbit?
- skystrife 9y agoBasically the same [1,2]. [1]: https://twitter.com/yoavgo/status/751178795323908096 https://twitter.com/yoavgo/status/751178795323908096 [2]: https://nlpers.blogspot.com/2016/08/fast-easy-baseline-text-categorization.html https://nlpers.blogspot.com/2016/08/fast-easy-baseline-text-...
- gerenuk 9y agoVowpal wabbit is approx. 4-8x faster than gensim but the accuracy will be less compared to gensim.
- _eht 9y agoI noticed some things were pretty terribly tagged earlier this week. Robots have a ways to go, this is good news.