3 ms·
Funny, I just built a HN article catcher that uses Diffbot to collect and classify submissions from the /new page [1]. I've been a Diffbot fan for quite a while
by mayank 13y ago
Funny, I just built a HN article catcher that uses Diffbot to collect and classify submissions from the /new page [1]. I've been a Diffbot fan for quite a while now (although their entity recognition/tag classifier needs a bit of work as you can see from the categorization on my catcher page below).
[1] http://lahiri.me/more http://lahiri.me/more
I should add that their API is fantastic, and far better than using BeautifulSoup/NLTK for extracting textual content from webpages.
- tswaterman 13y agoCool! How many articles, or what time period, did you use for this? It looks like you're using only a subset of the topic tags -- did you make a list of 'interesting stuff' to filter against?
- mayank 13y agoIt's been running for about a week I think, and I'm just taking the top 80 or so tags by article count. Glad you like it :)