3 ms·
Looks interesting. There is another good project that does this called Boilerpipe: http://code.google.com/p/boilerpipe/ http://code.google.com/p/boilerpipe/ I
by yesimahuman 16y ago
Looks interesting. There is another good project that does this called Boilerpipe: http://code.google.com/p/boilerpipe/ http://code.google.com/p/boilerpipe/
I recently developed a web service that performs these sorts of operations called Linguini (http://linguini.me http://linguini.me). The service hasn't been officially launched but it's in beta and usable. It can extract html/blog text through JSON web service calls and can tag names/companies/locations in that text.
- bravura 16y agoBoilerpipe is great. Also see the diffbot article API which is quite good: http://www.diffbot.com/docs/api/article http://www.diffbot.com/docs/api/article There is some deeper discussion of text extraction tools on Tomaz Kovacic's blog: http://tomazkovacic.com/blog/56/list-of-resources-article-text-extraction-from-html-documents/ http://tomazkovacic.com/blog/56/list-of-resources-article-te... One advantage of Goose, (the linked project), is that it tries to also extract the best image from the webpage. None of the competitors do this. I am not sure how good Goose's text extraction is, compared to boilerpipe or diffbot. A new tool just popped on my radar called justext: http://code.google.com/p/justext/ http://code.google.com/p/justext/ It was written by the BiWeC web-as-corpus people, who crawl the web and expose it through the SketchEngine for NLP research.