Boilerpipe is great. Also see the diffbot article API which is quite good: http://www.diffbot.com/docs/api/article http://www.diffbot.com/docs/api/article Ther

Boilerpipe is great. Also see the diffbot article API which is quite good: http://www.diffbot.com/docs/api/article http://www.diffbot.com/docs/api/article There is some deeper discussion of text extraction tools on Tomaz Kovacic's blog: http://tomazkovacic.com/blog/56/list-of-resources-article-text-extraction-from-html-documents/ http://tomazkovacic.com/blog/56/list-of-resources-article-te... One advantage of Goose, (the linked project), is that it tries to also extract the best image from the webpage. None of the competitors do this. I am not sure how good Goose's text extraction is, compared to boilerpipe or diffbot. A new tool just popped on my radar called justext: http://code.google.com/p/justext/ http://code.google.com/p/justext/ It was written by the BiWeC web-as-corpus people, who crawl the web and expose it through the SketchEngine for NLP research.