The Content Extractor is overly simple. You don't want all the Anchor tags for most things, you want the ones that aren't part of the page template. It took a

The Content Extractor is overly simple. You don't want all the Anchor tags for most things, you want the ones that aren't part of the page template. It took a lot of man hours to build the content extractor we use for our search engine. https://www.mashape.com/stremor/stremor-content-extractor https://www.mashape.com/stremor/stremor-content-extractor Also because not every site has a SiteMap, or well linked site structure you may have to turn to Social like FB and Twitter if you want to get everything.