Right, but that's not Google's fault, that's explicitly everyone who does a shit job of crawling's fault. Generally I wouldn't qualify a lot of those as search

Right, but that's not Google's fault, that's explicitly everyone who does a shit job of crawling's fault. Generally I wouldn't qualify a lot of those as search engine web crawlers but more web scrapers looking to re-use data, not just surface it.