3 ms·
> Some search engines provide a list of their scraper IP ranges Common Crawl's CCBot has published IP ranges. We aren't a search engine (although there are sea
by ccgreg 10mo ago
> Some search engines provide a list of their scraper IP ranges
Common Crawl's CCBot has published IP ranges. We aren't a search engine (although there are search engines using our data) and we like to describe our crawler as a crawler, not a "scraper".