> Not for services loading specific URLs due to (sometimes indirect) user requests. A crawler has a list of resources it periodically checks to see if it chang

> Not for services loading specific URLs due to (sometimes indirect) user requests. A crawler has a list of resources it periodically checks to see if it changed, and if it did, indexes it for user requests. Contrary to this totally-not-a-crawler, with its own database of existing resources, that periodically checks if anything changed, and if it did, caches content and builds chescksums.