3 ms·It's not an issue of being allowed to crawl.by Grimm1 6y agoIt's not an issue of being allowed to crawl.ziftface 6y agoOh I guess I misunderstood then. Why don't they crawl at Google's scale?Grimm1 6y agoMostly it comes down to their crawler itself and the cost of storing all of that data, it's sufficiently prohibitive.