4 ms·

Oh I guess I misunderstood then. Why don't they crawl at Google's scale?

• Grimm1 6y ago
  Mostly it comes down to their crawler itself and the cost of storing all of that data, it's sufficiently prohibitive.