3 ms·

For all the hate Google gets, it's crawlers are like 10x smarter than any other search engine. Wouldn't surprise me at all if there was some type of crawling is

• aunch 4y ago
  The strange thing is that other search engines like Baidu and Yandex get this right...
• colmmacc 4y ago
  Operating 8.8.8.8 likely gives Google a near-complete list of every active domain on the internet, and new domains show up there the second any 8.8.8.8 user queries it - which is likely well before anyone links to it. I have no idea if Google use this data for hints to their crawler and indexer, but if I worked there I sure would.
  • nkozyra 4y ago
    It would seem very odd to not use this tool which effectively gives a huge competitive advantage over a lot of other engines, although for obvious reasons I doubt they'd shout it from the rooftops.
  • krackers 4y ago
    This actually makes perfect sense! I've always wondered why they operate this service. There's nothing in the privacy policy that suggests they won't use the aggregated anonymized information for other purposes.
  • arp242 4y ago
    I doubt they actually discover domains that way; for a search engine just knowing a domain exists isn't all that useful. Besides, loads of spam crawlers can find new domains plenty fast enough: create a brand new domain with a blank page and a brand new email address, link it once somewhere, and see how long it takes to get spam because some spam crawler bot found it.