4 ms·
> Googlebot has a range of IP addresses that it publicly announces so websites can whitelist them. Google says[1] they do not do this: "Google doesn't post a
by quitethelogic 6y ago
> Googlebot has a range of IP addresses that it publicly announces so websites can whitelist them.
Google says[1] they do not do this:
"Google doesn't post a public list of IP addresses for website owners to allowlist."
[1]https://developers.google.com/search/docs/advanced/crawling/verifying-googlebot https://developers.google.com/search/docs/advanced/crawling/...
- johncolanduoni 6y agoFrom that same page they recommend using a reverse DNS lookup (and then a forward DNS lookup on the returned domain) to validate that it is google bot. So the effect is the same for anyone trying to impersonate googlebot (unless they can attack the DNS resolution of the site they’re scraping I guess).