3 ms·
>Unless you do it from a Google Cloud instance, that is. What's the reverse DNS for Google Cloud IPs? Google says to check that Googlebot's IP resolves to eith
by mootothemax 10y ago
>Unless you do it from a Google Cloud instance, that is.
What's the reverse DNS for Google Cloud IPs? Google says to check that Googlebot's IP resolves to either a .google.com or .googlebot.com domain.
https://support.google.com/webmasters/answer/80553?hl=en https://support.google.com/webmasters/answer/80553?hl=en
- hrrsn 10y ago.googleusercontent.com
- lend000 10y agoGood point -- although it makes sense why it isn't frequently implemented. DNS lookups aren't cheap for this kind of thing.
- Symbiote 10y agoIt would be sufficient to let some requests come through from "Googlebot", and then deal with them (block, rate-limit, whatever) once the DNS check has been completed.
- milankragujevic 10y agoCouldn't you use GWT Mobilizer to scrape a site then index that? Like this: http://i.imgur.com/ocR54Yq.jpg http://i.imgur.com/ocR54Yq.jpg