4 ms·
Yes, Google advertises its crawler IP ranges and it is quite easy to keep track of this and block them. But only if you control the infrastructure that your sit
by alt227 5mo ago
Yes, Google advertises its crawler IP ranges and it is quite easy to keep track of this and block them. But only if you control the infrastructure that your site runs on of course.
- rolph 5mo agostego your site, google sees the red herring version, intended users see the payload. this has been done before, quite often, but toward ends morally askew.
- account42 5mo agoYou think they'll stick to those IPs any more than robots.txt when it comes to that?
- alt227 5mo agoYes I do, I also think they will continue identifying themselves in the user-agent header. https://developers.google.com/crawling/docs/crawlers-fetchers/overview-google-crawlers https://developers.google.com/crawling/docs/crawlers-fetcher...