2 ms·
Allow? Deep down, do you think you have a choice? Mechanisms might exist to make you think you have one, the same way copywrite should prevent millions of book
by swarnie 6mo ago
Allow? Deep down, do you think you have a choice?
Mechanisms might exist to make you think you have one, the same way copywrite should prevent millions of books being gobbled up by TheZuck but ultimately do you really have a choice?
Rules and laws don't exists for you.
- alt227 6mo agoYes, Google advertises its crawler IP ranges and it is quite easy to keep track of this and block them. But only if you control the infrastructure that your site runs on of course.
- rolph 6mo agostego your site, google sees the red herring version, intended users see the payload. this has been done before, quite often, but toward ends morally askew.
- account42 6mo agoYou think they'll stick to those IPs any more than robots.txt when it comes to that?
- alt227 6mo agoYes I do, I also think they will continue identifying themselves in the user-agent header. https://developers.google.com/crawling/docs/crawlers-fetchers/overview-google-crawlers https://developers.google.com/crawling/docs/crawlers-fetcher...