3 ms·
That's surprising because Googlebot publishes IP ranges for its crawlers and it's fairly simple to block fake crawlers these days (super easy through Cloudflare
by beejiu 1y ago
That's surprising because Googlebot publishes IP ranges for its crawlers and it's fairly simple to block fake crawlers these days (super easy through Cloudflare, for example).
- OldfieldFund 1y agothis shouldn't work due to reverse DNS checks
- Raed667 1y agodoesn't google also run some "undercover" bots to verify that you don't serve very different versions of your website to users vs bots ?
- ewoodrich 1y agoIn my experience 12ft.io was pretty much useless after a honeymoon period of a few months when it first came out so I wouldn't be surprised. The Googlebot method used to work with almost everything but at some point major news orgs caught on in quick succession and I gave up even bothering to try it.