2 ms·
The vast majority of Googlebot user agents are lying. Real Googlebot is pretty well behaved in my experience. You should use reverse DNS or IP lists to check: h
by bobbiechen 4mo ago
The vast majority of Googlebot user agents are lying. Real Googlebot is pretty well behaved in my experience. You should use reverse DNS or IP lists to check: https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests https://developers.google.com/crawling/docs/crawlers-fetcher...
- simondotau 4mo agoIf you're using Cloudflare, set up a security rule to block requests that have "Googlebot" in the UA and are not recognised by CF as a real bot.