For Google it's pretty straight forward to throw away fake crawlers by just only allowing their published list of crawler IPs so you don't accidentally allow so

For Google it's pretty straight forward to throw away fake crawlers by just only allowing their published list of crawler IPs so you don't accidentally allow someone from a random GCP IP to crawl you if unwanted (https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests https://developers.google.com/crawling/docs/crawlers-fetcher...).