4 ms·
That's not from robots.txt, but their Bot Management feature which blocks things calling themselves Googlebot that don't come from known Google IPs.
by jamesog 2y ago
That's not from robots.txt, but their Bot Management feature which blocks things calling themselves Googlebot that don't come from known Google IPs.
- speedgoose 2y agoAre GCP IPs considered Google IPs?
- crop_rotation 2y agoNo I am very sure they are not.
- jgrahamc 2y agoNo.
- judge2020 2y agoFor reference https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot https://developers.google.com/search/docs/crawling-indexing/...