Googlebot supports their set of robots.txt instructions https://developers.google.com/search/docs/advanced/robots/create-robots-txt https://developers.google.co

Googlebot supports their set of robots.txt instructions https://developers.google.com/search/docs/advanced/robots/create-robots-txt https://developers.google.com/search/docs/advanced/robots/cr... Crawl Delay is not part of it.