https://meta.stackexchange.com/questions/404687/our-robots-txt-file-is-changing-to-allow-only-cloudflares-verified-bots https://meta.stackexchange.com/questions

https://meta.stackexchange.com/questions/404687/our-robots-txt-file-is-changing-to-allow-only-cloudflares-verified-bots https://meta.stackexchange.com/questions/404687/our-robots-t... >After this change, any trusted crawler will continue to see the old copy of robots.txt. I think this still works for crawlers like Google