robots.txt will just be ignored by bad actors. And the user-agent sent is completely in control of the bad actor. They can send their user-agent as "google bot

robots.txt will just be ignored by bad actors. And the user-agent sent is completely in control of the bad actor. They can send their user-agent as "google bot". You would need something like WAF from https://www.cloudflare.com/ https://www.cloudflare.com/ or AWS