3 ms·
Off the top of my head: - Cloudflare - Webserver-level user-agent blocking (Apache, nginx) - Application-level user-agent blocking (`if request.user_agent ==
by rocky_raccoon 3y ago
Off the top of my head:
- Cloudflare
- Webserver-level user-agent blocking (Apache, nginx)
- Application-level user-agent blocking (`if request.user_agent == 'OpenAI'`)
None of them are ideal since you can simply change your user agent, but all of them seem like better options than robots.txt to me.