4 ms·
Use Cloudflare (free) and just disable bots or add a captcha. You'll probably also save money if you enable and configure free caching.
by aetherspawn 2y ago
Use Cloudflare (free) and just disable bots or add a captcha.
You'll probably also save money if you enable and configure free caching.
- scoofy 2y agoWill this still allow Google and Bing and Archive bots?
- aetherspawn 2y agoI quickly checked for you, and my Cloudflare account currently has the following option: "Block AI Bots" under xxx.com > Security > Bots which I think does what you want. I am not paying for Cloudflare and it allows me to enable this option. If you use the (other) Bot Fight Mode or captcha options, then yes, you will block crawlers. AFAIK specific bot user agents can also be blocked (or allowed) using a Page Rule, which is also a free tier feature.