3 ms·
User Agent block just means they'd spoof their user agent.
by Dwedit 2y ago
User Agent block just means they'd spoof their user agent.
- marginalia_nu 2y agoThat generally gives you even more trouble with cloudflare. Behaving in any way inconsistent with your UA string is one of the easiest methods of identifying bots. Yeah you can use headless browsers, but then you're also using orders of magnitude more compute, and that's not really something that scales. The best way to avoid ending up in captcha-land is to say who you are, and respect robots.txt.