3 ms·
The issue is detecting them when they use random user agents and ip ranges.
by janalsncm 2y ago
The issue is detecting them when they use random user agents and ip ranges.
- bakugo 2y ago> when they use random user agents and ip ranges From what I've seen, most AI scrapers operate on known cloud IP ranges, usually amazon (Perplexity included), so just check for those.
- jsheard 2y agoI assume that's why Reddit appears to be cracking down on VPNs lately, they probably don't actually care about VPNs but they're throttling scraper traffic coming from datacenter IP address ranges, which VPN providers are also using.