3 ms·
Sounds like pretty sophisticated filtering! I’m working on a scraping project at the moment so looking at this too but from the other end. Super low volume tho
by Havoc 2y ago
Sounds like pretty sophisticated filtering!
I’m working on a scraping project at the moment so looking at this too but from the other end. Super low volume though so pretty tame - emphasis on success rate more than throughput
I bought a 4G dongle for use as last resort if nothing else gets through. And also investigating ipv6
- avastel 2y agoUsing a 4G dongle makes it easier to hide in the crowd indeed. Since your traffic will go through heavily shared mobile IPs, probably with thousands of users behind them, anti-bot vendors won't/shouldn't block per IP, but per fingerprint/session cookie instead.
- Havoc 2y agoAh hadn’t realised it’s the NAT. I thought it’s because the IPs are dynamic and rotate too much. Interesting. Currently planning on doing a layered approach. Cloud IPs first etc. Interesting challenge but also trying to be somewhat respectful about it since nobody likes aggressive bots