2 ms·
Scraping tools have become more powerful than ever, but bot restrictions have become equally more strict. It's hard to scrape reliably under any circumstance, o
by xnx 2y ago
Scraping tools have become more powerful than ever, but bot restrictions have become equally more strict. It's hard to scrape reliably under any circumstance, or even consistently without residential proxies.
- sakisv 2y agoWhen I first started it there was a couple of instances that my IP was blocked - despite being a residential IP behind CGNAT. I then started randomising every aspect of the scraping process that I could: The order that I visited the links, the sleep duration between almost every action, etc. As long as they don't implement a strict fingerprinting technique, that seems to be enough for now