3 ms·
IP bans are equivalent to residential door locks. They’re only deterring the most trivial attacks. In school I needed to scrape a few hundred thousand pages of
by KMnO4 5y ago
IP bans are equivalent to residential door locks. They’re only deterring the most trivial attacks.
In school I needed to scrape a few hundred thousand pages of a proteomics database website. For some reason you had to view each entry one at a time. There was IP throttling which banned you if you made requests too quickly. But slowing the script to 1 request per second would have taken days to scrape the site. So I paid <$5 for a list of 500 proxy servers and distributed it, completing the task in under half an hour.
- l33t2328 5y agoUsing proxies to hide your identity to get around a denial of access seems to get awfully close to violating the Computer Fraud and Abuse Act(in USA, at least). I’m surprised your school was okay with it.
- KMnO4 5y agoDon't worry, I don't live in the USA. Thanks for your concern though.
- borski 5y agoI agree it’s not perfect. It’s also significantly better than nothing.
- Gigachad 5y agoIt's also completely insufficient despite being better. There are so many scraper services that its just a matter of paying a small amount of money to make use of tens of thousands of IPs spread across ISPs and countries.
- inshadows 5y agoCan you share where did you get such a nice proxy for 500 proxies? TIA