4 ms·
Have there been efforts to set up something like the SMTP block lists for web scraping bots? Seems like this is something that needs to be crowd sourced. It'll
by flowerthoughts 2y ago
Have there been efforts to set up something like the SMTP block lists for web scraping bots? Seems like this is something that needs to be crowd sourced. It'll be much easier to find patterns in larger piles of data. A single real user is unlikely to do as much as quickly as a bot.
- johnea 2y agoThose are largely ineffective in the modern world. Most bot nets, like he mentions in the blag post, come from 1 IP address only 1 time. With thousands to tens of thousands of IP addresses in the bot net, there is just no way to block by IP address anymore.