2 ms·
I'm happy to page scrape other sites and not happy to have mine scraped. ;-) More seriously, if there're bots that you don't want scraping, just robots.txt the
by nostrademons 17y ago
I'm happy to page scrape other sites and not happy to have mine scraped. ;-)
More seriously, if there're bots that you don't want scraping, just robots.txt them away. If they ignore that, then they're being rather rude and you can figure out some way to auto-block them.