3 ms·
I'm neither a web dev nor a site owner, but OP literally asked for tips on ethical web scraping, not "what's the most I can get away with".
by lordgrenville 7y ago
I'm neither a web dev nor a site owner, but OP literally asked for tips on ethical web scraping, not "what's the most I can get away with".
- trickstra 7y agoThat's because OP is operating under the wrong assumption that sites won't ban an ethical scraper. The reality is that they will, and much faster than an unethical one. They don't care about your science project, they want ad revenue, conversions...