3 ms·I agree, but running a scraper does not necessarily mean disobeying robots.txt.by snowwrestler 11y agoI agree, but running a scraper does not necessarily mean disobeying robots.txt.