4 ms·
Pretty sure crawling and scraping is legal even if there's a robots.txt.
by simple_phrases 6y ago
Pretty sure crawling and scraping is legal even if there's a robots.txt.
- toomuchtodo 6y agoAs long as it’s public. If you need to bypass auth or similar, that’ll get you in trouble. https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-linkedin-protects-scraping-public-data https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-l...