24 ms·
Am I the only one that worries about licensing and legal issues when it comes to web scraping? I'd be terrified to build a product around it since one law suit
by remyp 9y ago
Am I the only one that worries about licensing and legal issues when it comes to web scraping? I'd be terrified to build a product around it since one law suit would threaten the core business.
- gyvastis 9y agoEverything that you can open in the browser you can scrape without any problem. Though keep in mind the number of requests you send to those parties should be thought about as it shouldn't vary greatly compared to a regular user. A user doesn't open 1000 pages in 60 seconds.
- anywherenotes 9y agodon't most sites claim they own data? Like could you legally scrape reddit and make your own site?
- zapperdapper 9y agoMe too. Just look at the legal battle between LinkedIn and hiQ. I have scraped, but not on the level where it would draw attention. Use the APIs if you can - Trip Advisor do offer an API, as do Bing (for search results). Unless you are scraping at huge scale I actually think the bigger problem is lack of semantics - people like diffbot[1] using AI/ML to try and solve this issue. [1] - https://www.diffbot.com https://www.diffbot.com