3 ms·
Do you guys have any concerns about websites' terms of use restrictions? I've done a bit of crawling and have always been curious how selling this sort of serv
by gjreda 13y ago
Do you guys have any concerns about websites' terms of use restrictions?
I've done a bit of crawling and have always been curious how selling this sort of service would handle those potential use restrictions / legal issues.
- tixocloud 13y agoYes, wouldn't you be a legal target for many companies once they have discovered that you are scraping their information? I've done crawling myself and wonder if I can actually resell the data without being afoul of the law.
- harvestmoon 13y agoI've been building a product search tool. I wanted to scrap data to power it. It'd be similar to the caching and scrapping Google does. But I spoke to a lawyer and she strongly advised against it (she said if the Terms of Service of a site say it isn't allowed, you have to follow that). Dunno. Confuses me how something is supposedly illegal yet many companies do all the time without a concern.
- vijayr 13y agoThese people seem to be making a decent business out of selling scraped data - http://www.aggdata.com/ http://www.aggdata.com/ There might be others too.
- drcube 13y agoDon't put information on the public internet if you don't want the public to read it. You can keep your private information nice and safe behind a firewall, or on a private network not attached to the internet. Easy peasy.
- grey-area 13y agoReading is very different from republishing.
- frabcus 13y agoThe answer is complicated, because the law isn't shaken out yet. Doubt it will be for centuries. I've written "Is scraping legal?" here, with details, and my own view morally. https://blog.scraperwiki.com/2012/04/is-scraping-legal/ https://blog.scraperwiki.com/2012/04/is-scraping-legal/
- rgovind 13y agoI believe you have to look at robots.txt file. If it lets you crawl, you should be fine.
- qq66 13y agoDoes the robots.txt have any legal significance? I would think that the Terms of Use can be more thorough and are more legally relevant.