3 ms·
I've raised this same point on a bunch of threads about scraping in the past, but... Scraping is fine if you ask the company and get permission! This may seem
by danpalmer 7y ago
I've raised this same point on a bunch of threads about scraping in the past, but...
Scraping is fine if you ask the company and get permission!
This may seem obvious, but so many conversations about scraping seem to start from the position that it is in some fundamental way, not allowed. This is not true.
Conversations also seem to start from the assumption that you need to scrape the whole web, which again is not true.
If you're teaching a machine learning course, perhaps you have a project on classifying... cars. Do you need to scrape the whole web to get a bunch of data about cars? No. Could you get away with scraping just Autotrader or a similar site? Maybe! Why not ask them! If you clearly state that it's for learning, that credit will be given, etc, you may find them quite amenable to it.
I work at a company built significantly around web scraping, and we have contracts with all of our scrape targets that confirm we are allowed to scrape them.