3 ms·
Check to see if the site that you're considering scraping has any sort of "terms and conditions", "terms of use", "legals", etc page. If it does, read that page
by nickh 18y ago
Check to see if the site that you're considering scraping has any sort of "terms and conditions", "terms of use", "legals", etc page. If it does, read that page in detail. If it doesn't, ask the site for permission to scrape.
If the site doesn't have one of those pages and you don't ask for permission, you're not only putting yourself in danger of legal action, but you're also depending on a data source that isn't reliable.
Remember, it'd only be a matter of time before they notice you scraping, and take measures to stop it.