Haven’t seen anyone mention this, but asking permission first is about the most ethical approach. If you think sites are unlikely to give you permission, that m

Haven’t seen anyone mention this, but asking permission first is about the most ethical approach. If you think sites are unlikely to give you permission, that might be an indication that what you’re doing has limited value. Offering to share your results with them could be a good plan. I work for a company that does a lot of web scraping, but we have a business contract with every company we scrape from.