The article avoids directly addressing this, but I think is the real elephant in the room -- whether you have the right to web scrape for research. The article

The article avoids directly addressing this, but I think is the real elephant in the room -- whether you have the right to web scrape for research. The article says, "You might be able to use what you scrape, but it’s worth checking that you can also legally share it. Ideally, the website content licence [sic] will be readily available." In practice, the data you want to scrape either has 1) no license/information about whether they are okay with you scraping it (and if they are okay with it, they usually offer the source data anyways), or more commonly they 2) strictly prohibit it in the terms of service, in which case it's not clear whether a scraper which mimics a browser falls under fair use. It's an area where when you ask your university attorney, they exchange a few emails with you and then avoid making any decision. I think it's just because it's not well tested in court (at least when I encountered these situations), and depends on whether you'd be subject to a takedown notice or lawsuit.