>Have you found that you aren't able to find accessible APIs to request against? I'm scraping from variety of different websites (1000+) that my org doesn't ow

>Have you found that you aren't able to find accessible APIs to request against? I'm scraping from variety of different websites (1000+) that my org doesn't own. Reconfiguring to hit APIs would be complex, and a maintenance problem, both of which I easily avoid by using selenium to drive an actual browser, at the expense of time. >Have you ever tried to contact the administrators to see if there's an API you could access? Just not feasible given the scope and breadth of the scraping. >Are you scraping data that would be against ToS if you tried to get it in a way that would benefit both you and the target web site? I inspect and respect the robots.txt