2 ms·
Sec.gov is blocking all requests from AWS?
I noticed sec.gov is blocking all requests from AWS since the 4 days ago. Has this been the same for you? How are you able to bypass this issue? Currently I can scrape sec.gov only from local computer. My ideal solution is that I can have a program that runs on some remote servers
- arthurcolle 5y agoYou could use a proxy run on another provider, Linode, DigitalOcean, etc. Or you could scrape sec.report What are you using SEC sites for, just EDGAR or something else?
- talolard 5y agoAdd a user agent to the request See here: https://www.sec.gov/os/accessing-edgar-data https://www.sec.gov/os/accessing-edgar-data