2 ms·
this won't stop the overall trend, but it can help you get around cloudflare's effective scraping blocking (copying my comment from a previous thread): If you'
by bjord 4y ago
this won't stop the overall trend, but it can help you get around cloudflare's effective scraping blocking (copying my comment from a previous thread):
If you're scraping with Python, try cloudscraper—among other things(!), it supports JS rendering (basically the bare-minimum check cloudflare does), without needing to run a full browser in the background. It's built on requests, so integration was pretty easy.
https://github.com/venomous/cloudscraper https://github.com/venomous/cloudscraper