4 ms·
> Can you elaborate? I haven't run into any roadblocks yet but I'm not scraping big sites or sending a massive number of requests. Cloudflare Bot Protection[1]
by shubhamjain 4y ago
> Can you elaborate? I haven't run into any roadblocks yet but I'm not scraping big sites or sending a massive number of requests.
Cloudflare Bot Protection[1] is a popular one. The website is guarded by a layer of code that needs to be executed before continuing. Normal browsers will follow through. It can be hard to bypass.
[1]: https://www.cloudflare.com/pg-lp/bot-mitigation-fight-mode/ https://www.cloudflare.com/pg-lp/bot-mitigation-fight-mode/
- GekkePrutser 4y agoSo that would break text browsers too, right? :( And users with JS disabled for privacy reasons.
- jamescampbell 4y agoI have a codebase that defeats cloudflare protection. Felt like I had keys to kingdom.