I'm working on a crawler and CloudFlare is the cause of 99% of all the headaches and random bugs I encounter doing simple HTTP requests. I literally have imple

I'm working on a crawler and CloudFlare is the cause of 99% of all the headaches and random bugs I encounter doing simple HTTP requests. I literally have implemented custom logic to deal with sites returning the "Server: Cloudflare" header.