I have written crawlers with this functionality (not for Cloudflare, but for sites that implemented the same technique themselves), and it didn't seem like a bi

I have written crawlers with this functionality (not for Cloudflare, but for sites that implemented the same technique themselves), and it didn't seem like a big problem.