I’m not involved with Cloudflare but IMO they probably calculate some score based on: honesty (not trying to hide user agent, etc), respecting robots.txt, prope

I’m not involved with Cloudflare but IMO they probably calculate some score based on: honesty (not trying to hide user agent, etc), respecting robots.txt, proper crawl rate, etc.