> but at the same time, CloudFlare makes it basically impossible for any new search engine other than Google to spider the site I hadn't heard about this, can

> but at the same time, CloudFlare makes it basically impossible for any new search engine other than Google to spider the site I hadn't heard about this, can anyone supply a link for more context? Is there anything a Cloudflare customer can do to "opt in" to being scraped by other search engine bots?