2 ms·
Does Cloudflare assume there is only one "legitimate search engine crawler"?
- jgrahamc 2y agoNo. There are tons of crawlers out there and we certainly don't think there's one. We maintain a list of verified bots here: https://radar.cloudflare.com/traffic/verified-bots https://radar.cloudflare.com/traffic/verified-bots and if you want your bot added you can do so via the "Add a bot" button.