4 ms·

I am mega curious how does Apple circumnavigate problem of getting flagged for crawling website. Doesn't CDN cloudfare eventually flag them? If proxies, what i

• ricardo81 6y ago
  Likely whitelisted as a 'good bot' in Cloudflare. If a bot isn't well known, doesn't respect robots.txt or crawls too fast then much more likely to get flagged, but applebot are none of those.