3 ms·
I think your last quote is incorrect, at least partially. I'm not sure they replace the entire target page: > This pre-generated content is seamlessly integrat
by pests 2y ago
I think your last quote is incorrect, at least partially. I'm not sure they replace the entire target page:
> This pre-generated content is seamlessly integrated as hidden links on existing pages via our custom HTML transformation process, without disrupting the original structure or content of the page.
> We also ensured that these links remain invisible to human visitors through carefully implemented attributes and styling. To further minimize the impact to regular visitors, we ensured that these links are presented only to suspected AI scrapers, while allowing legitimate users and verified crawlers to browse normally.
- metadat 2y agoHow does a crawler become verified and blessed by Cloudflare?
- pests 2y agoI’m not involved with Cloudflare but IMO they probably calculate some score based on: honesty (not trying to hide user agent, etc), respecting robots.txt, proper crawl rate, etc.
- hoherd 2y agoGood catch, I missed that part completely. I'm heading into a long airport layover so I should have time to reread the full article.