3 ms·
Mind sharing a decent robots.txt and/or user-agent list to block the AI crawlers?
by baobun 2y ago
Mind sharing a decent robots.txt and/or user-agent list to block the AI crawlers?
- hooloovoo_zoo 2y agoAny of the big chat models should be able to reproduce it :)
- jazzyjackson 2y agoLinked upthread: https://github.com/ai-robots-txt/ai.robots.txt/blob/main/robots.txt https://github.com/ai-robots-txt/ai.robots.txt/blob/main/rob...