> That's what robots.txt does It most certainly does not. robots.txt is almost totally worthless against genAI crawlers. Even being unindexed from search engin

> That's what robots.txt does It most certainly does not. robots.txt is almost totally worthless against genAI crawlers. Even being unindexed from search engines doesn't keep you safe.