ID10TError

3 ms·

1. 1.▲
   by ID10TError 6mo ago

   Good question — the short answer is that robots.txt only understands Allow/Disallow. It's binary: a bot can go here, or it can't. robots2.txt adds a permission layer that robots.txt was never designed for. For example, you mi
2. 2.▲
   Show HN: Robots2.txt – A robots.txt extension for AI agents and Age ratings(robots2.org)

   1 points by ID10TError 6mo ago|2 comments