> The way I see it, AI is not a robot and doesn't need to look at robots.txt I don't think you are seeing it very clearly then. Your secretary can also be a ro

> The way I see it, AI is not a robot and doesn't need to look at robots.txt I don't think you are seeing it very clearly then. Your secretary can also be a robot. What do you think an AI is if not a robot?? It doesn't "need" to look at robots.txt because nothing does.