Right. I don't care if AI (or anything else) indexes or learns from my sites. That's what they're there for. But yesterday I blocked an IP that hit one of my si

Right. I don't care if AI (or anything else) indexes or learns from my sites. That's what they're there for. But yesterday I blocked an IP that hit one of my sites 82000 times in an hour, or 22/second. And apparently it's a very stupid bot, because it kept redownloading CSS and other asset files every time it saw a link to them. There's no way the people behind that bot are going to follow any suggestions to make it behave better. After all, adding things like caching and rate-limiting to your web crawler might take a few hours, and who's got time for that.