Where I live, you can't really legally refuse serving an individual without a strong reason. And if you did, you'd better be able to prove that you didn't do it

Where I live, you can't really legally refuse serving an individual without a strong reason. And if you did, you'd better be able to prove that you didn't do it for discriminatory reasons. Here, bartenders can say you've had too many, they are supposed to stop selling you alcohol if they think it could be dangerous for you or other people. This is one of the cases where not only the business can refuse to serve you, it also has to. Now, I'm not a lawyer but I wouldn't expect robots.txt to be legally enforceable. Of course, in any case, the website can totally refuse to fulfill a request, rate limit, etc. I'd say this is not comparable to the bartender or a business refusing to serve you though. Responding to an HTTP request is not offering a commercial service. And if it is, whatever legal document that promises you the service can totally specify that they won't promise to answer your robot.