To be honest, I'm not certain of the purpose of the spam. Some portion of it would end up in the search results, sure. But I don't know if there's some second

To be honest, I'm not certain of the purpose of the spam. Some portion of it would end up in the search results, sure. But I don't know if there's some secondary benefit to, say, a casino showing a link coming from my site even if my site has a robots.txt saying that the address for that link isn't to be directly indexed. Is there such a benefit? If not then I'll just set up the robots.txt and observe whether that does indeed solve the problem. But I'd much prefer to just set up the permissions I know I want on my own running instance than spend time making inferences about the reasons bots are abusing my instance's inputs.