You're right that requiring a small fee to register can solve the problem for small communities (and so OP is wrong in the paragraphs that you quoted), but stil

You're right that requiring a small fee to register can solve the problem for small communities (and so OP is wrong in the paragraphs that you quoted), but still this solution doesn't work for OP's case as this requires login and he says that he doesn't want to know who is using his search engine (I guess for privacy reasons) and in this case indeed the bots can harm the proliferation of this kind of small scale services.