I see hundreds of thousands of requests from search engines daily without any burden, but I do block all other bots. It has nothing to do with paranoia, a lot o

I see hundreds of thousands of requests from search engines daily without any burden, but I do block all other bots. It has nothing to do with paranoia, a lot of bots are just evil making it impossible to act towards bots in good faith. For example there are forensic bots, copyright protection bots, brand protections bots and similar bots with the whole purpose of using crawled data to send you threats and "abuse" reports or drag you to court. There are bots that scan for vulnerabilities to exploit and bots that scan pages to find something bad to blacklist somewhere. There are bots that generate spammy links or even grab content for black hat seo, something search engines punish you for. There are bots that belong to companies that sell your data or try to gain competitive advantage through this without giving you anything in return, something you may or may not be comfortable with. There are even bots that gather data for political and intelligence purposes, literally causing death and destruction.