When I was at a modest sized public facing site, roughly 3/4 of requests were from bots. Most painful is over half were search result pages, which were much mo

When I was at a modest sized public facing site, roughly 3/4 of requests were from bots. Most painful is over half were search result pages, which were much more costly.