I am not the forum guy and my content is not really static. It's cached, but for this specific sub page I need at least one small query. Only a few requests are

I am not the forum guy and my content is not really static. It's cached, but for this specific sub page I need at least one small query. Only a few requests are rebuilding the page, but if every 10th is rebuilt and 9 other bots are hammering mostly cached pages I still have the same issue. The bot was never ment to query the same pages thousands of times. These pages were identical to him. There were already bot specific rules programmed in. This website already is heavily cached and optimized. Even thought it only has 16 database connections there maybe is one timeout every few months. Users usually don't open tabs much faster than the short requests take, only bing does. Really the time that went into optimizing it makes me kinda sad when someone questions that is was a effort Vs payoff thing. The bot ignored all rules I gave him and barely brought any benefit for me in terms of traffic. There is no payoff here, only effort.