Thank you for those insights, it's a topic I'm interested in. Agree with what you're saying about naive bots hitting websites/hosts/subnets too hard, in the con

Thank you for those insights, it's a topic I'm interested in. Agree with what you're saying about naive bots hitting websites/hosts/subnets too hard, in the context of site owners being hit by multiple bots for multiple reasons and them questioning the return they'll get. I'd be interested to know more info wrt DNS lookups. Did you apply a blanket rate limit on the number of DNS requests you'd make to any particular server? From past experience I know the .uk Nominet servers would temp-ban if you were doing more than a few hundred lookups per second. At the next host level down, was there a blanket limit or was it dependent on the number of domains that nameserver was responsible for?