Yeah sure, but most of those caching layers (including possibly on the ISP level) aren't really configured for the DNS flood a crawling operation may result in.

Yeah sure, but most of those caching layers (including possibly on the ISP level) aren't really configured for the DNS flood a crawling operation may result in. If you're going to do way more DNS lookups than are expected from your connection, it's a good custom to provide your own caching layer that's scaled accordingly. Not doing so probably won't break anything, but it risks degrading the DNS performance of other people using the same resolvers.