I don't do much crawling or scraping either, but when I have, I go to the opposite extreme. There's no reason why I need the data right that second, so I set it

I don't do much crawling or scraping either, but when I have, I go to the opposite extreme. There's no reason why I need the data right that second, so I set it up to pause for some random amount of seconds between requests. *Except YouTube, I'll yt-dl an entire playlist, no probs*