Domain specific but if you detect a bot you can start giving it false information. For example, a dictionary site. Someone tries to crawl your site after trig

Domain specific but if you detect a bot you can start giving it false information. For example, a dictionary site. Someone tries to crawl your site after triggering your "This is a bot" code, serve bad data to every 20 requests. Mispell a word, Mislabel a noun as a verb, give an incorrect definition. If you combine this with throttling then the value of scraping your site is greatly reduced. Also, most people won't come up with a super advanced crawler if they never get a "Permission denied, please stop crawling" message.