The "full solution" to this, of course, is micropayments. A bot which has to pay a tenth of a cent to you every time it visits one of your pages or something el

The "full solution" to this, of course, is micropayments. A bot which has to pay a tenth of a cent to you every time it visits one of your pages or something else the page 404s will quickly rack up a $10 bill crawling a whole 10,000 page site. If it tries to do that every day, or every hour, that's an excellent payday for you and a very compelling reason for almost all bots to blacklist your domain name. A human being who stops by to spend 20 minutes reading your blog once won't even notice they've spent 1.2 cents leafing through. This technology has existed for a while, and yet very few people have found it a good idea to wrap around. There is probably a good reason for that. The realistic solution is to probably just do some statistics and figure out who's getting on your nerves, and then ban them from your digital abode. Annoying, but people go a lot farther to protect their actual homes if they happen to live in high crime areas.