4 ms·
The point is to make it expensive to crawl your site. Anyone determined to do so is not blocked. But why would they be determined to do so for some random site?
by relistan 1y ago
The point is to make it expensive to crawl your site. Anyone determined to do so is not blocked. But why would they be determined to do so for some random site? The value to the AI crawler likely does not match the cost to crawl it. It will just move on to another site.
So the point is not to be faster than the bear. It’s to be faster than your fellow campers.
- genewitch 1y agoWhy not have them hash pow for btc then?
- sprremix 1y agoWhy must everything involve $'s?
- genewitch 1y agobecause there's a lot of rhetoric about how this "balances the imbalance between serving a request and making that request" and if we're having them do sha256, why not have them do sha256(sha256(data+random nonce)) and potentially earn the site owner some money?