> how can we make scraping less of a burden on individual hosts? Isn't this basically what content-addressable storage is for? Have the site provide the conten

> how can we make scraping less of a burden on individual hosts? Isn't this basically what content-addressable storage is for? Have the site provide the content hashes rather than the content and then put the content on IPFS/BitTorrent/whatever where the bots can get it from each other instead of bothering the site. Extra points if you can get popular browsers to implement support for this, since it also makes it a lot harder to censor things and a decent implementation (i.e. one that prefers closer sources/caches) would give most of the internet the efficiency benefits of a CDN without the centralization.