3 ms·
Essentially a digital museum?
by boredumb 4y ago
Essentially a digital museum?
- toomuchtodo 4y agoA distributed, highly resilient digital museum. The internet archive meets S3 meets BitTorrent. We’re most of the way there, just need improvements around durability, rebalancing around geographies, and the identity and metadata (item inventory) planes. (no association with IA, just an active contributor)
- zachh 4y agoDo you know if the IA is working on such projects / is there a way to get involved? Or is it mostly third parties doing so?
- toomuchtodo 4y agohttp://blog.archive.org/tag/distributed-web/ http://blog.archive.org/tag/distributed-web/ My personal opinion is that they’re going about this the wrong way, with IPFS, anything blockchain, etc. They can support this entire model with a DHT (supporting PKI for identity) and torrent advertising, the latest version of BitTorrent (which has great support for cross swarm file serving by hash), and RSS or other feeds for advertising the corpus or chunks of it. I maintain a copy in Cloudflare R2 of all IA item metadata, and would love to work on the ArchiveTeam equivalent of a Warrior appliance: appliance or container starts, consumes an RSS feed to request the least seeded items, downloads them up to disk space allowed, and then serve downloaded item files to the swarm.
- zachh 4y agoAgreed! Super helpful, thanks.
- boredumb 4y agoI could imagine something like IPFS and a strong community of volunteers hosting nodes with different partitions. I'm not sure how IPFS would scale to IA sized storage, if for nothing else there would have to be a good way to segment and balance data across volunteers nodes.