4 ms·
I have a design for a system where you can "donate" your disk space to a provider. Basically, you run the client, you say you want to make 1TB available to arch
by stavros 1y ago
I have a design for a system where you can "donate" your disk space to a provider. Basically, you run the client, you say you want to make 1TB available to archive.org, and their server can push the rarest content to your computer.
It's based on torrents, and you can easily make a content delivery system on top of this (so people can fetch data from this network).
I emailed a few archiving teams but nobody seemed interested, so I never made it.
- deleted 1y ago[deleted]
- toomuchtodo 1y agoIt's a hard problem to solve, because its easy to temporarily donate resources to archiving ops via the ArchiveTeam warrior, but a long term commitment to run persistent compute and storage to mirror a chunk of the internet archive. It's why I think Filecoin isn't going to work either; very little overlap between the people who feel its important to keep these archives alive versus people who would run distributed storage to collect financial compensation for doing so. Easier to send fiat to IA for them to invest (~$2/GB) and to pay to keep the disks spinning somewhere safe across the world. (ia volunteer, no affiliation otherwise)
- stavros 1y agoThe system I have in mind is strictly volunteer-run, and it automatically balances the files so that it minimises rare copies. You're right, though, long-term commitment is rare from volunteers. That's why the idea is to make short-term commitment so easy that you have a good enough pool of short-termers that it works out in the aggregate.
- toomuchtodo 1y agoAppreciate your work on this.
- stavros 1y agoEh I didn't really do any work, it's just a design right now, but I think it's a nice one. If any archive team wants to work with me on this, I'd be happy to make it a reality so we have a nice FOSS system for distributed, volunteer-led backups.
- toomuchtodo 1y agoI suggest emailing textfiles, he'll know who to connect you with in ArchiveTeam, and if there is an opportunity to connect with the decentralized web folks at ia. Strongly believe your architecture is superior to filecoin and IPFS due to relying on torrent primitives. (ia source of truth, storage system of last resort -> item index -> torrent index -> global torrent swarm)
- stavros 1y agoThanks, I will!
- stavros 1y agoI emailed him but haven't received a reply. In case you were curious for a bit more detail, here's a short design doc I wrote: https://gist.github.com/skorokithakis/68984ef699437c5129660d12d576e09e https://gist.github.com/skorokithakis/68984ef699437c5129660d...
- 1gn15 1y agoAnna's Archive has this system. This also sounds like Freenet.
- stavros 1y agoFreenet has a bunch of encryption, which is out of scope for this. What does Anna's Archive have, besides torrents?
- 1gn15 1y agoI'm a bit confused. Isn't this such a system where people can volunteer disk space? https://annas-archive.org/torrents https://annas-archive.org/torrents I think I'm misunderstanding you.
- stavros 1y agoMy system is more "I want to donate X GB" and it handles everything, filling that space up, getting the rarest torrents, getting updates, etc. Think of it as a central server managing a globally-distributed, unreliable JBOD in a "push" manner, rather than just downloading a torrent and being done.