3 ms·

I actually find that fairly tame. For a point of comparison, Wikipedia gets ~$150M in revenue a year, an "asset rise" (I presume this is what non-profits call

I actually find that fairly tame. For a point of comparison, Wikipedia gets ~$150M in revenue a year, an "asset rise" (I presume this is what non-profits call profit?) of ~$15M a year, and is sitting on about a quarter billion in the bank. Not that they want to, but I think Wikipedia could fund this using their current donations if they wanted. Hell, I almost wonder if one of the big storage providers would do it for free if they could do it in their staging environment so they get real traffic. It would be less good than real backups, but extra copies are still extra copies even if they're unreliable.

Aachen 2y ago
You're right, I guess it is tame and achievable so far as organisations go. I was imagining trying to get some friends together to have a decent percentage of the IA backed up, but that seems out of reach based on this napkin math. Not that that is necessarily demotivating, but it's going to depend on a lot of people intuitively seeing the value and keeping up their share
- everforward 2y ago
  Yeah, as a sort of pet project I don’t think backing up the whole thing is possible. You might be able to back up a significant portion of the unique data in IA if you limited it to text files. I think they probably have the highest information to file size ratio. It’s also probably the most likely to already be back up, though. Interesting issue; you might also get somewhere by cutting the 50TB up into 10GB torrents (or 100GB or whatever, something reasonable for a consumer hard drive) and maybe adding a script that checks the torrent swarm stats to recommend a torrent to download. Something where I run it, tell it I want to let it use 600GB, and it hands me torrent files for the least seeded 600GB. Maybe a super basic web UI so people can see how well backed up it is? Unsure if people would sign on or not; I probably would. I’ve got 10 or so TB of NFS I’m not using I could chuck at it. I would guess there are other data hoarders out there who would do the same, but only if it were somewhat easy. I’m probably not going to volunteer to do an hour of rtorrent cleanup a week to make sure I’m backing up the right things.
  - rolandog 2y ago
    I think a part of the scope of this project may have already been solved by The FreeNet Project (now HyphaNet) [0]. [0]: https://www.hyphanet.org/ https://www.hyphanet.org/
Intralexical 2y ago
They should probably consider it, really. A good portion of the text on Wikipedia relies on Wayback Machine links to remain verifiable. If they lose that, I guess the editors might have to comb every page for information which would need to be either resourced or deleted.

Aachen 2y ago

You're right, I guess it is tame and achievable so far as organisations go. I was imagining trying to get some friends together to have a decent percentage of the IA backed up, but that seems out of reach based on this napkin math. Not that that is necessarily demotivating, but it's going to depend on a lot of people intuitively seeing the value and keeping up their share

everforward 2y ago
Yeah, as a sort of pet project I don’t think backing up the whole thing is possible. You might be able to back up a significant portion of the unique data in IA if you limited it to text files. I think they probably have the highest information to file size ratio. It’s also probably the most likely to already be back up, though. Interesting issue; you might also get somewhere by cutting the 50TB up into 10GB torrents (or 100GB or whatever, something reasonable for a consumer hard drive) and maybe adding a script that checks the torrent swarm stats to recommend a torrent to download. Something where I run it, tell it I want to let it use 600GB, and it hands me torrent files for the least seeded 600GB. Maybe a super basic web UI so people can see how well backed up it is? Unsure if people would sign on or not; I probably would. I’ve got 10 or so TB of NFS I’m not using I could chuck at it. I would guess there are other data hoarders out there who would do the same, but only if it were somewhat easy. I’m probably not going to volunteer to do an hour of rtorrent cleanup a week to make sure I’m backing up the right things.
- rolandog 2y ago
  I think a part of the scope of this project may have already been solved by The FreeNet Project (now HyphaNet) [0]. [0]: https://www.hyphanet.org/ https://www.hyphanet.org/

Intralexical 2y ago

They should probably consider it, really. A good portion of the text on Wikipedia relies on Wayback Machine links to remain verifiable. If they lose that, I guess the editors might have to comb every page for information which would need to be either resourced or deleted.