4 ms·

1. yes, ipfs could solve that, but it relies on people hoting content. Previous examples of content-based addressing showed that little-accessed content tends t

1. yes, ipfs could solve that, but it relies on people hoting content. Previous examples of content-based addressing showed that little-accessed content tends to disappear as nodes go offline over the years. This would need to be solved, and I think the only way to solve it is to have a battery of centralized ipfs servers mirroring each other, which defeats the "fully distributed" setup 2. this would also need to be hosted and could be taken down. You'd need to mirror this too, but that's a simpler problem to solve (gigabytes instead of terabytes) 3. the upload portals and the RSS feeds would, again, be centralized or would have to change so regularly that they become impractical in the end you would end up with a dozen (a hundred? more?) different z-libraries, which would make it actually worse from a preservation standpoint, since only the most popular content would be shared, libraries that focused on rare/exotic/fringe material would be endangered of being lost since they have fewer volunteers/mirrors/seeds/... Also, freenet and other projects already showed that end-users allocating some storage and using that to spread data around is not an easy problem, the fluctuation in end-nodes is so big that it slows down the entire network to a crawl. I'm not sure this problem has been solved yet.

yamtaddle 4y ago
The core problem with IPFS and friends is that the vast majority of Web access these days occurs on battery-powered devices that need to sleep their connections often and for long periods or battery life will plummet. End users aren't going to accept even 10% worse battery life (and it'd likely be worse than that) just so they can participate in "the swarm" and have higher latency and slower transfer on all their requests.
pradn 4y ago
1. You're absolutely right that IPFS alone isn't a good way to guarantee durability. I think you'd need a second level of archiving done in bulk. Thousands of books collated into various torrents that people can help seed. This has already been done for LibGen. IPFS does provide a common hash space for everyone to rally around, and does make it easy to download single books. It's also easy to speed up downloads by using an IPFS gateway - the actual protocol is slow. I don't expect most users to actually pin files or anything. 2. Well, users and librarians need some way to find each other. That's true in any system. And that communication medium can allow certain kinds of attacks. (website on the public internet, word-of-mouth, Telegram groups) If all someone needs is an IPFS hash of a recent library metadata DB (a SQLite file), any way of communication will suffice. I think this approach allows for centralization (sure, keep a website up as long a the authorities don't care) but also gracefully allows for all manner of de-centralization (use any of the above methods to distribute the metadata DB). 3. Any many-to-one system with curation (librarians) will have weak points. The idea is you can set up upload portals across any communication medium (a regular website, a dark-net site, a Telegram group, email) - and the libraries take care of collating the information. The social grouping is what matters more (libraries vs uploaders vs downloaders) - and we want to make it tech agnostic and, therefore, more resilient. This system will be stable I think, for two reasons: 1. Network and branding effects will naturally create a few big libraries. People will use the familiar, useful libraries. See how few torrent search sites took up the bulk of traffic, back in the heyday of torrents. Most users will probably use a website, and the ones that are easiest to use will probably get the most traffic. 2. The resilience of the system is necessary only once in a while. A set of libraries will emerge, there'll be enforcement actions and they might break apart, and then new ones will pick up their pieces (easily bc the metadata and content is open). So we want to provide the open-ness for this to actually happen.

yamtaddle 4y ago

The core problem with IPFS and friends is that the vast majority of Web access these days occurs on battery-powered devices that need to sleep their connections often and for long periods or battery life will plummet. End users aren't going to accept even 10% worse battery life (and it'd likely be worse than that) just so they can participate in "the swarm" and have higher latency and slower transfer on all their requests.

pradn 4y ago

1. You're absolutely right that IPFS alone isn't a good way to guarantee durability. I think you'd need a second level of archiving done in bulk. Thousands of books collated into various torrents that people can help seed. This has already been done for LibGen. IPFS does provide a common hash space for everyone to rally around, and does make it easy to download single books. It's also easy to speed up downloads by using an IPFS gateway - the actual protocol is slow. I don't expect most users to actually pin files or anything. 2. Well, users and librarians need some way to find each other. That's true in any system. And that communication medium can allow certain kinds of attacks. (website on the public internet, word-of-mouth, Telegram groups) If all someone needs is an IPFS hash of a recent library metadata DB (a SQLite file), any way of communication will suffice. I think this approach allows for centralization (sure, keep a website up as long a the authorities don't care) but also gracefully allows for all manner of de-centralization (use any of the above methods to distribute the metadata DB). 3. Any many-to-one system with curation (librarians) will have weak points. The idea is you can set up upload portals across any communication medium (a regular website, a dark-net site, a Telegram group, email) - and the libraries take care of collating the information. The social grouping is what matters more (libraries vs uploaders vs downloaders) - and we want to make it tech agnostic and, therefore, more resilient. This system will be stable I think, for two reasons: 1. Network and branding effects will naturally create a few big libraries. People will use the familiar, useful libraries. See how few torrent search sites took up the bulk of traffic, back in the heyday of torrents. Most users will probably use a website, and the ones that are easiest to use will probably get the most traffic. 2. The resilience of the system is necessary only once in a while. A set of libraries will emerge, there'll be enforcement actions and they might break apart, and then new ones will pick up their pieces (easily bc the metadata and content is open). So we want to provide the open-ness for this to actually happen.