4 ms·

Can we create decentralized library (catalog) protocol, so it can't be taken down? I'm thinking about libgen/sci-hub/whatcd(RIP), but entirely decentralized, pr

Can we create decentralized library (catalog) protocol, so it can't be taken down? I'm thinking about libgen/sci-hub/whatcd(RIP), but entirely decentralized, protocol-based. We already have basically all necessary tools: BitTorrent/IPFS to distribute files, magnet links to link them, DHT to discover peers. We just need to combine them together in a smart way and maybe add some bits of missing functionality, like: - categorization and search; - voting for every entry; - maybe reputation system. Where peers regularly publishing good, high quality content get their 'karma'[1] upvoted. So when downloading you can choose people with higher reputation. Or when building catalog of materials in certain area of interest, you can filter only peers with certain threshold karma. - incentive for peers to keep seeding whatever they downloaded at least until achieving certain ratio (say, 2:1). Maybe by rewarding with 'torrent tokens' that you can spend on downloading, commenting etc.[2] - comments to every entry (published torrent). With means to combat spam, insults, irrelevant stuff etc. E.g. with voting system, where comments with, say, -3[3] votes become collapsed. Or/and with aforementioned 'tokens' that you spend when comment. - personal blacklists to block people whose torrents/comments you don't want to see. Maybe I'm reinventing the wheel here and something similar already exists, but for some reason not popular (since I never heard of it)? In that case we need to figure out why it didn't shoot and fix it to make it work. [1] Your ID will be a cryptographic key a la cryptocurrency wallet. [2] With rare but sought after torrents rewarding you with more tokens. [3] Personally adjustable.

lifeformed 5y ago
I am just a layman with all this, but could this be an actually useful application of blockchain? Create a DAO where reputation is earned through votes, and tokens are distributed to people who upload new good content that people want (a bounty system??), and you can spend them on downloads. I have no idea how any of this works so maybe it's a terrible idea.
tribler 5y ago
Yes, this is exactly what we are working on for 16 years now. Longest running torrent, trust and Tor-fork project: https://github.com/Tribler/tribler/wiki https://github.com/Tribler/tribler/wiki Its as hard as making a decentralised Google and decentralised YouTube at the same time. Over 75 master students and PhDs put their coding efforts in it at Delft University.
- mdprock 5y ago
  Looks great, congrats! sci-hub and libgen can be found on tribler?
  - tribler 5y ago
    Thnx! Sadly it's not that simple. What IPFS, Dat protocol, Tribler, and all others are missing is "adverserial decentralised information retrieval". For any keyword you type in, the match you want should show first. Trolling, Kremlin bots and copyright police forces should not be able to bring it down. Unsolved problem. How to create a privacy respecting relevance ranking or distributed clicklog.
kleer001 5y ago
I honestly know nothing about the space. Is this what you're talking about? At least as an MVP. : https://libgen.fun/dweb.html https://libgen.fun/dweb.html ?
peatmoss 5y ago
The file sharing aspect of this seems to be the least difficult portion of the problem. I’d love to see something that passes muster, to a librarian, on the catalog front. Sort of a wikipedia or open street map of library catalogs. The resources themselves might be found anywhere (physically at a library, in digital form for purchase, downloadable via torrent, etc.). A true library of babel. If I knew more about library science, I suspect catalog federation protocols all exist, but there is probably work to be done in making such a thing resilient.
- zozbot234 5y ago
  > Sort of a wikipedia or open street map of library catalogs. Wikidata has this in their scope (and it makes sense to keep the data there, since a book catalog needs to cross-reference entries for authors, topics etc.), and the Internet Archive OpenLibrary leverages their data.
CuriouslyC 5y ago
I think a lot of the social stuff is unnecessary and getting ahead of things. The core product IMO is just a resource database, with a nice resource description interface, peer syncing, and the ability to search and aggregate results from multiple databases together. A resource could be a web page blob, or a file, or plain text (e.g. a comment). All the social stuff, file viewers, blacklists, etc can be built on top of that, ideally by third parties in an open ecosystem.
lyoshenka 5y ago
This is exactly what LBRY is: https://lbry.com https://lbry.com

lifeformed 5y ago

I am just a layman with all this, but could this be an actually useful application of blockchain? Create a DAO where reputation is earned through votes, and tokens are distributed to people who upload new good content that people want (a bounty system??), and you can spend them on downloads. I have no idea how any of this works so maybe it's a terrible idea.

tribler 5y ago

Yes, this is exactly what we are working on for 16 years now. Longest running torrent, trust and Tor-fork project: https://github.com/Tribler/tribler/wiki https://github.com/Tribler/tribler/wiki Its as hard as making a decentralised Google and decentralised YouTube at the same time. Over 75 master students and PhDs put their coding efforts in it at Delft University.

mdprock 5y ago
Looks great, congrats! sci-hub and libgen can be found on tribler?
- tribler 5y ago
  Thnx! Sadly it's not that simple. What IPFS, Dat protocol, Tribler, and all others are missing is "adverserial decentralised information retrieval". For any keyword you type in, the match you want should show first. Trolling, Kremlin bots and copyright police forces should not be able to bring it down. Unsolved problem. How to create a privacy respecting relevance ranking or distributed clicklog.

kleer001 5y ago

I honestly know nothing about the space. Is this what you're talking about? At least as an MVP. : https://libgen.fun/dweb.html https://libgen.fun/dweb.html ?

peatmoss 5y ago

The file sharing aspect of this seems to be the least difficult portion of the problem. I’d love to see something that passes muster, to a librarian, on the catalog front. Sort of a wikipedia or open street map of library catalogs. The resources themselves might be found anywhere (physically at a library, in digital form for purchase, downloadable via torrent, etc.). A true library of babel. If I knew more about library science, I suspect catalog federation protocols all exist, but there is probably work to be done in making such a thing resilient.

zozbot234 5y ago
> Sort of a wikipedia or open street map of library catalogs. Wikidata has this in their scope (and it makes sense to keep the data there, since a book catalog needs to cross-reference entries for authors, topics etc.), and the Internet Archive OpenLibrary leverages their data.

CuriouslyC 5y ago

I think a lot of the social stuff is unnecessary and getting ahead of things. The core product IMO is just a resource database, with a nice resource description interface, peer syncing, and the ability to search and aggregate results from multiple databases together. A resource could be a web page blob, or a file, or plain text (e.g. a comment). All the social stuff, file viewers, blacklists, etc can be built on top of that, ideally by third parties in an open ecosystem.

lyoshenka 5y ago

This is exactly what LBRY is: https://lbry.com https://lbry.com