5 ms·

Actually torrents are pretty bad for storing "rare" files. Maybe someone should offer a fix.

by petra 11y ago

rakoo 11y ago
The problem is not really torrents. The problem is that seeds are ephemeral. But an equally ephemeral source for any protocol will always make the availability of the resource low. The solution is not to change the protocol; the solution is to have less ephemeral sources.
- blockchainer 11y ago
  Could this be a problem that a blockchain could help solve?
- TeamMCS 11y ago
  It would be interesting to see how a Usenet/Torrent adapter may work to at least supplement loss
- cpeterso 11y ago
  The Internet Archive might be a good host for rare (and not copyright infringing) torrents.
  - rakoo 11y ago
    It actually is, because it already provide torrent access to its content. See, for example, the Matrix ASCII (only the sample though): https://archive.org/details/TheMatrix-InAscii https://archive.org/details/TheMatrix-InAscii (1 more point for how terrific the Internet Archive is) EDIT: it turns out the complete Matrix ASCII is hosted on the IA, making its availability a non-problem for years to come, even though it's not the original torrent anymore
jimrandomh 11y ago
I think this is mostly a consequence of how the clients work. When I download something with BitTorrent, it goes into a Downloads folder. Then I want to move and rename the files to fit into an organized collection, but then the client won't be able to find it anymore. If my client were able to track files across moves and renames, I'd be much more willing to seed things permanently. (It would also need to have good upstream-bandwidth management, use a number of TCP connections that's less than O(n) in the number of torrents, and offer a few bulk operations like "stop seeding torrents with more than N seeds" and "resume seeding torrents with fewer than N seeds".)
- LukeShu 11y ago
  I've "solved" this by having torrents go to a "Torrents" folder, then symlinking those files into my organized collection.
  - doomrobo 11y ago
    I do the same thing but into multiple folders with a unionfs in read-only mode so I don't accidentally modify the files and cause them to be re-downloaded
  - icebraining 11y ago
    I hardlink them instead. Then I can prune them from the torrents folder without affecting the organized file.
    - LukeShu 11y ago
      Even if I stop seeding, I like to keep them in the torrents folder because it gives me essentially free integrity checking.
- harryh 11y ago
  You can deal with the renaming issue with a bunch of symlinks. It's pretty easy to script that up.
- petra 11y ago
  To make it work i think the client should have some hard drive space allocated for the purpose of "preservation" , which he can automatically use according to some reasonable policy(maybe only files i downloaded can be put in the folder) and it should allocate some share of my bandwidth for that purpose. Than the client automatically manages everything with coordination from the network. That's the only way this can scale for regular users, i think. And of course it may use social mechanisms like feedback "you're the reason why this rare work of art if available for everyone". As for scalability of TCP connections , you're probably right, not my area of expertise , but once this will be declared as an important BT problem with great practical value, i'm sure some researchers would be happy to contribute and might solve this.
deleted 11y ago
[deleted]
liotier 11y ago
In the good old days of using (Napster|Audiogalaxy|Gnutella) I could point the file-sharing application to my collection and presto - it was all shared and available to whoever was interested. Nowadays, I just share whatever torrent I have participated in lately, plus a few that seem popular but rare (those most uploaded from my client) - but I have a whole lot of rather rare music that I fail to share for lack of a way to do it. Is there a contemporary way to share a whole collection or is that really missing from the toolset ? Well - maybe I'll finally get around to setting up a heeadless gtk-gnutella node again.
- j3097736 11y ago
  eMule and derivated have the .emulecollection format, which is just a binary file grouping together multiple ed2k links to files. As for the network eMule/KAD seems do be doing ok with 200K+ peers while Gnutella nowadays seems to be populated mostly by slightly-compatible chinese clients, which bootstrap from traditional gnutella but are incompatible.
- SSLy 11y ago
  Soulseek is still up and kicking.
  - liotier 11y ago
    I thought about it and found it a bit too social and too insular - having "rooms" as a central feature feels odd to me who would rather use a global search of the whole file-sharing network. Also, having to run a desktop client was putting me off. But then I just found http://www.museek-plus.org http://www.museek-plus.org so I might try that... But it is disappearing from Debian: https://packages.qa.debian.org/m/museek+.html https://packages.qa.debian.org/m/museek+.html - not a good sign...
    - SSLy 11y ago
      But there is a global search of the whole network, chat rooms are separate thing. Well, it does not match wcd at all anyway.

Actually torrents are pretty bad for storing "rare" files. Maybe someone should offer a fix.

by petra 11y ago

Actually torrents are pretty bad for storing "rare" files. Maybe someone should offer a fix.

rakoo 11y ago

The problem is not really torrents. The problem is that seeds are ephemeral. But an equally ephemeral source for any protocol will always make the availability of the resource low. The solution is not to change the protocol; the solution is to have less ephemeral sources.

blockchainer 11y ago
Could this be a problem that a blockchain could help solve?
TeamMCS 11y ago
It would be interesting to see how a Usenet/Torrent adapter may work to at least supplement loss
cpeterso 11y ago
The Internet Archive might be a good host for rare (and not copyright infringing) torrents.
- rakoo 11y ago
  It actually is, because it already provide torrent access to its content. See, for example, the Matrix ASCII (only the sample though): https://archive.org/details/TheMatrix-InAscii https://archive.org/details/TheMatrix-InAscii (1 more point for how terrific the Internet Archive is) EDIT: it turns out the complete Matrix ASCII is hosted on the IA, making its availability a non-problem for years to come, even though it's not the original torrent anymore

jimrandomh 11y ago

I think this is mostly a consequence of how the clients work. When I download something with BitTorrent, it goes into a Downloads folder. Then I want to move and rename the files to fit into an organized collection, but then the client won't be able to find it anymore. If my client were able to track files across moves and renames, I'd be much more willing to seed things permanently. (It would also need to have good upstream-bandwidth management, use a number of TCP connections that's less than O(n) in the number of torrents, and offer a few bulk operations like "stop seeding torrents with more than N seeds" and "resume seeding torrents with fewer than N seeds".)

LukeShu 11y ago
I've "solved" this by having torrents go to a "Torrents" folder, then symlinking those files into my organized collection.
- doomrobo 11y ago
  I do the same thing but into multiple folders with a unionfs in read-only mode so I don't accidentally modify the files and cause them to be re-downloaded
- icebraining 11y ago
  I hardlink them instead. Then I can prune them from the torrents folder without affecting the organized file.
  - LukeShu 11y ago
    Even if I stop seeding, I like to keep them in the torrents folder because it gives me essentially free integrity checking.
harryh 11y ago
You can deal with the renaming issue with a bunch of symlinks. It's pretty easy to script that up.
petra 11y ago
To make it work i think the client should have some hard drive space allocated for the purpose of "preservation" , which he can automatically use according to some reasonable policy(maybe only files i downloaded can be put in the folder) and it should allocate some share of my bandwidth for that purpose. Than the client automatically manages everything with coordination from the network. That's the only way this can scale for regular users, i think. And of course it may use social mechanisms like feedback "you're the reason why this rare work of art if available for everyone". As for scalability of TCP connections , you're probably right, not my area of expertise , but once this will be declared as an important BT problem with great practical value, i'm sure some researchers would be happy to contribute and might solve this.

deleted 11y ago

[deleted]

liotier 11y ago

In the good old days of using (Napster|Audiogalaxy|Gnutella) I could point the file-sharing application to my collection and presto - it was all shared and available to whoever was interested. Nowadays, I just share whatever torrent I have participated in lately, plus a few that seem popular but rare (those most uploaded from my client) - but I have a whole lot of rather rare music that I fail to share for lack of a way to do it. Is there a contemporary way to share a whole collection or is that really missing from the toolset ? Well - maybe I'll finally get around to setting up a heeadless gtk-gnutella node again.

j3097736 11y ago
eMule and derivated have the .emulecollection format, which is just a binary file grouping together multiple ed2k links to files. As for the network eMule/KAD seems do be doing ok with 200K+ peers while Gnutella nowadays seems to be populated mostly by slightly-compatible chinese clients, which bootstrap from traditional gnutella but are incompatible.
SSLy 11y ago
Soulseek is still up and kicking.
- liotier 11y ago
  I thought about it and found it a bit too social and too insular - having "rooms" as a central feature feels odd to me who would rather use a global search of the whole file-sharing network. Also, having to run a desktop client was putting me off. But then I just found http://www.museek-plus.org http://www.museek-plus.org so I might try that... But it is disappearing from Debian: https://packages.qa.debian.org/m/museek+.html https://packages.qa.debian.org/m/museek+.html - not a good sign...
  - SSLy 11y ago
    But there is a global search of the whole network, chat rooms are separate thing. Well, it does not match wcd at all anyway.