11 ms·

After many years, Bittorrent doesn't cease to amaze me. Such a fantastic technology. Sad, though, that it is primarily known for illegitimate uses. This give

After many years, Bittorrent doesn't cease to amaze me. Such a fantastic technology. Sad, though, that it is primarily known for illegitimate uses. This gives it a bad reputation and makes it hard to promote Bittorrent for more worthwhile applications. For instance, Bittorrent would be a perfect platform for sharing scientific data but when I suggest that to colleagues, they don't take it seriously because Bittorrent is perceived to be that tool for downloading pirated movies. I wonder how this barrier for adoption in academia could be overcome. Perhaps, with a separate network that is targeted at academic uses (think alt-coins). Call this network ResearchTorrent and researchers could use the technology without ever having to make contact with the whole ecosystem around pirated material. Universities could support this network without risking bad PR and getting in conflict with the music/film industry.

tom_mellior 10y ago
http://academictorrents.com/ http://academictorrents.com/ claims to have 16.33 TB of research data.
- tmalsburg2 10y ago
  Last time I checked (3 years ago) this project didn't seem very active in terms of submission numbers and it appears that this hasn't really changed (7 submissions in Jan 2017). Not sure why, but the UI may be one reason.
  - tom_mellior 10y ago
    I don't think it's fair to expect scientific output to be comparable to Hollywood. Anyway, your colleagues think this is a pirate site?
jzelinskie 10y ago
I work on a BitTorrent tracker on the side[0] that's used for deploying software at a few companies, but has the unique feature of a middleware layer. Right now, there are two different middleware that require a signature of the infohash (files being shared) before allowing content to be shared. One is for JWTs[1] and upstreamed in the project and the other is for a blockchain and hasn't been upstreamed yet. Using features like this, they can run their tracker on the public internet and not worry about people sharing illegal content. I had never considered it, but this sounds like a great use case for academia, too. [0]: https://chihaya.io https://chihaya.io [1]: https://jwt.io https://jwt.io
- tmalsburg2 10y ago
  Very interesting. Does that mean that there would have to be a central authority that has to sign new torrents? For the present purpose that may not be ideal because this system wouldn't be decentralized, and one institution (or a small set of institutions) would have all control. Ideally, there would be some consensus mechanism that determines which torrents are bad and shouldn't be distributed. Also, what happens when the key for singing torrents is stolen? In the case of a company using such a system, that situation can be handled but in a more decentralized system with an emphasis on long-term storage the loss of a key may be a bigger problem.
  - jzelinskie 10y ago
    >Does that mean that there would have to be a central authority that has to sign new torrents? In this scenario, the source of truth is the location where your users are acquiring their .torrent files. >For the present purpose that may not be ideal because this system wouldn't be decentralized, and one institution (or a small set of institutions) would have all control. Ideally, there would be some consensus mechanism that determines which torrents are bad and shouldn't be distributed. Trackers are the centralized alternative to using the Mainline BitTorrent DHT. The performance advantage significant enough they're still in common use after the ubiquity of DHT support in client software. You could build a tracker that serves peer data that it has cached from crawling the DHT, which is an idea I don't think has been done yet. As for a consensus mechanism, that should probably be done out of band and client software could be configured to query it and react appropriately. >Also, what happens when the key for singing torrents is stolen? In the case of a company using such a system, that situation can be handled but in a more decentralized system with an emphasis on long-term storage the loss of a key may be a bigger problem. With the JWT system, you can rotate keys, but it will break clients announcing with the old key. Users would have to manually update all their torrents to participate in the swarm again. This is a shame, but there is no official signing scheme in BitTorrent. If the JWT system as chihaya has implemented gathered traction, it could be updated such that clients could actually fetch JWTs regularly from an auth endpoint and then it'd be possible to handle rotation the same way as most OIDC implementations.
userbinator 10y ago
It is also used for sharing plenty of free, large software too --- many Linux distros are available from torrents, for example.

tom_mellior 10y ago

http://academictorrents.com/ http://academictorrents.com/ claims to have 16.33 TB of research data.

tmalsburg2 10y ago
Last time I checked (3 years ago) this project didn't seem very active in terms of submission numbers and it appears that this hasn't really changed (7 submissions in Jan 2017). Not sure why, but the UI may be one reason.
- tom_mellior 10y ago
  I don't think it's fair to expect scientific output to be comparable to Hollywood. Anyway, your colleagues think this is a pirate site?

jzelinskie 10y ago

I work on a BitTorrent tracker on the side[0] that's used for deploying software at a few companies, but has the unique feature of a middleware layer. Right now, there are two different middleware that require a signature of the infohash (files being shared) before allowing content to be shared. One is for JWTs[1] and upstreamed in the project and the other is for a blockchain and hasn't been upstreamed yet. Using features like this, they can run their tracker on the public internet and not worry about people sharing illegal content. I had never considered it, but this sounds like a great use case for academia, too. [0]: https://chihaya.io https://chihaya.io [1]: https://jwt.io https://jwt.io

tmalsburg2 10y ago
Very interesting. Does that mean that there would have to be a central authority that has to sign new torrents? For the present purpose that may not be ideal because this system wouldn't be decentralized, and one institution (or a small set of institutions) would have all control. Ideally, there would be some consensus mechanism that determines which torrents are bad and shouldn't be distributed. Also, what happens when the key for singing torrents is stolen? In the case of a company using such a system, that situation can be handled but in a more decentralized system with an emphasis on long-term storage the loss of a key may be a bigger problem.
- jzelinskie 10y ago
  >Does that mean that there would have to be a central authority that has to sign new torrents? In this scenario, the source of truth is the location where your users are acquiring their .torrent files. >For the present purpose that may not be ideal because this system wouldn't be decentralized, and one institution (or a small set of institutions) would have all control. Ideally, there would be some consensus mechanism that determines which torrents are bad and shouldn't be distributed. Trackers are the centralized alternative to using the Mainline BitTorrent DHT. The performance advantage significant enough they're still in common use after the ubiquity of DHT support in client software. You could build a tracker that serves peer data that it has cached from crawling the DHT, which is an idea I don't think has been done yet. As for a consensus mechanism, that should probably be done out of band and client software could be configured to query it and react appropriately. >Also, what happens when the key for singing torrents is stolen? In the case of a company using such a system, that situation can be handled but in a more decentralized system with an emphasis on long-term storage the loss of a key may be a bigger problem. With the JWT system, you can rotate keys, but it will break clients announcing with the old key. Users would have to manually update all their torrents to participate in the swarm again. This is a shame, but there is no official signing scheme in BitTorrent. If the JWT system as chihaya has implemented gathered traction, it could be updated such that clients could actually fetch JWTs regularly from an auth endpoint and then it'd be possible to handle rotation the same way as most OIDC implementations.

userbinator 10y ago

It is also used for sharing plenty of free, large software too --- many Linux distros are available from torrents, for example.