4 ms·

Truly Decentralized BitTorrent Downloading Has Finally Arrived

58 points by eugenejen 16y ago

joe_the_user 16y ago
Sounds like an interesting application. Anyone have any idea what protocol this uses? The only thing I could figure was that it's written in python. I'm sure there are plenty of efforts similar to this. Any others deserving of mention?
- jamii 16y ago
  Most of the underlying protocols are described here, though it is a bit out of date: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.4174&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.... Essentially the core protocol is a gossip overlay. This generates random connections between peers in the overlay. Over these connections peers transmit information like recently seen torrents, recommendation lists, download/upload totals for other peers etc. Each peer proactively distributes known torrent files so that every peer builds up a database of torrents. Then search just works by broadcasting queries to nearby peers. The algorithms used have dozens of applications outside of filesharing. I wrote my MSc dissertation on a similar subject: http://scattered-thoughts.net/one/1283/644001/538941 http://scattered-thoughts.net/one/1283/644001/538941 The code for Tribler is naturally very specialised to their purpose. At some point I want to sit down and reimplement them in such a way that they can easily be reused and recombined for other applications. I have some vague ideas about building a prototype trust-based distributed database for dot-p2p.org based on similar principles.
  - joe_the_user 16y ago
    That's pretty amazing, actually. How hard would it be put this algorithm into a general-purpose library accessible by multiple applications?
    - jamii 16y ago
      I think that if you just want to use the same setup as Tribler you can just directly use their code. What I'm interested in is separating and generalizing the various protocols involved rather than having them all knotted together.
      - joe_the_user 16y ago
        Me too, It seems like a logical part of that would be to have each protocol documented and encapsulated in a library. Also, I'm working on a multi-protocol client which would aggregate a number of sources, so it would be especially nice to the have a library to call so that I could add one more source. In any case, your work looks fabulous and I will spend some time digesting your links.
- jamii 16y ago
  > Any others deserving of mention? OneSwarm is interesting in that it attempts to preserve user privacy. http://oneswarm.cs.washington.edu/ http://oneswarm.cs.washington.edu/
wahnfrieden 16y ago
How does this actually discover peers without any centralization?
- stcredzero 16y ago
  Word of mouth? (This could work well in environs like a university.) Twitter? All one would need is one peer to bootstrap.
- ars 16y ago
  It could be using the http://en.wikipedia.org/wiki/Kademlia http://en.wikipedia.org/wiki/Kademlia distributed hash table. I don't know know if it is or isn't, but it's one way to do it.
- jamii 16y ago
  You can bootstrap as soon as you have one peer. The software is distributed with the addresses of 4 existing servers which are run by the Tribler guys to make this easier but even if they go down people can still send invitations to their friends.
krakensden 16y ago
It looks like it's only aspirationally multiplatform, not Win/Mac/Lin like the article says.
- jamii 16y ago
  I've used it on both Windows 7 and Ubuntu. The Ubuntu version is a little fiddly to get working because Ubuntu ships with a crippled ECC library.
dknight 16y ago
How is it different from DC++ application wise?
- jamii 16y ago
  From the other thread: The core principle is similar. DC has centralized hubs which handle searches and upload files directly to users. Tribler has a distributed search engine which runs on every peer and uses bittorrent to share files. Effectively every single Tribler node behaves like a DC hub. As a result it is much harder to take down.
egorst 16y ago
87Kb (more than 1800 lines) of license agreement. I prefer just to click Cancel during installation.
slang701 16y ago
This is really great and you had better check it out below http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We... http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We... http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...;

58 points by eugenejen 16y ago

Sounds like an interesting application. Anyone have any idea what protocol this uses? The only thing I could figure was that it's written in python. I'm sure there are plenty of efforts similar to this. Any others deserving of mention?

jamii 16y ago
Most of the underlying protocols are described here, though it is a bit out of date: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.4174&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.... Essentially the core protocol is a gossip overlay. This generates random connections between peers in the overlay. Over these connections peers transmit information like recently seen torrents, recommendation lists, download/upload totals for other peers etc. Each peer proactively distributes known torrent files so that every peer builds up a database of torrents. Then search just works by broadcasting queries to nearby peers. The algorithms used have dozens of applications outside of filesharing. I wrote my MSc dissertation on a similar subject: http://scattered-thoughts.net/one/1283/644001/538941 http://scattered-thoughts.net/one/1283/644001/538941 The code for Tribler is naturally very specialised to their purpose. At some point I want to sit down and reimplement them in such a way that they can easily be reused and recombined for other applications. I have some vague ideas about building a prototype trust-based distributed database for dot-p2p.org based on similar principles.
- joe_the_user 16y ago
  That's pretty amazing, actually. How hard would it be put this algorithm into a general-purpose library accessible by multiple applications?
  - jamii 16y ago
    I think that if you just want to use the same setup as Tribler you can just directly use their code. What I'm interested in is separating and generalizing the various protocols involved rather than having them all knotted together.
    - joe_the_user 16y ago
      Me too, It seems like a logical part of that would be to have each protocol documented and encapsulated in a library. Also, I'm working on a multi-protocol client which would aggregate a number of sources, so it would be especially nice to the have a library to call so that I could add one more source. In any case, your work looks fabulous and I will spend some time digesting your links.
jamii 16y ago
> Any others deserving of mention? OneSwarm is interesting in that it attempts to preserve user privacy. http://oneswarm.cs.washington.edu/ http://oneswarm.cs.washington.edu/

wahnfrieden 16y ago

How does this actually discover peers without any centralization?

stcredzero 16y ago
Word of mouth? (This could work well in environs like a university.) Twitter? All one would need is one peer to bootstrap.
ars 16y ago
It could be using the http://en.wikipedia.org/wiki/Kademlia http://en.wikipedia.org/wiki/Kademlia distributed hash table. I don't know know if it is or isn't, but it's one way to do it.
jamii 16y ago
You can bootstrap as soon as you have one peer. The software is distributed with the addresses of 4 existing servers which are run by the Tribler guys to make this easier but even if they go down people can still send invitations to their friends.

krakensden 16y ago

It looks like it's only aspirationally multiplatform, not Win/Mac/Lin like the article says.

jamii 16y ago
I've used it on both Windows 7 and Ubuntu. The Ubuntu version is a little fiddly to get working because Ubuntu ships with a crippled ECC library.

dknight 16y ago

How is it different from DC++ application wise?

jamii 16y ago
From the other thread: The core principle is similar. DC has centralized hubs which handle searches and upload files directly to users. Tribler has a distributed search engine which runs on every peer and uses bittorrent to share files. Effectively every single Tribler node behaves like a DC hub. As a result it is much harder to take down.

egorst 16y ago

87Kb (more than 1800 lines) of license agreement. I prefer just to click Cancel during installation.

slang701 16y ago

This is really great and you had better check it out below http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We... http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We... http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...; http://www.xilisoft.com/ipad-magic.html?utm_source=John%26Wei&utm_medium=Text%2Blink%2B&utm_campaign=iPad%2BMagic%2B<br/&#62 http://www.xilisoft.com/ipad-magic.html?utm_source=John%26We...;