3 ms·

Dragonfly: an intelligent P2P based file distribution system

43 points by jinqueeny 8y ago

dvasdekis 8y ago
Anyone have any views on if this is materially different to Dat for the same challenge? https://github.com/datproject/dat https://github.com/datproject/dat
- boramalper 8y ago
  As I understand, (1) Dragonfly’a aim for P2P is for technical reasons (i.e. better network utilisation, distributing the load, etc.) whereas Dat’s aim is decentralisation; (2) Dragonfly thus assumes that peers (e.g. servers in your datacenter) trust each other whereas Dat cannot make such assumptions.
jzelinskie 8y ago
The context is that Dragonfly has recently been proposed to the CNCF Sandbox[0]. Checkout the HN previous thread[1]. I've worked on P2P distribution systems and found that not many people are able to reuse these systems because of assumptions about the network that are not consistent across all environments. Eager to hear if anyone besides Alibaba is using Dragonfly as a counterpoint. [0]: https://github.com/cncf/toc/issues/130 https://github.com/cncf/toc/issues/130 [1]: https://news.ycombinator.com/item?id=17303055 https://news.ycombinator.com/item?id=17303055
- sdenton4 8y ago
  FWIW, Google's Borg infra also uses p2p package distribution: "In addition, Borg distributes packages to machines in parallel using tree and torrent-like protocols." https://pdos.csail.mit.edu/6.824/papers/borg.pdf https://pdos.csail.mit.edu/6.824/papers/borg.pdf
sali0 8y ago
Forgive this noob, but I was wondering what use cases infrastructure like this is used for. I have only read a little about containers and have seen plenty of projects working with them, but then I see that this project is also related. Can anyone point me to where I can read about the use cases of all of these and how they fit in real world applications? Thank you!
- voxadam 8y ago
  VM and container image distribution for clusters.
earenndil 8y ago
Previous discussion (269 points, 58 comments) https://news.ycombinator.com/item?id=17303055 https://news.ycombinator.com/item?id=17303055
koalalorenzo 8y ago
Does anybody know the differences with IPFS? https://ipfs.io/ https://ipfs.io/
- boramalper 8y ago
  See my reply comparing Dragonfly with Dat: https://news.ycombinator.com/item?id=17915016 https://news.ycombinator.com/item?id=17915016

43 points by jinqueeny 8y ago

Anyone have any views on if this is materially different to Dat for the same challenge? https://github.com/datproject/dat https://github.com/datproject/dat

boramalper 8y ago
As I understand, (1) Dragonfly’a aim for P2P is for technical reasons (i.e. better network utilisation, distributing the load, etc.) whereas Dat’s aim is decentralisation; (2) Dragonfly thus assumes that peers (e.g. servers in your datacenter) trust each other whereas Dat cannot make such assumptions.

jzelinskie 8y ago

The context is that Dragonfly has recently been proposed to the CNCF Sandbox[0]. Checkout the HN previous thread[1]. I've worked on P2P distribution systems and found that not many people are able to reuse these systems because of assumptions about the network that are not consistent across all environments. Eager to hear if anyone besides Alibaba is using Dragonfly as a counterpoint. [0]: https://github.com/cncf/toc/issues/130 https://github.com/cncf/toc/issues/130 [1]: https://news.ycombinator.com/item?id=17303055 https://news.ycombinator.com/item?id=17303055

sdenton4 8y ago
FWIW, Google's Borg infra also uses p2p package distribution: "In addition, Borg distributes packages to machines in parallel using tree and torrent-like protocols." https://pdos.csail.mit.edu/6.824/papers/borg.pdf https://pdos.csail.mit.edu/6.824/papers/borg.pdf

sali0 8y ago

Forgive this noob, but I was wondering what use cases infrastructure like this is used for. I have only read a little about containers and have seen plenty of projects working with them, but then I see that this project is also related. Can anyone point me to where I can read about the use cases of all of these and how they fit in real world applications? Thank you!

voxadam 8y ago
VM and container image distribution for clusters.

earenndil 8y ago

Previous discussion (269 points, 58 comments) https://news.ycombinator.com/item?id=17303055 https://news.ycombinator.com/item?id=17303055

koalalorenzo 8y ago

Does anybody know the differences with IPFS? https://ipfs.io/ https://ipfs.io/

boramalper 8y ago
See my reply comparing Dragonfly with Dat: https://news.ycombinator.com/item?id=17915016 https://news.ycombinator.com/item?id=17915016