3 ms·

Quay has supported Docker image pulls over BitTorrent for a couple years now https://coreos.com/blog/torrent-pulls https://coreos.com/blog/torrent-pulls. Docke

Quay has supported Docker image pulls over BitTorrent for a couple years now https://coreos.com/blog/torrent-pulls https://coreos.com/blog/torrent-pulls. Docker themselves have discussed making the official registry extensible enough to support BitTorrent pulls, but I don't know if anything ever happened there. Facebook has been using BitTorrent for deploys for something like 9 years now. They configured the tracker to prefer sharing peers with longer matching subnet prefixes, to keep bandwidth off the backbone as much as possible.

fizwhiz 8y ago
Is there a paper or talk you could link to that describes their system?
- jzelinskie 8y ago
  Justin linked the blog post[0] which is probably the best written description. The short of it is that when you upload layers to Quay, it stream calculates the BitTorrent pieces. Private layers are given unique swarms isolated by namespace and peer discovery is protected by a tracker[1] that contains middleware validating JWTs passed in the announce URL of the torrent. A custom client[2] can be used to simplify downloading and importing of images into the local docker CAS. Honestly, most organizations don't have sophisticated enough networks that the benefits outweigh the complexity of p2p orchestration. This is why it's popular at Alibaba, Facebook, Twitter, but most people are still just using the OCI distribution protocol[3]. Feel free to contact me (Keybase is in my profile) if interested. I'd love to get more people on the path to p2p, but it's often a solution looking for a problem. [0]: https://coreos.com/blog/torrent-pulls https://coreos.com/blog/torrent-pulls [1]: http://chihaya.io http://chihaya.io [2]: https://github.com/coreos/quayctl https://github.com/coreos/quayctl [3]: https://github.com/opencontainers/distribution-spec https://github.com/opencontainers/distribution-spec

fizwhiz 8y ago

Is there a paper or talk you could link to that describes their system?

jzelinskie 8y ago
Justin linked the blog post[0] which is probably the best written description. The short of it is that when you upload layers to Quay, it stream calculates the BitTorrent pieces. Private layers are given unique swarms isolated by namespace and peer discovery is protected by a tracker[1] that contains middleware validating JWTs passed in the announce URL of the torrent. A custom client[2] can be used to simplify downloading and importing of images into the local docker CAS. Honestly, most organizations don't have sophisticated enough networks that the benefits outweigh the complexity of p2p orchestration. This is why it's popular at Alibaba, Facebook, Twitter, but most people are still just using the OCI distribution protocol[3]. Feel free to contact me (Keybase is in my profile) if interested. I'd love to get more people on the path to p2p, but it's often a solution looking for a problem. [0]: https://coreos.com/blog/torrent-pulls https://coreos.com/blog/torrent-pulls [1]: http://chihaya.io http://chihaya.io [2]: https://github.com/coreos/quayctl https://github.com/coreos/quayctl [3]: https://github.com/opencontainers/distribution-spec https://github.com/opencontainers/distribution-spec