4 ms·

Kraken – P2P Docker Registry

84 points by tehlike 6y ago

robertlagrant 6y ago
Not to be confused with GitKraken (https://www.gitkraken.com https://www.gitkraken.com) or KrakenD (https://www.krakend.io https://www.krakend.io)
- BenoitP 6y ago
  Or Kraken the cryptocurrency exchange (https://www.kraken.com/ https://www.kraken.com/)
  - js4ever 6y ago
    Or kraken.io, the image optimizer. This cool name seems to be quite popular
    - justinzollars 6y ago
      Or kraken the rum (https://www.krakenrum.com/ https://www.krakenrum.com/)
      - brian_herman__ 6y ago
        Or the original Kraken from H.P. Lovecraft https://lovecraft.fandom.com/wiki/Cthylla https://lovecraft.fandom.com/wiki/Cthylla
- Timpy 6y ago
  I definitely assumed this was git related before clicking
toyg 6y ago
This is so cool. Will I ever use it? Probably not, I’m a hobbyist at the moment. Do I think everything should be P2P, particularly if infrastructure-critical like a registry? Hell yes.
- majewsky 6y ago
  Most organizations are not Google or Facebook or Uber, so setting up a P2P network of registries is ghoulish overkill. Any image registry with a HA setup (e.g. API pods in Kubernetes and backing storage in Ceph or a NetApp cluster or whatever) is probably enough. Source: I've implemented a federating image registry for use at $dayjob (https://github.com/sapcc/keppel https://github.com/sapcc/keppel). Before doing so, I evaluated the landscape and also looked at Kraken briefly.
  - jacques_chester 6y ago
    Thanks for the keppel link. Impressive work.
  - LockAndLol 6y ago
    Since you have experience writing your own registry, do you think it would be difficult to support IPFS (or a similar distributed storage technology) for pulling images? I'm thinking HTTP for metadata and IPFS for storage could reduce the load on registries significantly, but I know very little about registries.
    - majewsky 6y ago
      Conceptually, no big deal. Most registries already deliver blob contents from a CDN or object store (via HTTP redirect). However, the registry protocol (i.e. the Open Containers Distribution API) only knows HTTP, so you'd have to use a HTTP-to-IPFS proxy to use the IPFS backing store with existing clients.
makeworld 6y ago
> Kraken's goal is to reduce global max download time and communication overhead in a stable environment, while BitTorrent was designed for an unpredictable and adversarial environment, so it needs to preserve more copies of scarce data and defend against malicious or bad behaving peers. Looks like it can't just be deployed on the Internet for us to use as an alternative, it's mostly for trusted internal use. Too bad.
- deleted 6y ago
  [deleted]
tiew9Vii 6y ago
When I saw Dockerhub announce they are throttling pulls I did think using something like IPFS (InterPlanetary File System)[1] would be ideal for a public Docker image repository. Dockerhub must be a huge expense for the free repositories, switching to P2P would significantly reduce the running costs. [1] https://ipfs.io/ https://ipfs.io/
- toomuchtodo 6y ago
  https://github.com/miguelmota/ipdr https://github.com/miguelmota/ipdr

84 points by tehlike 6y ago

Not to be confused with GitKraken (https://www.gitkraken.com https://www.gitkraken.com) or KrakenD (https://www.krakend.io https://www.krakend.io)

BenoitP 6y ago
Or Kraken the cryptocurrency exchange (https://www.kraken.com/ https://www.kraken.com/)
- js4ever 6y ago
  Or kraken.io, the image optimizer. This cool name seems to be quite popular
  - justinzollars 6y ago
    Or kraken the rum (https://www.krakenrum.com/ https://www.krakenrum.com/)
    - brian_herman__ 6y ago
      Or the original Kraken from H.P. Lovecraft https://lovecraft.fandom.com/wiki/Cthylla https://lovecraft.fandom.com/wiki/Cthylla
Timpy 6y ago
I definitely assumed this was git related before clicking

toyg 6y ago

This is so cool. Will I ever use it? Probably not, I’m a hobbyist at the moment. Do I think everything should be P2P, particularly if infrastructure-critical like a registry? Hell yes.

majewsky 6y ago
Most organizations are not Google or Facebook or Uber, so setting up a P2P network of registries is ghoulish overkill. Any image registry with a HA setup (e.g. API pods in Kubernetes and backing storage in Ceph or a NetApp cluster or whatever) is probably enough. Source: I've implemented a federating image registry for use at $dayjob (https://github.com/sapcc/keppel https://github.com/sapcc/keppel). Before doing so, I evaluated the landscape and also looked at Kraken briefly.
- jacques_chester 6y ago
  Thanks for the keppel link. Impressive work.
- LockAndLol 6y ago
  Since you have experience writing your own registry, do you think it would be difficult to support IPFS (or a similar distributed storage technology) for pulling images? I'm thinking HTTP for metadata and IPFS for storage could reduce the load on registries significantly, but I know very little about registries.
  - majewsky 6y ago
    Conceptually, no big deal. Most registries already deliver blob contents from a CDN or object store (via HTTP redirect). However, the registry protocol (i.e. the Open Containers Distribution API) only knows HTTP, so you'd have to use a HTTP-to-IPFS proxy to use the IPFS backing store with existing clients.

makeworld 6y ago

> Kraken's goal is to reduce global max download time and communication overhead in a stable environment, while BitTorrent was designed for an unpredictable and adversarial environment, so it needs to preserve more copies of scarce data and defend against malicious or bad behaving peers. Looks like it can't just be deployed on the Internet for us to use as an alternative, it's mostly for trusted internal use. Too bad.

deleted 6y ago
[deleted]

tiew9Vii 6y ago

When I saw Dockerhub announce they are throttling pulls I did think using something like IPFS (InterPlanetary File System)[1] would be ideal for a public Docker image repository. Dockerhub must be a huge expense for the free repositories, switching to P2P would significantly reduce the running costs. [1] https://ipfs.io/ https://ipfs.io/

toomuchtodo 6y ago
https://github.com/miguelmota/ipdr https://github.com/miguelmota/ipdr