8 ms·

Kraken, an Open Source Peer-to-Peer Docker Registry

204 points by kungfudoi 8y ago

leetrout 8y ago
I’ve said this at other companies I’ve worked at — the datacenter / enterprise software distribution is underserved by P2P. I wish more companies used BitTorrent internally for distribution of tools and software. I feel like homogenous environments like those that benefit from centralized config management like puppet would probably see performance gains with P2P technologies introduced. This looks like a great contribution from Uber engineering and I look forward to playing with it!
- jzelinskie 8y ago
  This is partially why I built chihaya[0]. The idea is by having a powerful middleware model for traditional BitTorrent software, devs can easily extend the protocol for whatever they need internally. As a sibling comment mentions, an older fork of Chihaya is one of the tools used at FB for distribution in their orchestration system, Tupperware. In practice, internal networks are such a mess that leveraging p2p is often more trouble than it's worth. [0]: https://chihaya.io https://chihaya.io
  - dman 8y ago
    Could you elaborate on the issues with local networks?
    - deleted 8y ago
      [deleted]
  - steeve 8y ago
    I've used it for a side project and it worked great, thank you for it!
- dymk 8y ago
  Facebook uses BitTorrent for binary and artifact distribution within their DCs
  - kosinus 8y ago
    This is absolutely great if it fits your case, but otherwise maybe not the best time investment. We happen to do event-based stuff at $work, with short running servers. Deploys go to several AWS regions, coordinated from servers in Europe. Bittorrent is really amazing for that sort of thing.
- 52-6F-62 8y ago
  I just had flashbacks to Bearshare on U of T's campus
- stingraycharles 8y ago
  On this note, we’ve been happily using Resilio (formerly BTSync) here at QuasarDB, as an alternative to Dropbox and the likes. It’s P2P and without a central cloud, and serves us very well.
- JMTQp8lwXL 8y ago
  Enterprises want to consume software dependencies they can trust. In a peer-to-peer model, anybody can publish anything. The value in having some sort of intermediary (e.g., Artifactory, etc) is they can act as a gateway before vulnerable dependencies make it into your build pipeline.
  - ses1984 8y ago
    But torrent has built in integrity checking. You should still check signatures on packages. Your distribution and storage media should be totally independent of the packages you choose to use and how you validate them.
    - JMTQp8lwXL 8y ago
      I think the idea is that enterprises would still prefer to offload that responsibility to a third-party provider. I agree with what you're saying, from a risk management perspective, you shouldn't be trusting that your dev's will be pulling in safe third party code (e.g., from the NPM public registry) instead of forcing them to go through your dependency gateway (e.g., artifactory). Under the hood, a tool like Artifactory could take advantage of this means of distribution, be responsible for signature checking, etc.
      - baroffoos 8y ago
        I didn't think anyone was suggesting to pull random stuff out of a p2p network but instead that you build your docker image and then use p2p to share it to thousands of servers. There are no trust issues here because you are the creator of the data being downloaded.
  - tlrobinson 8y ago
    > In a peer-to-peer model, anybody can publish anything. That doesn't have to be the case. You can certainly have private peer-to-peer networks that are authenticated and/or restricted to your internal network. Facebook did this back in 2010: https://torrentfreak.com/facebook-uses-bittorrent-and-they-love-it-100625/ https://torrentfreak.com/facebook-uses-bittorrent-and-they-l...
discordianfish 8y ago
I'm intrigued by p2p distribution like this but it makes me wonder if it's really more cost effective to switch/route all these small torrent packets instead of just using fileservers with 10G interfaces and maybe tiered caching.
- justinsaccount 8y ago
  fileservers don't send packets? "... a test where a 3G Docker image with 2 layers is downloaded by 2600 hosts concurrently (5200 blob downloads)" they show this finishing in 20s, so that's 7,800 GB transferred in 20 seconds. That works out to 390 GB/sec. A 10Gbps interface can transfer about 1GB/sec, so you'd need 195 dual attached fileservers, or about 100 if they were running at 40gbps.
  - jsight 8y ago
    > fileservers don't send packets? This line is far funnier than it should be. It reminds me of the time that I was told about how awful some architecture was because it would overload the network. Their improved architecture sent lots of smaller files over NFS instead. That was novel.
    - thekhatribharat 8y ago
      Well, probably they meant network congestion. Network congestion depends on both time distribution and link distribution of data, which differs widely with the choice of network protocols and network topologies (even if total data flowing through network remains the same)
      - discordianfish 8y ago
        Yep. I'll admit, I didn't crunch the numbers but routing torrent traffic isn't cheap. I haven't found a definitive resources but AFAIK/IIRC torrent uses at least an order of magnitude more packets per second for the same throughput as HTTP.
justboxing 8y ago
Gonna be really hard for devs to find tutorials[1] or other related things on this, given the name ( 'Kraken' ) is already a well-established crypto exchange. Same is the case with 'Discourse'. Almost impossible to find relevant content cos any search for 'discourse + keyword' invariably shows content related to "online discourse (conversation)" [1] https://www.google.com/search?q=kraken+docs https://www.google.com/search?q=kraken+docs
- flurdy 8y ago
  My initial thought was that Kraken had released a Docker registry which was odd. But now I realise it is a separate project by Uber that is called Kraken. But as it is a project I am okay with the name clash. It kind of suits the distributed nature of its arms. I know there will be some confusion but hopefully, people's google-fu and search terms will be able to distinguish from Kraken the cryptocurrency company, Kraken the docker registery tool and Kraken the mythical mega octopus, and more.
- augbog 8y ago
  It also used to be a pretty popular Node.js express suite http://krakenjs.com/ http://krakenjs.com/
- purple_ducks 8y ago
  https://kraken.io/ https://kraken.io/ - online image optimiser
- res0nat0r 8y ago
  "kraken docker uber" brings up the homepage as a first hit.
- deleted 8y ago
  [deleted]
- lawhite 8y ago
  Not to mention Krakend being an open source API Gateway
- marenkay 8y ago
deleted 8y ago
[deleted]
marenkay 8y ago
Why would anyone want to use this over Dragonfly, which seems to have way more battle testing? Is this another case of NIH?
- ShakataGaNai 8y ago
  This short answer seems to be "Timing". #1 - Dragonfly started November 2017 where as "Kraken was first deployed at Uber in early 2018". Which presumes Uber was working on this probably much earlier than Dragonfly even started. #2 - Dragonfly is still listed as a Sandbox CNCF project, where as Uber is running this in production for roughly a year it seems. #3 - Dragonfly was just refactored completely from Java to Go. So its kind of hard to say it's "battle tested" at all, at this point in time. It's basically an entirely new project as of 2 days ago.
  - marenkay 8y ago
    https://www.alibabacloud.com/blog/behind-alibabas-double-11-mysterious-dragonfly-technology-%C2%AEc-pb-grade-large-file-distribution-system_594074 https://www.alibabacloud.com/blog/behind-alibabas-double-11-... indicates Dargonfly existed much longer and was used in scenarios much larger than what Uber presented. What got into CNCF may be a newer iteration based on what Alibaba has been running but that means it should at least hold a candle to what is described in the post. With that said, one can surmise the Java version is the one they used in production. Apart from that, P2P image distribution is probably limited to internal uses that are not critical. Kind of missing indications of much difference in reliability to say an internal CDN this would give.
    - monoian 8y ago
      Kraken's readme included comparison with Dragonfly: https://github.com/uber/kraken#comparison-with-other-projects https://github.com/uber/kraken#comparison-with-other-project...
webmonkeyuk 8y ago
I'm chuckling inside thinking about how many people go and install/use this versus how many people actually work at a scale that they need to use this.
- munchbunny 8y ago
  To some extent testing this out when you need it is a bit too late. If you anticipate having a problem, it's useful to play with solutions before you actually have said problem. That's different from applying a 50,000 node solution to a 50 node problem though.
- puzzle 8y ago
  Given that e.g. AWS charges you for cross-region ECR image pulling, this can make a difference for scrappy companies that push large images on green (=multiple times a day, with lots of cache misses) to multiple regions. That's even if your deployments have just tens of replicas. Larger companies probably worry about other parts of the bill.
- farisjarrah 8y ago
  It makes sense to plan ahead for increased scale. If you are working for a VC backed company whose mission goals are to grow grow grow scale scale scale, then you cant exactly build for the infrastructure you currently are using. Its perfectly acceptable to build out an overbuilt infra, as long as your costs aren't shooting you in the foot. You know whats worse then paying too much for infrastructure? Loosing money and clients because your infrastructure breaks anytime you get a real workload on it.
  - franciscop 8y ago
    But even worse is not being able to release because the system complexity has shot through the roof. Plan (and test!) for 10x scale at a time, then optimize to squeeze another 5-10x while you build the 1000x system.
jnsaff2 8y ago
Reminds me of twitter murder: https://github.com/lg/murder https://github.com/lg/murder
warp_factor 8y ago
Cool project. I'm always wondering though, if those optimizations are really needed or if they are created by engineers that want to have fun and do premature optimizations. Ever wondered why a "WebApp" like Uber needs so many engineers and new projects?
- barnabee 8y ago
  I equally wonder where we'd be if no-one ever indulged in a premature optimisation…
  - warp_factor 8y ago
    premature optimization is super fun. I did it a lot and I would advise to engineers to do it if they can. But when I look at companies like Airbnb and Uber for example, I cannot really understand why they require that type of tools. Yes they are big, even very big, but nowhere the size of a Facebook or a Google. Most of those projects seem to be engineers having fun at work, and marketing themselves by publishing cool blog posts.
- puzzle 8y ago
  I was at Google when this became the standard more than a decade ago (https://www.usenix.org/sites/default/files/conference/protected-files/lisa_2014_talk.pdf https://www.usenix.org/sites/default/files/conference/protec...). For jobs where you had tens or hundreds of replicas, i.e. for services with lots of traffic, it made the rollout of new binaries in Borg much faster, especially if you were deploying to a cluster in Asia and your packages were built and stored in the US. Even in the US, this being before Firehose/Jupiter fabrics, network congestion had had noticeable impact. MPM traffic was low priority, so for large enough binaries (and most of them were, because of static linking) you could easily see a Borg task spend seconds or even tens of seconds in "downloading" state if the borglet hadn't already seen and cached locally that version of the package. It was about time this became available also for Docker and Kubernetes.
- thekhatribharat 8y ago
  WhatsApp is a good example, they had a small engineering team handling tens of millions of users.
stuff4ben 8y ago
Having been at the hurting side of both a 500+TB globally distributed Artifactory cluster and a relatively smaller 30TB Quay cluster, I for one welcome this new contender! Scaling binary object stores globally is no small feat. FWIW, JFrog the company behind Artifactory is way better than CoreOS/RedHat from a vendor support aspect.
- regnerba 8y ago
  O_O 500+TB Artifactory cluster! Hahahaha ours is like 30GB and just RPMs. Was it much of a challenge to scale Artifactory to that level?
ianstallings 8y ago
This has to be the most insane shit I've seen in quite a while.
instaheat 8y ago
Kraken, not to be confused with the Bitcoin and Cryptocurrency exchange.
- oregontechninja 8y ago
  Also not to be confused with the GUI git client "gitkraken"
  - classicsnoot 8y ago
    Or the botnet that appeared and disappeared in a few hours back in 2008. https://en.m.wikipedia.org/wiki/Kraken_botnet?wprov=sfla1 https://en.m.wikipedia.org/wiki/Kraken_botnet?wprov=sfla1
  - barbecue_sauce 8y ago
    Or the php framework. Or kraken.js. Or the responsive css boilerplate. Or the API gateway (KrakenD). Or the Joomla theme. Or the ransomware-as-a-service (RaaS) affiliate program.
    - franciscop 8y ago
      I just had a talk last week with my coworker about how many tech projects seem to be called Kraken and BAM, here's another.
      - WrtCdEvrydy 8y ago
        Same as projects call Phoenix.
- jhbadger 8y ago
  Or the commonly used bioinformatics tool "kraken" which is used to predict the species a bit of DNA came from (useful in metagenomic studies of the microbiome or environment where a sample may contain DNA from thousands of organisms) https://ccb.jhu.edu/software/kraken/ https://ccb.jhu.edu/software/kraken/
zerotolerance 8y ago
A P2P software supply chain had better have a content signing and key distribution framework in place.
hardwaresofton 8y ago
Naively I would have thought a docker registry with on-disk storage managed by ipfs would have been a low-effort way to meet this requirement. Unfortunately you'd need something to manage the pinning settings, but it feels like a relatively small addition to some other registry that could be hacked together in a small amount of time.
- deleted 8y ago
  [deleted]
- zcw100 8y ago
  You mean something like ipfs cluster? https://cluster.ipfs.io https://cluster.ipfs.io
  - hardwaresofton 8y ago
    I didn't know ipfs cluster existed -- thanks for the reference. I was more thinking of a harbor[0] like cobbling of these technologies together -- harbor combines a bunch of F/OSS tools into one powerful registry solution. Here are a few: - Distribution for storing images - Notary for signing - Clair for static analysis Based on this I was thinking of was basically Distribution + ipfs cluster + a small management daemon. The daemon is only there to tie the other pieces together and present a unified interface but the bulk of the work can be done with the other pieces. [0]: https://github.com/goharbor/harbor/wiki/Architecture-Overview-of-Harbor https://github.com/goharbor/harbor/wiki/Architecture-Overvie... [1]: https://github.com/docker/distribution https://github.com/docker/distribution [2]: https://github.com/theupdateframework/notary https://github.com/theupdateframework/notary [3]: https://github.com/coreos/clair https://github.com/coreos/clair
jvassev 8y ago
Wouldn't IPFS be a more natural fit - and also trivial to implement. Not HN material but here it is anyway https://github.com/jvassev/image2ipfs https://github.com/jvassev/image2ipfs. I guess what IFPS is missing and Kraken has are "pluggable storage options, and instead of managing data blobs, Kraken plugs into reliable blob storage options like S3, HDFS"

204 points by kungfudoi 8y ago

I’ve said this at other companies I’ve worked at — the datacenter / enterprise software distribution is underserved by P2P. I wish more companies used BitTorrent internally for distribution of tools and software. I feel like homogenous environments like those that benefit from centralized config management like puppet would probably see performance gains with P2P technologies introduced. This looks like a great contribution from Uber engineering and I look forward to playing with it!

jzelinskie 8y ago
This is partially why I built chihaya[0]. The idea is by having a powerful middleware model for traditional BitTorrent software, devs can easily extend the protocol for whatever they need internally. As a sibling comment mentions, an older fork of Chihaya is one of the tools used at FB for distribution in their orchestration system, Tupperware. In practice, internal networks are such a mess that leveraging p2p is often more trouble than it's worth. [0]: https://chihaya.io https://chihaya.io
- dman 8y ago
  Could you elaborate on the issues with local networks?
  - deleted 8y ago
    [deleted]
- steeve 8y ago
  I've used it for a side project and it worked great, thank you for it!
dymk 8y ago
Facebook uses BitTorrent for binary and artifact distribution within their DCs
- kosinus 8y ago
  This is absolutely great if it fits your case, but otherwise maybe not the best time investment. We happen to do event-based stuff at $work, with short running servers. Deploys go to several AWS regions, coordinated from servers in Europe. Bittorrent is really amazing for that sort of thing.
52-6F-62 8y ago
I just had flashbacks to Bearshare on U of T's campus
stingraycharles 8y ago
On this note, we’ve been happily using Resilio (formerly BTSync) here at QuasarDB, as an alternative to Dropbox and the likes. It’s P2P and without a central cloud, and serves us very well.
JMTQp8lwXL 8y ago
Enterprises want to consume software dependencies they can trust. In a peer-to-peer model, anybody can publish anything. The value in having some sort of intermediary (e.g., Artifactory, etc) is they can act as a gateway before vulnerable dependencies make it into your build pipeline.
- ses1984 8y ago
  But torrent has built in integrity checking. You should still check signatures on packages. Your distribution and storage media should be totally independent of the packages you choose to use and how you validate them.
  - JMTQp8lwXL 8y ago
    I think the idea is that enterprises would still prefer to offload that responsibility to a third-party provider. I agree with what you're saying, from a risk management perspective, you shouldn't be trusting that your dev's will be pulling in safe third party code (e.g., from the NPM public registry) instead of forcing them to go through your dependency gateway (e.g., artifactory). Under the hood, a tool like Artifactory could take advantage of this means of distribution, be responsible for signature checking, etc.
    - baroffoos 8y ago
      I didn't think anyone was suggesting to pull random stuff out of a p2p network but instead that you build your docker image and then use p2p to share it to thousands of servers. There are no trust issues here because you are the creator of the data being downloaded.
- tlrobinson 8y ago
  > In a peer-to-peer model, anybody can publish anything. That doesn't have to be the case. You can certainly have private peer-to-peer networks that are authenticated and/or restricted to your internal network. Facebook did this back in 2010: https://torrentfreak.com/facebook-uses-bittorrent-and-they-love-it-100625/ https://torrentfreak.com/facebook-uses-bittorrent-and-they-l...

discordianfish 8y ago

I'm intrigued by p2p distribution like this but it makes me wonder if it's really more cost effective to switch/route all these small torrent packets instead of just using fileservers with 10G interfaces and maybe tiered caching.

justinsaccount 8y ago
fileservers don't send packets? "... a test where a 3G Docker image with 2 layers is downloaded by 2600 hosts concurrently (5200 blob downloads)" they show this finishing in 20s, so that's 7,800 GB transferred in 20 seconds. That works out to 390 GB/sec. A 10Gbps interface can transfer about 1GB/sec, so you'd need 195 dual attached fileservers, or about 100 if they were running at 40gbps.
- jsight 8y ago
  > fileservers don't send packets? This line is far funnier than it should be. It reminds me of the time that I was told about how awful some architecture was because it would overload the network. Their improved architecture sent lots of smaller files over NFS instead. That was novel.
  - thekhatribharat 8y ago
    Well, probably they meant network congestion. Network congestion depends on both time distribution and link distribution of data, which differs widely with the choice of network protocols and network topologies (even if total data flowing through network remains the same)
    - discordianfish 8y ago
      Yep. I'll admit, I didn't crunch the numbers but routing torrent traffic isn't cheap. I haven't found a definitive resources but AFAIK/IIRC torrent uses at least an order of magnitude more packets per second for the same throughput as HTTP.

justboxing 8y ago

Gonna be really hard for devs to find tutorials[1] or other related things on this, given the name ( 'Kraken' ) is already a well-established crypto exchange. Same is the case with 'Discourse'. Almost impossible to find relevant content cos any search for 'discourse + keyword' invariably shows content related to "online discourse (conversation)" [1] https://www.google.com/search?q=kraken+docs https://www.google.com/search?q=kraken+docs

flurdy 8y ago
My initial thought was that Kraken had released a Docker registry which was odd. But now I realise it is a separate project by Uber that is called Kraken. But as it is a project I am okay with the name clash. It kind of suits the distributed nature of its arms. I know there will be some confusion but hopefully, people's google-fu and search terms will be able to distinguish from Kraken the cryptocurrency company, Kraken the docker registery tool and Kraken the mythical mega octopus, and more.
augbog 8y ago
It also used to be a pretty popular Node.js express suite http://krakenjs.com/ http://krakenjs.com/
purple_ducks 8y ago
https://kraken.io/ https://kraken.io/ - online image optimiser
res0nat0r 8y ago
"kraken docker uber" brings up the homepage as a first hit.
deleted 8y ago
[deleted]
lawhite 8y ago
Not to mention Krakend being an open source API Gateway
marenkay 8y ago

deleted 8y ago

[deleted]

marenkay 8y ago

Why would anyone want to use this over Dragonfly, which seems to have way more battle testing? Is this another case of NIH?

ShakataGaNai 8y ago
This short answer seems to be "Timing". #1 - Dragonfly started November 2017 where as "Kraken was first deployed at Uber in early 2018". Which presumes Uber was working on this probably much earlier than Dragonfly even started. #2 - Dragonfly is still listed as a Sandbox CNCF project, where as Uber is running this in production for roughly a year it seems. #3 - Dragonfly was just refactored completely from Java to Go. So its kind of hard to say it's "battle tested" at all, at this point in time. It's basically an entirely new project as of 2 days ago.
- marenkay 8y ago
  https://www.alibabacloud.com/blog/behind-alibabas-double-11-mysterious-dragonfly-technology-%C2%AEc-pb-grade-large-file-distribution-system_594074 https://www.alibabacloud.com/blog/behind-alibabas-double-11-... indicates Dargonfly existed much longer and was used in scenarios much larger than what Uber presented. What got into CNCF may be a newer iteration based on what Alibaba has been running but that means it should at least hold a candle to what is described in the post. With that said, one can surmise the Java version is the one they used in production. Apart from that, P2P image distribution is probably limited to internal uses that are not critical. Kind of missing indications of much difference in reliability to say an internal CDN this would give.
  - monoian 8y ago
    Kraken's readme included comparison with Dragonfly: https://github.com/uber/kraken#comparison-with-other-projects https://github.com/uber/kraken#comparison-with-other-project...

webmonkeyuk 8y ago

I'm chuckling inside thinking about how many people go and install/use this versus how many people actually work at a scale that they need to use this.

munchbunny 8y ago
To some extent testing this out when you need it is a bit too late. If you anticipate having a problem, it's useful to play with solutions before you actually have said problem. That's different from applying a 50,000 node solution to a 50 node problem though.
puzzle 8y ago
Given that e.g. AWS charges you for cross-region ECR image pulling, this can make a difference for scrappy companies that push large images on green (=multiple times a day, with lots of cache misses) to multiple regions. That's even if your deployments have just tens of replicas. Larger companies probably worry about other parts of the bill.
farisjarrah 8y ago
It makes sense to plan ahead for increased scale. If you are working for a VC backed company whose mission goals are to grow grow grow scale scale scale, then you cant exactly build for the infrastructure you currently are using. Its perfectly acceptable to build out an overbuilt infra, as long as your costs aren't shooting you in the foot. You know whats worse then paying too much for infrastructure? Loosing money and clients because your infrastructure breaks anytime you get a real workload on it.
- franciscop 8y ago
  But even worse is not being able to release because the system complexity has shot through the roof. Plan (and test!) for 10x scale at a time, then optimize to squeeze another 5-10x while you build the 1000x system.

jnsaff2 8y ago

Reminds me of twitter murder: https://github.com/lg/murder https://github.com/lg/murder

warp_factor 8y ago

Cool project. I'm always wondering though, if those optimizations are really needed or if they are created by engineers that want to have fun and do premature optimizations. Ever wondered why a "WebApp" like Uber needs so many engineers and new projects?

barnabee 8y ago
I equally wonder where we'd be if no-one ever indulged in a premature optimisation…
- warp_factor 8y ago
  premature optimization is super fun. I did it a lot and I would advise to engineers to do it if they can. But when I look at companies like Airbnb and Uber for example, I cannot really understand why they require that type of tools. Yes they are big, even very big, but nowhere the size of a Facebook or a Google. Most of those projects seem to be engineers having fun at work, and marketing themselves by publishing cool blog posts.
puzzle 8y ago
I was at Google when this became the standard more than a decade ago (https://www.usenix.org/sites/default/files/conference/protected-files/lisa_2014_talk.pdf https://www.usenix.org/sites/default/files/conference/protec...). For jobs where you had tens or hundreds of replicas, i.e. for services with lots of traffic, it made the rollout of new binaries in Borg much faster, especially if you were deploying to a cluster in Asia and your packages were built and stored in the US. Even in the US, this being before Firehose/Jupiter fabrics, network congestion had had noticeable impact. MPM traffic was low priority, so for large enough binaries (and most of them were, because of static linking) you could easily see a Borg task spend seconds or even tens of seconds in "downloading" state if the borglet hadn't already seen and cached locally that version of the package. It was about time this became available also for Docker and Kubernetes.
thekhatribharat 8y ago
WhatsApp is a good example, they had a small engineering team handling tens of millions of users.

stuff4ben 8y ago

Having been at the hurting side of both a 500+TB globally distributed Artifactory cluster and a relatively smaller 30TB Quay cluster, I for one welcome this new contender! Scaling binary object stores globally is no small feat. FWIW, JFrog the company behind Artifactory is way better than CoreOS/RedHat from a vendor support aspect.

regnerba 8y ago
O_O 500+TB Artifactory cluster! Hahahaha ours is like 30GB and just RPMs. Was it much of a challenge to scale Artifactory to that level?

ianstallings 8y ago

This has to be the most insane shit I've seen in quite a while.

instaheat 8y ago

Kraken, not to be confused with the Bitcoin and Cryptocurrency exchange.

oregontechninja 8y ago
Also not to be confused with the GUI git client "gitkraken"
- classicsnoot 8y ago
  Or the botnet that appeared and disappeared in a few hours back in 2008. https://en.m.wikipedia.org/wiki/Kraken_botnet?wprov=sfla1 https://en.m.wikipedia.org/wiki/Kraken_botnet?wprov=sfla1
- barbecue_sauce 8y ago
  Or the php framework. Or kraken.js. Or the responsive css boilerplate. Or the API gateway (KrakenD). Or the Joomla theme. Or the ransomware-as-a-service (RaaS) affiliate program.
  - franciscop 8y ago
    I just had a talk last week with my coworker about how many tech projects seem to be called Kraken and BAM, here's another.
    - WrtCdEvrydy 8y ago
      Same as projects call Phoenix.
jhbadger 8y ago
Or the commonly used bioinformatics tool "kraken" which is used to predict the species a bit of DNA came from (useful in metagenomic studies of the microbiome or environment where a sample may contain DNA from thousands of organisms) https://ccb.jhu.edu/software/kraken/ https://ccb.jhu.edu/software/kraken/

zerotolerance 8y ago

A P2P software supply chain had better have a content signing and key distribution framework in place.

hardwaresofton 8y ago

Naively I would have thought a docker registry with on-disk storage managed by ipfs would have been a low-effort way to meet this requirement. Unfortunately you'd need something to manage the pinning settings, but it feels like a relatively small addition to some other registry that could be hacked together in a small amount of time.

deleted 8y ago
[deleted]
zcw100 8y ago
You mean something like ipfs cluster? https://cluster.ipfs.io https://cluster.ipfs.io
- hardwaresofton 8y ago
  I didn't know ipfs cluster existed -- thanks for the reference. I was more thinking of a harbor[0] like cobbling of these technologies together -- harbor combines a bunch of F/OSS tools into one powerful registry solution. Here are a few: - Distribution for storing images - Notary for signing - Clair for static analysis Based on this I was thinking of was basically Distribution + ipfs cluster + a small management daemon. The daemon is only there to tie the other pieces together and present a unified interface but the bulk of the work can be done with the other pieces. [0]: https://github.com/goharbor/harbor/wiki/Architecture-Overview-of-Harbor https://github.com/goharbor/harbor/wiki/Architecture-Overvie... [1]: https://github.com/docker/distribution https://github.com/docker/distribution [2]: https://github.com/theupdateframework/notary https://github.com/theupdateframework/notary [3]: https://github.com/coreos/clair https://github.com/coreos/clair

jvassev 8y ago

Wouldn't IPFS be a more natural fit - and also trivial to implement. Not HN material but here it is anyway https://github.com/jvassev/image2ipfs https://github.com/jvassev/image2ipfs. I guess what IFPS is missing and Kraken has are "pluggable storage options, and instead of managing data blobs, Kraken plugs into reliable blob storage options like S3, HDFS"