9 ms·

Why Docker and CoreOS’ split was predictable

201 points by dantiberian 12y ago

chatmasta 12y ago
Great post. I completely agree, and was making this same argument to someone yesterday. There are a few important trends and facts: - Containerization allows more tenants-per-server than virtualization - The entire business model of cloud computing is maximizing tenants-per-server - Because containerization has a direct cost benefit to cloud providers, they will adopt it quickly - Quick adoption of containerization will trigger a "new wave" similar to when Linux gained mass adoption - Cloud companies are scrambling to invest in hot containerization rather than build their own - Docker has developer street cred and a solid team - Post-dotcom, post-web2.0, companies invest in open source software now. This is new. - Cloud companies compete with each other - Different cloud companies invested in Docker, CoreOS - Docker, CoreOS are competing My logic is a bit fragmented, but all the necessary facts are there, and the conclusion that logically follows from them is that it makes rational business sense for CoreOS to diverge from Docker. There's billions of dollars at stake in the latest "platform war" (previous wars include desktop, server, mobile) -- the battle for the cloud. Containerization directly reduces costs and the "metric that matters" for cloud hosts, tenants-per-server. Therefore any open source company with a good containerization solution is going to attract a lot of investment. But bigcorp A does not want to rely on strategic direction of bigcorp B. Therefore they diverge their open source offerings. It makes sense from a business perspective, and it's kind of sad from an open source perspective, but also good if you believe in free market economics. The question is, how much do free market economics apply in open source?
- dantiberian 12y ago
  Containerization directly reduces costs and the "metric that matters" for cloud hosts, tenants-per-server. the flip side of this coin is that it reduces the cost for every cloud provider. In essence containers moves the locus of competition further up the stack, commoditising the layers below. I'll bet there were a lot of people who were upset when shipping containers were standardised too.
  - mrbill 12y ago
    I'll bet there were a lot of people who were upset when shipping containers were standardised too. There's a GREAT book on the subject: "The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger" http://www.amazon.com/Box-Shipping-Container-Smaller-Economy/dp/0691136408 http://www.amazon.com/Box-Shipping-Container-Smaller-Economy...
    - zrail 12y ago
      This is a fantastic book. I reviewed it last year: https://www.petekeen.net/book-review-the-box https://www.petekeen.net/book-review-the-box
      - mrbill 12y ago
        What I found most interesting was how long it took to come to agreements on "standard" sizes. Lots of infighting and politics.
- kiyoto 12y ago
  >The question is, how much do free market economics apply in open source? This is something I ask myself a lot (as someone focused 100% on open source at a company whose core business is not open source support). For all the hype that open source has enjoyed as a business model, it is much smaller/untested/too early to tell compared to the proprietary model. The most financially successful open source company to date is Red Hat, which, at 10B USD+, is a great success by anyone's measure. But that's the best any open source pure play company has done. If you look in the other direction, Oracle is worth 180B USD, and even a fledgling BI company like Tableau is in the high single digit billion. As much as I'd like open source companies to be an Oracle-grade blockbuster success, the brief story is not exactly promising.
  - dantiberian 12y ago
    As well as Red Hat has done, they are dwarfed by many other companies. This post about open source compressing the market size by 10x made a lot of sense, and was also a little depressing. http://kellblog.com/2014/11/18/it-aint-easy-making-money-in-open-source-thoughts-on-the-hortonworks-s-1/ http://kellblog.com/2014/11/18/it-aint-easy-making-money-in-...
    - ojilles 12y ago
      But isn't that exactly how innovation and disruption works? Taking a market and saying "we'll do it for 10x less".
      - wmf 12y ago
        It depends what kind of innovation. Disruption often involves business or marketing innovation combined with technical non-innovation. Shrinking a market by 10x can be a good way to ensure that there won't be enough profit left to invest in R&D.
  - skriticos2 12y ago
    I guess it's a question of what the company is selling. Google for example uses huge stacks of open source to run their business. And even though they are far from being pure open source, they do contribute back to the community in multiple ways, for example with contributions to the Linux kernel and open source projects like Chromium and Go. And then there are projects like Qt, that mix open source tools with enterprise support. But I think that the field of IT is still extremely young and that in the future larger companies will discover that collaborating on open source platform and infrastructure projects will give them much better synergy effects and lower IT costs by removing the license cost and at the same time lowering the support cost. Compared to history, the 20-30 years that mainstream computing has is really just a spark and there is so much change coming that will be directed by market and competition.
- beachstartup 12y ago
  > - The entire business model of cloud computing is maximizing tenants-per-server this is not exactly true. the business model is to maximize the revenue per server, not the number of tenants. this can be done in a number of ways.
  - chatmasta 12y ago
    Good point. Of course, one of those ways is by charging the same price-per-tenant but increasing the number of tenants-per-server. If multiple companies do this, we should expect the price-per-tenant floor to decrease.
    - Someone1234 12y ago
      I liken containers to adding an extra row of seats on a commercial aircraft. Right now the airline (host) operates one aircraft (server) and can fit X number of seats on it (split due to both consumption but also overhead from the seat's depth (hypervisor overhead)). If containers have lower overhead than hypervisors then it would be akin to an airline buying thinner seats and getting an extra row in. However it does remain to be seen how much more efficient containers are over hypervisors, and also how secure they are (there are lot's of unanswered questions on the security front).
      - derefr 12y ago
        I don't think most people are comparing containers to hypervisors (VMs); they're comparing using just VMs to using containers on top of VMs. Especially, they're comparing running one-VM-from-some-IaaS per workload, to running several of those workloads (possibly from multiple tenants) on top of the same VM-from-some-IaaS. If it wasn't insane to run a hypervisor within a hypervisor, this platform war would have started years ago, with Heroku taking AWS VMs and splitting them into tiny Heroku VMs.
  - simonpantzare 12y ago
    My understanding of Amazon's new container service is that it lets users spawn a bunch of instances that the service manages (job scheduling and so on). This is not maximizing tenants per server nor maximizing revenue per server, IMO, as it mainly lets users increase the workload per instance. Without the container service, unless they run the similar infrastructure themselves, users would have to run additional instances to have the same isolation and ease of deployment (if they use Puppet, Chef, Ansible, or whatnot instead of some Docker cluster management software).
kiyoto 12y ago
This analysis is spot on. There is nothing wrong with what Docker or CoreOS are doing. As the OP pointed out, stuff like this happens all the time in computing. What makes it awkward is that you can't say this stuff and hope to convince the critically important group of people for open source software/companies: user communities. Users care a lot about their software and community around it growing and improving, but they won't be convinced by the needs of venture-backed, growth-oriented corporate strategy. (I am not saying this is good or bad but stating the fact).
ErikHuisman 12y ago
Click here if you get an 404: http://danielcompton.net/2014/12/02/modular-integrated-docker-coreos%20(daniel%20compton's%20conflicted%20copy%202014-12-03) http://danielcompton.net/2014/12/02/modular-integrated-docke...
- dantiberian 12y ago
  Back up now. Sorry about that.
Karunamon 12y ago
Author changed the post title I think, the link is now dead. http://danielcompton.net http://danielcompton.net still shows the article in first position on his blog.
- dantiberian 12y ago
  Some post management didn't go quite right, thanks for the tip, it's back up now.
jasode 12y ago
I appreciate the author's insights of taking previous examples of technology shifts, extracting a generic pattern from them, and then applying it to the new Docker direction. But I don't fully buy into it. To me, the new emphasis on "orchestrating & managing" containers is an obvious and unavoidable progression of taming new layers of complexity as they appear. The lower level technology of containers solved one set of complexities (app deploy onto 1 operating system), but they also create new complexities: tame the explosion of a thousand containers in the data center. How is this different from MS System Center, HP OpenView, and IBM Tivoli as managers/watchdogs of Windows & Linux boxes or KVM-over-ip to remotely reboot/powercycle racks of servers? Yes, those "enterprisey management console" products are "value added" and successful. Yes, there was competition between OpenView & Tivoli and competition between KVM vendors. But they weren't the kind of products that captured mindshare as glamorous; it was just something the IT department picked to "keep the lights running". Those management products also did not mint billionaires (millionaires yes, but not billionaires.) So the recent analysis from blogs that pits Docker vs CoreOs as a "strategic-winner-take-all" platform war like Internet Explorer vs Netscape doesn't feel quite accurate. It seems like the more interesting software fight is datacenter cloud o/s which is even higher level than Docker's orchestration-of-containers focus. Example of this would be OpenStack[1]. Proprietary ones would be Facebook internal cloud stack and Amazon AWS. This "datacenter-as-a-platform" would subsume the container standards fight. There may be something I'm missing in the Docker vs CoreOS that makes me discount this competition while others are playing it up. For example, is there some plugin architecture that would lock in developer mindshare? [1]http://en.wikipedia.org/wiki/OpenStack http://en.wikipedia.org/wiki/OpenStack
- dantiberian 12y ago
  It seems like the more interesting software fight is datacenter cloud o/s which is even higher level than Docker's orchestration-of-containers focus. I hadn't thought about it, but this makes a lot of sense as the next place of competition where value can be added. It's also where CoreOS is trying to be. It would also suggest that the level above that is abstracting datacenter's away to just manage a single global system made up of multiple DC's. There may be something I'm missing in the Docker vs CoreOS that makes me discount this competition while others are playing it up. Docker and CoreOS aren't strictly competing with each other, CoreOS wanted to use Docker as a commodity component (as it was designed), Docker eventually realised that becoming a commodity businesses wasn't going to be very profitable and moved into CoreOS' territory.
serve_yay 12y ago
Great analysis, thank you.
CoconutPilot 12y ago
I have a different theory. Originally Docker was going to be a container and CoreOS was a platform to run containers. They were both pieces in a traditional software stack. Docker had a round of funding and the investors pushed Docker in a different direction: App store. Docker was no longer just a piece in the stack and CoreOS could see the writing on the wall: libOS. Docker would marginalize CoreOS the same way LXC was marginalized. CoreOS is going on the offensive because they were tipped off on Docker's hand. Occam's Razor.
- perlpimp 12y ago
  So which platform can one use to deploy docker cloud? We already have a set of CoreOS servers running docker containers... with rocket replacing docker and automagic updates in coreOS this seems kinda dangerous.
gtirloni 12y ago
When working with FreeBSD jails a decade ago, I always thought they were enough to build my servers and needed just more polishing. Then hypervisors caught on and it was the virtualization era. Now soft-virtualization is coming back and it's becoming popular to say "hypervisors are overhead". It's funny how we could have taken a different direction ten years ago but I think I understand. We needed to go through the "VMs will run any of your existing software, don't worry". With mass adoption, new software can start to be written to run inside soft-virtualization (with not that many modifications, if any). If only.. oh well.

201 points by dantiberian 12y ago

Great post. I completely agree, and was making this same argument to someone yesterday. There are a few important trends and facts: - Containerization allows more tenants-per-server than virtualization - The entire business model of cloud computing is maximizing tenants-per-server - Because containerization has a direct cost benefit to cloud providers, they will adopt it quickly - Quick adoption of containerization will trigger a "new wave" similar to when Linux gained mass adoption - Cloud companies are scrambling to invest in hot containerization rather than build their own - Docker has developer street cred and a solid team - Post-dotcom, post-web2.0, companies invest in open source software now. This is new. - Cloud companies compete with each other - Different cloud companies invested in Docker, CoreOS - Docker, CoreOS are competing My logic is a bit fragmented, but all the necessary facts are there, and the conclusion that logically follows from them is that it makes rational business sense for CoreOS to diverge from Docker. There's billions of dollars at stake in the latest "platform war" (previous wars include desktop, server, mobile) -- the battle for the cloud. Containerization directly reduces costs and the "metric that matters" for cloud hosts, tenants-per-server. Therefore any open source company with a good containerization solution is going to attract a lot of investment. But bigcorp A does not want to rely on strategic direction of bigcorp B. Therefore they diverge their open source offerings. It makes sense from a business perspective, and it's kind of sad from an open source perspective, but also good if you believe in free market economics. The question is, how much do free market economics apply in open source?

dantiberian 12y ago
Containerization directly reduces costs and the "metric that matters" for cloud hosts, tenants-per-server. the flip side of this coin is that it reduces the cost for every cloud provider. In essence containers moves the locus of competition further up the stack, commoditising the layers below. I'll bet there were a lot of people who were upset when shipping containers were standardised too.
- mrbill 12y ago
  I'll bet there were a lot of people who were upset when shipping containers were standardised too. There's a GREAT book on the subject: "The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger" http://www.amazon.com/Box-Shipping-Container-Smaller-Economy/dp/0691136408 http://www.amazon.com/Box-Shipping-Container-Smaller-Economy...
  - zrail 12y ago
    This is a fantastic book. I reviewed it last year: https://www.petekeen.net/book-review-the-box https://www.petekeen.net/book-review-the-box
    - mrbill 12y ago
      What I found most interesting was how long it took to come to agreements on "standard" sizes. Lots of infighting and politics.
kiyoto 12y ago
>The question is, how much do free market economics apply in open source? This is something I ask myself a lot (as someone focused 100% on open source at a company whose core business is not open source support). For all the hype that open source has enjoyed as a business model, it is much smaller/untested/too early to tell compared to the proprietary model. The most financially successful open source company to date is Red Hat, which, at 10B USD+, is a great success by anyone's measure. But that's the best any open source pure play company has done. If you look in the other direction, Oracle is worth 180B USD, and even a fledgling BI company like Tableau is in the high single digit billion. As much as I'd like open source companies to be an Oracle-grade blockbuster success, the brief story is not exactly promising.
- dantiberian 12y ago
  As well as Red Hat has done, they are dwarfed by many other companies. This post about open source compressing the market size by 10x made a lot of sense, and was also a little depressing. http://kellblog.com/2014/11/18/it-aint-easy-making-money-in-open-source-thoughts-on-the-hortonworks-s-1/ http://kellblog.com/2014/11/18/it-aint-easy-making-money-in-...
  - ojilles 12y ago
    But isn't that exactly how innovation and disruption works? Taking a market and saying "we'll do it for 10x less".
    - wmf 12y ago
      It depends what kind of innovation. Disruption often involves business or marketing innovation combined with technical non-innovation. Shrinking a market by 10x can be a good way to ensure that there won't be enough profit left to invest in R&D.
- skriticos2 12y ago
  I guess it's a question of what the company is selling. Google for example uses huge stacks of open source to run their business. And even though they are far from being pure open source, they do contribute back to the community in multiple ways, for example with contributions to the Linux kernel and open source projects like Chromium and Go. And then there are projects like Qt, that mix open source tools with enterprise support. But I think that the field of IT is still extremely young and that in the future larger companies will discover that collaborating on open source platform and infrastructure projects will give them much better synergy effects and lower IT costs by removing the license cost and at the same time lowering the support cost. Compared to history, the 20-30 years that mainstream computing has is really just a spark and there is so much change coming that will be directed by market and competition.
beachstartup 12y ago
> - The entire business model of cloud computing is maximizing tenants-per-server this is not exactly true. the business model is to maximize the revenue per server, not the number of tenants. this can be done in a number of ways.
- chatmasta 12y ago
  Good point. Of course, one of those ways is by charging the same price-per-tenant but increasing the number of tenants-per-server. If multiple companies do this, we should expect the price-per-tenant floor to decrease.
  - Someone1234 12y ago
    I liken containers to adding an extra row of seats on a commercial aircraft. Right now the airline (host) operates one aircraft (server) and can fit X number of seats on it (split due to both consumption but also overhead from the seat's depth (hypervisor overhead)). If containers have lower overhead than hypervisors then it would be akin to an airline buying thinner seats and getting an extra row in. However it does remain to be seen how much more efficient containers are over hypervisors, and also how secure they are (there are lot's of unanswered questions on the security front).
    - derefr 12y ago
      I don't think most people are comparing containers to hypervisors (VMs); they're comparing using just VMs to using containers on top of VMs. Especially, they're comparing running one-VM-from-some-IaaS per workload, to running several of those workloads (possibly from multiple tenants) on top of the same VM-from-some-IaaS. If it wasn't insane to run a hypervisor within a hypervisor, this platform war would have started years ago, with Heroku taking AWS VMs and splitting them into tiny Heroku VMs.
- simonpantzare 12y ago
  My understanding of Amazon's new container service is that it lets users spawn a bunch of instances that the service manages (job scheduling and so on). This is not maximizing tenants per server nor maximizing revenue per server, IMO, as it mainly lets users increase the workload per instance. Without the container service, unless they run the similar infrastructure themselves, users would have to run additional instances to have the same isolation and ease of deployment (if they use Puppet, Chef, Ansible, or whatnot instead of some Docker cluster management software).

kiyoto 12y ago

This analysis is spot on. There is nothing wrong with what Docker or CoreOS are doing. As the OP pointed out, stuff like this happens all the time in computing. What makes it awkward is that you can't say this stuff and hope to convince the critically important group of people for open source software/companies: user communities. Users care a lot about their software and community around it growing and improving, but they won't be convinced by the needs of venture-backed, growth-oriented corporate strategy. (I am not saying this is good or bad but stating the fact).

ErikHuisman 12y ago

Click here if you get an 404: http://danielcompton.net/2014/12/02/modular-integrated-docker-coreos%20(daniel%20compton's%20conflicted%20copy%202014-12-03) http://danielcompton.net/2014/12/02/modular-integrated-docke...

dantiberian 12y ago
Back up now. Sorry about that.

Karunamon 12y ago

Author changed the post title I think, the link is now dead. http://danielcompton.net http://danielcompton.net still shows the article in first position on his blog.

dantiberian 12y ago
Some post management didn't go quite right, thanks for the tip, it's back up now.

jasode 12y ago

I appreciate the author's insights of taking previous examples of technology shifts, extracting a generic pattern from them, and then applying it to the new Docker direction. But I don't fully buy into it. To me, the new emphasis on "orchestrating & managing" containers is an obvious and unavoidable progression of taming new layers of complexity as they appear. The lower level technology of containers solved one set of complexities (app deploy onto 1 operating system), but they also create new complexities: tame the explosion of a thousand containers in the data center. How is this different from MS System Center, HP OpenView, and IBM Tivoli as managers/watchdogs of Windows & Linux boxes or KVM-over-ip to remotely reboot/powercycle racks of servers? Yes, those "enterprisey management console" products are "value added" and successful. Yes, there was competition between OpenView & Tivoli and competition between KVM vendors. But they weren't the kind of products that captured mindshare as glamorous; it was just something the IT department picked to "keep the lights running". Those management products also did not mint billionaires (millionaires yes, but not billionaires.) So the recent analysis from blogs that pits Docker vs CoreOs as a "strategic-winner-take-all" platform war like Internet Explorer vs Netscape doesn't feel quite accurate. It seems like the more interesting software fight is datacenter cloud o/s which is even higher level than Docker's orchestration-of-containers focus. Example of this would be OpenStack[1]. Proprietary ones would be Facebook internal cloud stack and Amazon AWS. This "datacenter-as-a-platform" would subsume the container standards fight. There may be something I'm missing in the Docker vs CoreOS that makes me discount this competition while others are playing it up. For example, is there some plugin architecture that would lock in developer mindshare? [1]http://en.wikipedia.org/wiki/OpenStack http://en.wikipedia.org/wiki/OpenStack

dantiberian 12y ago
It seems like the more interesting software fight is datacenter cloud o/s which is even higher level than Docker's orchestration-of-containers focus. I hadn't thought about it, but this makes a lot of sense as the next place of competition where value can be added. It's also where CoreOS is trying to be. It would also suggest that the level above that is abstracting datacenter's away to just manage a single global system made up of multiple DC's. There may be something I'm missing in the Docker vs CoreOS that makes me discount this competition while others are playing it up. Docker and CoreOS aren't strictly competing with each other, CoreOS wanted to use Docker as a commodity component (as it was designed), Docker eventually realised that becoming a commodity businesses wasn't going to be very profitable and moved into CoreOS' territory.

serve_yay 12y ago

Great analysis, thank you.

CoconutPilot 12y ago

I have a different theory. Originally Docker was going to be a container and CoreOS was a platform to run containers. They were both pieces in a traditional software stack. Docker had a round of funding and the investors pushed Docker in a different direction: App store. Docker was no longer just a piece in the stack and CoreOS could see the writing on the wall: libOS. Docker would marginalize CoreOS the same way LXC was marginalized. CoreOS is going on the offensive because they were tipped off on Docker's hand. Occam's Razor.

perlpimp 12y ago
So which platform can one use to deploy docker cloud? We already have a set of CoreOS servers running docker containers... with rocket replacing docker and automagic updates in coreOS this seems kinda dangerous.

gtirloni 12y ago

When working with FreeBSD jails a decade ago, I always thought they were enough to build my servers and needed just more polishing. Then hypervisors caught on and it was the virtualization era. Now soft-virtualization is coming back and it's becoming popular to say "hypervisors are overhead". It's funny how we could have taken a different direction ten years ago but I think I understand. We needed to go through the "VMs will run any of your existing software, don't worry". With mass adoption, new software can start to be written to run inside soft-virtualization (with not that many modifications, if any). If only.. oh well.