3 ms·

I think it's clear in everyone's minds that right from the start Kubernetes was making Docker a first class citizen right? Borg did not use Docker. Borg may wel

by tombh 5y ago

I think it's clear in everyone's minds that right from the start Kubernetes was making Docker a first class citizen right? Borg did not use Docker. Borg may well be the technical ancestor of Kubernetes, but Kubernetes owes significantly more to the paradigm shift and cultural revolution that Docker brought. Therefore I believe it's more relevant to consider Docker the inspiration and ancestor of Kubernetes.

joshuamorton 5y ago
Borg didn't use docker containers, but does use containers, they're just a form of containers that are propriety and predate docker by a decade (but like they're functionally really similar to docker containers, at least from the perspective of the orchestration layer). So docker is maybe the thing that allowed kubernetes to catch on, but that's just because the external world was catching up to what borg had been doing for more than a decade, and so kubernetes was Google sticking a decade or more of expertise in container orchestration into the OSS world.
- tombh 5y ago
  I just don't think it's fair to define Docker by merely its ability to isolate resources (that which distinguishes containers from VMs). Docker is an entire ecosystem of best practices, standards, tooling, packaging, documentation and community support. Google caught up to Docker not the other way round.
  - joshuamorton 5y ago
    > Docker is an entire ecosystem of best practices, standards, tooling, packaging, documentation and community support. When kubernetes was released, the majority of that ecosystem didn't yet exist. Kubernetes influenced many of those best practices, standards, and tooling. So in shaping the ecosystem, yes, docker was catching up to Google. Like I think this would be a reasonable argument if Kubernetes was released today, but it was released only about a year after Docker was. (heck, iirc, even docker as a tool was originally built on LXC, which I think google functionally upstreamed as part of borg hypervisor stuff, but I could be mistaken there. I do know that LXC was based on Google work though: https://www.nextplatform.com/2016/03/22/decade-container-control-google/ https://www.nextplatform.com/2016/03/22/decade-container-con...) Edit: Ok yeah I think I figured it out, cgroups is the borg upstreamed feature (https://news.ycombinator.com/item?id=9506800 https://news.ycombinator.com/item?id=9506800), LXC and evenutally docker are built atop cgroups, etc. etc. So the entire modern containerization infra is built atop a feature borg upstreamed.
    - tombh 5y ago
      We might be going in circles a bit here. Sorry if I'm misunderstanding something. Docker was released with Dockerfiles, registry support (maybe not Docker Hub, I don't remember), a generous CLI; `start`, `build`, `ps`, etc. So I would very much say the majority of the ecosystem existed before Kubernetes. Yes LXC and cgroups were Google contributions before the existence of Docker. But jails and containers in general were also general ideas before LXC and cgroups, I think most notably in BSD. I feel like we must have had different experiences of that period 8 or so years ago. I distinctly remember the hype and bandwagon feeling of Docker's release. I remember various projects trying to capitalise on Docker to make PaaS-type projects; Swarm, Deis, Flynn (even my own Peas project), before Kubernetes came along. It seems so clear to me that Google were reactive rather than proactive. I really don't think Kubernetes would be a thing without Docker.
      - joshuamorton 5y ago
        I don't really dispute much you say here (although I'd argue that calling Dockers API an "ecosystem" and "best practices" is a stretch, do you write your dockerfiles the same way you did today as in 2013?). Like yes, k8s had to come after docker. What I dispute is how any of that is the spiritual successor of heroku (or GAE or any other PaaS). And even then I think you understimate googles contribution of "we've literally been doing container orchestration for longer than anyone else, here's tooling that is built with lessons we've learned in mind". That's far more than just "money and influence", and I don't think it's "exploitative".
        tombh 5y ago
        That's a very fair criticism of my use of "exploited". And also I accept that "ecosystem" could be a descriptive stretch for Docker's release day. I think in the same way that you're concerned I'm misrepresenting Google, I'm concerned that the documentary misrepresents the landscape of developers and projects that set the scene for Google's Kubernetes. Yes Google made significant contributions to the story that led up to Kubernetes, but crucially, I believe they didn't nurture the culture, they didn't create the mindset that makes developers (and managers) ready to think in a containerised way. And I think that's important because the smaller voices that did make those innovations easily get drowned out by the might of Google.
  - lokar 5y ago
    Not really. It's not obvious from the outside, but from a Google developer POV borg (and workqueue mostly) had everything docker had. It just achieved it differently (monrepo, standard build, custom name service, etc). Google pre-k8s achieved "container" orchestration by controlling the whole stack in detail. K8S provided a way to get most of that with mostly arbitrary code.
    - tombh 5y ago
      You're really missing my point. It's not always enough to create a technically successful project. Docker was a _cultural_ revolution, it had its finger on the pulse, and without that revolution Kubernetes would not have the traction it does.
      - ithkuil 5y ago
        Indeed. The striking difference is that Google's internal culture is (?used to be, at least at the time this documentary takes place) that of: build your static binaries, perhaps bundle a couple of other files in a package (basically a tarball) if you really must, and run in the code in a "container" whose system image is not controlled by the author of the code package but by the place where you deploy the code (the compute infrastructure, ime. the Borg cell). When the admins of the compute infrastructure decide to roll out a new system image they'll let you know and you'll have to coordinate in order to maintain proper hygiene. The public version of this embraced a fundamentally different status quo: software is a mess. People pull in dependency in many different ways. You may end up using some python code that has a C extension that in order to build requires some C library whose installation instructions are a mix of run some install script after having installed some apt package. This idea that you build a container with a container (and accepting that the resulting mess is "ok") is indeed something that has to be granted as a form of innovation to docker. It created a whole new industry of people trying to tame the dependency hell and supply chain woes that have been just swept under a carpet.
throwaway894345 5y ago
Linux containers were important, but Docker gets a lot of credit for popularizing them. Even still, Kubernetes makes containers dramatically more useful than just running a bunch of Docker containers on EC2 instances or whatever. Not only does it schedule containers, but it enables controllers and operators for enabling increasingly higher levels of abstraction. I don’t think Heroku has any analog (nor should it because Heroku is a PaaS).
- tombh 5y ago
  But there were/are many projects that offer those higher levels of abstraction on top of Docker; Deis, Flynn, Rancher, Fleet. I just don't think Kubernetes was that innovative in that regard. It won out because I mean of course it's a fantastic piece of software, that's a given, but the differentiating factor is Google's brand, not any particular technical novelty.

by tombh 5y ago

joshuamorton 5y ago

Borg didn't use docker containers, but does use containers, they're just a form of containers that are propriety and predate docker by a decade (but like they're functionally really similar to docker containers, at least from the perspective of the orchestration layer). So docker is maybe the thing that allowed kubernetes to catch on, but that's just because the external world was catching up to what borg had been doing for more than a decade, and so kubernetes was Google sticking a decade or more of expertise in container orchestration into the OSS world.

tombh 5y ago
I just don't think it's fair to define Docker by merely its ability to isolate resources (that which distinguishes containers from VMs). Docker is an entire ecosystem of best practices, standards, tooling, packaging, documentation and community support. Google caught up to Docker not the other way round.
- joshuamorton 5y ago
  > Docker is an entire ecosystem of best practices, standards, tooling, packaging, documentation and community support. When kubernetes was released, the majority of that ecosystem didn't yet exist. Kubernetes influenced many of those best practices, standards, and tooling. So in shaping the ecosystem, yes, docker was catching up to Google. Like I think this would be a reasonable argument if Kubernetes was released today, but it was released only about a year after Docker was. (heck, iirc, even docker as a tool was originally built on LXC, which I think google functionally upstreamed as part of borg hypervisor stuff, but I could be mistaken there. I do know that LXC was based on Google work though: https://www.nextplatform.com/2016/03/22/decade-container-control-google/ https://www.nextplatform.com/2016/03/22/decade-container-con...) Edit: Ok yeah I think I figured it out, cgroups is the borg upstreamed feature (https://news.ycombinator.com/item?id=9506800 https://news.ycombinator.com/item?id=9506800), LXC and evenutally docker are built atop cgroups, etc. etc. So the entire modern containerization infra is built atop a feature borg upstreamed.
  - tombh 5y ago
    We might be going in circles a bit here. Sorry if I'm misunderstanding something. Docker was released with Dockerfiles, registry support (maybe not Docker Hub, I don't remember), a generous CLI; `start`, `build`, `ps`, etc. So I would very much say the majority of the ecosystem existed before Kubernetes. Yes LXC and cgroups were Google contributions before the existence of Docker. But jails and containers in general were also general ideas before LXC and cgroups, I think most notably in BSD. I feel like we must have had different experiences of that period 8 or so years ago. I distinctly remember the hype and bandwagon feeling of Docker's release. I remember various projects trying to capitalise on Docker to make PaaS-type projects; Swarm, Deis, Flynn (even my own Peas project), before Kubernetes came along. It seems so clear to me that Google were reactive rather than proactive. I really don't think Kubernetes would be a thing without Docker.
    - joshuamorton 5y ago
      I don't really dispute much you say here (although I'd argue that calling Dockers API an "ecosystem" and "best practices" is a stretch, do you write your dockerfiles the same way you did today as in 2013?). Like yes, k8s had to come after docker. What I dispute is how any of that is the spiritual successor of heroku (or GAE or any other PaaS). And even then I think you understimate googles contribution of "we've literally been doing container orchestration for longer than anyone else, here's tooling that is built with lessons we've learned in mind". That's far more than just "money and influence", and I don't think it's "exploitative".
      - tombh 5y ago
        That's a very fair criticism of my use of "exploited". And also I accept that "ecosystem" could be a descriptive stretch for Docker's release day. I think in the same way that you're concerned I'm misrepresenting Google, I'm concerned that the documentary misrepresents the landscape of developers and projects that set the scene for Google's Kubernetes. Yes Google made significant contributions to the story that led up to Kubernetes, but crucially, I believe they didn't nurture the culture, they didn't create the mindset that makes developers (and managers) ready to think in a containerised way. And I think that's important because the smaller voices that did make those innovations easily get drowned out by the might of Google.
- lokar 5y ago
  Not really. It's not obvious from the outside, but from a Google developer POV borg (and workqueue mostly) had everything docker had. It just achieved it differently (monrepo, standard build, custom name service, etc). Google pre-k8s achieved "container" orchestration by controlling the whole stack in detail. K8S provided a way to get most of that with mostly arbitrary code.
  - tombh 5y ago
    You're really missing my point. It's not always enough to create a technically successful project. Docker was a _cultural_ revolution, it had its finger on the pulse, and without that revolution Kubernetes would not have the traction it does.
    - ithkuil 5y ago
      Indeed. The striking difference is that Google's internal culture is (?used to be, at least at the time this documentary takes place) that of: build your static binaries, perhaps bundle a couple of other files in a package (basically a tarball) if you really must, and run in the code in a "container" whose system image is not controlled by the author of the code package but by the place where you deploy the code (the compute infrastructure, ime. the Borg cell). When the admins of the compute infrastructure decide to roll out a new system image they'll let you know and you'll have to coordinate in order to maintain proper hygiene. The public version of this embraced a fundamentally different status quo: software is a mess. People pull in dependency in many different ways. You may end up using some python code that has a C extension that in order to build requires some C library whose installation instructions are a mix of run some install script after having installed some apt package. This idea that you build a container with a container (and accepting that the resulting mess is "ok") is indeed something that has to be granted as a form of innovation to docker. It created a whole new industry of people trying to tame the dependency hell and supply chain woes that have been just swept under a carpet.

throwaway894345 5y ago

Linux containers were important, but Docker gets a lot of credit for popularizing them. Even still, Kubernetes makes containers dramatically more useful than just running a bunch of Docker containers on EC2 instances or whatever. Not only does it schedule containers, but it enables controllers and operators for enabling increasingly higher levels of abstraction. I don’t think Heroku has any analog (nor should it because Heroku is a PaaS).

tombh 5y ago
But there were/are many projects that offer those higher levels of abstraction on top of Docker; Deis, Flynn, Rancher, Fleet. I just don't think Kubernetes was that innovative in that regard. It won out because I mean of course it's a fantastic piece of software, that's a given, but the differentiating factor is Google's brand, not any particular technical novelty.