4 ms·

Except k8s is really, really not a PaaS. It can be used to build a PaaS in style of Heroku, but it's not one. It's lineage spans all the way back to IBM ASP in

by p_l 5y ago

Except k8s is really, really not a PaaS. It can be used to build a PaaS in style of Heroku, but it's not one. It's lineage spans all the way back to IBM ASP in 1967, arguably even to 7094/7044 directly-coupled system (but that didn't have multiple worker nodes), hitting various systems on the way (TOPS-20 Galaxy, VMScluster, HPC scheduling systems for MPP supercomputers and probably many more I can't recall). It's about efficient use of multiple machines, or as Google & ex-Google employees called it in a book released at similar time as Heroku went from private beta to GA, "Datacenter as a Computer". 12 factors were important, but they just as well can apply to noncontainerised software (arguably, they are just as valid in the ~2000 world of EJB java servers, except replace "environment variables" and the like with JNDI). That said, Rails exploding in popularity around the same time as EC2 became an option heavily helped push the generic case out of the hell of PHP with occasional CGI (ASP usually meant you had more resources than shared hosting from hell)

tombh 5y ago
I think you're missing my point. We're on HN right, where we often discuss the difference between technical merit and market fit. There's a time and a place for technically correct solutions, which Google excel at, but a successful business needs more than that, it needs to capture a zeitgeist, which Google sometimes fail to do, see Google Plus for example. Of course the technical lineage of Kubernetes is Borg and parents (which I admit I've never heard of), but I put it to HN that Kubernetes' lineage is not what created the market fit. In fact I think it's disingenuous of the documentary to imply that (in fairness it does give time to Docker), hence why I'm so keen to give my version of events. Your point about how The 12 Factors apply just as well to non-containerised software is exactly the beauty of them! The 12 Factors don't come from management, they're not an attempt (at least directly) to capture profit from a market. They are somehow fundamental axioms about writing software in this current era, they're maybe even self-evident, universal truths. It turns out that there is a fundamental connection between the ability for a piece of software to be efficiently scaled and for that piece of software to be collaborated on. 2 or more developer machines share essential characteristics with 2 or more scaled containers, which share requirements with 2 or more CI instances ― reproducibility, behaviour guaranteed by tests, ease of downloading, security, configurability through ENV. You see how we felt like Heroku understood us the humble developer, it made us DevOps, all whilst helping us satisfy the bottom line of managers? Kubernetes is truly an extraordinary piece of software, but it hasn't, nor do I believe it ever will, achieve anything of The 12 Factors' lasting legacy.
- p_l 5y ago
  It is, in my opinion, at best converged evolution. Maybe because 12 factors are such self-evident truths - though if true, I'd at best credit them as formulation into a listicle of the fundamental behaviour of JCL batch programs on JES2/3 (don't have enough experience with other systems of that class to compare, but I suspect certain commonality). What my point is that while Heroku might be a very applicable relation to some, it's very explicitly a Platform as a Service, in a way pushing a bit back towards the model that we were escaping from (by invoking Rails name) in mid-2000 to cheap full-OS machines, from the confines "PaaSsy" LAMP. Kubernetes didn't capture a zeitgeist by introducing any of the features (though it definitely pushed containers up into enterprise world a lot). It was in no way first, or even second, despite coming from home of linux containers. It doesn't even force you to use 12 factors - I should know something about it, having done what many called stupid, as in I did "lift and shift" of over 60 "classical" applications that in no way fit 12 factors onto it. Now, trodding a bit further away from concrete facts and into more a world of personal opinion... Kubernetes captured the zeitgeist by working. There was immense desire across all kinds of business, developments, etc. for something that fit nebulous label of "private cloud". Often with local deployments involved. And to do it cheaper (or at least, easier to automate) than existing "heavy" virtualisation stacks. Not everyone could use at all, or just burn money on AWS, many VPS providers were more like classic VMware virtualization clusters in experience. Some had money to burn on Heroku and actually fit running on it (or were lucky to fit in the cheap band of usage). Thus you had OpenStack and the dwarfs: CloudStack, Eucalyptus, OpenNebula, Ganeti, with more adventurous ones building on Mesos or fully custom. All tackling the problem of, essentially, providing simpler and faster (in turnaround time) way of handling users access to resources. The most moneyed and successful (at least from pov of Ops) was OpenStack. And it was a hot mess. Upgrading an openstack cluster was a meme, and not a nice one. Meanwhile containers gain steam, for good or bad, and people start to move from moving from the quite bad "single app VM instances", to containerised approach (12 factors already hit harder due to the single-app VM approach popularised by AWS ASGs coupled with cloud-init). Into this enters kubernetes. Yes, it might seem hard to setup - but it's no OpenStack, it brings no licensing purchasing mess of VMware, it initially kills one of the more annoying aspects (though it will return) which is the overlay networking. It brings unified API, simple operating model (wish I kept the old 1.0-1.4 presentations on hand that went bottom up!). We can deploy stateless services in somewhat sensible way (and if you know how, they can be stateful too). The saner internal design and API commonality can be arguably compared to the effect of the original "heroku" command that allowed you to just deploy from a git repo - just from the point of Ops team. By next year, we move from simple ReplicationController to ReplicaSets + Deployments, as well as gaining Ingress beta and first rush of "pluggable" componentry in the form of Ingress Controllers. It's arguably the time where you can start dealing with K8s as simplistic replacement for Heroku - except it's not some closed magic operated by the Wizard of Oz, but something you can deploy on bunch of Raspberry Pi in middle of presentation at a conference after suffering a long trip (we even forgot, iirc, to bring our planned kube-master machine and had to improvise). Early versions of TPRs (priors of CRDs) make me give what were probably some of the first talks about operators (before CoreOS people coined the term) - but I used the same terminology as K8s used (Controllers). By 2017, the recommended way to deploy openstack starts to be "deploy openstack on top of k8s", and DC/OS starts to visibly lose. K8s makes certain things forced upon you, but you find that it doesn't matter and tends to make sense (Pods having one IP, etc.). Secondary/Tertiary adoption follows, bad memes start about kubernetes (like you only need it for scale, that it's cloud specific, etc. etc.) but it doesn't really matter. Soon, operators start flowing in, thanks to simplicity of the API model (this evolves later to things like Crossplane and even separating kube-apiserver into standalone program, to run without rest of k8s). K8s manifests might feel like JCL, but you don't care because of how bloody more complex the same things used to be before. Heroku might have provided easy way to deploy constrained application for a developer, but, at least in my experience, k8s became the lever with which I could move a lot of servers with little effort, whether to run large batch jobs, complex meshes of microservices, CI build jobs, or build a heroku-style PaaS. As developers learnt to follo 12 factors, the job became easier. P.S. Funnily enough, probably because accidents of history kept me on the Ops side, but IME kubernetes adoption always had push from the sysadmin side, not necessarily developer side. P.P.S. As nice as Heroku is, there's a reason why it's my main example of burning money, especially contrasting how it's billed as "cheaper than paying for an Ops engineer". In the world of global pricing, what works as cheap in Silicon Valley is... not cheap in most of the world. P.P.P.S This kinda went ranty, I guess I should maybe write a blog post on this perspective...

by p_l 5y ago

tombh 5y ago

I think you're missing my point. We're on HN right, where we often discuss the difference between technical merit and market fit. There's a time and a place for technically correct solutions, which Google excel at, but a successful business needs more than that, it needs to capture a zeitgeist, which Google sometimes fail to do, see Google Plus for example. Of course the technical lineage of Kubernetes is Borg and parents (which I admit I've never heard of), but I put it to HN that Kubernetes' lineage is not what created the market fit. In fact I think it's disingenuous of the documentary to imply that (in fairness it does give time to Docker), hence why I'm so keen to give my version of events. Your point about how The 12 Factors apply just as well to non-containerised software is exactly the beauty of them! The 12 Factors don't come from management, they're not an attempt (at least directly) to capture profit from a market. They are somehow fundamental axioms about writing software in this current era, they're maybe even self-evident, universal truths. It turns out that there is a fundamental connection between the ability for a piece of software to be efficiently scaled and for that piece of software to be collaborated on. 2 or more developer machines share essential characteristics with 2 or more scaled containers, which share requirements with 2 or more CI instances ― reproducibility, behaviour guaranteed by tests, ease of downloading, security, configurability through ENV. You see how we felt like Heroku understood us the humble developer, it made us DevOps, all whilst helping us satisfy the bottom line of managers? Kubernetes is truly an extraordinary piece of software, but it hasn't, nor do I believe it ever will, achieve anything of The 12 Factors' lasting legacy.

p_l 5y ago
It is, in my opinion, at best converged evolution. Maybe because 12 factors are such self-evident truths - though if true, I'd at best credit them as formulation into a listicle of the fundamental behaviour of JCL batch programs on JES2/3 (don't have enough experience with other systems of that class to compare, but I suspect certain commonality). What my point is that while Heroku might be a very applicable relation to some, it's very explicitly a Platform as a Service, in a way pushing a bit back towards the model that we were escaping from (by invoking Rails name) in mid-2000 to cheap full-OS machines, from the confines "PaaSsy" LAMP. Kubernetes didn't capture a zeitgeist by introducing any of the features (though it definitely pushed containers up into enterprise world a lot). It was in no way first, or even second, despite coming from home of linux containers. It doesn't even force you to use 12 factors - I should know something about it, having done what many called stupid, as in I did "lift and shift" of over 60 "classical" applications that in no way fit 12 factors onto it. Now, trodding a bit further away from concrete facts and into more a world of personal opinion... Kubernetes captured the zeitgeist by working. There was immense desire across all kinds of business, developments, etc. for something that fit nebulous label of "private cloud". Often with local deployments involved. And to do it cheaper (or at least, easier to automate) than existing "heavy" virtualisation stacks. Not everyone could use at all, or just burn money on AWS, many VPS providers were more like classic VMware virtualization clusters in experience. Some had money to burn on Heroku and actually fit running on it (or were lucky to fit in the cheap band of usage). Thus you had OpenStack and the dwarfs: CloudStack, Eucalyptus, OpenNebula, Ganeti, with more adventurous ones building on Mesos or fully custom. All tackling the problem of, essentially, providing simpler and faster (in turnaround time) way of handling users access to resources. The most moneyed and successful (at least from pov of Ops) was OpenStack. And it was a hot mess. Upgrading an openstack cluster was a meme, and not a nice one. Meanwhile containers gain steam, for good or bad, and people start to move from moving from the quite bad "single app VM instances", to containerised approach (12 factors already hit harder due to the single-app VM approach popularised by AWS ASGs coupled with cloud-init). Into this enters kubernetes. Yes, it might seem hard to setup - but it's no OpenStack, it brings no licensing purchasing mess of VMware, it initially kills one of the more annoying aspects (though it will return) which is the overlay networking. It brings unified API, simple operating model (wish I kept the old 1.0-1.4 presentations on hand that went bottom up!). We can deploy stateless services in somewhat sensible way (and if you know how, they can be stateful too). The saner internal design and API commonality can be arguably compared to the effect of the original "heroku" command that allowed you to just deploy from a git repo - just from the point of Ops team. By next year, we move from simple ReplicationController to ReplicaSets + Deployments, as well as gaining Ingress beta and first rush of "pluggable" componentry in the form of Ingress Controllers. It's arguably the time where you can start dealing with K8s as simplistic replacement for Heroku - except it's not some closed magic operated by the Wizard of Oz, but something you can deploy on bunch of Raspberry Pi in middle of presentation at a conference after suffering a long trip (we even forgot, iirc, to bring our planned kube-master machine and had to improvise). Early versions of TPRs (priors of CRDs) make me give what were probably some of the first talks about operators (before CoreOS people coined the term) - but I used the same terminology as K8s used (Controllers). By 2017, the recommended way to deploy openstack starts to be "deploy openstack on top of k8s", and DC/OS starts to visibly lose. K8s makes certain things forced upon you, but you find that it doesn't matter and tends to make sense (Pods having one IP, etc.). Secondary/Tertiary adoption follows, bad memes start about kubernetes (like you only need it for scale, that it's cloud specific, etc. etc.) but it doesn't really matter. Soon, operators start flowing in, thanks to simplicity of the API model (this evolves later to things like Crossplane and even separating kube-apiserver into standalone program, to run without rest of k8s). K8s manifests might feel like JCL, but you don't care because of how bloody more complex the same things used to be before. Heroku might have provided easy way to deploy constrained application for a developer, but, at least in my experience, k8s became the lever with which I could move a lot of servers with little effort, whether to run large batch jobs, complex meshes of microservices, CI build jobs, or build a heroku-style PaaS. As developers learnt to follo 12 factors, the job became easier. P.S. Funnily enough, probably because accidents of history kept me on the Ops side, but IME kubernetes adoption always had push from the sysadmin side, not necessarily developer side. P.P.S. As nice as Heroku is, there's a reason why it's my main example of burning money, especially contrasting how it's billed as "cheaper than paying for an Ops engineer". In the world of global pricing, what works as cheap in Silicon Valley is... not cheap in most of the world. P.P.P.S This kinda went ranty, I guess I should maybe write a blog post on this perspective...