3 ms·

The problem is the devops culture that has burdened development teams with having to juggle a lot of complexity. The solution is having some separation of conce

The problem is the devops culture that has burdened development teams with having to juggle a lot of complexity. The solution is having some separation of concerns. Development teams should not have to spend a lot of time on devops. That's something that should just work that you buy from someone. You pay for the privilege of doing more interesting things. Kubernetes becomes a problem when you have people who are not operations people with many years of experience with this stuff trying to do this while learning how to do it at the same time. The related problem is that having people spend time on this is orders of magnitudes more expensive than it is to run an actual cluster, which is also not cheap. A week of devops time easily equates months/years of cloud hosting time for a modestly sized setup using e.g. Google Cloud Run. And lets face it, it's never just a week. Many teams have full time dev ops people costing 100-200$K/year, each. Great if you are running a business generating millions of revenue. Not so great if you are running a project that has yet to generate a single dollar of revenue and is a long time away from actually getting there. That describes most startups out there. I actually managed to stay below the Cloud Run freemium layer for a while making it close to free. Took me 2 minutes to setup CI/CD. Comes with logging, auto scaling, alerting, etc. Best of all, it freed me up to do more interesting things. Technically I'm using Kubernetes. Except of course I'm not. I spent zero time fiddling with kubernetes specific config. All I did was tell Google Cloud run to go create me a CI/CD pipeline from this git repository and scale it. 3 minute job to click together. Service was up and running right after the build succeeded. Great stuff. That's how devops should be: spend a minimum of time on it in exchange for acceptable results.

mgkimsal 5y ago
> Great if you are running a business generating millions of revenue. It's not even great in that situation. Millions in profit, perhaps, but that $200k+ would probably better be spent elsewhere - enhancing functionality, increasing sales, support, etc.
parasubvert 5y ago
"Development teams should not have to spend a lot of time on devops. That's something that should just work that you buy from someone." This is the fundamental disagreement. DevOps was a reaction to developers that build software that was nearly impossible to operate because they treated Ops as servants that paid to do the dirty work, rather than peers with a set of valuable skills that cover a scope beyond what many Dev teams have. And it was a reaction to Ops being ground down into becoming the "department of no", when really they should be at the table with the development team as a way towards a collaborative reality check. A model where one team gets to completely ignore the complexities of operational reality is a broken, inhumane, and unsustainable model. That said, it's also unsustainable to expose all complexity to dev teams that don't have the skills or incentive to manage this. Progressive disclosure and composable abstractions are the tool to remedy this. Kubernetes was never intended to be exposed directly to app developers, it was a system developer's platform toolkit. Exposing it is misunderstanding + laziness on the part of some operations teams. The intent was always to build higher PaaS-like abstractions such as Knative (which is what Google Cloud Run is based on).
- zdw 5y ago
  Treating ops as a separate janitorial service and how that goes south is nicely captured in this article: https://machinesplusminds.blogspot.com/2012/08/the-carpets-are-so-clean-we-dont-need.html https://machinesplusminds.blogspot.com/2012/08/the-carpets-a...
- midrus 5y ago
  As a frontend developer, I love to run applications in production, being able to get a terminal to my server, setup metrics, and do all these devopsy things. But it is a totally different experience from doing this with Appengine, Heroku, Tsuru, etc... than with a custom in house built kubernetes plus a thousand custom home made tools and 10 different repositories with custom undocumented YAML files and another 3000 "gotchas" of things that don't work yet, we're on it, we need to migrate to the new version,etc. So I symphatize with the parent comment in the sense that, in this custom built mountain of stuff, I don't want to do deveops... if you give me an easy to use, well tested, well documented, stable production infrastructure as the ones I mentioend, then I'm all in. I also agree with you on your last paragraphs about not exposing the raw thing to the developers. This is the key. The problem is when the systems gurus want you to understand to the same level everything they understand, your frontend coworkers want you to be on the latest of every library, your product manager wants you to perfectly understand the product, your manager expect you to be the best at dealing with people, and you still have to smile and be happy about team building... oh, and don't forget the Agile Coach expecting you to also be good at all the team dynamics and card games. I'm all in in operating the applications my team builds. Having to operate custom in house kubernetes clusterfucks is not my job.
  - parasubvert 5y ago
    100%. I spent 5+ years of my life helping cloud foundry take off, and saw the enormous benefits of having your own private Heroku. But the market overwhelmingly decided it wanted to play with a lower level foundation (those CF instances mostly are still chugging along running hundreds of thousands of containers, but they’re in their own world… “legacy”?). Let’s own it and not delude ourselves that the current state of Kubernetes is the end state. It’s like saying the Linux syscall interface is too complex for app developers. Well yes! It’s for system developers. We as an industry are working to improve that.

mgkimsal 5y ago

> Great if you are running a business generating millions of revenue. It's not even great in that situation. Millions in profit, perhaps, but that $200k+ would probably better be spent elsewhere - enhancing functionality, increasing sales, support, etc.

parasubvert 5y ago

"Development teams should not have to spend a lot of time on devops. That's something that should just work that you buy from someone." This is the fundamental disagreement. DevOps was a reaction to developers that build software that was nearly impossible to operate because they treated Ops as servants that paid to do the dirty work, rather than peers with a set of valuable skills that cover a scope beyond what many Dev teams have. And it was a reaction to Ops being ground down into becoming the "department of no", when really they should be at the table with the development team as a way towards a collaborative reality check. A model where one team gets to completely ignore the complexities of operational reality is a broken, inhumane, and unsustainable model. That said, it's also unsustainable to expose all complexity to dev teams that don't have the skills or incentive to manage this. Progressive disclosure and composable abstractions are the tool to remedy this. Kubernetes was never intended to be exposed directly to app developers, it was a system developer's platform toolkit. Exposing it is misunderstanding + laziness on the part of some operations teams. The intent was always to build higher PaaS-like abstractions such as Knative (which is what Google Cloud Run is based on).

zdw 5y ago
Treating ops as a separate janitorial service and how that goes south is nicely captured in this article: https://machinesplusminds.blogspot.com/2012/08/the-carpets-are-so-clean-we-dont-need.html https://machinesplusminds.blogspot.com/2012/08/the-carpets-a...
midrus 5y ago
As a frontend developer, I love to run applications in production, being able to get a terminal to my server, setup metrics, and do all these devopsy things. But it is a totally different experience from doing this with Appengine, Heroku, Tsuru, etc... than with a custom in house built kubernetes plus a thousand custom home made tools and 10 different repositories with custom undocumented YAML files and another 3000 "gotchas" of things that don't work yet, we're on it, we need to migrate to the new version,etc. So I symphatize with the parent comment in the sense that, in this custom built mountain of stuff, I don't want to do deveops... if you give me an easy to use, well tested, well documented, stable production infrastructure as the ones I mentioend, then I'm all in. I also agree with you on your last paragraphs about not exposing the raw thing to the developers. This is the key. The problem is when the systems gurus want you to understand to the same level everything they understand, your frontend coworkers want you to be on the latest of every library, your product manager wants you to perfectly understand the product, your manager expect you to be the best at dealing with people, and you still have to smile and be happy about team building... oh, and don't forget the Agile Coach expecting you to also be good at all the team dynamics and card games. I'm all in in operating the applications my team builds. Having to operate custom in house kubernetes clusterfucks is not my job.
- parasubvert 5y ago
  100%. I spent 5+ years of my life helping cloud foundry take off, and saw the enormous benefits of having your own private Heroku. But the market overwhelmingly decided it wanted to play with a lower level foundation (those CF instances mostly are still chugging along running hundreds of thousands of containers, but they’re in their own world… “legacy”?). Let’s own it and not delude ourselves that the current state of Kubernetes is the end state. It’s like saying the Linux syscall interface is too complex for app developers. Well yes! It’s for system developers. We as an industry are working to improve that.