4 ms·

This is one of those rare articles about DevOps that isn't bullshit. The irony...! This part, while true, bothers me: "Simple, just hire some more frontend an

This is one of those rare articles about DevOps that isn't bullshit. The irony...! This part, while true, bothers me: "Simple, just hire some more frontend and backend engineers to develop a great internal PaaS with all the golden paths your operations team is architecting while you are trying to build your actual product that runs on top of it." I've been on that team, multiple times. It doesn't really work, because it's yet another silo, and it takes years to bear fruit. We shouldn't be developing platforms, we should be buying them. It's like building and running a factory so you can have an assembly line to make widgets. Most people who want to make widgets today actually contract that out to a company in China who already runs a factory where they adjust their tooling to make your widgets. That's what economies of scale and specialization do: you pay somebody else to do the hard, expensive thing that isn't a core part of your business. You get to market faster, you waste less money, and carry less risk. We're doing business wrong. You should not be building factories, you should be hiring a factory. Do I mean "hire consultants?" No. What we need doesn't seem to exist. We need companies who do white glove integration of a software product with a bespoke PaaS. Like Heroku, but with more attention to architecture, monitoring, alerting, security, ci/cd, etc. Someone who will give you the tools, train your people, do integration alongside them, and provide support. And they could do this on a cloud account that the customer owns, with IAM guardrails to prevent them fucking up the platform. There is no value added to your product by building a factory. You might reduce cost, eventually, if your business is huge enough. But there's a small number of such businesses out there, and yours likely isn't one of them. So let's start some platform companies and make those the default choice.

quadrifoliate 3y ago
> Do I mean "hire consultants?" No. What we need doesn't seem to exist. We need companies who do white glove integration of a software product with a bespoke PaaS. Like Heroku, but with more attention to architecture, monitoring, alerting, security, ci/cd, etc. Someone who will give you the tools, train your people, do integration alongside them, and provide support. And they could do this on a cloud account that the customer owns, with IAM guardrails to prevent them fucking up the platform. I think one of the reasons this doesn't exist is that you are describing every single component of the software development process here. I have a personal theory that software developers have developed severe tunnel vision about code in the last 10-15 years. Before then, most developers had some knowledge about deployment practices, shell scripts, apache, that sort of thing. They roughly understood how something like a web server worked; writing a toy one was in many CS curricula. Whereas now software developers' number 1 skill set seems to be "I know how to solve complicated algorithmic problems". Which is fine and dandy, but it takes so much time to master that it drives out all other deployment basics that many of us acquired while tinkering with software. I am talking bare minimum here, by the way -- I have worked with developers a year or two out of college who did not know what a shell script was because they had spent their entire degree inside either an IDE or Leetcode. This, combined with the fact that complexity has increased exponentially in the operations tooling space means that you sort of need knowledge about operations knowledge in your software engineering team along with complicated algorithmic problems. You could do this by hiring some very expensive consultants like you describe. I think you could also do that by altering your hiring pipeline to de-emphasize algorithmic problem solving, and increase the importance of ops knowledge and testing.
- regularfry 3y ago
  > I have a personal theory that software developers have developed severe tunnel vision about code in the last 10-15 years. Before then, most developers had some knowledge about deployment practices, shell scripts, apache, that sort of thing. They roughly understood how something like a web server worked; writing a toy one was in many CS curricula. You're describing a period (well, not far off - maybe a little earlier) when the norm was to chuck a .war file over the wall to a sysadmin team who'd plug it into tomcat, and who absolutely definitely wouldn't give you access to the server and you had to ask very nicely to even see logs when it broke. The tunnel vision has always been there, as long as the hiring environment has thought that specialism is a good thing. What's needed is for people to do more time in tiny orgs where there's no room not to learn how everything works, and where there's no space for complicated solutions that don't fit in one person's head.
aprdm 3y ago
I think you have a valid point, but what you want to buy doesn’t exist because it’s too complex to be made. Every company has a very different culture and set of tools/skill sets. It can be done for simple web apps — that’s sort of what heroku did and I loved heroku. For a whole company integration that is a bit more complex ? I don’t think so .. maybe one day we will have a tool like sales force and platform engineers will be customizing it instead
- throwawaaarrgh 3y ago
  What about Kubernetes? One single tool stack that does everything, and now everyone uses it, from big to small companies. That is effectively a PaaS. Another one is OpenStack. Did you know there are web hosting providers who will give you your own OpenStack install and charge you based on only what you use? It's a cloud provider, but again, it's one set of tools that everyone can use, regardless of size. All I'm saying is, form a company that specializes in these technologies, add a shit ton of documentation and web UIs, and also actually write some code, tell the customer how to change their app to work better, etc. There are already well known DevOps consulting companies that do the latter, but they are not advertising themselves as a full service platform. I think they should, and I think we should use their platform, rather than build our own per company. Like the tool ecosystems mentioned above, there is room for more standardization, while simultaneously room for people to tell your software devs how to use it right. There's no shame in changing your company's culture (or setting it from the start) to align with DevOps principles. Might as well be the culture and principles of people who know what they're doing :)
  - aprdm 3y ago
    We aren’t talking about PaaS or infrastructure. Kubernetes and infra is a very small part of the puzzle

quadrifoliate 3y ago

> Do I mean "hire consultants?" No. What we need doesn't seem to exist. We need companies who do white glove integration of a software product with a bespoke PaaS. Like Heroku, but with more attention to architecture, monitoring, alerting, security, ci/cd, etc. Someone who will give you the tools, train your people, do integration alongside them, and provide support. And they could do this on a cloud account that the customer owns, with IAM guardrails to prevent them fucking up the platform. I think one of the reasons this doesn't exist is that you are describing every single component of the software development process here. I have a personal theory that software developers have developed severe tunnel vision about code in the last 10-15 years. Before then, most developers had some knowledge about deployment practices, shell scripts, apache, that sort of thing. They roughly understood how something like a web server worked; writing a toy one was in many CS curricula. Whereas now software developers' number 1 skill set seems to be "I know how to solve complicated algorithmic problems". Which is fine and dandy, but it takes so much time to master that it drives out all other deployment basics that many of us acquired while tinkering with software. I am talking bare minimum here, by the way -- I have worked with developers a year or two out of college who did not know what a shell script was because they had spent their entire degree inside either an IDE or Leetcode. This, combined with the fact that complexity has increased exponentially in the operations tooling space means that you sort of need knowledge about operations knowledge in your software engineering team along with complicated algorithmic problems. You could do this by hiring some very expensive consultants like you describe. I think you could also do that by altering your hiring pipeline to de-emphasize algorithmic problem solving, and increase the importance of ops knowledge and testing.

regularfry 3y ago
> I have a personal theory that software developers have developed severe tunnel vision about code in the last 10-15 years. Before then, most developers had some knowledge about deployment practices, shell scripts, apache, that sort of thing. They roughly understood how something like a web server worked; writing a toy one was in many CS curricula. You're describing a period (well, not far off - maybe a little earlier) when the norm was to chuck a .war file over the wall to a sysadmin team who'd plug it into tomcat, and who absolutely definitely wouldn't give you access to the server and you had to ask very nicely to even see logs when it broke. The tunnel vision has always been there, as long as the hiring environment has thought that specialism is a good thing. What's needed is for people to do more time in tiny orgs where there's no room not to learn how everything works, and where there's no space for complicated solutions that don't fit in one person's head.

aprdm 3y ago

I think you have a valid point, but what you want to buy doesn’t exist because it’s too complex to be made. Every company has a very different culture and set of tools/skill sets. It can be done for simple web apps — that’s sort of what heroku did and I loved heroku. For a whole company integration that is a bit more complex ? I don’t think so .. maybe one day we will have a tool like sales force and platform engineers will be customizing it instead

throwawaaarrgh 3y ago
What about Kubernetes? One single tool stack that does everything, and now everyone uses it, from big to small companies. That is effectively a PaaS. Another one is OpenStack. Did you know there are web hosting providers who will give you your own OpenStack install and charge you based on only what you use? It's a cloud provider, but again, it's one set of tools that everyone can use, regardless of size. All I'm saying is, form a company that specializes in these technologies, add a shit ton of documentation and web UIs, and also actually write some code, tell the customer how to change their app to work better, etc. There are already well known DevOps consulting companies that do the latter, but they are not advertising themselves as a full service platform. I think they should, and I think we should use their platform, rather than build our own per company. Like the tool ecosystems mentioned above, there is room for more standardization, while simultaneously room for people to tell your software devs how to use it right. There's no shame in changing your company's culture (or setting it from the start) to align with DevOps principles. Might as well be the culture and principles of people who know what they're doing :)
- aprdm 3y ago
  We aren’t talking about PaaS or infrastructure. Kubernetes and infra is a very small part of the puzzle