4 ms·

This is sadly the state of the current mainstream DevOps "movement". It is no longer anything that could be described as a movement, only as an industry, a set

by darkr 7y ago

This is sadly the state of the current mainstream DevOps "movement". It is no longer anything that could be described as a movement, only as an industry, a set of tools, and a thing a few years back that CTOs in laggard companies announce to the board that they need to be doing. It was supposed to be about efficiency and repeatability, and approaching ops with a SWE mindset; but most importantly taking ownership of your shit, end-to-end; which is something that the best and most effective SysAdmins and Software Engineers always did. We don't hire "DevOps Engineers", rather "Platform Engineers", in which we have a hard requirement that you are approaching competence as a software engineer in at least one language, and in at least one paradigm (e.g you should be able to tell me about type systems, data structures, polymorphism, higher order functions, composition vs inheritance, referential transparency, TDD etc). We also expect that all of our backend software engineers deploy and maintain their own infra (as code), using the guardrails/services/systems provided by our platform team. Deploying a new database cluster for a user-facing service is a pull request from a backend engineer, not a Jira ticket for a "DevOps Engineer" There are companies out there "doing it right", but they are in the minority.

_bxg1 7y ago
This is something I've been wondering about. I recently interviewed for a full-stack dev position, and I felt I did quite well on all of the development questions. It wasn't a devops role - they had a whole separate devops position - but I got passed over for another candidate and when I got the news it was suggested that I should "get some devops experience" and maybe try again in the future. I thought that was weird. I know generally what docker/containers do and what purpose things like Kubernetes serve, I'd just never used them. I figured I'd be able to pick up whatever I needed for doing minimal devops tasks in the course of the job. Is it common to expect more than that from "developers"? This was neither a small nor a foolish company.
- darkr 7y ago
  I can’t speak for how common it is, but in my opinion, it is not unreasonable to ask that developers have at least a reasonable understanding of how the systems that they build actually work, throughout at least the majority of the layers of abstraction that they run upon. It is though quite unreasonable to expect experience with specific tools, unless you’ve asked for them on the job spec. The title “full stack engineer” is a whole other can of worms, not a million miles away from this DevOps thread.
  - cmiles74 7y ago
    I agree with brundolf, these are tools and most people can get a handle on how they work in an afternoon or two. We aren't talking about expecting new hires set this infrastructure up from scratch, just use the tools that are already in place.
- wolco 7y ago
  It's a poor excuse if they did not list those skills as required. People need a reason to reject and will use a variety reasons that may not apply or matter for the job you are applying for as long as it sounds good on paper.
- Aeolun 7y ago
  > full-stack dev position How can you be a full-stack dev if you haven’t deployed (and kept running) anything in the past few years? Unless you just kept manually rsync-ing files to a server all that time.
  - sah2ed 7y ago
    You are making a subtle mixup between “I am” vs “I can”. He didn’t write that he is a full stack dev, he wrote that he interviewed for a full stack dev role: > “I recently interviewed for a full-stack dev position, and I felt I did quite well on all of the development questions.”
    - Aeolun 7y ago
      That is a fair point. My main idea was that the questions make sense in context. The ‘you’ in my message should be read as a generic you.
sudosteph 7y ago
This was bound to happen though. Everyone wants an engineer who can do everything competently, but few places are actually willing to compensate to recruit and keep the people who have those skills. Even worse, some of the places that can afford the right people - they have internal politics that prevent these people from being able to drive meaningful change once in the role. Good people will leave environments like that and they'll be left with people who are either under-utilized or who caused internal conflict in the first place. Every executive loves to hear "ownership", "get rid of silos", "speed up release time", "repeatable deployments". They don't like hearing "increase salaries dramatically", "piss off some existing employees", and "hiring is going to be even harder". So what happens instead? Middle-management types decide to train their existing teams to use tools that are associated with "DevOps", update some job titles, and tell the investors and executives they now have a DevOps team. The existing employees are now happy to learn new tools and feel appreciated, the migration to the new tools may or may not solve some old lingering pain points (while also introducing some new, un-predicted pain points that the new DevOps team will happily resolve and write RCAs about, allowing leadership to think that "dev culture" is taking hold). The fact of the matter is, "doing it right" is very expensive and not necessarily going to save every company costs or increase their revenue in the long-run. Sometimes just using better tools and and adopting better processes is enough to see benefits, and that's ok. If running infrastructure is your company's core competency, then it makes sense to invest in extremely skilled people at all levels that touch infra. But those extremely skilled people are expensive, prone to turnover, and tend to be picky about where they work. So I don't should shame companies and engineers for shallow adoption of DevOps tooling and imply they're subverting the DevOps "movement" or whatever. There is room for many roles under the DevOps umbrella, and just because some places aren't immediately restructuring everything, doesn't mean they aren't learning won't contribute back to the greater community at some point in the future.
- yee_hawps 7y ago
  I have been lurking HN for a couple of years, but this comment made me create an account. I am all too familiar with the problems you're stating here. It is quite frustrating, really. People like myself are hired to change the system, destroy silos - then, they (management, generally) see that we're talented at building infrastructure, or some other task that we do, and they throw us into a traditional sysadmin role, and are confused why they can't hold on to a DevOps/SRE/WhateverEngineer for very long. Then they tell their managers that they have/are doing DevOps because they had a guy build some CI/CD pipelines and build out servers, probably manually 1-by-1 because the tools for automating that aren't allowed and the "DevOps" guy doesn't have entitlements to automate it. Not that I'd know anything about that...
carlsborg 7y ago
Werner Vogels has a recent blog post [1] entitled "Modern Applications at AWS" where he says "To succeed in using application development to increase agility and innovation speed, organizations must adopt five elements, in any order: microservices; purpose-built databases; automated software release pipelines; a serverless operational model; and automated, continuous security." I see the devops role at large cos evolving into dev-sec-ops for the "automated, continuous security" tooling and processes. (Besides the usual tooling for delivering and supporting a reliable network service, cloud engineering, system-level issue resolution in dev and test, load and performance and containerized test automation in the pipelines, failover fire-drills, etc.) [1] https://www.allthingsdistributed.com/2019/08/modern-applications-at-aws.html https://www.allthingsdistributed.com/2019/08/modern-applicat...
- Aeolun 7y ago
  > microservices; purpose-built databases; automated software release pipelines; a serverless operational model; and automated, continuous security. Two of those things are absolutely irrelevant to succeed.
  - caramel_ 7y ago
    I’m guessing microservices and serverless? Curious as a developer trying to get a better handle on how systems should be built/designed.
    - Aeolun 7y ago
      Yes, that was my intent anyway. I’m sure different people read that differently.
- iliketocomplain 7y ago
  That's not an article, that's an ad for AWS. As most ads, if taken as advice, it's extremely bad advice.
wpietri 7y ago
I'm 100% in favor of your view of DevOps, and its devolution from movement to dubious agglomeration of vendors and consultants is something I've seen before. Like DevOps, the Agile movement started out as a bunch of smart, dedicated people seeking a new way to work. But once the excitement spread out of that passionate early-adopter group, it changed radically for the worse. I think that's because once you get to mainstream adopters, they're not interested in deep change. They want to keep doing what they're doing, but 10% better. Vendors and consultants retool to serve that market, inevitably watering things down and frequently missing the point entirely. This really frustrated me when it happened in the Agile movement. [1] But I've come to accept that as long as our industry is structured the way it is, it's going to keep happening. It's honestly kind of depressing, but the good news is that anybody willing to build a culture of excellence and put in marginally more work can get much, much better results than their competitors. [1] I wrote more about that here: http://williampietri.com/writing/2011/agiles-second-chasm-and-how-we-fell-in/ http://williampietri.com/writing/2011/agiles-second-chasm-an...
- arwhatever 7y ago
  This linked article was excellent and I encourage anyone here to read it.
james_s_tayler 7y ago
>There are companies out there "doing it right", but they are in the minority Probably true of absolutely everything in the end.
swtrs 7y ago
Funny, our company recently turned our wonderfully positioned Platform Engineers into Dev Ops Engineering concerned almost entirely with Chef.
mukti 7y ago
I'm involved in a few different areas at my company, but when I get involved in hiring DevOps types, I try to look for the person you described as a "Platform Engineer." There are far too many times where I talk to people who know what tools are, but its just a black box that they plug things into, and they only know to use it because that's what someone at their local DevOps meetup group said was cool/useful. I want people who can help devs, not just deploy their code, or give them a system to run on. They shouldn't be afraid of learning a new language (or just looking at it), and should have competency in one already.
Kiro 7y ago
What makes you so sure your company is "doing it right"?

by darkr 7y ago

_bxg1 7y ago

This is something I've been wondering about. I recently interviewed for a full-stack dev position, and I felt I did quite well on all of the development questions. It wasn't a devops role - they had a whole separate devops position - but I got passed over for another candidate and when I got the news it was suggested that I should "get some devops experience" and maybe try again in the future. I thought that was weird. I know generally what docker/containers do and what purpose things like Kubernetes serve, I'd just never used them. I figured I'd be able to pick up whatever I needed for doing minimal devops tasks in the course of the job. Is it common to expect more than that from "developers"? This was neither a small nor a foolish company.

darkr 7y ago
I can’t speak for how common it is, but in my opinion, it is not unreasonable to ask that developers have at least a reasonable understanding of how the systems that they build actually work, throughout at least the majority of the layers of abstraction that they run upon. It is though quite unreasonable to expect experience with specific tools, unless you’ve asked for them on the job spec. The title “full stack engineer” is a whole other can of worms, not a million miles away from this DevOps thread.
- cmiles74 7y ago
  I agree with brundolf, these are tools and most people can get a handle on how they work in an afternoon or two. We aren't talking about expecting new hires set this infrastructure up from scratch, just use the tools that are already in place.
wolco 7y ago
It's a poor excuse if they did not list those skills as required. People need a reason to reject and will use a variety reasons that may not apply or matter for the job you are applying for as long as it sounds good on paper.
Aeolun 7y ago
> full-stack dev position How can you be a full-stack dev if you haven’t deployed (and kept running) anything in the past few years? Unless you just kept manually rsync-ing files to a server all that time.
- sah2ed 7y ago
  You are making a subtle mixup between “I am” vs “I can”. He didn’t write that he is a full stack dev, he wrote that he interviewed for a full stack dev role: > “I recently interviewed for a full-stack dev position, and I felt I did quite well on all of the development questions.”
  - Aeolun 7y ago
    That is a fair point. My main idea was that the questions make sense in context. The ‘you’ in my message should be read as a generic you.

sudosteph 7y ago

This was bound to happen though. Everyone wants an engineer who can do everything competently, but few places are actually willing to compensate to recruit and keep the people who have those skills. Even worse, some of the places that can afford the right people - they have internal politics that prevent these people from being able to drive meaningful change once in the role. Good people will leave environments like that and they'll be left with people who are either under-utilized or who caused internal conflict in the first place. Every executive loves to hear "ownership", "get rid of silos", "speed up release time", "repeatable deployments". They don't like hearing "increase salaries dramatically", "piss off some existing employees", and "hiring is going to be even harder". So what happens instead? Middle-management types decide to train their existing teams to use tools that are associated with "DevOps", update some job titles, and tell the investors and executives they now have a DevOps team. The existing employees are now happy to learn new tools and feel appreciated, the migration to the new tools may or may not solve some old lingering pain points (while also introducing some new, un-predicted pain points that the new DevOps team will happily resolve and write RCAs about, allowing leadership to think that "dev culture" is taking hold). The fact of the matter is, "doing it right" is very expensive and not necessarily going to save every company costs or increase their revenue in the long-run. Sometimes just using better tools and and adopting better processes is enough to see benefits, and that's ok. If running infrastructure is your company's core competency, then it makes sense to invest in extremely skilled people at all levels that touch infra. But those extremely skilled people are expensive, prone to turnover, and tend to be picky about where they work. So I don't should shame companies and engineers for shallow adoption of DevOps tooling and imply they're subverting the DevOps "movement" or whatever. There is room for many roles under the DevOps umbrella, and just because some places aren't immediately restructuring everything, doesn't mean they aren't learning won't contribute back to the greater community at some point in the future.

yee_hawps 7y ago
I have been lurking HN for a couple of years, but this comment made me create an account. I am all too familiar with the problems you're stating here. It is quite frustrating, really. People like myself are hired to change the system, destroy silos - then, they (management, generally) see that we're talented at building infrastructure, or some other task that we do, and they throw us into a traditional sysadmin role, and are confused why they can't hold on to a DevOps/SRE/WhateverEngineer for very long. Then they tell their managers that they have/are doing DevOps because they had a guy build some CI/CD pipelines and build out servers, probably manually 1-by-1 because the tools for automating that aren't allowed and the "DevOps" guy doesn't have entitlements to automate it. Not that I'd know anything about that...

carlsborg 7y ago

Werner Vogels has a recent blog post [1] entitled "Modern Applications at AWS" where he says "To succeed in using application development to increase agility and innovation speed, organizations must adopt five elements, in any order: microservices; purpose-built databases; automated software release pipelines; a serverless operational model; and automated, continuous security." I see the devops role at large cos evolving into dev-sec-ops for the "automated, continuous security" tooling and processes. (Besides the usual tooling for delivering and supporting a reliable network service, cloud engineering, system-level issue resolution in dev and test, load and performance and containerized test automation in the pipelines, failover fire-drills, etc.) [1] https://www.allthingsdistributed.com/2019/08/modern-applications-at-aws.html https://www.allthingsdistributed.com/2019/08/modern-applicat...

Aeolun 7y ago
> microservices; purpose-built databases; automated software release pipelines; a serverless operational model; and automated, continuous security. Two of those things are absolutely irrelevant to succeed.
- caramel_ 7y ago
  I’m guessing microservices and serverless? Curious as a developer trying to get a better handle on how systems should be built/designed.
  - Aeolun 7y ago
    Yes, that was my intent anyway. I’m sure different people read that differently.
iliketocomplain 7y ago
That's not an article, that's an ad for AWS. As most ads, if taken as advice, it's extremely bad advice.

wpietri 7y ago

I'm 100% in favor of your view of DevOps, and its devolution from movement to dubious agglomeration of vendors and consultants is something I've seen before. Like DevOps, the Agile movement started out as a bunch of smart, dedicated people seeking a new way to work. But once the excitement spread out of that passionate early-adopter group, it changed radically for the worse. I think that's because once you get to mainstream adopters, they're not interested in deep change. They want to keep doing what they're doing, but 10% better. Vendors and consultants retool to serve that market, inevitably watering things down and frequently missing the point entirely. This really frustrated me when it happened in the Agile movement. [1] But I've come to accept that as long as our industry is structured the way it is, it's going to keep happening. It's honestly kind of depressing, but the good news is that anybody willing to build a culture of excellence and put in marginally more work can get much, much better results than their competitors. [1] I wrote more about that here: http://williampietri.com/writing/2011/agiles-second-chasm-and-how-we-fell-in/ http://williampietri.com/writing/2011/agiles-second-chasm-an...

arwhatever 7y ago
This linked article was excellent and I encourage anyone here to read it.

james_s_tayler 7y ago

>There are companies out there "doing it right", but they are in the minority Probably true of absolutely everything in the end.

swtrs 7y ago

Funny, our company recently turned our wonderfully positioned Platform Engineers into Dev Ops Engineering concerned almost entirely with Chef.

mukti 7y ago

I'm involved in a few different areas at my company, but when I get involved in hiring DevOps types, I try to look for the person you described as a "Platform Engineer." There are far too many times where I talk to people who know what tools are, but its just a black box that they plug things into, and they only know to use it because that's what someone at their local DevOps meetup group said was cool/useful. I want people who can help devs, not just deploy their code, or give them a system to run on. They shouldn't be afraid of learning a new language (or just looking at it), and should have competency in one already.

Kiro 7y ago

What makes you so sure your company is "doing it right"?