4 ms·

I don’t think it’s possible anymore to have a nuanced conversation about DevOps or SRE without getting bogged down in the nomenclature. I’ve never had a fun dis

I don’t think it’s possible anymore to have a nuanced conversation about DevOps or SRE without getting bogged down in the nomenclature. I’ve never had a fun discussion about it and never seen anyone change their mind or even just walk away happy. I think the conversation needs to happen component by component, function by function to make progress. Is this issue oncall? Let’s talk about oncall. Is the issue overly complicated deployments? Let’s talk about that instead. I think there is both immeasurable good in having a more complete picture of an application and immeasurable pain in completely throwing away specialization. The happy medium is going to differ for each team and trying to cookie cutter yourself based on some made up, smooshed together word like DevOps or copy what Google did with SRE (which actually varies quite a bit team to team in my experience) is going to end in pain.

fatherzine 4y ago
As you note, fundamentally, "dev" and "ops" require different skillsets. "Dev" experience does not translate to "ops". An experienced "ops" might be able to rootcause an issue by pattern matching its markers to previous experience. A green "dev" is just that, green, and will usually operate with the handicap of a huge learning curve ahead. Sites like https://serverfault.com https://serverfault.com make it a bit easier to poke around, but are not a substitute for hard earned experience. To further the conundrum, the output of an "ops" day's work is a 2 lines change to some otherwise obscure config. Some people will have more "ops" experience than others in a given area, thus the magical ability to come up with said 2 lines change much faster than others. Perversely, this creates tremendous pressure on the "dev"s with less experience, which sadly manifests in overtime and strenuous effort compared to regular "dev" work, with little to show management at the end of the day. Worse, if the fix is subtly broken, there will be additional penalties accrued. Thus "dev"s develop an emotional aversion to anything resembling "ops", and for good measure. In pre Silicon-Valley-gold-rush era, we didn't expect surgeons to jump in and fix the electrical panel, even if the fix is simply to push the 9th button on the circuit breaker back into place.
- tuckerman 4y ago
  I don’t entirely disagree, but we do expect all surgeons to have gone through medical school and have a fairly general understanding of the human body. I think what is being asked of devs and ops folks is actually in between these metaphors. I agree that specialization is an amazing asset when used properly, which often means looking at your exact team composition and leveraging their strengths. Having some ops-minded devs leads to some amazingly scalable and simple systems, and having some very dev-minded ops folks can lead to faster incident resolution or amazing tools. I think what shouldn’t come back is either the complete forgoing of any production responsibilities of a dev team (your software doesn’t work unless it works in prod and is maintainable and operable) or the return of large NOCs that tackle reliability with blood and sweat instead of code. Beyond that, I think it should be case by case.
  - fatherzine 4y ago
    * YES: Having /some/ ops-minded devs leads to some amazingly scalable and simple systems * YES: Having /some/ very dev-minded ops folks can lead to faster incident resolution or amazing tools. IMHO the key is in recognizing "devs" and "ops" are disjunct skillsets, furthermore, as you noted in the first post, each of "dev" and "ops" has further skillset subdivisions. Don't expect all devs to wear all ops hats at once, at a moment's notice. NB, the converse doesn't happen in practice, I have never seen ops being expected to fix subtle bugs in prod code, against the clock. Encourage and recognize specialization, use the team, instead of individuals, to cover width times depth. Keep a eye for people who spread out themselves too thin, aka "heroes". At the very best "heroes" will burn out before time, usually leaving some half-baked tangle in their wake.

fatherzine 4y ago

As you note, fundamentally, "dev" and "ops" require different skillsets. "Dev" experience does not translate to "ops". An experienced "ops" might be able to rootcause an issue by pattern matching its markers to previous experience. A green "dev" is just that, green, and will usually operate with the handicap of a huge learning curve ahead. Sites like https://serverfault.com https://serverfault.com make it a bit easier to poke around, but are not a substitute for hard earned experience. To further the conundrum, the output of an "ops" day's work is a 2 lines change to some otherwise obscure config. Some people will have more "ops" experience than others in a given area, thus the magical ability to come up with said 2 lines change much faster than others. Perversely, this creates tremendous pressure on the "dev"s with less experience, which sadly manifests in overtime and strenuous effort compared to regular "dev" work, with little to show management at the end of the day. Worse, if the fix is subtly broken, there will be additional penalties accrued. Thus "dev"s develop an emotional aversion to anything resembling "ops", and for good measure. In pre Silicon-Valley-gold-rush era, we didn't expect surgeons to jump in and fix the electrical panel, even if the fix is simply to push the 9th button on the circuit breaker back into place.

tuckerman 4y ago
I don’t entirely disagree, but we do expect all surgeons to have gone through medical school and have a fairly general understanding of the human body. I think what is being asked of devs and ops folks is actually in between these metaphors. I agree that specialization is an amazing asset when used properly, which often means looking at your exact team composition and leveraging their strengths. Having some ops-minded devs leads to some amazingly scalable and simple systems, and having some very dev-minded ops folks can lead to faster incident resolution or amazing tools. I think what shouldn’t come back is either the complete forgoing of any production responsibilities of a dev team (your software doesn’t work unless it works in prod and is maintainable and operable) or the return of large NOCs that tackle reliability with blood and sweat instead of code. Beyond that, I think it should be case by case.
- fatherzine 4y ago
  * YES: Having /some/ ops-minded devs leads to some amazingly scalable and simple systems * YES: Having /some/ very dev-minded ops folks can lead to faster incident resolution or amazing tools. IMHO the key is in recognizing "devs" and "ops" are disjunct skillsets, furthermore, as you noted in the first post, each of "dev" and "ops" has further skillset subdivisions. Don't expect all devs to wear all ops hats at once, at a moment's notice. NB, the converse doesn't happen in practice, I have never seen ops being expected to fix subtle bugs in prod code, against the clock. Encourage and recognize specialization, use the team, instead of individuals, to cover width times depth. Keep a eye for people who spread out themselves too thin, aka "heroes". At the very best "heroes" will burn out before time, usually leaving some half-baked tangle in their wake.