5 ms·

I don't know if this is O-Ring theory as much as a "six sigma" style process. The quality of each output depends on the quality of the inputs, and imperfections

by exelius 11y ago

I don't know if this is O-Ring theory as much as a "six sigma" style process. The quality of each output depends on the quality of the inputs, and imperfections grow exponentially when things fall out of tolerances. The whole point of DevOps is really to get your software developers to share in the pain that certain design / development choices can cause. In the absence of some DevOps-like process, dev teams will naturally shovel all their tech debt on the operations team. Rather than build their software to handle multi-tenancy, they'll just say "Eh, deploy another VM with a different configuration". Not that that's necessarily the wrong answer, but because it's less work for them, they don't take into account how hard it will be for the operations team to manage. If the dev team is responsible for both building the software, the tools used to manage the deployment, and responsibility for incident resolution, they'll make decisions that are better for the company and its customers. That's my take, anyway.

stcredzero 11y ago
It's amazing how the logic behind "sharing the pain" is applicable in so many situations. BigCo database admins need to protect the integrity and availability of their data, so they don't mind creating more bureaucracy for programmers. Bureaucrats in general want to cover their butts and simplify their workflows, so they push complexity onto their "customers." I'm sure others can think of many more examples.
- exelius 11y ago
  Well, IMO it's less about bureaucracy than it is saying "just because we've created these artificial divisions between development and ops doesn't mean our customers will be understanding when stuff goes wrong". Operational silos lead to poor customer experiences, and devops is one way to try to break them down.
thwarted 11y ago
The whole point of DevOps is really to get your software developers to share in the pain that certain design / development choices can cause. This is a great characterization and one of the few views of DevOps I can get behind. Unfortunately, in my experience some higher up emits an edict to "do DevOps" and for whatever reason that has fallen on operations (my area) to "implement" (a hazy concept and task) with little buy in from the design/development side; meanwhile, some developers complain that they didn't sign up to "be on call", despite that they are only being tasked with such for their own team's code. In other situations, developers take it upon themselves to deploy stuff without feedback or involvement from the operations experts, and you end up with Shadow IT. It has to be an organization-wide endeavor, and there has to be a culture of involvement of all the areas of expertise.
- brazzledazzle 11y ago
  >one of the few views of DevOps I can get behind. What parts do you have a problem with? If you don't mind me asking. I'm looking at it as a way to change things in my organization and I would like to see some criticism of it beyond the most common one which is basically "my management is a bunch of idiots and lacks reading comprehension".
  - thwarted 11y ago
    Mostly that it is ill-defined and thus open to interpretation-du-jour, although I think we may be close to being past that point as an industry. It's really the same problem that agile had/has, being turned into capital-A Agile. There's not some predefined, packaged set of things you do that suddenly make your organization able to claim "we do DevOps" overnight, but it seems that's what many people want. There's a lot of rhetoric on HN about devops, especially around startups, but in my experience, a small team, or even a larger team at a company that started some time after 2005, has most likely been practicing and doing things like CI, deep monitoring, log-everything, accessible metrics, distributed responsibility, SOA, deploy-on-demand, insert-other-favorite-stereotypical-devopsy-thing, or at least considering how they can leverage even some part of these things in their environment, since day one. And to those organizations there is no other way to do it. The DevOps movement didn't arise to answer the needs of the small organization, but rather the entrenched processes that lead to gross inefficiencies and engineer disempowerment at larger organizations. Small, younger organizations don't have those problems at the scale the larger, more entrenched organizations do. The reason I like The whole point of DevOps is really to get your software developers to share in the pain that certain design / development choices can cause is because it communicates a mindset that you can use to evaluate what you currently do. "Are we using tool X" or "devops is process Y" may be actionable, but don't help you actually work towards alleviating any pain. I've seen "operations" provide tools that were explicitly asked for by developers that then never get used because they don't have the time or inclination to learn how to use them, but because that tool is available they get to say they are part of "the DevOps Movement". More specific examples are more appropriate for a blog post than a HN comment. I think the goal in general is best characterized as getting all the stakeholders involved earlier on at all points in the process and listening to them and incorporating their feedback and insight. One of the harder things there is actually identifying who the actual stakeholders are and affording them the time to actually be involved. It's very easy to, say, keep the DBA out of the loop because you don't think your changes are that involved or won't impact the larger data storage/management infrastructure. Or use a different database technology because you thought it sounded cool but the organization has no usage or operational expertise with that tool and you've listed it as one of your requirements without enough time in your deadline for other people to come up to speed on it. But of course you want to use that tool because...devops! A common refrain I've heard from groups who claim to be dedicated to devops is a long the lines of "Yes, yes, we realize that such and such pain point exists and is causing problems across the organization, but we'll get to that next time. We just need to get this one out of the door, and I promise we'll address the concerns of these other parties for the next project". Really, my problems with "DevOps" is not much different than any random blog post that has articulated the issues with it, and such blog posts are more likely to have articulated it better than I could hope to in a comment thread.
serge2k 11y ago
I still don't buy this justification. Sure, maybe a little. To me it's still mostly just a way to avoid spending money on an operations team.

by exelius 11y ago

stcredzero 11y ago

It's amazing how the logic behind "sharing the pain" is applicable in so many situations. BigCo database admins need to protect the integrity and availability of their data, so they don't mind creating more bureaucracy for programmers. Bureaucrats in general want to cover their butts and simplify their workflows, so they push complexity onto their "customers." I'm sure others can think of many more examples.

exelius 11y ago
Well, IMO it's less about bureaucracy than it is saying "just because we've created these artificial divisions between development and ops doesn't mean our customers will be understanding when stuff goes wrong". Operational silos lead to poor customer experiences, and devops is one way to try to break them down.

thwarted 11y ago

The whole point of DevOps is really to get your software developers to share in the pain that certain design / development choices can cause. This is a great characterization and one of the few views of DevOps I can get behind. Unfortunately, in my experience some higher up emits an edict to "do DevOps" and for whatever reason that has fallen on operations (my area) to "implement" (a hazy concept and task) with little buy in from the design/development side; meanwhile, some developers complain that they didn't sign up to "be on call", despite that they are only being tasked with such for their own team's code. In other situations, developers take it upon themselves to deploy stuff without feedback or involvement from the operations experts, and you end up with Shadow IT. It has to be an organization-wide endeavor, and there has to be a culture of involvement of all the areas of expertise.

brazzledazzle 11y ago
>one of the few views of DevOps I can get behind. What parts do you have a problem with? If you don't mind me asking. I'm looking at it as a way to change things in my organization and I would like to see some criticism of it beyond the most common one which is basically "my management is a bunch of idiots and lacks reading comprehension".
- thwarted 11y ago
  Mostly that it is ill-defined and thus open to interpretation-du-jour, although I think we may be close to being past that point as an industry. It's really the same problem that agile had/has, being turned into capital-A Agile. There's not some predefined, packaged set of things you do that suddenly make your organization able to claim "we do DevOps" overnight, but it seems that's what many people want. There's a lot of rhetoric on HN about devops, especially around startups, but in my experience, a small team, or even a larger team at a company that started some time after 2005, has most likely been practicing and doing things like CI, deep monitoring, log-everything, accessible metrics, distributed responsibility, SOA, deploy-on-demand, insert-other-favorite-stereotypical-devopsy-thing, or at least considering how they can leverage even some part of these things in their environment, since day one. And to those organizations there is no other way to do it. The DevOps movement didn't arise to answer the needs of the small organization, but rather the entrenched processes that lead to gross inefficiencies and engineer disempowerment at larger organizations. Small, younger organizations don't have those problems at the scale the larger, more entrenched organizations do. The reason I like The whole point of DevOps is really to get your software developers to share in the pain that certain design / development choices can cause is because it communicates a mindset that you can use to evaluate what you currently do. "Are we using tool X" or "devops is process Y" may be actionable, but don't help you actually work towards alleviating any pain. I've seen "operations" provide tools that were explicitly asked for by developers that then never get used because they don't have the time or inclination to learn how to use them, but because that tool is available they get to say they are part of "the DevOps Movement". More specific examples are more appropriate for a blog post than a HN comment. I think the goal in general is best characterized as getting all the stakeholders involved earlier on at all points in the process and listening to them and incorporating their feedback and insight. One of the harder things there is actually identifying who the actual stakeholders are and affording them the time to actually be involved. It's very easy to, say, keep the DBA out of the loop because you don't think your changes are that involved or won't impact the larger data storage/management infrastructure. Or use a different database technology because you thought it sounded cool but the organization has no usage or operational expertise with that tool and you've listed it as one of your requirements without enough time in your deadline for other people to come up to speed on it. But of course you want to use that tool because...devops! A common refrain I've heard from groups who claim to be dedicated to devops is a long the lines of "Yes, yes, we realize that such and such pain point exists and is causing problems across the organization, but we'll get to that next time. We just need to get this one out of the door, and I promise we'll address the concerns of these other parties for the next project". Really, my problems with "DevOps" is not much different than any random blog post that has articulated the issues with it, and such blog posts are more likely to have articulated it better than I could hope to in a comment thread.

serge2k 11y ago

I still don't buy this justification. Sure, maybe a little. To me it's still mostly just a way to avoid spending money on an operations team.