12 ms·

I was one of the biggest proponents of DevOps (even started and the DevOps orange county meetup) but ... after working at AWS for 5+ years, my entire perspectiv

I was one of the biggest proponents of DevOps (even started and the DevOps orange county meetup) but ... after working at AWS for 5+ years, my entire perspective shifted. While DevOps does work in some organizations, sometimes what you really need to do is operationalize your software development team. Don't get me wrong, system developers/engineers/operations are needed but for sometimes for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring).

0x457 4y ago
> for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring). But that's literally what DevOps is? DevOps isn't a special DevOps team that writes terraform.
calvinmorrison 4y ago
> development team's operational posture (e.g. run books, alarming, monitoring. Isn't DevOps in large part forcing the developers to have responsibility over the code they push and giving short and frequent pushes to easily rectify problems? Making developers do support work reduces support in the long run because they'll get frustrated and automate solutions.
- stetrain 4y ago
  Yeah, I feel like the term has been taken over to mean using kubernetes and having continuous deployment pipelines for your microservices. My understanding of root of the term is about ownership and responsibility, where the team that develops the app also owns how the app deploys and runs in production, including monitoring and responding to issues. If the team decides that the right way to deploy their app is once every two weeks using a shell script and a zip file, cool. As long as it's documented and monitored so you know if it's working.
  - calvinmorrison 4y ago
    Yeah, what's worked well for me is having ops or security people draw up guidelines (for security, it may be something as trivial as require ssh key auth, for ops it may be: write a script that matches an existing deployment procedure and we can get your app up and running very quickly).
- lazide 4y ago
  And/or become burnt out and develop a severe ‘learned helpness’ complex. It depends on the environment they’re in, and if they have the tools and support to be able to fix issues. Many companies, they don’t.
oneplane 4y ago
There seems to be an increasing disconnect between the marketing value of a term ("DevOps") and what we're actually supposed to be doing, which might best be described as blameless accountability. Even in information security it basically revolves around the same things before you can even do any sensible work and improvements to processes. - Know what you have - Know where it is - Know what it is doing - Know what it is supposed to be doing If you don't have any of those pieces you're just guessing and hoping for the best. For software that might be "know what services you are building, know where you're running them, know how they are doing, know how they are supposed to be doing". But if we call that "DevOps" is suddenly marketable.
- lazide 4y ago
  The issue of course is those things are hard.
  - oneplane 4y ago
    They are only hard if they don't get the same priority and attention you'd see in regulated situations. Replace software or information security with "bars of gold" and suddenly it's not so hard. That's because marketing why you should know the what/where/how/why of your gold to business decision makers is apparently much more palatable than abstract concepts found in IT, even if the value is the same. If you get the resources (including time, money and manpower) do just do it, it can be done. But you usually don't get all three (and maybe not even one), and therefore you're stuck with a task that cannot be executed well.
    - lazide 4y ago
      Did general aviation become cheaper or more available to folks when it got more heavily regulated? Fatality rates dropped, but it basically froze the technology and capability into the late 70’s/early 80’s and locked in just a few companies. The overall number of people working in that industry is tiny now. It’s almost a complete dead end. It’s the classic trade off - cheaper, better, faster. Pick any 2 if you’re lucky. Most only get to pick 1.
plugin-baby 4y ago
> for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring). Oops, I thought that was DevOps.
- fennecfoxen 4y ago
  That's actual-devops. Grandparent poster has worked at firms which have abandoned all its lessons and just use it to mean something like "regular old ops teams, but they use software". So he's using the term like all the big companies do, basically. This is also where you will find things like a "CI/CD" team that doesn't do continuous delivery: they just build Jenkins pipelines, and run them as part of the manual cluster bringup every 2-4 weeks. Or "RESTful" to mean "APIs which use JSON and HTTP verbs" regardless of any hypermedia. And let's not even get started on "agile"...
  - jbverschoor 4y ago
    'word', as they say
  - JamesBarney 4y ago
    Here's a 400 page book and a 12 week training course on all the processes we used to implement Agile's "People over process" principle.
    - Banana699 4y ago
- __MatrixMan__ 4y ago
  Me too. In my experience it went poorly because us devs are now handling our own deployments and building alerting systems and such, but the "devops" people won't give us the level of access necessary to do that with tools designed for the job, so we end up building custom ops-related tooling into the app and writing backdoors that let us use it without having wake the ops folk. When you're accustomed to having to build everything you need, and you're tasked with ops work, you end up reinventing a lot of wheels in a rather weird way and compromising your app in the process. Or at least that's what we did.
dec0dedab0de 4y ago
Which definition of DevOps are you against? and what changed your mind against it?
lowbloodsugar 4y ago
The original definition of DevOps was what you describe in your second paragraph - development teams being responsible for operations. Now DevOps means "We've created a DevOps team! They do DevOps!" Frequently its the managers of old school operation teams changing their names to DevOps teams to get a promotion.
- jen20 4y ago
  It may seem pedantic, but it's an important difference: that's _not_ what the original definition was. Instead, it was a cross-functional team of developers and operators being responsible for a whole system.
  - lowbloodsugar 4y ago
    Yes, I should not have said a "development team", as it makes a distinction between "people who create new software" and "people who run that software in production environment". Your definition is better.
- CoconutPilot 4y ago
  I always liked this early definition of DevOps: "Giving developers operational responsibilities has greatly enhanced the quality of the services, both from a customer and a technology point of view. The traditional model is that you take your software to the wall that separates development and operations, and throw it over and then forget about it. Not at Amazon. You build it, you run it." - Werner Vogels, Amazon CTO (2006) DevOps meant that Developers are also responsible for Operations. But that meaning is lost to time, just like hacking meaning programming.
bot41 4y ago
> the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring). this is devops
NonNefarious 4y ago

0x457 4y ago

> for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring). But that's literally what DevOps is? DevOps isn't a special DevOps team that writes terraform.

calvinmorrison 4y ago

> development team's operational posture (e.g. run books, alarming, monitoring. Isn't DevOps in large part forcing the developers to have responsibility over the code they push and giving short and frequent pushes to easily rectify problems? Making developers do support work reduces support in the long run because they'll get frustrated and automate solutions.

stetrain 4y ago
Yeah, I feel like the term has been taken over to mean using kubernetes and having continuous deployment pipelines for your microservices. My understanding of root of the term is about ownership and responsibility, where the team that develops the app also owns how the app deploys and runs in production, including monitoring and responding to issues. If the team decides that the right way to deploy their app is once every two weeks using a shell script and a zip file, cool. As long as it's documented and monitored so you know if it's working.
- calvinmorrison 4y ago
  Yeah, what's worked well for me is having ops or security people draw up guidelines (for security, it may be something as trivial as require ssh key auth, for ops it may be: write a script that matches an existing deployment procedure and we can get your app up and running very quickly).
lazide 4y ago
And/or become burnt out and develop a severe ‘learned helpness’ complex. It depends on the environment they’re in, and if they have the tools and support to be able to fix issues. Many companies, they don’t.

oneplane 4y ago

There seems to be an increasing disconnect between the marketing value of a term ("DevOps") and what we're actually supposed to be doing, which might best be described as blameless accountability. Even in information security it basically revolves around the same things before you can even do any sensible work and improvements to processes. - Know what you have - Know where it is - Know what it is doing - Know what it is supposed to be doing If you don't have any of those pieces you're just guessing and hoping for the best. For software that might be "know what services you are building, know where you're running them, know how they are doing, know how they are supposed to be doing". But if we call that "DevOps" is suddenly marketable.

lazide 4y ago
The issue of course is those things are hard.
- oneplane 4y ago
  They are only hard if they don't get the same priority and attention you'd see in regulated situations. Replace software or information security with "bars of gold" and suddenly it's not so hard. That's because marketing why you should know the what/where/how/why of your gold to business decision makers is apparently much more palatable than abstract concepts found in IT, even if the value is the same. If you get the resources (including time, money and manpower) do just do it, it can be done. But you usually don't get all three (and maybe not even one), and therefore you're stuck with a task that cannot be executed well.
  - lazide 4y ago
    Did general aviation become cheaper or more available to folks when it got more heavily regulated? Fatality rates dropped, but it basically froze the technology and capability into the late 70’s/early 80’s and locked in just a few companies. The overall number of people working in that industry is tiny now. It’s almost a complete dead end. It’s the classic trade off - cheaper, better, faster. Pick any 2 if you’re lucky. Most only get to pick 1.

plugin-baby 4y ago

> for a small four person start up, the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring). Oops, I thought that was DevOps.

fennecfoxen 4y ago
That's actual-devops. Grandparent poster has worked at firms which have abandoned all its lessons and just use it to mean something like "regular old ops teams, but they use software". So he's using the term like all the big companies do, basically. This is also where you will find things like a "CI/CD" team that doesn't do continuous delivery: they just build Jenkins pipelines, and run them as part of the manual cluster bringup every 2-4 weeks. Or "RESTful" to mean "APIs which use JSON and HTTP verbs" regardless of any hypermedia. And let's not even get started on "agile"...
- jbverschoor 4y ago
  'word', as they say
- JamesBarney 4y ago
  Here's a 400 page book and a 12 week training course on all the processes we used to implement Agile's "People over process" principle.
  - Banana699 4y ago
__MatrixMan__ 4y ago
Me too. In my experience it went poorly because us devs are now handling our own deployments and building alerting systems and such, but the "devops" people won't give us the level of access necessary to do that with tools designed for the job, so we end up building custom ops-related tooling into the app and writing backdoors that let us use it without having wake the ops folk. When you're accustomed to having to build everything you need, and you're tasked with ops work, you end up reinventing a lot of wheels in a rather weird way and compromising your app in the process. Or at least that's what we did.

dec0dedab0de 4y ago

Which definition of DevOps are you against? and what changed your mind against it?

lowbloodsugar 4y ago

The original definition of DevOps was what you describe in your second paragraph - development teams being responsible for operations. Now DevOps means "We've created a DevOps team! They do DevOps!" Frequently its the managers of old school operation teams changing their names to DevOps teams to get a promotion.

jen20 4y ago
It may seem pedantic, but it's an important difference: that's _not_ what the original definition was. Instead, it was a cross-functional team of developers and operators being responsible for a whole system.
- lowbloodsugar 4y ago
  Yes, I should not have said a "development team", as it makes a distinction between "people who create new software" and "people who run that software in production environment". Your definition is better.
CoconutPilot 4y ago
I always liked this early definition of DevOps: "Giving developers operational responsibilities has greatly enhanced the quality of the services, both from a customer and a technology point of view. The traditional model is that you take your software to the wall that separates development and operations, and throw it over and then forget about it. Not at Amazon. You build it, you run it." - Werner Vogels, Amazon CTO (2006) DevOps meant that Developers are also responsible for Operations. But that meaning is lost to time, just like hacking meaning programming.

bot41 4y ago

> the best investment is improving the development team's operational posture (e.g. run books, alarming, monitoring). this is devops

NonNefarious 4y ago