4 ms·

Depending on on-call crisis management is pretty much the opposite of DevOps. DevOps isn't about being on-call; it's about making on-call unnecessary.

• icedchai 7y ago
  This sounds like pure fantasy. How are you going to run a 24/7 operation if nobody is on call?
  • beat 7y ago
    And what's wrong with fantasy? More correctly, it's something of an unattainable goal. Can we get rid of on-call? Probably not. Can we drastically reduce the frequency and scope of on-call actions? Absolutely! Can we get to where every single time the on-call person has to take action, there's a post-mortem to understand what went wrong, and fix the automation and monitoring so it doesn't happen again?
  • hsk823 7y ago
    Okay, how about making on call routine enough that nothing goes wrong during those on call times effectively making them unncessary. If you are constantly working while off the clock but "on call", there's something wrong with that picture.
  • peterwwillis 7y ago
    Not 'nobody on call', just 'unnecessary on call'. Have you ever gotten paged for a transient error that went away by the time you checked it? Or paged for the same error 5 weeks in a row? These, and really most pages, are fixed by DevOps. In a DevOps system, a feedback loop is used to address all alerts as bugs to be immediately and permanently fixed. After a while, if something breaks, it's because someone just changed something, so it's happening during working hours. Alerts pop up in slack and are acknowledged before a page is sent out. So nobody is getting called. If your infrastructure is ephemeral and managed as code, you use CI/CD to deploy all changes, and you aren't resource-constrained (at this point, cloud-native infra is only constrained by budget) you shouldn't have stuff crashing randomly at 3am, so there should be very few pages.
    • logifail 7y ago
      > you aren't resource-constrained (at this point, cloud-native infra is only constrained by budget) Isn't "budget" just another type of resource?