3 ms·

This article should be called "being on a sucky team sucks". It sounds like OP just has experience with one bad on call. I've also been there. I even heard of

This article should be called "being on a sucky team sucks". It sounds like OP just has experience with one bad on call. I've also been there. I even heard of teams with +100 high severity issues per week. But it doesn't need to be. My current team has the best on call I've ever experienced. It's a mix of a lucky product and some discipline. It isn't rocket science really. If a team is drowning in ops here are two easy techniques I've seen work well. 1. Whoever is on call, their job isn't only to answer pages, but also to improve the system. This solves the dichotomy of feature work vs ops fix. The ops fix is your job for that week. 2. Have team wide (even org wide!) fix-it days, where everyone works exclusively in operational issues. Again, we didn't invent this. Look at Google's SRE books on reducing toil. You can adjust the ratios of feature vs ops work as needed. And if you are in leadership, please acknowledge, celebrate and reward operational work. People tend to work in what they perceive as being valued.

bobkazamakis 4y ago
>1. Whoever is on call, their job isn't only to answer pages, but also to improve the system Way to make being on-call more punishing as well as detract from both goals -- your clients receive worse support and the amount of work done on the fix is inversely proportional to the time you're spending with clients who are calling you because there is a problem, not because they just want to talk. 2 actually works; tech debt hackathons are a great thing, and sometimes ops and dev hands don't communicate as effectively as they need to throughout the SDLC.
- angarg12 4y ago
  The point of 1 is that whoever is oncall shouldn't be working on features/sprint tasks, but actually fixing the underlying problems of the system. This is second to addressing the pages. Of course if your system is so deficient that engineers spend 100% of their time putting out fires, then you need to address that first. That's what reducing toil from the SRE books is about.

bobkazamakis 4y ago

>1. Whoever is on call, their job isn't only to answer pages, but also to improve the system Way to make being on-call more punishing as well as detract from both goals -- your clients receive worse support and the amount of work done on the fix is inversely proportional to the time you're spending with clients who are calling you because there is a problem, not because they just want to talk. 2 actually works; tech debt hackathons are a great thing, and sometimes ops and dev hands don't communicate as effectively as they need to throughout the SDLC.

angarg12 4y ago
The point of 1 is that whoever is oncall shouldn't be working on features/sprint tasks, but actually fixing the underlying problems of the system. This is second to addressing the pages. Of course if your system is so deficient that engineers spend 100% of their time putting out fires, then you need to address that first. That's what reducing toil from the SRE books is about.