4 ms·

I can troubleshoot during business hours. I can triag during business hours. I can asset tradeoffs and mitigation strategies during business hours. Being wok

I can troubleshoot during business hours. I can triag during business hours. I can asset tradeoffs and mitigation strategies during business hours. Being woken up in the middle of the night never builds empathy and if you pay enough attention to the industry, we're leadning more towards trauma, depression, anxiety and self-diagnosing ADHD. BTW, me being anti-on call is not anti-resiliency... in fact, I don't think on-call is resilient at all with how most organizations operate since the resilience isn't adaptive capacity, but "do your day job and be on call" - no matter HOW much we talk about "your on call week is the week your on call and you focus on that" - it never happens. Never ever.

vitus 5y ago
I get that any one of us can do all of these things during normal business hours (assuming that said situations arise then). But "can" isn't the same as "will". There are plenty of things I'll never get around to if I can't explicitly set aside time for oncall. > Being woken up in the middle of the night never builds empathy and if you pay enough attention to the industry, we're leadning more towards trauma, depression, anxiety and self-diagnosing ADHD. Being woken up in the middle of the night leaves me grumpy and motivates me the following work day to figure out how I can avoid getting woken up again for the same issue. This is somewhat like dogfooding the oncall experience. If I'm trying to troubleshoot something but the logs are unhelpful or the runbook is too vague for my sleep-deprived brain to figure out, I can fix that for next time. If it takes me half an hour to answer a simple question, it's worth considering how to improve our monitoring and dashboards. If I can't follow my own documented process, how can I expect a team who didn't build these systems to do so? As I said, I don't think oncall is for everyone. Oncall as a hidden job requirement is especially problematic, especially if your team / company doesn't expect (and/or mandate!) that your regular project work will be on the back burner during your shifts. Ideally oncall should be opt-in, but there's a balancing act with not having enough people to avoid rapid burnout. Google limits how much you can earn per oncall compensation (for SWEs, 2 weeks per quarter) -- if you're regularly exceeding that limit, then your oncall rotation isn't healthy. There's a lot of horror stories in these comments about oncall rotations, and quite frankly I wouldn't want to join one of those either. ... Another program Google has that's like oncall on steroids: Mission Control [0]. It's an exchange program where SWEs embed themselves in SRE teams, up to and including taking on their pager. It's a fantastic opportunity for gaining a deeper understanding / appreciation of production, but it can potentially be an even greater imposition. If you're an unattached 20-something, you might welcome the opportunity to live in Sydney for 6 months, but if you've got kids, that's not reasonable. (Of course, with the pandemic, relocation to a different country isn't really a thing at this time.) [0] https://cloud.google.com/blog/products/gcp/adventures-in-sre-land-welcome-to-google-mission-control https://cloud.google.com/blog/products/gcp/adventures-in-sre...
- pseudalopex 5y ago
  You can't set aside business hours if you don't set aside weekends and nights? Professionalism, natural empathy, and performance evaluations motivate people enough in my experience. Team exchanges are great. Why would it have to be 6 months abroad though?

vitus 5y ago

I get that any one of us can do all of these things during normal business hours (assuming that said situations arise then). But "can" isn't the same as "will". There are plenty of things I'll never get around to if I can't explicitly set aside time for oncall. > Being woken up in the middle of the night never builds empathy and if you pay enough attention to the industry, we're leadning more towards trauma, depression, anxiety and self-diagnosing ADHD. Being woken up in the middle of the night leaves me grumpy and motivates me the following work day to figure out how I can avoid getting woken up again for the same issue. This is somewhat like dogfooding the oncall experience. If I'm trying to troubleshoot something but the logs are unhelpful or the runbook is too vague for my sleep-deprived brain to figure out, I can fix that for next time. If it takes me half an hour to answer a simple question, it's worth considering how to improve our monitoring and dashboards. If I can't follow my own documented process, how can I expect a team who didn't build these systems to do so? As I said, I don't think oncall is for everyone. Oncall as a hidden job requirement is especially problematic, especially if your team / company doesn't expect (and/or mandate!) that your regular project work will be on the back burner during your shifts. Ideally oncall should be opt-in, but there's a balancing act with not having enough people to avoid rapid burnout. Google limits how much you can earn per oncall compensation (for SWEs, 2 weeks per quarter) -- if you're regularly exceeding that limit, then your oncall rotation isn't healthy. There's a lot of horror stories in these comments about oncall rotations, and quite frankly I wouldn't want to join one of those either. ... Another program Google has that's like oncall on steroids: Mission Control [0]. It's an exchange program where SWEs embed themselves in SRE teams, up to and including taking on their pager. It's a fantastic opportunity for gaining a deeper understanding / appreciation of production, but it can potentially be an even greater imposition. If you're an unattached 20-something, you might welcome the opportunity to live in Sydney for 6 months, but if you've got kids, that's not reasonable. (Of course, with the pandemic, relocation to a different country isn't really a thing at this time.) [0] https://cloud.google.com/blog/products/gcp/adventures-in-sre-land-welcome-to-google-mission-control https://cloud.google.com/blog/products/gcp/adventures-in-sre...

pseudalopex 5y ago
You can't set aside business hours if you don't set aside weekends and nights? Professionalism, natural empathy, and performance evaluations motivate people enough in my experience. Team exchanges are great. Why would it have to be 6 months abroad though?