5 ms·

This answer always downplays what "on call" really means. Even if you only get paged once a year, if you're "on call" and the expectation is that within X minut

This answer always downplays what "on call" really means. Even if you only get paged once a year, if you're "on call" and the expectation is that within X minutes you are at a computer working, this affects the life you live. You can't go camping, out to a nice dinner, on vacation, etc without lugging a laptop, internet connection, etc with you. I've heard managers often tout that "if your code is stable you'll never get paged". But that hides the fact that you're still on the hook to be available. So either you start shirking your responsibility or you reshape your life around some percent of non-working hours being owned by the company still.

rednerrus 7y ago
Or you put your laptop in the trunk of your car when you're on call and have tethering on your phone. Expecting another group to maintain your code seems an awful lot like throwing it over the wall.
- matlock 7y ago
  If thats priced into the contract and made clear from the beginning sure. But it doesn't come for free.
- afthonos 7y ago
  This is why SRE organizations typically have stringent requirements for stability before accepting a rotation, and will give it back if reliability drops below a certain level. SREs have global responsibility to the system and have a correspondingly global view. If your system broke down because of a bug you wrote, that’s on you. If a system three times removed from yours that you didn’t know depended on you broke because you changed a non-API behavior, that’s where SREs shine: they know how to quickly isolate the problem, roll back the necessary systems, and define how to avoid similar problems in the future.
- lazyasciiart 7y ago
  And you keep your phone on during a film, and at the nice dinner, and during your daughter's wedding, and never go camping without phone service. That is exactly what is being described by "This answer always downplays what "on call" really means."
  - rednerrus 7y ago
    During your rotation, you have to have your phone with you. I go to the movies and to nice dinners. I would definitely get my rotation covered for a wedding or camping. You should have faith in the systems that you build. If you don't want to carry a pager for them, why does someone else want to do it?
sethammons 7y ago
Not sure what was being downplayed. You are def on call and have n minutes to respond, so you need to stay in range of cell towers or wifi. However, when your system is more reliable, you wake up in the middle of the night less and less. We rotate through on call on our team. One week out of six or so. During that time I keep my laptop and my phone that can tether with me. I've been on call and at amusement parks. I got paged once and had to go out to the car and work an hour. Calculated risk on my part. What I should have done is ask a team mate if they would cover me for the several hours I was at the park.
- viklove 7y ago
  My strategy is to not work places where I need to be on call, and quit if it ever becomes a requirement. I've never been paid enough to give up my free time.
  - pm90 7y ago
    I agree on principle with this strategy. However, I do want to work in places where I am responsible for operating what I build. I want to know how well the service I built is doing. If I promise it will be up 99.99% of the time but is not, then I need to know if its down so I can figured out why it went down. Being on call for systems you build also leads to better software. It makes me design software in ways that are helpful to investigate when it fails. e.g. my error messages provide a lot of context. There is investment in making sure the service is able to terminate gracefully. Metrics are instrumented to locate when things might be off. What this means is that for things that I build, I often don't get paged as often. Its in my interest to ensure that the service is honest, well designed and well built. When I do get paged, its for something that's seriously wrong. Anyways, I've been lucky to be in this position. What I've observed for many teams is that they inherit flaky systems which they have to make more reliable; but in the meanwhile every OC shift is an absolute slog, requiring 40 hours/week dedication by people.
    - horsawlarway 7y ago
      My approach has been to request ownership before agreeing to be on call. If I'm going to be responsible for what I build at all times, then I'm going to be compensated for what it's earning at all times. That means options, stock, or profit sharing (with a large preference for stock). Otherwise I feel like this relationship is strictly abusive. Alternatively, my contracting rate is 150 an hour. I'll agree to work up to 70 hours a week, but everything after 50 is compensated hourly. On call counts, regardless of whether I actually get pinged.
pm90 7y ago
On call is one of the things that nobody told me about honestly. You’re absolutely right that it affects ones life: you need to ensure availability for OC work. Managers who downplay this are liars. That being said: the SRE model has encouraged orgs to improve observability and set clear expectations and it often coaxes teams into building more reliable systems under the threat of limitless OC toil.
POiNTx 7y ago
At my previous company, we could switch our on call day with other people on the team. This mitigated the issue you're talking about. I never had an issue scheduling a switch with someone for the year I was on call there. That's just one specific company though, not sure if other teams allow this.

rednerrus 7y ago

Or you put your laptop in the trunk of your car when you're on call and have tethering on your phone. Expecting another group to maintain your code seems an awful lot like throwing it over the wall.

matlock 7y ago
If thats priced into the contract and made clear from the beginning sure. But it doesn't come for free.
afthonos 7y ago
This is why SRE organizations typically have stringent requirements for stability before accepting a rotation, and will give it back if reliability drops below a certain level. SREs have global responsibility to the system and have a correspondingly global view. If your system broke down because of a bug you wrote, that’s on you. If a system three times removed from yours that you didn’t know depended on you broke because you changed a non-API behavior, that’s where SREs shine: they know how to quickly isolate the problem, roll back the necessary systems, and define how to avoid similar problems in the future.
lazyasciiart 7y ago
And you keep your phone on during a film, and at the nice dinner, and during your daughter's wedding, and never go camping without phone service. That is exactly what is being described by "This answer always downplays what "on call" really means."
- rednerrus 7y ago
  During your rotation, you have to have your phone with you. I go to the movies and to nice dinners. I would definitely get my rotation covered for a wedding or camping. You should have faith in the systems that you build. If you don't want to carry a pager for them, why does someone else want to do it?

sethammons 7y ago

Not sure what was being downplayed. You are def on call and have n minutes to respond, so you need to stay in range of cell towers or wifi. However, when your system is more reliable, you wake up in the middle of the night less and less. We rotate through on call on our team. One week out of six or so. During that time I keep my laptop and my phone that can tether with me. I've been on call and at amusement parks. I got paged once and had to go out to the car and work an hour. Calculated risk on my part. What I should have done is ask a team mate if they would cover me for the several hours I was at the park.

viklove 7y ago
My strategy is to not work places where I need to be on call, and quit if it ever becomes a requirement. I've never been paid enough to give up my free time.
- pm90 7y ago
  I agree on principle with this strategy. However, I do want to work in places where I am responsible for operating what I build. I want to know how well the service I built is doing. If I promise it will be up 99.99% of the time but is not, then I need to know if its down so I can figured out why it went down. Being on call for systems you build also leads to better software. It makes me design software in ways that are helpful to investigate when it fails. e.g. my error messages provide a lot of context. There is investment in making sure the service is able to terminate gracefully. Metrics are instrumented to locate when things might be off. What this means is that for things that I build, I often don't get paged as often. Its in my interest to ensure that the service is honest, well designed and well built. When I do get paged, its for something that's seriously wrong. Anyways, I've been lucky to be in this position. What I've observed for many teams is that they inherit flaky systems which they have to make more reliable; but in the meanwhile every OC shift is an absolute slog, requiring 40 hours/week dedication by people.
  - horsawlarway 7y ago
    My approach has been to request ownership before agreeing to be on call. If I'm going to be responsible for what I build at all times, then I'm going to be compensated for what it's earning at all times. That means options, stock, or profit sharing (with a large preference for stock). Otherwise I feel like this relationship is strictly abusive. Alternatively, my contracting rate is 150 an hour. I'll agree to work up to 70 hours a week, but everything after 50 is compensated hourly. On call counts, regardless of whether I actually get pinged.

pm90 7y ago

On call is one of the things that nobody told me about honestly. You’re absolutely right that it affects ones life: you need to ensure availability for OC work. Managers who downplay this are liars. That being said: the SRE model has encouraged orgs to improve observability and set clear expectations and it often coaxes teams into building more reliable systems under the threat of limitless OC toil.

POiNTx 7y ago

At my previous company, we could switch our on call day with other people on the team. This mitigated the issue you're talking about. I never had an issue scheduling a switch with someone for the year I was on call there. That's just one specific company though, not sure if other teams allow this.