6 ms·

My quick guide to pager duty. Step 1: Find a new job. There are way too many opportunities out there to subject yourself to this nonsense. Pager duty is essen

by hiou 10y ago

My quick guide to pager duty. Step 1: Find a new job. There are way too many opportunities out there to subject yourself to this nonsense. Pager duty is essentially wage theft.

CalChris 10y ago
It ranks up there with unpaid internships (which should be illegal).
- protomyth 10y ago
  Unpaid internships are a way for companies / organizations to know that the intern comes from an affluent background because poor people cannot afford to do those types of internships. Good luck getting rid of something that keeps it all in the club.
  - adrianratnapala 10y ago
    Yep. Personally. I'm a free-market type and so I struggle with any moral objection to these internships. But I find it amusing that internships are so popular within institutions (govt, media, big tech) full of people with politics that should find fault with them. By contrast: when I was the work experience boy at an electrical goods maker, I would have accepted no pay (I needed it for Uni). But they made sure to pay me out of a sense of professionalism.
    - protomyth 10y ago
      > Personally. I'm a free-market type and so I struggle with any moral objection to these internships. But I find it amusing that internships are so popular within institutions (govt, media, big tech) full of people with politics that should find fault with them. Same story here. I am truly amazed at the people who talk to me about opportunity and diversity and still have unpaid internships. It is an amazingly effective filter for a certain class of individual. > By contrast: when I was the work experience boy at an electrical goods maker, I would have accepted no pay (I needed it for Uni). But they made sure to pay me out of a sense of professionalism. The vocational fields got it right with paid apprenticeships. Shows the value of your work increasing as your skill and knowledge does.
- biggerfisch 10y ago
  In the US, unpaid internships that provide "immediate advantage from the activities of the intern" _are_ illegal.
Chris2048 10y ago
This only counts if you aren't highly compensated for the unpredictable hours.
AndrewKemendo 10y ago
So who should support emergency issues? Nobody?
- hiou 10y ago
  The same people who do this is other industries. Paid staff scheduled to work those hours. Honest question: You didn't know that?
  - q3k 10y ago
    How does being on pager duty not coincide with being paid to work these hours?
    - CodeMage 10y ago
      There is at least one big company out there, where every dev is part of what they call the "on call rotation" -- where being "on call" typically means being on pager duty -- without any special compensation for that. The only exemption is if you're working on a project that isn't yet in production. They tell you about it during their recruitment process and they communicate it very clearly: take it or leave it, there are no special cases. It sucks, but a lot of people take it anyway.
      - greedo 10y ago
        Yup. We're on call for a week at a time. Only compensation is the "privilege" of working from home on the Friday of the week we're on call. No extra pay, no comp time, nada. Luckily the number of calls is very low, maybe 1-2 per week.
  - AndrewKemendo 10y ago
    Another poster answered the question but your honest question isn't that honest. DevOps is paid for emergency ops - it's part of the job description.
    - 10y ago
jstanley 10y ago
Completely agree. If it's unimportant enough that the company can't afford to pay you a large multiple of your normal salary to work on it out-of-hours, it's unimportant enough that it can wait until you're next in the office.
- AndrewKemendo 10y ago
  Where in the article is it talking about who is taking the call or how much they are paid at all?
  - jstanley 10y ago
    I didn't read the article, I was responding to the comment.
kyrra 10y ago
I think this is because most companies do pager duty wrong. I highly recommend the google SRE book[0] (notes here[1], chapter 11 covers oncall/pager). One thing mentioned in this book is compensation for being oncall. At Google we get fairly decent pay compensation for holding the pager, enough where it can incentivize people to be on the rotation. (I'm a software engineer at google who is oncall at this moment) [0] http://shop.oreilly.com/product/0636920041528.do http://shop.oreilly.com/product/0636920041528.do [1] http://danluu.com/google-sre-book/ http://danluu.com/google-sre-book/
- mikhaill 10y ago
  Thanks for the sources, I'll have to check them out. We've recently spent a considerable amount of time tracking, analyzing and evolving our PagerDuty alerting[0], learnings of which we've shared in a blog post. [0] https://goshippo.com/blog/evolution-our-pagerduty-playbook-fewer-alerts-more-uptime/ https://goshippo.com/blog/evolution-our-pagerduty-playbook-f...
- user5994461 10y ago
  > At Google we get fairly decent pay compensation for holding the pager, enough where it can incentivize people to be on the rotation. Google is a big company. I expect them to 1) Have people in all timezone so that there is no night shift 2) Have many people on rotation so each individual is rarely on shift. That's not comparable to smaller companies.
  - kyrra 10y ago
    Just some anecdotal about my team at Google. I definitely do night shifts (we do 1 week long rotations), but with a 30-minute SLA on responding to pages. I've had bad nights where my first page happens at 11pm, then they keep going until 4am. The better part is that we are oncall once a quarter or so (12 people on the rotation). The thing is, there is basically a waiting list to join the rotation. Compensation is nice for those that are motivated by it. But it also exposes you to a lot of the infrastructure that you normally don't deal with (so it's a great way to learn). We have a higher-up-the-stack SRE team that does 12-hour shifts so it's not as bad for them. They handle the larger scale issues that are beyond job specific issues (ex: datacenter issues). I can understand this sucking if you are on a small engineering team where you can't do things like this (I've got friends at companies that employ < 10 developers, I've heard the stories). I guess I wasn't thinking about it for smaller eng teams where the number of people available to support the product isn't there.
Theodores 10y ago
Last time I was on paid pager duty there was a slim likelihood that there would be any chance of the 3 a.m. call. Therefore we had a strict rota as it would have been unfair if someone was getting all of the free cash. This only amounted to £300 a month due to everyone wanting to be on the rota, however, I am sure that paid for my Christmas by the end of the year. Plus the minor incidents that did happen also turned out to be good team moments, everyone would hear that 'you fixed it'. Okay you did reboot-retry on that file server that wasn't responding and everything was fine five minutes later, but a lot of knowledge went into pressing that reboot button and, as far as manager types are concerned, you saved the day. To their non-technical minds that could be voodoo wizardry so they are pleased, feather in cap given. I also believe that doing some out of hours emergency support is good for one's own education, you are able to think quickly on your feet as it is a heightened urgent situation at hand. Just being placed in this position helps you gain experience of this type of problem solving. Total focus on the task in hand is easily achievable, one is not thinking about lunch or going home and only half focused. Once you have that experience then find a new job!
- shermanyo 10y ago
  > Okay you did reboot-retry on that file server that wasn't responding and everything was fine five minutes later, but a lot of knowledge went into pressing that reboot button and, as far as manager types are concerned, you saved the day. You likely understood the consequences of your action too (and if not, you just learned! :P )
module0000 10y ago
I agree that it is wage theft the majority of the time. I had a previous gig that paid a 20% salary bonus for months you were on pager duty. If the pager went off, you got a 10% weekly bonus that increased with each incident up to 40% maximum. People fought like cats and dogs to get on pager duty at that place...I wish more businesses took that approach.
- wging 10y ago
  That seems like the wrong incentive, if you were also the people responsible for making sure pageable events didn't happen.
Symbiote 10y ago
Step 0: Agree, on the condition that any alert received outside working hours becomes a priority task for someone the next day -- whether that means them fixing a bug, adjusting the alert, or investigating and explaining why the cause is extremely unlikely to recur. This is working well for me, has improved the service for users, and has made our monitoring system much more useful. (Alerts used to be about as accurate as "Main website broken", but are now more like "microservice X is taking >10s to respond".)
hsod 10y ago
I don't see how it's wage theft if you were aware of it when you accepted the job.
codingdave 10y ago
> Pager duty is essentially wage theft. It is only reasonable to hold that stance if it goes both ways. If you consider it wage theft to expect you to work on-call hours, then you also must consider it proper work ethic to always work 40 hours a week, never taking an afternoon to... well, do anything. 40 hours, no more, no less. Don't be late in the morning, either. If that is how you want to work, there are certainly jobs that offer it.

by hiou 10y ago

My quick guide to pager duty. Step 1: Find a new job. There are way too many opportunities out there to subject yourself to this nonsense. Pager duty is essentially wage theft.

CalChris 10y ago

It ranks up there with unpaid internships (which should be illegal).

protomyth 10y ago
Unpaid internships are a way for companies / organizations to know that the intern comes from an affluent background because poor people cannot afford to do those types of internships. Good luck getting rid of something that keeps it all in the club.
- adrianratnapala 10y ago
  Yep. Personally. I'm a free-market type and so I struggle with any moral objection to these internships. But I find it amusing that internships are so popular within institutions (govt, media, big tech) full of people with politics that should find fault with them. By contrast: when I was the work experience boy at an electrical goods maker, I would have accepted no pay (I needed it for Uni). But they made sure to pay me out of a sense of professionalism.
  - protomyth 10y ago
    > Personally. I'm a free-market type and so I struggle with any moral objection to these internships. But I find it amusing that internships are so popular within institutions (govt, media, big tech) full of people with politics that should find fault with them. Same story here. I am truly amazed at the people who talk to me about opportunity and diversity and still have unpaid internships. It is an amazingly effective filter for a certain class of individual. > By contrast: when I was the work experience boy at an electrical goods maker, I would have accepted no pay (I needed it for Uni). But they made sure to pay me out of a sense of professionalism. The vocational fields got it right with paid apprenticeships. Shows the value of your work increasing as your skill and knowledge does.
biggerfisch 10y ago
In the US, unpaid internships that provide "immediate advantage from the activities of the intern" _are_ illegal.

Chris2048 10y ago

This only counts if you aren't highly compensated for the unpredictable hours.

AndrewKemendo 10y ago

So who should support emergency issues? Nobody?

hiou 10y ago
The same people who do this is other industries. Paid staff scheduled to work those hours. Honest question: You didn't know that?
- q3k 10y ago
  How does being on pager duty not coincide with being paid to work these hours?
  - CodeMage 10y ago
    There is at least one big company out there, where every dev is part of what they call the "on call rotation" -- where being "on call" typically means being on pager duty -- without any special compensation for that. The only exemption is if you're working on a project that isn't yet in production. They tell you about it during their recruitment process and they communicate it very clearly: take it or leave it, there are no special cases. It sucks, but a lot of people take it anyway.
    - greedo 10y ago
      Yup. We're on call for a week at a time. Only compensation is the "privilege" of working from home on the Friday of the week we're on call. No extra pay, no comp time, nada. Luckily the number of calls is very low, maybe 1-2 per week.
- AndrewKemendo 10y ago
  Another poster answered the question but your honest question isn't that honest. DevOps is paid for emergency ops - it's part of the job description.
  - 10y ago

jstanley 10y ago

Completely agree. If it's unimportant enough that the company can't afford to pay you a large multiple of your normal salary to work on it out-of-hours, it's unimportant enough that it can wait until you're next in the office.

AndrewKemendo 10y ago
Where in the article is it talking about who is taking the call or how much they are paid at all?
- jstanley 10y ago
  I didn't read the article, I was responding to the comment.

kyrra 10y ago

I think this is because most companies do pager duty wrong. I highly recommend the google SRE book[0] (notes here[1], chapter 11 covers oncall/pager). One thing mentioned in this book is compensation for being oncall. At Google we get fairly decent pay compensation for holding the pager, enough where it can incentivize people to be on the rotation. (I'm a software engineer at google who is oncall at this moment) [0] http://shop.oreilly.com/product/0636920041528.do http://shop.oreilly.com/product/0636920041528.do [1] http://danluu.com/google-sre-book/ http://danluu.com/google-sre-book/

mikhaill 10y ago
Thanks for the sources, I'll have to check them out. We've recently spent a considerable amount of time tracking, analyzing and evolving our PagerDuty alerting[0], learnings of which we've shared in a blog post. [0] https://goshippo.com/blog/evolution-our-pagerduty-playbook-fewer-alerts-more-uptime/ https://goshippo.com/blog/evolution-our-pagerduty-playbook-f...
user5994461 10y ago
> At Google we get fairly decent pay compensation for holding the pager, enough where it can incentivize people to be on the rotation. Google is a big company. I expect them to 1) Have people in all timezone so that there is no night shift 2) Have many people on rotation so each individual is rarely on shift. That's not comparable to smaller companies.
- kyrra 10y ago
  Just some anecdotal about my team at Google. I definitely do night shifts (we do 1 week long rotations), but with a 30-minute SLA on responding to pages. I've had bad nights where my first page happens at 11pm, then they keep going until 4am. The better part is that we are oncall once a quarter or so (12 people on the rotation). The thing is, there is basically a waiting list to join the rotation. Compensation is nice for those that are motivated by it. But it also exposes you to a lot of the infrastructure that you normally don't deal with (so it's a great way to learn). We have a higher-up-the-stack SRE team that does 12-hour shifts so it's not as bad for them. They handle the larger scale issues that are beyond job specific issues (ex: datacenter issues). I can understand this sucking if you are on a small engineering team where you can't do things like this (I've got friends at companies that employ < 10 developers, I've heard the stories). I guess I wasn't thinking about it for smaller eng teams where the number of people available to support the product isn't there.

Theodores 10y ago

Last time I was on paid pager duty there was a slim likelihood that there would be any chance of the 3 a.m. call. Therefore we had a strict rota as it would have been unfair if someone was getting all of the free cash. This only amounted to £300 a month due to everyone wanting to be on the rota, however, I am sure that paid for my Christmas by the end of the year. Plus the minor incidents that did happen also turned out to be good team moments, everyone would hear that 'you fixed it'. Okay you did reboot-retry on that file server that wasn't responding and everything was fine five minutes later, but a lot of knowledge went into pressing that reboot button and, as far as manager types are concerned, you saved the day. To their non-technical minds that could be voodoo wizardry so they are pleased, feather in cap given. I also believe that doing some out of hours emergency support is good for one's own education, you are able to think quickly on your feet as it is a heightened urgent situation at hand. Just being placed in this position helps you gain experience of this type of problem solving. Total focus on the task in hand is easily achievable, one is not thinking about lunch or going home and only half focused. Once you have that experience then find a new job!

shermanyo 10y ago
> Okay you did reboot-retry on that file server that wasn't responding and everything was fine five minutes later, but a lot of knowledge went into pressing that reboot button and, as far as manager types are concerned, you saved the day. You likely understood the consequences of your action too (and if not, you just learned! :P )

module0000 10y ago

I agree that it is wage theft the majority of the time. I had a previous gig that paid a 20% salary bonus for months you were on pager duty. If the pager went off, you got a 10% weekly bonus that increased with each incident up to 40% maximum. People fought like cats and dogs to get on pager duty at that place...I wish more businesses took that approach.

wging 10y ago
That seems like the wrong incentive, if you were also the people responsible for making sure pageable events didn't happen.

Symbiote 10y ago

Step 0: Agree, on the condition that any alert received outside working hours becomes a priority task for someone the next day -- whether that means them fixing a bug, adjusting the alert, or investigating and explaining why the cause is extremely unlikely to recur. This is working well for me, has improved the service for users, and has made our monitoring system much more useful. (Alerts used to be about as accurate as "Main website broken", but are now more like "microservice X is taking >10s to respond".)

hsod 10y ago

I don't see how it's wage theft if you were aware of it when you accepted the job.

codingdave 10y ago

> Pager duty is essentially wage theft. It is only reasonable to hold that stance if it goes both ways. If you consider it wage theft to expect you to work on-call hours, then you also must consider it proper work ethic to always work 40 hours a week, never taking an afternoon to... well, do anything. 40 hours, no more, no less. Don't be late in the morning, either. If that is how you want to work, there are certainly jobs that offer it.