5 ms·
> Pay. People on call should get paid extra for it I've been on call for 20+ years. I've never gotten paid extra for it. I just figure it's baked into the no
by jedberg 8y ago
> Pay. People on call should get paid extra for it
I've been on call for 20+ years. I've never gotten paid extra for it. I just figure it's baked into the normal paycheck. As long as everyone on the team is doing on call about the same amount, it doesn't really matter. At the end of the year, it usually works out pretty evenly.
> Scheduling. When I have been on call, it has always been one week at a time,
I agree with this one. At Netflix we tried a bunch of different schemes, going from just a few hours on call at a time to a week at a time. The week seemed to work out best for everyone.
> Escalation. There should always be an escalation path if there is a real crisis.
At Netflix, our escalation path was always the on-call engineer, me (the team lead), and then my manager and then their manager. It almost never got past the first engineer, and in the rare cases it did, pretty much everyone on the team was ok with getting a call at any time to help in a crisis, so usually we'd just call whoever would have the most relevant expertise for the current issue. Oftentimes another one of us was already on the call listening anyway. It rarely rolled up to me.
My point being, beyond the rigidity of one person being designated on call, if you really want it to work well you need to be flexible and trust that your team is made of competent people that you can rely on, and they need to be cool with getting a call when they aren't on call, assuming that they might call on you one day.
- Neil44 8y agoIf the on call hours were part of your initial contract and renumeration then that's OK. Otherwise they're changing your terms of employment in their favour, taking more of your life than they're paying you for.
- lawnchair_larry 8y agoTell me more about this “contract” you speak of. We must work in very different industries if you get anything remotely like what you describe.
- pjc50 8y agoIf you're not getting a written contract you're not working in the industry, you're the victim of a fraud.
- vinceguidry 8y agoIn the US we call employment contracts "employment agreements", and we don't tend to think of them as contracts, even though they're legally enforceable as contracts. One of the weird things about living here.
- cableshaft 8y agoThey are absolutely contracts and people who don't treat them as such are just asking for trouble. I don't care what an employer tries to call it. Anything that has legal language and they're expecting me to sign my name on it is a legal contract and I treat it as such.
- logfromblammo 8y agoEvery such paper I have received has explicitly noted the following, paraphrased: - This is not a contract; any contract with us must be signed by the CEO. (Paper is not signed by the CEO.) - You are an "at will" employee. The employment relationship may be ended at any time, by any party, for any reason, or no reason at all. There are no notice requirements, and any and all obligations of one party to the other are severed at the moment of separation. - We may change the terms and conditions of your employment at any time. If you don't like it, you are free to leave. As employment "contracts" go, these were slightly less useful to me than a roll of toilet paper.
- cableshaft 8y agoI guess that makes sense. I've worked for a lot of small companies, though, so most of these I've signed have also been signed by the CEO. I wouldn't be surprised if they tried to treat it like a contract if they wanted to use it against you in court, though.
- gaius 8y agoalways the on-call engineer, me (the team lead), and then my manager and then their manager. The best scheme in my experience is to have a 24/7 fully staffed L1 working shifts for initial response triage then waking the appropriate L2 to deal with the actual problem if it isn’t covered in the runbook. No good waking the DBA if you can’t login to the database because a router has crashed and failed to failover. The next morning the L2 guy updates the runbook so if possible if the same thing recurs L1 can just handle it. What really doesn’t work is having the on-call support also doing out-of-hours work such as maintenance or whatever. That’s a recipe for disaster. People willing to make the sacrifice of on-call are a precious commodity to be used wisely, or they’ll walk. Paying only in the event of a callout doesn’t work either, that person has still had to decline other activities in order to make themselves available.
- jedberg 8y agoIf you have 24/7 coverage using run books, perhaps a better system would be to take the money you're spending on the 5ish people you need to do 24/7 coverage, and instead pay two people twice as much to codify everything in the run book so that it happens automatically. Then have the alerts go straight to the L2.
- gaius 8y agoThe infra for that job had far too many moving parts to make that feasible unfortunately. Multiple fibre providers, satellites, microwave relays, etc etc. The L1 guys would be on the phone alot in the event of a serious incident. Even today there still needs to be humans in the loop, I don’t see that changing.
- wikibob 8y agoGoogle pays people, both in cash, and/or in compensatory time off. This is specifically called out in the SRE book [0]. They've noted that it's important to pay compensation, both to be fair to the employees, and as a closed-loop feedback mechanism to ensure the business prioritizes fixing pages. This concept of business feedback is also discussed in a chapter of the terrific Seeking SRE book, chapter "Against On-Call: A Polemic" [1]. > Compensation Adequate compensation needs to be considered for out-of-hours support. Different organizations handle on-call compensation in different ways; > Google offers time-off-in-lieu or straight cash compensation, capped at some proportion of overall salary. > The compensation cap represents, in practice, a limit on the amount of on-call work that will be taken on by any individual. > This compensation structure ensures incentivization to be involved in on-call duties as required by the team, but also promotes a balanced on-call work distribution and limits potential drawbacks of excessive on-call work, such as burnout or inadequate time for project work. [0] https://landing.google.com/sre/sre-book/chapters/being-on-call/ https://landing.google.com/sre/sre-book/chapters/being-on-ca... (search Compensation) [1] http://shop.oreilly.com/product/0636920063964.do http://shop.oreilly.com/product/0636920063964.do
- deleted 8y ago[deleted]
- purple_ducks 8y ago> I've been on call for 20+ years. I've never gotten paid extra for it. I just figure it's baked into the normal paycheck So much for 40 hour work week. People would be spinning in their graves if they knew what had become of employer-employee relationship.
- eertami 8y agoEven 40 sounds too high to be honest, I've only been in the industry 4 years but already refuse anything more than 30hrs.
- illiac_1962 8y agoSame here. Thirty hours max.
- michaelt 8y agothey need to be cool with getting a call when they aren't on call This is all a question of how often it happens. If I'm getting an emergency call when I'm not on call twice a year, well, these things happen. If I'm getting such a call 12 times a year, I gotta wonder if there's a problem with our testing practices, our prioritisation/tech debt, our L1 support, our high-level architecture, our infrastructure, the teams that interface with our system, our training practices or our hiring standards. Some of those will be within my power to fix. Others, maybe I figure it's easier for me to move companies.
- rconti 8y agoI've been part of a formal on-call rotation in Ops departments at 3 companies, and only one of them actually compensated us specifically for it, and that was back in the 90s. I like the idea of compensating by the week; seems like compensating per incident gives a bit of a perverse incentive.