9 ms·

You don't get anything out of burning out. You just burn out. And that time is not coming back. This post seems like an apologist talking. Production support a

You don't get anything out of burning out. You just burn out. And that time is not coming back. This post seems like an apologist talking. Production support alone is not that much of a problem. What the author skipped (conveniently? or forgot to mention?) is - it's really the "on call" phenomenon that's the problem. The "typical" on-call - where when you are on-call you are magically on-call 24x7. Yes, during your sleeping hours as well; as if that's less important and the company can avoid spending money to hire dedicated support for those hours and instead make you suffer (yes, it's just that - there's no other name for it like "satisfaction", "learning", "growing" or any of those buzzwords). You want engineers to do production support? Well, let them do it during normal office hours and only few times a month. Or heck, let them do it for weeks but let them punch in and punch out normal office hours. Let them choose to do only one half of the day and have someone else willing to do the another half. There's no excuse for burning out engineers (esp. unsuspecting youngsters) by pushing them into ungodly hours of work ruining their health among other things while trying to constantly tell them - "do you even realise what a service to humanity you are doing!". It's just exploitation.

dopylitty 6y ago
The point about on-call is really critical and really on-point. If a company thinks an application is important enough to run 24x7 then it should staff for 24x7 support. Stealing wages from workers by expecting them to be available 24x7 (on-call) is an absolute abuse. It also leads to burn out, poor performance during the day (how is a dev's development ability when they were up at 2:30am on an incident call the night before?), and clouded thinking causing mistakes or impacting recovery time during incidents.
- HenryBemis 6y ago
  The companies do this for the money. And the people what work in those companies have no real sense of the risks, or they just care more for the numbers and want to roll the dice. I would not trust someone who I just woke up at 2am to do something. He/she is mid-sleep. They will be prone to errors, they will be super tired, and I just ruined their next 1.5 days that it will take them to recover from that. This is not a job where you live boxes where intellect is not needed as much, (strength and stamina will also be affected by a mid-night alarm). You want your folks to be 100% on par, otherwise they may make things worse.
  - HenryBemis 6y ago
    Type: live = "lift boxes" (just spotted that, apologies I can't edit and correct)
- MattGaiser 6y ago
  They should also allow time and budget for building an application that can run 24/7 without too many errors. "We need [insert thing manager asks for here] immediately" has consequences.
  - mtberatwork 6y ago
    It doesn't have consequences though, which is why it keeps happening.
- nineteen999 6y ago
  > If a company thinks an application is important enough to run 24x7 then it should staff for 24x7 support And where it really matters, they do. My team and I build and manage a large Emergency Services telecommunication network. We have Tier 1/2 operators on shift work 24/7. Tier 3 staff (programmers, system integrators and administrators) are their escalation point for critical issues outside of business hours. > Stealing wages from workers by expecting them to be available 24x7 (on-call) The Tier 3's that are on-call in our environment are on a rotating roster are compensated nicely for being prepared to answer the phone outside business hours. Frequently they don't get called during their week at all and it's free money. > how is a dev's development ability when they were up at 2:30am on an incident call the night before? Easy, as well as the financial compensation, we give them time in lieu. Two hours callout in the middle of the night, two (paid) hours given back on their next working day, or whenever they prefer, subject to availability of other staff. There are simple solutions to these problems, and where they matter, they are applied. Granted things are very black and white for us as lives are potentially at stake, but any company that wants to have 24/7 engineers available needs to pay for that kind of support.
  - piokoch 6y ago
    "Frequently they don't get called during their week at all and it's free money." That's, I think, is a wrong perspective, when people are on call, they have to be somewhere near their computer/internet connection and be ready to work (so it is not just you can go to a party and if call happens do some quick fix in a toilet). On-calls cannot do what they want with their time, so they don't get money for free.
    - nineteen999 6y ago
      Yeah this is true although our compensation for their time is pretty generous. As a team we are pretty flexible too and if somebody has an event they want to go to they can always find someone else on the team to cover or swap with them for the night.
  - machello13 6y ago
    > Easy, as well as the financial compensation, we give them time in lieu. Two hours callout in the middle of the night, two (paid) hours given back on their next working day, or whenever they prefer, subject to availability of other staff. Every company should do this but none I've worked at do. To be honest, I just take the makeup time myself.
deleted 6y ago
[deleted]
lifeisstillgood 6y ago
The cure - Unionisation seriously
- Galanwe 6y ago
  The cure - proper labor law. Seriously
  - Ma8ee 6y ago
    Which is one of the things that Unions are good at pushing for. Who do you think create laws? A single guy walking up to his Member of Congress.
    - lifeisstillgood 6y ago
      Yeah this is like two hands drawing each other.
    - Galanwe 6y ago
      I guess it depends on everyone's personal experience with unions. From where I come from, unions are mainly a way for lazy employees to get immunity while doing nothing all day long. I would still maintain that if you live in a country where it's legal to be called at any time, any day, then you have a third world class labor law - go downvote US :>
  - dtoma 6y ago
    Historically, I don't think we've made much progress in terms of pro-worker labor laws without said workers first having to get together to create bargaining power. By default the power is in the hands of the ruling class, and they have no interest in handing out paid holidays, reasonable working hours, or sick leave.
    - Galanwe 6y ago
      Well, there is still that old thing called universal election, where basically people can vote for their leader. If other things are considered more valuable than proper labor law (like, say, building a wall), then I guess voters get what they deserved.
m463 6y ago
Various places I've worked put certain people on-call. They were salaried but somehow paid extra for the time they were on-call. It seemed to be voluntary and the people who did it liked the extra cash. On the other hand I had a friend at a very large and well known company. He got a job offer and was hired into one department, but he wanted to take a little time off between jobs before he started. They somehow convinced him to start saying there was a holiday coming up and he could take the time off then. and as soon as he came in he started getting calls 1am 2am 3am etc... so he left. And they cajoled him back saying things were different and he finally bought it and went back. same thing happened again, and he quit for a final time. One part of the problem was that he was a US citizen working with a bunch of H1B visa folks and the company could get away with that sort of stuff. H1B folks will say yes sir, no sir because their dreams of living in the US are tied to keeping their job at all costs. and then the bad work culture festers.
hinkley 6y ago
It’s usually a case if taking responsibility for things you have no power over. When they ask it’s usually after a giant hole has been dug, and patterns have been set. If you knew from the outset that you would be the support team then you’d have prioritized some other tickets. You’d have increased the estimates on others. You would have refused to work on these three, you would have argued vigorously about these four decisions, and you would insisted your boss fire “That Guy” months ago because his code is garbage and his only real skill is articulate deflection. This group of folks wants several somethings for nothing. One of them is labor, another is somewhere to assign blame. They are grooming you for failure and we all deserve better.
hinkley 6y ago
> "do you even realise what a service to humanity you are doing!". I don’t know who needs to hear this besides me twenty years ago, but if you want to do charity then go home at 6pm and volunteer at a real charity. Don’t do it for a wannabe robber baron who will not share with you. Don’t do it for someone where even an emotional payoff is years away or may never come. Find something else you care about and help some people just because. Not because you’re getting under-paid and over-guilted to do it.
- tasogare 6y ago
  Well even "real charity", including very famous ones, are quite shady too: volunteers do work for free while donations goes towards C-exec 5-6 digits salaries.
burnt_husk 6y ago
Having worked in the tech sector for a while now, having burned out once, and having been on-call 24x7 far more than is healthy, I would say exploitation is the name of the game to a lot of managers. You are less of a person and more of a means to an end. A tool to achieve something, and some tools are disposable. It can be of career advantage to a manager to burn out engineers. Maybe instead of spreading 24x7 on-call across 3 teams in three timezones, you put it on 1 team in 1 timezone. By doing so a manager can achieve a lot with less resources, and hopefully secure their own elevation up the corporate ladder before the cost of their strategy becomes evident. The cost of burn out I think remains hidden, in technology there's a constant flux of staff anyway, teams being being created and dissolved, in all the noise a few people being exhausted and bailing from the company is hardly noticed. Perhaps they said something before they left, but it's best for everyone in middle management if the burnt out individual is labeled the problem, they were a bad culture fit you see, a grumbler who didn't have what it took.
danielheath 6y ago
Burning out from on-call comes from not being able to fix the underlying causes. I'm happy to hold the pager if I've also got the right to block/rollback deploys until the system is stable - my current job has had two out-of-hours pages in the past year, and we're in the alexa top 10k so it's not like there's no traffic.
- taylodl 6y ago
  Bingo! There's nothing more motivating to "prioritize" fixing a problem than being woken up at 2 a.m. being affected by it. I've worked in places where one team writes the code and another team supports it and it's always the same: the code is absolute garbage. It ends up being too complex and having way too many moving parts, and is impossible to diagnose. I've worked in places where the team who writes it, supports it - 24x7. Their code is always super simple, easy to diagnose, and easy to maintain. I've been at this for 35 years at many companies and working with many teams and it's always the same: if you want good software then make the team creating it also support it. In every case I've experienced it leads to software requiring little to no support, easy to maintain, and easy to extend. Why? Because nobody wants to get up in the middle of the night or work weekends and moreover, they'd rather be adding features than limping along with existing features.
- mjayhn 6y ago
  I'm in ops and this is a huge, huge headache for those of us on our end at companies where we're not given Google SRE powers to control release trajectory based on failure budgets of some sort. I've worked at too many places that had no SWEs on-call for the on-call alerts that I get, which the vast majority of the time involves throwing a bandaid (as in redirecting traffic, etc) in front of an internal bug that I hope eventually gets fixed once the RFO/etc has been submitted before it hits my NEXT on-call rotation or my poor coworkers. Without SWEs on my rotation they don't understand the immediacy. They aren't the ones getting their Christmas week interrupted every 4 hours while ops keeps the house running. In Ops having your entire day ruined by various on-call alerts usually feels like you're working without any breaks and nobody even cares. Anyone want a bad golang developer, wannabe ex-ops person who knows a lot about platform reliability and o11y and wants to focus on the golang end finally? I'll make your teams automation and o11y purr no matter where it is (bm, cloud, global pops, serverless..)..
afberg 6y ago
I think there's only one way to solve this — which I've been unsuccessfully advocating for at my current company — and that is paid voluntary on call schedules. It creates an actual market for on call work where engineers can simply say no to the extra cash if they don't like work taking up their nights and weekends. If the company is having trouble with no engineers wanting to be on call the pay is simply too low and needs to be increased. It's a job like any other and should be compensated as such. In the end I honestly believe it will be beneficial for the company not having engineers burn out so quickly. Compensation also clearly sets the expectations — if you're being paid to do it you'll take it more seriously. Just my 2 cents
- tybit 6y ago
  I work somewhere that does this and it works surprisingly well. It’s rare to find an employer that’s willing to look after their staff though.
- nimrody 6y ago
  But does it give engineers a good incentive to improve product quality and reduce the number of production incidents? Where I work we are not on-call. Nevertheless, I try to help the ops team when they encounter issues. This does make you improve logging and error handling since you know it takes a lot more time when it's difficult to filter logs for the interesting events. Engineers not exposed to production issues and customers will never understand why you need these extra measures.
  - tjpnz 6y ago
    Our on-call system gives everyone an allowance per service plus additional time off if you're on-call on weekends and public holidays. You get both regardless of whether you get paged. Getting paid to do nothing is a great incentive for pushing quality, especially when you're on-call for more than one service in a given week.
zeckalpha 6y ago
If the engineers aren’t oncall, who is? Is it okay to exploit non-engineers? If anything, it is less exploitative to have those who are empowered to improve their situation oncall.
- detaro 6y ago
  Given that the parent comment is clear about the problem being the 24x7 expectation: Someone paid to work or be available that shift, engineer or not?
  - gertrunde 6y ago
    There are too many variables and factors involved really. Is someone covering the full 24 hours, or is it just out of hours? Are they under a one week in X rota, or expected to be permanently available? Are they expected to cover anything/everything, or are they just the escalation point for their specialist area? What other support is available? (i.e. if the shit really hits the fan, are you left to deal with it alone?) On average, how many times are callouts expected? There's a big difference between half a dozen times a week and half a dozen times a year. How are the extra duties remunerated/compensated? Is there time off in lieu? There's a massive spectrum there ranging from hugely unpleasant and not worth the money to not particularly onerous and helpful extra cash/time off.
  - zeckalpha 6y ago
    No, they said “normal office hours” are for engineers. Someone needs to take the other shifts.
- deleted 6y ago
  [deleted]
- magicalhippo 6y ago
  Our support folks handle the on-call support. They do one week each on rotation, they get some extra compensation and the following friday off. If it's a serious issue they can't handle they might wake up one of us programmers, but usually they can find some temporary fix or workaround until the next morning.
mtberatwork 6y ago
> There's no excuse for burning out engineers (esp. unsuspecting youngsters) by pushing them into ungodly hours of work ruining their health among other things while trying to constantly tell them - "do you even realise what a service to humanity you are doing!". In the US, this is happening across the board and not just in tech. The expectation to always be available [often without monetary compensation] is sadly the new normal. Without strong labor laws in place, this implicit form of exploitation will never cease.
- rdtwo 6y ago
  Everyone is doing it is the excuse
throwaway0a5e 6y ago
I'm not gonna advocate for having engineers directly field customer calls at 3am (because that's a recipe for unhappy customers) but as someone who fields customer support calls/tickets as a large part of my job I feel very comfortable saying that a "sizeable enough to be a big problem" will not build systems that are supportable without engineering escalations unless they get support trying to escalate an issue or conference them in at 3am from time to time.
hanniabu 6y ago
You're talking as if they care about the exploitation. It's done purposely. They simply don't care when it means greater profits. They also believe there's an endless line of devs they can burn out and throw away.
fatnoah 6y ago
>The "typical" on-call - where when you are on-call you are magically on-call 24x7. I ran the Engineering org for a startup and we had a small, 3 person Ops team that handled initial triage of events. About 75% of these issues were Engineering-related. My solution was to a) create an on-call rotation for Engineering and b) allow the Engineers to prioritize reliability work. It sounds like a no-brainer, but I had to fight with the rest of the exec team to allow b) to happen, since it came at the expense of the product roadmap. I eventually won the fight and our nightly on-call volume went from 1-2 incidents per day to 1-2 every few months. Vindication came about a year later when we were acquired by a large company. As part of the due diligence process (including 18 hours with me going over technical details in front of 30 senior folks from the acquiring company) we got major kudos for having a level of reliability that far exceeded what they typically saw for a company our size.

dopylitty 6y ago

The point about on-call is really critical and really on-point. If a company thinks an application is important enough to run 24x7 then it should staff for 24x7 support. Stealing wages from workers by expecting them to be available 24x7 (on-call) is an absolute abuse. It also leads to burn out, poor performance during the day (how is a dev's development ability when they were up at 2:30am on an incident call the night before?), and clouded thinking causing mistakes or impacting recovery time during incidents.

HenryBemis 6y ago
The companies do this for the money. And the people what work in those companies have no real sense of the risks, or they just care more for the numbers and want to roll the dice. I would not trust someone who I just woke up at 2am to do something. He/she is mid-sleep. They will be prone to errors, they will be super tired, and I just ruined their next 1.5 days that it will take them to recover from that. This is not a job where you live boxes where intellect is not needed as much, (strength and stamina will also be affected by a mid-night alarm). You want your folks to be 100% on par, otherwise they may make things worse.
- HenryBemis 6y ago
  Type: live = "lift boxes" (just spotted that, apologies I can't edit and correct)
MattGaiser 6y ago
They should also allow time and budget for building an application that can run 24/7 without too many errors. "We need [insert thing manager asks for here] immediately" has consequences.
- mtberatwork 6y ago
  It doesn't have consequences though, which is why it keeps happening.
nineteen999 6y ago
> If a company thinks an application is important enough to run 24x7 then it should staff for 24x7 support And where it really matters, they do. My team and I build and manage a large Emergency Services telecommunication network. We have Tier 1/2 operators on shift work 24/7. Tier 3 staff (programmers, system integrators and administrators) are their escalation point for critical issues outside of business hours. > Stealing wages from workers by expecting them to be available 24x7 (on-call) The Tier 3's that are on-call in our environment are on a rotating roster are compensated nicely for being prepared to answer the phone outside business hours. Frequently they don't get called during their week at all and it's free money. > how is a dev's development ability when they were up at 2:30am on an incident call the night before? Easy, as well as the financial compensation, we give them time in lieu. Two hours callout in the middle of the night, two (paid) hours given back on their next working day, or whenever they prefer, subject to availability of other staff. There are simple solutions to these problems, and where they matter, they are applied. Granted things are very black and white for us as lives are potentially at stake, but any company that wants to have 24/7 engineers available needs to pay for that kind of support.
- piokoch 6y ago
  "Frequently they don't get called during their week at all and it's free money." That's, I think, is a wrong perspective, when people are on call, they have to be somewhere near their computer/internet connection and be ready to work (so it is not just you can go to a party and if call happens do some quick fix in a toilet). On-calls cannot do what they want with their time, so they don't get money for free.
  - nineteen999 6y ago
    Yeah this is true although our compensation for their time is pretty generous. As a team we are pretty flexible too and if somebody has an event they want to go to they can always find someone else on the team to cover or swap with them for the night.
- machello13 6y ago
  > Easy, as well as the financial compensation, we give them time in lieu. Two hours callout in the middle of the night, two (paid) hours given back on their next working day, or whenever they prefer, subject to availability of other staff. Every company should do this but none I've worked at do. To be honest, I just take the makeup time myself.

deleted 6y ago

[deleted]

lifeisstillgood 6y ago

The cure - Unionisation seriously

Galanwe 6y ago
The cure - proper labor law. Seriously
- Ma8ee 6y ago
  Which is one of the things that Unions are good at pushing for. Who do you think create laws? A single guy walking up to his Member of Congress.
  - lifeisstillgood 6y ago
    Yeah this is like two hands drawing each other.
  - Galanwe 6y ago
    I guess it depends on everyone's personal experience with unions. From where I come from, unions are mainly a way for lazy employees to get immunity while doing nothing all day long. I would still maintain that if you live in a country where it's legal to be called at any time, any day, then you have a third world class labor law - go downvote US :>
- dtoma 6y ago
  Historically, I don't think we've made much progress in terms of pro-worker labor laws without said workers first having to get together to create bargaining power. By default the power is in the hands of the ruling class, and they have no interest in handing out paid holidays, reasonable working hours, or sick leave.
  - Galanwe 6y ago
    Well, there is still that old thing called universal election, where basically people can vote for their leader. If other things are considered more valuable than proper labor law (like, say, building a wall), then I guess voters get what they deserved.

m463 6y ago

Various places I've worked put certain people on-call. They were salaried but somehow paid extra for the time they were on-call. It seemed to be voluntary and the people who did it liked the extra cash. On the other hand I had a friend at a very large and well known company. He got a job offer and was hired into one department, but he wanted to take a little time off between jobs before he started. They somehow convinced him to start saying there was a holiday coming up and he could take the time off then. and as soon as he came in he started getting calls 1am 2am 3am etc... so he left. And they cajoled him back saying things were different and he finally bought it and went back. same thing happened again, and he quit for a final time. One part of the problem was that he was a US citizen working with a bunch of H1B visa folks and the company could get away with that sort of stuff. H1B folks will say yes sir, no sir because their dreams of living in the US are tied to keeping their job at all costs. and then the bad work culture festers.

hinkley 6y ago

It’s usually a case if taking responsibility for things you have no power over. When they ask it’s usually after a giant hole has been dug, and patterns have been set. If you knew from the outset that you would be the support team then you’d have prioritized some other tickets. You’d have increased the estimates on others. You would have refused to work on these three, you would have argued vigorously about these four decisions, and you would insisted your boss fire “That Guy” months ago because his code is garbage and his only real skill is articulate deflection. This group of folks wants several somethings for nothing. One of them is labor, another is somewhere to assign blame. They are grooming you for failure and we all deserve better.

hinkley 6y ago

> "do you even realise what a service to humanity you are doing!". I don’t know who needs to hear this besides me twenty years ago, but if you want to do charity then go home at 6pm and volunteer at a real charity. Don’t do it for a wannabe robber baron who will not share with you. Don’t do it for someone where even an emotional payoff is years away or may never come. Find something else you care about and help some people just because. Not because you’re getting under-paid and over-guilted to do it.

tasogare 6y ago
Well even "real charity", including very famous ones, are quite shady too: volunteers do work for free while donations goes towards C-exec 5-6 digits salaries.

burnt_husk 6y ago

Having worked in the tech sector for a while now, having burned out once, and having been on-call 24x7 far more than is healthy, I would say exploitation is the name of the game to a lot of managers. You are less of a person and more of a means to an end. A tool to achieve something, and some tools are disposable. It can be of career advantage to a manager to burn out engineers. Maybe instead of spreading 24x7 on-call across 3 teams in three timezones, you put it on 1 team in 1 timezone. By doing so a manager can achieve a lot with less resources, and hopefully secure their own elevation up the corporate ladder before the cost of their strategy becomes evident. The cost of burn out I think remains hidden, in technology there's a constant flux of staff anyway, teams being being created and dissolved, in all the noise a few people being exhausted and bailing from the company is hardly noticed. Perhaps they said something before they left, but it's best for everyone in middle management if the burnt out individual is labeled the problem, they were a bad culture fit you see, a grumbler who didn't have what it took.

danielheath 6y ago

Burning out from on-call comes from not being able to fix the underlying causes. I'm happy to hold the pager if I've also got the right to block/rollback deploys until the system is stable - my current job has had two out-of-hours pages in the past year, and we're in the alexa top 10k so it's not like there's no traffic.

taylodl 6y ago
Bingo! There's nothing more motivating to "prioritize" fixing a problem than being woken up at 2 a.m. being affected by it. I've worked in places where one team writes the code and another team supports it and it's always the same: the code is absolute garbage. It ends up being too complex and having way too many moving parts, and is impossible to diagnose. I've worked in places where the team who writes it, supports it - 24x7. Their code is always super simple, easy to diagnose, and easy to maintain. I've been at this for 35 years at many companies and working with many teams and it's always the same: if you want good software then make the team creating it also support it. In every case I've experienced it leads to software requiring little to no support, easy to maintain, and easy to extend. Why? Because nobody wants to get up in the middle of the night or work weekends and moreover, they'd rather be adding features than limping along with existing features.
mjayhn 6y ago
I'm in ops and this is a huge, huge headache for those of us on our end at companies where we're not given Google SRE powers to control release trajectory based on failure budgets of some sort. I've worked at too many places that had no SWEs on-call for the on-call alerts that I get, which the vast majority of the time involves throwing a bandaid (as in redirecting traffic, etc) in front of an internal bug that I hope eventually gets fixed once the RFO/etc has been submitted before it hits my NEXT on-call rotation or my poor coworkers. Without SWEs on my rotation they don't understand the immediacy. They aren't the ones getting their Christmas week interrupted every 4 hours while ops keeps the house running. In Ops having your entire day ruined by various on-call alerts usually feels like you're working without any breaks and nobody even cares. Anyone want a bad golang developer, wannabe ex-ops person who knows a lot about platform reliability and o11y and wants to focus on the golang end finally? I'll make your teams automation and o11y purr no matter where it is (bm, cloud, global pops, serverless..)..

afberg 6y ago

I think there's only one way to solve this — which I've been unsuccessfully advocating for at my current company — and that is paid voluntary on call schedules. It creates an actual market for on call work where engineers can simply say no to the extra cash if they don't like work taking up their nights and weekends. If the company is having trouble with no engineers wanting to be on call the pay is simply too low and needs to be increased. It's a job like any other and should be compensated as such. In the end I honestly believe it will be beneficial for the company not having engineers burn out so quickly. Compensation also clearly sets the expectations — if you're being paid to do it you'll take it more seriously. Just my 2 cents

tybit 6y ago
I work somewhere that does this and it works surprisingly well. It’s rare to find an employer that’s willing to look after their staff though.
nimrody 6y ago
But does it give engineers a good incentive to improve product quality and reduce the number of production incidents? Where I work we are not on-call. Nevertheless, I try to help the ops team when they encounter issues. This does make you improve logging and error handling since you know it takes a lot more time when it's difficult to filter logs for the interesting events. Engineers not exposed to production issues and customers will never understand why you need these extra measures.
- tjpnz 6y ago
  Our on-call system gives everyone an allowance per service plus additional time off if you're on-call on weekends and public holidays. You get both regardless of whether you get paged. Getting paid to do nothing is a great incentive for pushing quality, especially when you're on-call for more than one service in a given week.

zeckalpha 6y ago

If the engineers aren’t oncall, who is? Is it okay to exploit non-engineers? If anything, it is less exploitative to have those who are empowered to improve their situation oncall.

detaro 6y ago
Given that the parent comment is clear about the problem being the 24x7 expectation: Someone paid to work or be available that shift, engineer or not?
- gertrunde 6y ago
  There are too many variables and factors involved really. Is someone covering the full 24 hours, or is it just out of hours? Are they under a one week in X rota, or expected to be permanently available? Are they expected to cover anything/everything, or are they just the escalation point for their specialist area? What other support is available? (i.e. if the shit really hits the fan, are you left to deal with it alone?) On average, how many times are callouts expected? There's a big difference between half a dozen times a week and half a dozen times a year. How are the extra duties remunerated/compensated? Is there time off in lieu? There's a massive spectrum there ranging from hugely unpleasant and not worth the money to not particularly onerous and helpful extra cash/time off.
- zeckalpha 6y ago
  No, they said “normal office hours” are for engineers. Someone needs to take the other shifts.
deleted 6y ago
[deleted]
magicalhippo 6y ago
Our support folks handle the on-call support. They do one week each on rotation, they get some extra compensation and the following friday off. If it's a serious issue they can't handle they might wake up one of us programmers, but usually they can find some temporary fix or workaround until the next morning.

mtberatwork 6y ago

> There's no excuse for burning out engineers (esp. unsuspecting youngsters) by pushing them into ungodly hours of work ruining their health among other things while trying to constantly tell them - "do you even realise what a service to humanity you are doing!". In the US, this is happening across the board and not just in tech. The expectation to always be available [often without monetary compensation] is sadly the new normal. Without strong labor laws in place, this implicit form of exploitation will never cease.

rdtwo 6y ago
Everyone is doing it is the excuse

throwaway0a5e 6y ago

I'm not gonna advocate for having engineers directly field customer calls at 3am (because that's a recipe for unhappy customers) but as someone who fields customer support calls/tickets as a large part of my job I feel very comfortable saying that a "sizeable enough to be a big problem" will not build systems that are supportable without engineering escalations unless they get support trying to escalate an issue or conference them in at 3am from time to time.

hanniabu 6y ago

You're talking as if they care about the exploitation. It's done purposely. They simply don't care when it means greater profits. They also believe there's an endless line of devs they can burn out and throw away.

fatnoah 6y ago

>The "typical" on-call - where when you are on-call you are magically on-call 24x7. I ran the Engineering org for a startup and we had a small, 3 person Ops team that handled initial triage of events. About 75% of these issues were Engineering-related. My solution was to a) create an on-call rotation for Engineering and b) allow the Engineers to prioritize reliability work. It sounds like a no-brainer, but I had to fight with the rest of the exec team to allow b) to happen, since it came at the expense of the product roadmap. I eventually won the fight and our nightly on-call volume went from 1-2 incidents per day to 1-2 every few months. Vindication came about a year later when we were acquired by a large company. As part of the due diligence process (including 18 hours with me going over technical details in front of 30 senior folks from the acquiring company) we got major kudos for having a level of reliability that far exceeded what they typically saw for a company our size.