4 ms·

I know it's not a popular idea, but here it goes anyway: senior software engineers, please say NO to being on call. It just puts some much pressure on engineers

I know it's not a popular idea, but here it goes anyway: senior software engineers, please say NO to being on call. It just puts some much pressure on engineers (like me) who don't want to be on call in exchange of more money: managers then think of you as someone who is not very commited to the company, bla bla bla... I'm commited to the company I work at any given time, from 9am to 5pm. That's all the time a company can get from me.

leokennis 6y ago
In my opinion, it’s almost mandatory for a well functioning software development environment to have developers on call (next to ops people). Bugs that get you woken up in the middle of the night are fixed in no time. Documentation that is lacking, or spaghetti code, both preventing you from quickly fixing an issue instead of spending hours on it? Fixed the next day. If the stakes of writing bad code are low, code quality will suffer. No better fix for those low stakes than being on call.
- sdevonoes 6y ago
  > No better fix for those low stakes than being on call. I prefer a company with a strong tech culture: code review, thinkg long-term (marathons instead of sprints) and design systems with architecture-first on mind (instead of feature-first), automated testing, QA people, don't deploy on Fridays...
  - pc86 6y ago
    If you've got testing, code reviews, QA, and CI/CD, Friday deployments should be totally fine.
    - sdevonoes 6y ago
      The point is "Friday deployments" are not needed (unless we are talking about hotfixes). What's the use case? Can't it wait until Monday? Why are all in such a rush?
      - pc86 6y ago
        I would say that if things are tested appropriately (it's a big if, and not an easy thing to do honestly), when you commit a new feature it should go through the testing process and right into production. If that makes you nervous then there's more testing that needs to be added. Canary deployments with extensive monitoring and automated rollbacks and make this almost entirely hands-off. For most software systems the goal should be to abstract away the actual act of "deploying" and let the infrastructure handle that on its own.
      - tharkun__ 6y ago
        Because the company makes your rating/pay dependent on BS metrics that drive the wrong behavior, such as merging on Friday so that your cycle time isn't too high. Might mean a paycut. Friday deploy gone wrong? Fix the issue, issue another PR that is approved and also deployed in no time? You're the hero and your metrics are also awesome. Cycle time baby! :(
      - potatoz2 6y ago
        Because then you have double the changes to deploy on Monday. And if you push that logic further (no deploys on Thursday, no deploys after 3pm, no deploys +-3 days around holidays, etc.) you end up with very high density deploys, which are riskier.
- pc86 6y ago
  There are absolutely some industries where this makes sense (life-saving health care, national defense, things like Amazon where you literally stop making money when things go down). 90% of developers don't work in these industries or on these types of projects. Hell, I work in healthcare, for a private company, and there is zero reason for any of us to be on call. Patient quality doesn't suffer one iota if our software is down, which it rarely is. It really sounds like you're advocating that developers be on call to "fix" the "problem" that there's no reason for them to be on call?
  - potatoz2 6y ago
    What they're saying is that if someone else is oncall, you tend not to deal with the quality issues of the system (it's a negative externality). If the system you own doesn't require any on-call at all, then great. But for those that do (let's say your a web developer for nytimes.com), the code and system owners should be on-call, not a separate ops team. Ideally being on-call is something you barely notice because nothing happens.
viraptor 6y ago
So once both senior and non-senior engineers opt out of on-call, who do you expect to take care of issues that happen out of hours? (In case of companies who deal with 24h traffic and transactions)
- sdevonoes 6y ago
  Dedicated people hired as such.
  - viraptor 6y ago
    That's realistically out of reach for non-enterprise companies. If they're not primarily standard senior engineers doing typical day work, you need at least 4 people to cover the on-call week + redundancies for leave. You're talking about employing a whole extra team just to handle an occasional callout.
whatsmyusername 6y ago
I'm in ops. Devs don't need to be on call, but if it gets to the point where I need you and you refuse to assist, expect your property to get zero time or attention going forward. Dedicated on call exists because this is an incredibly common problem. People talk about being a team, until something's on fire at a weird hour. If it's a megacorp, sure yeah whatever. It's not like our compensation is tied to product performance. I'll do my due dilligence, document everything, and just leave it down till business hours handoff (this isn't hyperbole, this is exactly how I would handle it). If it's a startup? Leave me hanging and you'll instantly burn all the good will I have toward you. And I've got a direct horn to your boss and the CEO.

leokennis 6y ago

In my opinion, it’s almost mandatory for a well functioning software development environment to have developers on call (next to ops people). Bugs that get you woken up in the middle of the night are fixed in no time. Documentation that is lacking, or spaghetti code, both preventing you from quickly fixing an issue instead of spending hours on it? Fixed the next day. If the stakes of writing bad code are low, code quality will suffer. No better fix for those low stakes than being on call.

sdevonoes 6y ago
> No better fix for those low stakes than being on call. I prefer a company with a strong tech culture: code review, thinkg long-term (marathons instead of sprints) and design systems with architecture-first on mind (instead of feature-first), automated testing, QA people, don't deploy on Fridays...
- pc86 6y ago
  If you've got testing, code reviews, QA, and CI/CD, Friday deployments should be totally fine.
  - sdevonoes 6y ago
    The point is "Friday deployments" are not needed (unless we are talking about hotfixes). What's the use case? Can't it wait until Monday? Why are all in such a rush?
    - pc86 6y ago
      I would say that if things are tested appropriately (it's a big if, and not an easy thing to do honestly), when you commit a new feature it should go through the testing process and right into production. If that makes you nervous then there's more testing that needs to be added. Canary deployments with extensive monitoring and automated rollbacks and make this almost entirely hands-off. For most software systems the goal should be to abstract away the actual act of "deploying" and let the infrastructure handle that on its own.
    - tharkun__ 6y ago
      Because the company makes your rating/pay dependent on BS metrics that drive the wrong behavior, such as merging on Friday so that your cycle time isn't too high. Might mean a paycut. Friday deploy gone wrong? Fix the issue, issue another PR that is approved and also deployed in no time? You're the hero and your metrics are also awesome. Cycle time baby! :(
    - potatoz2 6y ago
      Because then you have double the changes to deploy on Monday. And if you push that logic further (no deploys on Thursday, no deploys after 3pm, no deploys +-3 days around holidays, etc.) you end up with very high density deploys, which are riskier.
pc86 6y ago
There are absolutely some industries where this makes sense (life-saving health care, national defense, things like Amazon where you literally stop making money when things go down). 90% of developers don't work in these industries or on these types of projects. Hell, I work in healthcare, for a private company, and there is zero reason for any of us to be on call. Patient quality doesn't suffer one iota if our software is down, which it rarely is. It really sounds like you're advocating that developers be on call to "fix" the "problem" that there's no reason for them to be on call?
- potatoz2 6y ago
  What they're saying is that if someone else is oncall, you tend not to deal with the quality issues of the system (it's a negative externality). If the system you own doesn't require any on-call at all, then great. But for those that do (let's say your a web developer for nytimes.com), the code and system owners should be on-call, not a separate ops team. Ideally being on-call is something you barely notice because nothing happens.

viraptor 6y ago

So once both senior and non-senior engineers opt out of on-call, who do you expect to take care of issues that happen out of hours? (In case of companies who deal with 24h traffic and transactions)

sdevonoes 6y ago
Dedicated people hired as such.
- viraptor 6y ago
  That's realistically out of reach for non-enterprise companies. If they're not primarily standard senior engineers doing typical day work, you need at least 4 people to cover the on-call week + redundancies for leave. You're talking about employing a whole extra team just to handle an occasional callout.

whatsmyusername 6y ago

I'm in ops. Devs don't need to be on call, but if it gets to the point where I need you and you refuse to assist, expect your property to get zero time or attention going forward. Dedicated on call exists because this is an incredibly common problem. People talk about being a team, until something's on fire at a weird hour. If it's a megacorp, sure yeah whatever. It's not like our compensation is tied to product performance. I'll do my due dilligence, document everything, and just leave it down till business hours handoff (this isn't hyperbole, this is exactly how I would handle it). If it's a startup? Leave me hanging and you'll instantly burn all the good will I have toward you. And I've got a direct horn to your boss and the CEO.