4 ms·
We took it one step further on my team. The best way to ensure that the pager(duty) doesn't go off at 2am in the morning is to put developers in the first on-ca
by curun1r 10y ago
We took it one step further on my team. The best way to ensure that the pager(duty) doesn't go off at 2am in the morning is to put developers in the first on-call group. Not only did it work, but the developers got a much more nuanced view of how systems operate. When we first started, I'd see developers using ping to determine whether a nameservice entry was correct. After a few months of handling almost all of our own ops, developers knew how each piece of the puzzle worked and how it all fit together rather than the hazy, largely abstracted view they had before.
But we learned that we needed it to go the opposite direction too. We had ops people who, when given a corporate-wide mandate to apply a security patch or some such task, would log into every machine and apply the patch, despite the fact that we'd been practicing immutable infrastructure with zero-downtime deployments. We hadn't given them the necessary exposure to the dev side to understand that you had to apply those fixes to a base image and trigger a redeploy. There was a lot of finger pointing a couple of weeks later after it was discovered that the fixes were overwritten by an application deploy.
- adrianN 10y agoI hope you increased their salaries together with the new 2am pager duty.