3 ms·
> pager culture where you're awoken in the middle of the night For multinational corporations, this has never made sense to me. Why do we insist on relying gro
by falcolas 9y ago
> pager culture where you're awoken in the middle of the night
For multinational corporations, this has never made sense to me. Why do we insist on relying groggy, "at the bottom of their performance curve" engineers to keep your system up, instead of someone in the middle of their work day?
IMO, if you're waking someone up in the middle of the night, something is horribly wrong, and it's not the issue you woke that person up for.
- dsfyu404ed 9y agoMost don't page people at 1am for the initial page. People only get paged at 1am when the problem has been narrowed down to some specific software and all the people with expertise in that thing are in the same time zone.
- mabbo 9y agoHaving done Amazon on-call for five years, I can give some background on that. First, the front-line pager duty is usually hit by an ops eng team first. Each of those teams has an India group and an North America group. If they can't resolve the issue, they page in the developers. Having good Ops Eng guys supporting you is a blessing. There's also a very strong incentive for teams to write good software and test before deploying when you know damn well that if you're deploying something that doesn't work, you're the guy who will have to deal with it at 3am. Having strong integration and system tests suddenly becomes incredibly important. Edit: and as dsfyu404ed said, the Ops Eng guys will be smart enough to narrow down where the problem is usually. If you're being paged, it's usually your team's fault.