5 ms·

Tips and Best Practices for being On-Call

25 points by semicolon 16y ago

tt 16y ago
It doesn't matter very much how awesome your device is. You need at least a second person who would also get the alert as a backup (assuming this is a critical system that you really care about, otherwise you wouldn't want to be on-call for it in the first place). As the primary responder, your responsibility is to acknowledge any alert within a minute or so. Otherwise, the backup person would have to jump in. To ensure "accountability," you may have a manager who would be a third backup and can escalate up from there.
- semicolon 16y ago
  Yes, a good escalation policy would have a primary responder, a backup or secondary, and then one or managers, going up the hierarchy. PagerDuty supports that and my next post will be on that topic. Having a good device is important too though; if you sleep through or miss an alert, it may take another 10-20 minutes or so (depending on the escalation policy) before the alert escalates to the next person. This slack time could be pretty important depending on the severity of the problem.
tylerrooney 16y ago
I would recommend against "picking sharp or piercing ringtones". I once had a horrific on-call schedule and, like any mammal, I developed a Pavlovian response to the sound of my pager which resembled those door buzzers that retail stores use. Friends of mine told me that I used to jump when I heard them go off. So realize that whatever that ringtone is you'll associate it with this kind of anger: http://www.jeckels.com/photoDetail?PhotoId=4586&CategoryIndex=50&ReferringCategoryId=447 http://www.jeckels.com/photoDetail?PhotoId=4586&Category...
- semicolon 16y ago
  Nice; it looks like that pager was an Amazon one too. There was a time when the sound of my pager (or even my cell phone's SMS ringtone) would get my heart beating faster. Maybe this would be a good ringtone for being on-call: http://www.youtube.com/watch?v=5bNE-5TVAmg http://www.youtube.com/watch?v=5bNE-5TVAmg
vacri 16y ago
Forgot to mention that you also need a patient spouse.

25 points by semicolon 16y ago

tt 16y ago

It doesn't matter very much how awesome your device is. You need at least a second person who would also get the alert as a backup (assuming this is a critical system that you really care about, otherwise you wouldn't want to be on-call for it in the first place). As the primary responder, your responsibility is to acknowledge any alert within a minute or so. Otherwise, the backup person would have to jump in. To ensure "accountability," you may have a manager who would be a third backup and can escalate up from there.

semicolon 16y ago
Yes, a good escalation policy would have a primary responder, a backup or secondary, and then one or managers, going up the hierarchy. PagerDuty supports that and my next post will be on that topic. Having a good device is important too though; if you sleep through or miss an alert, it may take another 10-20 minutes or so (depending on the escalation policy) before the alert escalates to the next person. This slack time could be pretty important depending on the severity of the problem.

tylerrooney 16y ago

I would recommend against "picking sharp or piercing ringtones". I once had a horrific on-call schedule and, like any mammal, I developed a Pavlovian response to the sound of my pager which resembled those door buzzers that retail stores use. Friends of mine told me that I used to jump when I heard them go off. So realize that whatever that ringtone is you'll associate it with this kind of anger: http://www.jeckels.com/photoDetail?PhotoId=4586&CategoryIndex=50&ReferringCategoryId=447 http://www.jeckels.com/photoDetail?PhotoId=4586&Category...

semicolon 16y ago
Nice; it looks like that pager was an Amazon one too. There was a time when the sound of my pager (or even my cell phone's SMS ringtone) would get my heart beating faster. Maybe this would be a good ringtone for being on-call: http://www.youtube.com/watch?v=5bNE-5TVAmg http://www.youtube.com/watch?v=5bNE-5TVAmg

vacri 16y ago

Forgot to mention that you also need a patient spouse.