We use PagerDuty (http://pagerduty.com http://pagerduty.com) at HipChat and while I absolutely loathe being woken up by it, it's helped us identify issues durin

We use PagerDuty (http://pagerduty.com http://pagerduty.com) at HipChat and while I absolutely loathe being woken up by it, it's helped us identify issues during off-peak hours much more quickly. But no matter what systems you have in place or how many hundreds of deploys you've done, there's always a new way for things to break.