5 ms·
This is exactly right. Ops are making decisions about how the app runs because they are the ones get paged at 4 am when it suddenly does not.
by havetocharge 10y ago
This is exactly right. Ops are making decisions about how the app runs because they are the ones get paged at 4 am when it suddenly does not.
- CaptSpify 10y agoOne of the best strategies I've seen with this is: put devs on call. As soon as they are woken up at 4 am, you can see entire teams turn around their attitude on how things are run.
- seanp2k2 10y agoYep, this. PagerDuty and Slack can help. Many alerting and monitoring services now work with these. Info / debug stuff goes to slack, urgent and important stuff goes to PD. Make a quadrant of how important and how time-sensitive each alert is to respond to and make them go to the respective mechanism. Email can stand in for Slack, but people tend to ignore emails and filter them if they start getting spammed with important-but-not-their-problem things, then they'll miss the important-and-their-problem alerts later because their filter matches those too. Suddenly, everyone is writing tests, testing locally, deploying canaries, getting demos ready 2 days ahead of time, and not deploying at 6PM on Friday.
- CaptSpify 10y agoSorry, but I fail to see how A) That is exclusive/special to things like slack or pd. Seems like a lot of tools could handle this. B) How those tools are related to being woken up. I exclusively use email for work notifications. Chat is purely for BS and chit-chat. The last thing I would do when woken up at 3 is check chat.