3 ms·
I like making the devs who write the application the SREs who are going to be supporting it and making them carry a pager. If its worth it to the team to write
by lamontcg 12y ago
I like making the devs who write the application the SREs who are going to be supporting it and making them carry a pager.
If its worth it to the team to write code with edge conditions that will errantly wake them up in the middle of night occasionally they can make that decision to put that tax on their own lives, not someone else's.
Then the SREs are in every single code review.
- quanticle 12y agoThis works only as long as you have a single application, or a bunch of independent applications, each with their own team. With any kind of "platform" or "framework" or shared service architecture, the incentive will be for the devs on the application teams to do as little logging as possible, because what will inevitably happen is that failures and errors with "unknown" causes will be marked as platform failures and will wake the platform team in the middle of the night. At that point, you're back to where you started. I've been in that scenario multiple times, and believe me, there are few things worse than trying to debug someone else's code at three in the morning to try to determine if the page that woke me up was a legitimate platform issue or if it's due to application code that's misbehaving or misusing the platform. Making devs carry pagers certainly helps, but it's a mistake to think that it's a panacea, and that forcing devs to carry pagers will suddenly make them write code with perfect logging and perfect error handling.
- dredmorbius 12y agoNobody's arguing that it's a panacea. Only that raising awareness of operational issues, reliability, and lost sleep does tend to put feet to the fire.