3 ms·
Why bypass the code review for high priority fixes? That's when you need it the most.
by xendo 9y ago
Why bypass the code review for high priority fixes? That's when you need it the most.
- dmlittle 9y agoDepending on what the issue is you might want a quick fix right away which can sometimes be hacky and then put in a long-term fix that is code reviewed by others to make sure it’s the right solution.
- deleted 9y ago[deleted]
- joshuamorton 9y agoThe big example is config changes during ongoing incidents. You may need to flip a flag or update a config right now, but waiting for review, even if it takes just a few minutes, can blow slas. There's ways to handle that somewhat though.
- kevan 9y agoAlso at a big 5. We've found that skipping reviews causes more problems than it solves in emergency situations. It's really tempting to bypass the process because you're sure you can fix it fast, but this mindset results in more missteps in aggregate. If a couple minutes really makes that much of a difference you should probably page multiple people in from the start of an issue.
- joshuamorton 9y agoGenerally speaking, your un-reviewed change isn't a "fix", but something to stop the bleeding. It might move traffic out of a problematic area, or disable an experiment that's leading to unusual metrics. In other words, its a rollback, not a fix-forward. (and I'll note that where I work, these changes still need to be reviewed soon after they're submitted). I agree that all fixes should be reviewed before submission.
- throwaway84742 9y agoThis is absolutely the wrong thing to do. When mitigating issues on call anything that goes live bypassing canary has to receive much more scrutiny, not less. It is best not to do such things, but if you have to do them, review the change extra hard.
- joshuamorton 9y agoSee my other comment to a sibling expressing a similar sentiment. Rollbacks and drains may require submissions to change configurations, but aren't bypassing canary, they're specifically reverting to known good systems.