Any major product at Google has an on-call team with offices in multiple time zones. Typically, two engineers from different time zones will be on-call at any g

Any major product at Google has an on-call team with offices in multiple time zones. Typically, two engineers from different time zones will be on-call at any given time—they switch back and forth between primary and secondary, so nobody is primary on-call during the middle of the night, local time. For example, you might have an SRE team in Mountain View, and an SRE team in Dublin. Maybe the engineer in Mountain View is primary on-call until midnight, and then it switches to the engineer in Dublin, who starts their shift at 8:00 AM. If it’s getting code changes, it’s not getting code changes right now. The software engineers may be in Mountain View, but they won’t patch this and push out a new version during the night. Someone (read: an SRE) may change a configuration or push out a temporary fix now, and any code changes will only go out after significant testing. Generally speaking, you don’t hotfix high-profile production services. You rollback, you set up filters, you disable features, you turn off component services, you run in a degraded capacity—but if you want to actually change the code, you take your time and do it right. Rushing out a code change in the middle of the night is liable to make things worse, and nobody wants to do it.