2 ms·
Out of interest why have an on call SRE when you have Google employees around the world. It's a reasonable time somewhere? The advantage is having someone who i
by quickthrower2 8y ago
Out of interest why have an on call SRE when you have Google employees around the world. It's a reasonable time somewhere? The advantage is having someone who is more 'with it' when the problem arises.
I can guess the answer - maybe it is a trust issue, they want to minimize the number of people with rollback privileges.
- nostrademons 8y agoPartially trust, but also because SRE is a specialized role with specialized tools and a specialized skillset. (And a slightly different mindset from regular engineers - SREs are very attuned to "what can go wrong?", while engineers are supposed to be attuned to "what can go right?".) You need to be more conversant with the high-level details of how the system operates (eg. which binaries are run, what are their resource requirements, what are common failure modes, what can cause cascading failures, when is everything deployed), and much less conversant with the details of how a particular piece of code runs, since the default action when it breaks is just "shut it off and let the engineer debug". Plus SREs are paid more (because they have on-call hours), so it wouldn't really save Google any money to have any every engineer act as a potential SRE. That said, when I was there many teams of smaller, riskier, or less critical services actually did have engineers carry the pager, sometimes with reduced SLAs (and an interlock built into the code so that an outage in their service wouldn't eg. take down websearch).