3 ms·
> If you think a playbook is bad, try being oncall for the first time for a massively (Google-scale) distributed system without a playbook. You are not Google
by notyourday 7y ago
> If you think a playbook is bad, try being oncall for the first time for a massively (Google-scale) distributed system without a playbook.
You are not Google scale. Don't invent a pen that writes in space when a pencil would do the trick.
- marcinzm 7y agoI'd argue 100% automation is a space pen while run books are a pencil.
- notyourday 7y agoYou do not need 100% automation. What you need is a systematic approach to handling problems followed by fixing the root cause. Runbooks came from techops in broadcasting, power plant operations, etc where there was a clear division between operators who pushed buttons, ran cables, etc and those that made decisions about buttons to push and cables to run. Dumb hands + runbooks created "smart hands". If your SRE runs like that it is not SRE. Look at the incident handling: 1. Identify the issue 2. Implement a workaround to restore the service 3. Identify the root cause 4. Implement a fix for the root cause 5. Remove the workaround Runbooks cover 1. and 2.
- easymodex 7y agoDid you just assume my scale?
- p_l 7y agoAnd check your metaphors before using them. The pencil was more expensive.
- ggambetta 7y ago> You are not Google scale. I don't know, I think I was kind of Google scale when working as a Google SRE.
- notyourday 7y ago1) https://www.transposit.com/ https://www.transposit.com/ is not Google. 2) Your impact on Google scale was nearly 0 because you were one of a thousand SREs at Google.
- williamDafoe 7y agoThere are only 500.
- notyourday 7y agoMy bad
- GaryNumanVevo 7y agoThere’s no tolerance for rude and unsubstantiated comments here