I joined a new team about two years back. They were very reliant on runbooks for operational things. Problem was, at least half of them were out of date. Nee

I joined a new team about two years back. They were very reliant on runbooks for operational things. Problem was, at least half of them were out of date. Need to access logs on a machine when they aren't being sent someplace like {Datadog, Cloudwatch, etc.}? Here are the SSH instructions. But wait, this key is no longer in the secure vault. etc. etc. with no updated information on how to get the new IAM or whatever permissions needed for SSH access. I think calling it a culture problem is a cop-out, unfortunately. Documentation woes, in and of themselves, exist everywhere in this industry. I don't know what the solution is, but I just don't think it's a culture thing. If anything, it should be a mental health thing. You're on-call and get paged at 3AM about some metric drastically falling and you can't get the logs on your usual log system and need to log into a machine or cluster directly. And since your team is heavily runbook-reactive (new term I just coined to describe a team reliant on runbooks to solve "known" problems rather than learned knowledge), you're totally stuck and under a lot of pressure precisely because the runbook is no longer applicable. This is never fun for anyone. One possible approach is setting up a type of Red vs Blue team monthly or quarterly event. Half the team spends the week randomly "breaking" things (ideally in a stage or other, more isolated environment) and the other half gets the pages, learns what runbooks are out of date or otherwise no longer correct, fix the issue both operationally and in the runbooks. Additionally, in the process one team learns about what kinds of issues can break the system(s) and the other team learns about what kinds of fixes can return the system(s) to working order. Then switch off roles every other time.