My team runs incident drills as part of our training. The person running the drill (the instructor) breaks something in our pre-production environment. This cou

My team runs incident drills as part of our training. The person running the drill (the instructor) breaks something in our pre-production environment. This could be pushing a buggy PR, changing some configuration, taking a node offline, etc. The instructor and the learner then hop in a video call. The learner shares their screen, then works through the incident with help from the instructor. They acknowledge the incident in our management tool, use logs and metrics to track it down, take steps to mitigate it, then mark the incident as mitigated in our management tool. This builds skills and confidence for the learner, and has the additional benefit of exposing pain points in our tooling.