3 ms·
You could create an agent template for each incident you've ever had, with context pre-cached with the postmortem report, full code change, and any other inform
by daxfohl 7mo ago
You could create an agent template for each incident you've ever had, with context pre-cached with the postmortem report, full code change, and any other information about the incident. Then for every new PR you could clone agents from all those templates and ask whether the PR could cause something similar to the pre-loaded incident. If any of them say yes, reject the PR unless there's a manual override. You'd never have a repeat incident.
Obviously it's probably cost-prohibitive to do an all to all analysis for every PR, but I imagine with some intelligent optimizations around likelihood and similarity analysis something along those lines would be possible and practical.
- iLoveOncall 7mo agoYou vastly underestimate the complexity of systems in a company like Amazon. COEs and Operation Readiness Reviews are already the documents that you mention, but they are largely useless in preventing incidents.
- senderista 7mo agoActually CoEs would make an amazing training corpus for code reviews.
- 8note 7mo agocode review is too late to give some of that feedback, and design/requirements documents dont have nearly the standardization of presentation and feedback tools for that to be useable. Amazon does have those things, and has fine tuning on models based on those postmortems. Noisy reviews are also a problem causer. the PR doesnt know what scale a chunk of code is running at, without having access to 20 more packages and other details.