2 ms·
That's the conclusion of the report: no one was previously aware that an "unrelated" field in an "unrelated" activity would have production-wide consequences.
by bonesss 9y ago
That's the conclusion of the report: no one was previously aware that an "unrelated" field in an "unrelated" activity would have production-wide consequences.
The detected an issue within 4 minutes, but it took an hour and a half to diagnose the issue, find the unwanted and unforeseen change, and revert it. That's not an absurd amount of time for a complex system with millions of users.
I would hazard to guess that this long-running system isn't even taken in as part of the companies 'change management' routines, in so far as it's job was number filtering and the operation was 'routine'. At my work we produce production change reports for production changes, but we don't fill them out every time we run a stable application...