3 ms·

not knowing anything about crates, this is a very amateurish and subpar postmortem; provides neither a simple, executive summary nor a technical deep-dive. prob

not knowing anything about crates, this is a very amateurish and subpar postmortem; provides neither a simple, executive summary nor a technical deep-dive. problems arent annotated (or even clearly identified) and so cant be correlated to solutions or a POAM. everything is written in a very passive, non-action oriented voice. i do not leave this confident that issues have been identified, triaged, and queued for repair or mitigation. > Deploying the change took way longer than expected what does this mean? how much longer is "way longer"? what was your RTO goal, how did you fail to meet it, and how will you meet it in the future? > We should also strive to reduce the amount of time PRs sit in master without being live. not a plan. everyones striving at whatever their doing. were you not striving before? how will your accomplish your goal of minimizing non-live time? > It took 1 hour and 31 minutes from the start of the incident to the deploy of the fix. "deploy of the fix" is an unclear statement. does that mean when code was pushed, or when you verified your clients no longer had problems?

jen20 7y ago
There is a clear timeline shown at the end which answers the last of these questions — it was 1 hour and 31 minutes until all ill effects of the bug were addressed and service was fully restored.
- kryogen1c 7y ago
  > There is a clear timeline shown at the end Yes, i saw and it did not clarify my question to which you are responding which is obvious because... > all ill effects of the bug were addressed and service was fully restored. ...this is not true. The timeline says a customer problem was submitted the next day, and additional action was required to ensure the "service was fully restored"
codetrotter 7y ago
On the flip side, as someone that has a few crates on crates.io, and is familiar with at least some of the internal workings of git and how commit data is pushed, the postmortem reads entirely reasonably to me.

jen20 7y ago

There is a clear timeline shown at the end which answers the last of these questions — it was 1 hour and 31 minutes until all ill effects of the bug were addressed and service was fully restored.

kryogen1c 7y ago
> There is a clear timeline shown at the end Yes, i saw and it did not clarify my question to which you are responding which is obvious because... > all ill effects of the bug were addressed and service was fully restored. ...this is not true. The timeline says a customer problem was submitted the next day, and additional action was required to ensure the "service was fully restored"

codetrotter 7y ago

On the flip side, as someone that has a few crates on crates.io, and is familiar with at least some of the internal workings of git and how commit data is pushed, the postmortem reads entirely reasonably to me.