3 ms·

Someone at our place recently commented that the ops team during an incident strongly feels like NASA mission control in critical moments[1]. I wanted to protes

by tetha 7y ago

Someone at our place recently commented that the ops team during an incident strongly feels like NASA mission control in critical moments[1]. I wanted to protest, but that's surprisingly accurate. > And if anyone is screaming or gets emotional, they should not be in that room. If someone starts yelling around in my incident war room for no reason, they get thrown out. I'm a calm and quiet person, but bugging around during a major incident is one of the few things that make me mad. 1: https://youtu.be/Y0yOTanzx-s?t=3059 https://youtu.be/Y0yOTanzx-s?t=3059

Moru 7y ago
When the insident is over, does it look like 55:50 in that video? :-)
- tetha 7y ago
  We recently had a 15 month long project almost fail due to some stupid shit and a really wonky error no one understood so far. <Almost fail> as in "Keep the customer on the phone we have 3 possible things to try and don't hang up! No one leaves that call until I'm out of hacks to deploy!" That evening we had the entire ops team in the houston-mode for several hours. And yes, once we had a workaround in place the customer accepted, we reacted like that. Except we also had our critical-incident whiskey go around. Then the CEO walked in to congratulate us on that project. Whoops. But he's a good sport, so good times. :)
di4na 7y ago
It is not surprising at all. Mission Control was forged in the fire (literally for Apollo 1) and they are one of the most visible "incident team" we know about. I highly advise to read Gene Kranz memoirs "Failure is not an Option" if you work in that kind of environment.
- ethbro 7y ago
  I heard recently that he never said that. Apparently it was mentioned when the Apollo 13 script writers were gathering stories at NASA, they liked it, and then gave it to the Kranz character. Who then decided, "Hey, if everyone thinks I said it..." and titled his memoir.
  - di4na 7y ago
    Yep exactly.

by tetha 7y ago

Moru 7y ago

When the insident is over, does it look like 55:50 in that video? :-)

tetha 7y ago
We recently had a 15 month long project almost fail due to some stupid shit and a really wonky error no one understood so far. <Almost fail> as in "Keep the customer on the phone we have 3 possible things to try and don't hang up! No one leaves that call until I'm out of hacks to deploy!" That evening we had the entire ops team in the houston-mode for several hours. And yes, once we had a workaround in place the customer accepted, we reacted like that. Except we also had our critical-incident whiskey go around. Then the CEO walked in to congratulate us on that project. Whoops. But he's a good sport, so good times. :)

di4na 7y ago

It is not surprising at all. Mission Control was forged in the fire (literally for Apollo 1) and they are one of the most visible "incident team" we know about. I highly advise to read Gene Kranz memoirs "Failure is not an Option" if you work in that kind of environment.

ethbro 7y ago
I heard recently that he never said that. Apparently it was mentioned when the Apollo 13 script writers were gathering stories at NASA, they liked it, and then gave it to the Kranz character. Who then decided, "Hey, if everyone thinks I said it..." and titled his memoir.
- di4na 7y ago
  Yep exactly.