4 ms·
I think it also matters on what the response to the incident was, in particular the answer to the questions of: who was responsible and how can we prevent this
by hermitdev 4y ago
I think it also matters on what the response to the incident was, in particular the answer to the questions of: who was responsible and how can we prevent this from happening again?
If the answer to the first question is a bunch of finger pointing or the answer to the second is along the lines of "nothing", heads will likely roll.
My first boss as a FTE at a trading shop made a change to a production database outside of process before a weekend. Come Monday morning, production trading systems couldn't come up for market open. We were also an options market maker. He refused to accept responsibility for his actions and denied there was anything done to prevent it. He was quickly behind closed doors in his boss's office. I had to undo the changes he made the week before (hooray for audit tables) as soon as I could. We were potentially liable for a substantial penalty from the exchange for being out of the market, and every minute mattered. Don't know if we ever did get penalized, but he was fired and out the door by 10AM that day.
I've certainly seen and made lots of mistakes over the decades since, but that's the only one for which I've seen someone canned. Response matters.