7 ms·

I had a project that was really important once. I made a tiny mistake that had a big consequence - a couple of hours of potential lost revenues from our custome

I had a project that was really important once. I made a tiny mistake that had a big consequence - a couple of hours of potential lost revenues from our customers. I fixed my mistake with both my boss and CEO nearby. I said after I pushed the fix that I really need more resources around it. That little light of "yeah, this is important" that should have flickered didn't. :) I will not be surprised if nothing gets fixed with the issue at NYSE.

smcin 4y ago
But how much actual lost customer revenue? Also, did the customer even notice or not? You're reminding me of the difference between engineers and non-technical managers; to many of the latter something's only a problem if/when the customer or senior mgmt are on the phone complaining about it. Until then it's all naysaying engineers being too pessimistic about process and risk.
- bigpeopleareold 4y ago
  No one gave me a figure, but the product going down does have a direct impact on customer revenues. So, while no one actually gave me direct numbers, it was felt quite a bit. The most notable thing from the incident was where certain account managers in Japan had to do formal mea culpas because of this mistake. So, in other words, they were on calls, but I got back just "you can't have this happen again - do something." Could someone else at least do code review? I was depressed that I got a lot of "well, we don't know the code base" - the issue was lost in the shuffle also with management. So, I just grew more pessimistic about process and risk. It's not healthy and I would not bottle it up now, but I did, and that was my mistake. I ended up hating that project, but at least I got to train someone else to work on it (who has a team around him.)
1980phipsi 4y ago
If you were to ask what is the probability that this specific error happens again, I would think it would be pretty low. Probably lower than a week ago. If you were to ask what is the probability that some significant, costly error happens again, I don't think the probability is that much lower than a week ago.

smcin 4y ago

But how much actual lost customer revenue? Also, did the customer even notice or not? You're reminding me of the difference between engineers and non-technical managers; to many of the latter something's only a problem if/when the customer or senior mgmt are on the phone complaining about it. Until then it's all naysaying engineers being too pessimistic about process and risk.

bigpeopleareold 4y ago
No one gave me a figure, but the product going down does have a direct impact on customer revenues. So, while no one actually gave me direct numbers, it was felt quite a bit. The most notable thing from the incident was where certain account managers in Japan had to do formal mea culpas because of this mistake. So, in other words, they were on calls, but I got back just "you can't have this happen again - do something." Could someone else at least do code review? I was depressed that I got a lot of "well, we don't know the code base" - the issue was lost in the shuffle also with management. So, I just grew more pessimistic about process and risk. It's not healthy and I would not bottle it up now, but I did, and that was my mistake. I ended up hating that project, but at least I got to train someone else to work on it (who has a team around him.)

1980phipsi 4y ago

If you were to ask what is the probability that this specific error happens again, I would think it would be pretty low. Probably lower than a week ago. If you were to ask what is the probability that some significant, costly error happens again, I don't think the probability is that much lower than a week ago.