4 ms·

That "CYA" response really doesn't feel like "us" to me, at least not the AWS side. I can't speak to the experiences on the store side, nor on the devices side.

That "CYA" response really doesn't feel like "us" to me, at least not the AWS side. I can't speak to the experiences on the store side, nor on the devices side. Over in AWS, we practice the "blameless postmortem" model you'll read about from time to time, and it is very rare to see any professional or personal consequences beyond some light joking come about from "breaking prod" To that point, I know personally the engineers who triggered several of the AWS outages you will have read about in the news over the last decade and more. Some of them are still with us, some have been promoted since then.

delecti 5y ago
Your experience also matches mine from my time in the advertising org. There's no benefit to placing blame, just figure out what went wrong, how to fix it, and how to prevent something like it from happening again. In fact it was one of the few parts of the culture that I tried to take with me after leaving.
- teawrecks 5y ago
  Reminds me of that classic parable about the CEO and the worker who just made a $100,000 error. The worker says, "I assume you'll be firing me now" and the CEO says, "Fire you?! I just spent $100,000 training you!"
itg 5y ago
Depends on the team. In my team in AWS, if something was your fault, the manager would call you out in front of the entire team. Then they wondered why attrition was so high.
imwillofficial 5y ago
I’m agreeing with the “it’s not us” vibe. I work on a team that has some planet wide scale software pipelines. I broke it, hard. I’m not a software engineer so I had no idea how to undo what I had done. People from 3 different teams, two of them experts from other teams who heard my call for help hopped on a call and spent all day helping me fix the issue I made and then some. This was a hard stop on a cortical package pipeline for 24 hours and at no point did I feel like my job was at stake despite it clearly being my fault, and due to me running a command I know I shouldn’t have and cutting corners.
deleted 5y ago
[deleted]
dontTellBezos 5y ago
As an example; the individual who caused this outage was not fired for it, even though the final cause was user error. https://aws.amazon.com/message/41926/ https://aws.amazon.com/message/41926/
atopuzov 5y ago
Sometimes COEs are used to punish engineers. Eg. take a look at so called "moustache" COE (which I was part of).

delecti 5y ago

Your experience also matches mine from my time in the advertising org. There's no benefit to placing blame, just figure out what went wrong, how to fix it, and how to prevent something like it from happening again. In fact it was one of the few parts of the culture that I tried to take with me after leaving.

teawrecks 5y ago
Reminds me of that classic parable about the CEO and the worker who just made a $100,000 error. The worker says, "I assume you'll be firing me now" and the CEO says, "Fire you?! I just spent $100,000 training you!"

itg 5y ago

Depends on the team. In my team in AWS, if something was your fault, the manager would call you out in front of the entire team. Then they wondered why attrition was so high.

imwillofficial 5y ago

I’m agreeing with the “it’s not us” vibe. I work on a team that has some planet wide scale software pipelines. I broke it, hard. I’m not a software engineer so I had no idea how to undo what I had done. People from 3 different teams, two of them experts from other teams who heard my call for help hopped on a call and spent all day helping me fix the issue I made and then some. This was a hard stop on a cortical package pipeline for 24 hours and at no point did I feel like my job was at stake despite it clearly being my fault, and due to me running a command I know I shouldn’t have and cutting corners.

deleted 5y ago

[deleted]

dontTellBezos 5y ago

As an example; the individual who caused this outage was not fired for it, even though the final cause was user error. https://aws.amazon.com/message/41926/ https://aws.amazon.com/message/41926/

atopuzov 5y ago

Sometimes COEs are used to punish engineers. Eg. take a look at so called "moustache" COE (which I was part of).