7 ms·

Nobody knows how to do a failure analysis. I used to work in r&d, now that I’m building websites and mobile apps the culture doesn’t care. Pointing out obvious

Nobody knows how to do a failure analysis. I used to work in r&d, now that I’m building websites and mobile apps the culture doesn’t care. Pointing out obvious design limitations will, more often that not, make me the asshole. Not even trying to delay ship or get future rework scheduled, just having it documented is too much. Out of sight out of mind.

Dr_ReD 4y ago
It's not that they don't know how to. It's that there's an economic incentive to not care. So they don't.
- pif 4y ago
  Exactly, and the incentive is that customers do not care. Build quality, keep redundancy, and in the next pandemic your business will flourish... unless you'll have had to close it before 'cause of people buying cheap and not good!
  - falcolas 4y ago
    I assert that customers do care. Customers were quite annoyed that they couldn't buy vehicles, regardless of the price. Or do you just not remember the complaints? On the business side, Toyota - the progenator of JIT manufacturing who still warehoused chips to deal with a shortage exactly like what happened - flourished at a time where others floundered. As an added bonus, they were still able to remain profitable prior to the shortage, despite having warehoused chips. I think the "buy cheap not good" is more of a proactive excuse from companies who want to sell cheap goods to maximize their profit. Kind of an extension of the "never ask customers what they want, they'd just want better horses" or "they can have it in any color so long as it's black" tropes.
    - pif 4y ago
      > I assert that customers do care. Customers were quite annoyed that they couldn't buy vehicles This is not what I meant. Customers did not care, before the pandemic.
canucklady 4y ago
It's not that people don't know it's a SPOF. The issue is that if you fail in a way that is common, nobody blames you. Cell phone 2FA is so ubiquitous that when it doesn't work clients wonder if they're the one fucking up. We had a massive internet outage in Canada recently and nobody blamed individual shops for not being able to take credit cards, they blamed the phone company. If you roll your own thing, even if it's objectively better, you're painting a target on your back. Now when your new thing fails it's your fault. There's also something to be said for how modern financial capitalism has squeezed redundancy out of everything. Supply chains are just-in-time, businesses rely on a million vendors for everything, if a single link in the chain fails it usually has a huge blast radius because of broad consolidation in these upstream suppliers. Basically there are a lot of small businesses that depend on like 5 large companies (think Sysco, telecoms, etc) and if those large companies fail every small company fails in the same way, so there's no incentive to derisk (because your competitors will all also be failing in the same way).
- dumpsterlid 4y ago
  You hit the nail on the head
- gameshot911 4y ago
  Let's not forget the benefits either. Reduced redundancy is _good_ (as long as nothing fails in the chain, ofc). It enables society to make more, for lower costs. And it works remarkably well, overall.
  - falcolas 4y ago
    But thats the problem being expressed - things will fail. Things will always fail, and ignoring that is the equivalent of burying your head in the sand and thinking your ass is covered.
    - gameshot911 4y ago
      If things go right 8 times out of 10 overall, that's a net win. Expected value - I get Product X $2 cheaper if things go right, but lose $6 if things go wrong. Expected value = 2*.8-6*.2 = .4. Overall net positive.
bob1029 4y ago
> Pointing out obvious design limitations will, more often that not, make me the asshole. Being an "asshole" isn't always a bad thing, assuming you mean "frowned upon for providing dissent along lines of unhappy, but factual, technical realities which are applicable in the current context". If this is the new definition of asshole, then I am the king of assholes.
- taneq 4y ago
  As long as your new definition of 'asshole' also comes with a new definition of 'bad thing' that doesn't include 'will piss off your users and cause you/your client to lose money'...
  - bob1029 4y ago
    Agreed. I think how you deliver your unhappy truths is 99% of keeping the other party from going bananas. There are very few things that cannot be reframed in a more positive light. Being able to sell a "no" is much more important than being able to sell someone on a shiny piece of bullshit.
- 0xbadcafebee 4y ago
  Asshole here too. Left previous job, people sent messages thanking me for being the only person asking hard questions.
taneq 4y ago
Plenty of people know how to do failure mode analysis. In most webcentric scenarios, though, nobody cares because it's all about the average case. If 99 of your users have a great experience and 1 has a bad-but-not-bad-enough-to-make-the-front-page experience, that's a huge win over 100 users having an okay experience. I hate it too, but that's the market.
toss1 4y ago
True! It's Risk. Humans are typically insanely bad at understanding risk, how to assess it, and how to mitigate it. For most, if something is risky, it's just a black box, perhaps with some odds, and it's either feared or ignored (or both). Few understand the critical difference between using knowledge, skill, technology, and planning to manage and mitigate risk, vs rolling the dice and getting away with it for a long time. Using knowledge, skill, technology, and planning to manage and mitigate risk has gotten humans to great heights - we can make entire careers living in environments in which random behaviors will kill us in seconds (e.g., aerospace, underwater, large construction, metals refining, etc.). But most people think much more naively, and even do so in professional environments unless required otherwise (and even sometimes where they are). So, they just think that if something isn't obviously a risk, or only happens occasionally, ignore it. This is just rolling the dice. Even if you get away with it for a long time, that only makes you more falsely confident and therefore more vulnerable. From the last mild pandemic, there are over a million dead and close to half million suffering long disease from this attitude. And you see it in your work - you simply point out an obvious very damaging failure mode, even just to document it, and you get a hostile reaction. I'm not sure what to do about it. Perhaps rock climbing should be a required activity in school, learning to viscerally manage risk, and showing that it is usually actually easy to to?

Dr_ReD 4y ago

It's not that they don't know how to. It's that there's an economic incentive to not care. So they don't.

pif 4y ago
Exactly, and the incentive is that customers do not care. Build quality, keep redundancy, and in the next pandemic your business will flourish... unless you'll have had to close it before 'cause of people buying cheap and not good!
- falcolas 4y ago
  I assert that customers do care. Customers were quite annoyed that they couldn't buy vehicles, regardless of the price. Or do you just not remember the complaints? On the business side, Toyota - the progenator of JIT manufacturing who still warehoused chips to deal with a shortage exactly like what happened - flourished at a time where others floundered. As an added bonus, they were still able to remain profitable prior to the shortage, despite having warehoused chips. I think the "buy cheap not good" is more of a proactive excuse from companies who want to sell cheap goods to maximize their profit. Kind of an extension of the "never ask customers what they want, they'd just want better horses" or "they can have it in any color so long as it's black" tropes.
  - pif 4y ago
    > I assert that customers do care. Customers were quite annoyed that they couldn't buy vehicles This is not what I meant. Customers did not care, before the pandemic.

canucklady 4y ago

It's not that people don't know it's a SPOF. The issue is that if you fail in a way that is common, nobody blames you. Cell phone 2FA is so ubiquitous that when it doesn't work clients wonder if they're the one fucking up. We had a massive internet outage in Canada recently and nobody blamed individual shops for not being able to take credit cards, they blamed the phone company. If you roll your own thing, even if it's objectively better, you're painting a target on your back. Now when your new thing fails it's your fault. There's also something to be said for how modern financial capitalism has squeezed redundancy out of everything. Supply chains are just-in-time, businesses rely on a million vendors for everything, if a single link in the chain fails it usually has a huge blast radius because of broad consolidation in these upstream suppliers. Basically there are a lot of small businesses that depend on like 5 large companies (think Sysco, telecoms, etc) and if those large companies fail every small company fails in the same way, so there's no incentive to derisk (because your competitors will all also be failing in the same way).

dumpsterlid 4y ago
You hit the nail on the head
gameshot911 4y ago
Let's not forget the benefits either. Reduced redundancy is _good_ (as long as nothing fails in the chain, ofc). It enables society to make more, for lower costs. And it works remarkably well, overall.
- falcolas 4y ago
  But thats the problem being expressed - things will fail. Things will always fail, and ignoring that is the equivalent of burying your head in the sand and thinking your ass is covered.
  - gameshot911 4y ago
    If things go right 8 times out of 10 overall, that's a net win. Expected value - I get Product X $2 cheaper if things go right, but lose $6 if things go wrong. Expected value = 2*.8-6*.2 = .4. Overall net positive.

bob1029 4y ago

> Pointing out obvious design limitations will, more often that not, make me the asshole. Being an "asshole" isn't always a bad thing, assuming you mean "frowned upon for providing dissent along lines of unhappy, but factual, technical realities which are applicable in the current context". If this is the new definition of asshole, then I am the king of assholes.

taneq 4y ago
As long as your new definition of 'asshole' also comes with a new definition of 'bad thing' that doesn't include 'will piss off your users and cause you/your client to lose money'...
- bob1029 4y ago
  Agreed. I think how you deliver your unhappy truths is 99% of keeping the other party from going bananas. There are very few things that cannot be reframed in a more positive light. Being able to sell a "no" is much more important than being able to sell someone on a shiny piece of bullshit.
0xbadcafebee 4y ago
Asshole here too. Left previous job, people sent messages thanking me for being the only person asking hard questions.

taneq 4y ago

Plenty of people know how to do failure mode analysis. In most webcentric scenarios, though, nobody cares because it's all about the average case. If 99 of your users have a great experience and 1 has a bad-but-not-bad-enough-to-make-the-front-page experience, that's a huge win over 100 users having an okay experience. I hate it too, but that's the market.

toss1 4y ago

True! It's Risk. Humans are typically insanely bad at understanding risk, how to assess it, and how to mitigate it. For most, if something is risky, it's just a black box, perhaps with some odds, and it's either feared or ignored (or both). Few understand the critical difference between using knowledge, skill, technology, and planning to manage and mitigate risk, vs rolling the dice and getting away with it for a long time. Using knowledge, skill, technology, and planning to manage and mitigate risk has gotten humans to great heights - we can make entire careers living in environments in which random behaviors will kill us in seconds (e.g., aerospace, underwater, large construction, metals refining, etc.). But most people think much more naively, and even do so in professional environments unless required otherwise (and even sometimes where they are). So, they just think that if something isn't obviously a risk, or only happens occasionally, ignore it. This is just rolling the dice. Even if you get away with it for a long time, that only makes you more falsely confident and therefore more vulnerable. From the last mild pandemic, there are over a million dead and close to half million suffering long disease from this attitude. And you see it in your work - you simply point out an obvious very damaging failure mode, even just to document it, and you get a hostile reaction. I'm not sure what to do about it. Perhaps rock climbing should be a required activity in school, learning to viscerally manage risk, and showing that it is usually actually easy to to?