4 ms·

When the software I write fails, and it does, someone doesn't get to read a web page. Nobody dies or even gets very inconvenienced. This story reminds me of how

When the software I write fails, and it does, someone doesn't get to read a web page. Nobody dies or even gets very inconvenienced. This story reminds me of how good a thing that is. After 40 years of practice I'm still not competent to write such high stakes code. I hope the people who do write it are far more competent than me, but still at least as skeptical about their own capability.

AgloeDreams 7y ago
An interesting trap is people in the field of social media (FB/ Twitter) not realizing that they are in that high-stakes environment where bad privacy settings and leaks can cause death.
- fredley 7y ago
  I mean with regards to Facebook in particular, bad settings can cause genocide. https://www.bbc.co.uk/news/blogs-trending-45449938 https://www.bbc.co.uk/news/blogs-trending-45449938
- twitter_anon 7y ago
  Some of us do, as we come from (or work with people who come from) states that will perpetrate violence against dissidents. Unfortunately, there still aren't enough of us. We're trying to change that.
massysett 7y ago
(I don't know how competent you are, but it's likely that) nobody is more competent than you are. We solve high-stakes situations not by installing different people, but by installing better systems and by double-checking things. In my job we ensure the quality of high-risk things by having multiple layers of review and quality control. It doesn't matter how good the people are who are assigned to the task. We need double-checks. Lower-risk things are not subject to this level of quality control. Most computer software is low-risk and so it gets very little review. Unfortunately some computer software is much higher risk but does not seem to get review commensurate with its risk.
- Traster 7y ago
  Yeah, this is so important to understand. The fundamentals of creating safe systems revolve around putting in place good procedures and systems, not just by 'being better'. A great example is one of the ways that surgery was made safer was simply by introducing check-list to ensure all pre-surgery checks were done. Surgeons didn't get smarter or more diligent, they simply had to check off all the basic things before a surgery and that forced them into a situation where they couldn't miss obvious things. Just introducing that basic checklist according to studies reduced in-hospital mortality for surgeries by close to 50%.[1] [1]:https://www.who.int/patientsafety/safesurgery/faq_introduction/en/#Q4.3 https://www.who.int/patientsafety/safesurgery/faq_introducti...
jakeogh 7y ago
You are right, if this was a code issue. But it's not a code issue. It's a fundamental misunderstanding of the situation. It's not a problem that can be solved with code. People vastly underestimate the power of brains due to lifetimes of science fiction anthropomorphizing machines. https://news.ycombinator.com/item?id=21250424 https://news.ycombinator.com/item?id=21250424

AgloeDreams 7y ago

An interesting trap is people in the field of social media (FB/ Twitter) not realizing that they are in that high-stakes environment where bad privacy settings and leaks can cause death.

fredley 7y ago
I mean with regards to Facebook in particular, bad settings can cause genocide. https://www.bbc.co.uk/news/blogs-trending-45449938 https://www.bbc.co.uk/news/blogs-trending-45449938
twitter_anon 7y ago
Some of us do, as we come from (or work with people who come from) states that will perpetrate violence against dissidents. Unfortunately, there still aren't enough of us. We're trying to change that.

massysett 7y ago

(I don't know how competent you are, but it's likely that) nobody is more competent than you are. We solve high-stakes situations not by installing different people, but by installing better systems and by double-checking things. In my job we ensure the quality of high-risk things by having multiple layers of review and quality control. It doesn't matter how good the people are who are assigned to the task. We need double-checks. Lower-risk things are not subject to this level of quality control. Most computer software is low-risk and so it gets very little review. Unfortunately some computer software is much higher risk but does not seem to get review commensurate with its risk.

Traster 7y ago
Yeah, this is so important to understand. The fundamentals of creating safe systems revolve around putting in place good procedures and systems, not just by 'being better'. A great example is one of the ways that surgery was made safer was simply by introducing check-list to ensure all pre-surgery checks were done. Surgeons didn't get smarter or more diligent, they simply had to check off all the basic things before a surgery and that forced them into a situation where they couldn't miss obvious things. Just introducing that basic checklist according to studies reduced in-hospital mortality for surgeries by close to 50%.[1] [1]:https://www.who.int/patientsafety/safesurgery/faq_introduction/en/#Q4.3 https://www.who.int/patientsafety/safesurgery/faq_introducti...

jakeogh 7y ago

You are right, if this was a code issue. But it's not a code issue. It's a fundamental misunderstanding of the situation. It's not a problem that can be solved with code. People vastly underestimate the power of brains due to lifetimes of science fiction anthropomorphizing machines. https://news.ycombinator.com/item?id=21250424 https://news.ycombinator.com/item?id=21250424