9 ms·

One of the most important pieces of legislation in the UK that helped improve safety in workplaces was the Health and Safety at Work Act 1974, which placed a du

by ssl232 3y ago

One of the most important pieces of legislation in the UK that helped improve safety in workplaces was the Health and Safety at Work Act 1974, which placed a duty of care on organisations to look after the wellbeing of their staff. One of the tenets is to take near misses seriously. According to a health and safety engineer in a course I attended, near misses are like gifts from the heavens, as they show you exactly how something could go wrong despite no one getting hurt, giving organisations a chance to improve their processes. If a workplace accident does occur, the Health and Safety Executive (who enforce the act in large organisations) can levy enormous fines for preventable accidents especially where there is evidence of a near miss beforehand.

bertil 3y ago
Very true. In general, I think that piece has a bit of a European blind side when starting by “The United States leads the world in airline safety.”
- deleted 3y ago
  [deleted]
josephg 3y ago
With the number of data breaches we see cropping up, I wonder if a similar law could be written to hold companies liable for the safe handling of personal data.
- izacus 3y ago
  TBH having software engineer output in general be liable to some minimum safety, correctness and quality standard would be a god send for the world. But of course the developers will revolt against that.
  - mnau 3y ago
    How do you define "minimum safety, correctness and quality standard"?
    - deleted 3y ago
      [deleted]
    - pintxo 3y ago
      By having professionals agree on a minimal set, and writing them down?
      - mnau 3y ago
        So you propose no idea of "minimal set". If a liability is proposed, it has to be reasonably binary state: compliant/non-compliant. Just like every time, there is no concrete proposal what constitute "minimal set". That's like "make education better", with no concrete plan. We can agree on goal, but on on the method.
        josephg 3y ago
        There are several ways we could write a list of best practices. But the simplest would be to simply attach a financial cost to leaking any personal data to the open internet. This is essentially how every other industry already works: If my building falls down, the company which made it is financially liable. If I get sick from food poisoning, I can sue the companies responsible for giving me that food. And so on. We could also write a list of "best practices" - like they do in the construction and aviation industries. Things like: - Never store personal data on insecure devices (eg developer laptops which have FDE disabled or weak passwords) - Install (or at least evaluate) all security updates from your OS vendor and software dependencies - Salt all passwords in the database And so on. If you locked some competent security engineers in a room for a few days, it would be pretty easy to come up with a reasonable list of practices. There would have to be some judgement in how they're applied, just like in the construction industry. But if companies are held liable if customer data was leaked as a result of best practices not being followed, well, I imagine the situation would improve quite quickly. Its boring work. Compliance is always boring. But it might be better than the current situation, and our endless list of data breaches.
  - pintxo 3y ago
    I don't think it would be the developers that would revolt. Well some would for sure, but the real opposition will come from the C-level, as this would give their employees way too much power to say no.
  - EthicalSimilar 3y ago
    Would just cause more companies to outsource anyway.
    - josephg 3y ago
      It might do the opposite. Imagine if the company was held responsible for the quality of the software regardless of how it was made. I suspect it would be much easier to meet any quality standards if software was written in-house.
      - lmz 3y ago
        It would get outsourced to the first provider who can indemnify the company against any failures in the software. Whether or not any provider would dare to provide such a service however...
        josephg 3y ago
        However it happens, it still attaches a legal & financial cost to lazy security practices. And makes it actually in companies' best interest to do security auditing. I think that would be a net win for computer security - and consumers everywhere.
  - m_fayer 3y ago
    The agile-peddling consultants and managers would certainly be the first to the barricades.
    - izacus 3y ago
      As long as they're first to the stocks too, we're fine with it ;)
  - beAbU 3y ago
    If you want to call yourself "engineer" then at a minimum all those standards and minimum requirements should apply, no questions asked. I've heard stories of hauling civil engineers out of retirement and in front of a tribunal after some structure that collapsed before it's time, or it was found that the original design was flawed in some way. An engineer "signing off" on something actually means something, with actual stakes. Of course the "developers" will revolt against this, because they are not engineers. A developer does not get to sign off on anything, and everything they do must be scrutinised by the engineer in charge before it can be deployed. Doing this for your common run-of-the-mill CRUD app or HTML website is obviously overkill. Just like you don't need an engineer to replace a broken window or hang a new door. But when it comes to things that actually matter, like data safety and privacy, you must ensure you bring in the right culture (let alone job title).
    - avgcorrection 3y ago
      > If you want to call yourself "engineer" then at a minimum all those standards and minimum requirements should apply, no questions asked. I don’t call myself that so I’m all good. My general vocation is just “technologist”. Plus: not a protected title in my jurisdiction.
    - renewiltord 3y ago
      All those engineers need to be hauled up because they’re killing people. Software engineers by contrast are high performance: barely any fatalities and so much value created. It’s why it’s not construction firms that are the most valuable companies but software companies. You can count on software engineers to build things that won’t kill, for the most part. Other kinds of engineers, on the other hand, are constantly killing people. We need higher standards for engineers. They could learn from software folks. If you can’t do it safely don’t do it.
      - josephg 3y ago
        I have a friend who works in a hospital. Apparently the hospital software they use constantly freezes for like, 30+ seconds while they're trying to get work done. Meanwhile my friend has had her passport, entire medical history and all sorts of personal information leaked by Medibank then Optus, within a few months of each other. Neither company as far as I can tell has been held in any way to account for the blunder. Meanwhile the Post Office Scandal is rocking the UK - where a software mistake landed a bunch of completely innocent people in jail and led to multiple suicides. And don't even get me started on the impact of social media. We might not kill as many people as engineers. Maybe. But we certainly cause more than our share of harm to society.
  - toast0 3y ago
    I'm am engineer by many definitions, although I don't have or wear a stripey hat, so not the most important definition. I'm not licensed though and have no signoff rights or responsibilities. If I were to consider signing off on my work or the work of my colleagues after review, the industry would have to be completely different. I have been in the industry for more than 20 years and I can count the number of times I've had a complete specification for a project I worked on with zero hands. I can't sign off that the work is to spec it the spec is incomplete or nonexistant. Writing, consuming, and verifying specs costs time and money and adds another layer of people into the process. The cost of failure and the cost to remediate failures discovered after release for most software is too low to justify the cost of rigor. There's exceptions: software in systems involved with life safety, avionics, and the like obviously have a high cost for failure, and you can't take a crashed plane, turn it off and on and then transport the passengers. You don't get a civil engineer to sign off on a one-story house in most cases either.
    - dave4420 3y ago
      Any software system that might produce legal evidence should also be an exception… it feels a bit too much of a common case to call it an exception though. (This is in reference to the Horizon scandal, yes.)
  - trey-jones 3y ago
    I think you can prevent a lot of revolt by just finding a way of putting this other than "software engineer output" should be held to standards. The whole point of this article is that the individual is not the problem. Rather, the system should prevent the individual from being the problem. Nobody wants software quality to be regulated by people who don't know anything about software. So, probably we should be regulating ourselves. It's not an easy problem though. Can you really state a minimum safety standard for software that is quantifiable? - All tests passing? Not good enough - 100% test coverage? Not good enough - AI approved code? ha
  - deleted 3y ago
    [deleted]
  - ticviking 3y ago
    Our employers will revolt because we will either cost more or be slower. The price of an developer is already very high compared to other skilled labor
  - caskstrength 3y ago
    > But of course the developers will revolt against that. Why would developers revolt? OTOH users would definitely be unhappy when every little app for their iPhone suddenly costs as much as an airplane ticket.
  - HeyLaughingBoy 3y ago
    > and quality standard This is my life: https://en.wikipedia.org/wiki/IEC_62304 https://en.wikipedia.org/wiki/IEC_62304
    - alfanick 3y ago
      Oh I also unfortunately know this standard by heart, hi!
  - rcxdude 3y ago
    The EU is doing something like this with some proposed legislation. Basically it's a CE mark for software. The current version is very much a bare minimum, though. But it does impose liability on people selling software.
- jacquesm 3y ago
  In the EU this already happened and slowly but surely it is having an effect. Some US companies are still fighting it tooth and nail though.
kqr 3y ago
Indeed. - An accident is a combination of the system being in a hazardous state and unfavourable environmental conditions that turn the hazard into an accident. - A near miss is when the system was in the hazardous state but the environmental conditions were in our favour so the accident didn't happen. We can't control environmental conditions, we can just make sure to keep the system out of the hazardous state. This means a near miss contains all we need to perform a proper post mortem. The environmental conditions are really immaterial.
varjag 3y ago
(This is a harmonized EU legislation)
m_fayer 3y ago
By this logic, the recent Boeing 737 incident should be considered a gift from the heavens. Which I think is actually a good way to look at it, though we don’t yet know whether this gift will be squandered.
- joshfee 3y ago
  I think a company taking things seriously would view it as a gift from the heavens, but the fact that Boeing is seeking an exemption for certification doesn't really seem like they're putting safety ahead of profits.
  - kqr 3y ago
    Like many large corporations they are no longer in the product development business but the capital management business.
  - jacobr1 3y ago
    On the other hand their customers and regulators are taking it seriously, which will enable mitigation.
mathstuf 3y ago
> One of the most important pieces of legislation in the UK that helped improve safety in workplaces was the Health and Safety at Work Act 1974, which placed a duty of care on organisations to look after the wellbeing of their staff. I've not been following closely, but I suspect the Post Office did not follow this. Are there ramifications happening along these lines for the Post Office fiasco? Or is it already a big enough thing that this would just be "oh, and also that too I guess" kind of thing?
- HideousKojima 3y ago
  Amongst many other complications in the UK Post Office scandal that make it different from what's being discussed here: the victims in the post office scandal were franchisees, not employees.
dr_orpheus 3y ago
Near misses are absolutely like gifts from the heavens. I was part of a post-investigation for a near miss that would've looked like this if things had gone a bit differently: https://en.wikipedia.org/wiki/NOAA-19#Damage_during_manufacture https://en.wikipedia.org/wiki/NOAA-19#Damage_during_manufact...
- deleted 3y ago
  [deleted]
toss1 3y ago
YEs. Almost everyone has seen at factories the prominently posted signs updated daily: "XX Days Since a Time Lost Accident". Some years ago, I noticed in some more technically advanced shops a change: "XX Days Since a Near Miss", along with an invitation and method to report near-misses. Seems like progress; a near-miss noticed is indeed a real gift, if followed-up.
gav 3y ago
One interesting read on the topic of near misses and High-Reliability Organizations is the paper "Going Solid", which has a great summary here: https://cohost.org/mononcqc/post/888958-paper-going-solid https://cohost.org/mononcqc/post/888958-paper-going-solid
mattrick 3y ago
I've been obsessed with the US Chemical Safety Board videos on YouTube that describe in great detail the events that lead up to industrial accidents. One of the common themes I've seen among them is that there's usually some sort of warning sign or near miss that goes ignored by the people responsible for them since they don't cause any major damage. Then a few days or months later that system fails in an entirely predictable way with catastrophic consequences. A good example of this is the fatal phosgene gas release at a DuPont chemical plant[1]. [1]: https://www.youtube.com/watch?v=ISNGimMXL7M https://www.youtube.com/watch?v=ISNGimMXL7M
- nayuki 3y ago
  Yeah, shoutouts to the USCSB channel; I've been watching them for years. https://www.youtube.com/@USCSB/videos https://www.youtube.com/@USCSB/videos Back to aviation, I can recommend these channels that explain air accidents: https://www.youtube.com/@MentourPilot/videos https://www.youtube.com/@MentourPilot/videos , https://www.youtube.com/@MiniAirCrashInvestigation/videos https://www.youtube.com/@MiniAirCrashInvestigation/videos , https://www.youtube.com/@AirSafetyInstitute/videos https://www.youtube.com/@AirSafetyInstitute/videos
- rcxdude 3y ago
  It is worth keeping in mind that you don't see the other side of the equation in these reports: how many warning signs and near misses that didn't result in a major accident. Part of that is just the odds, and why people and organisations can become complacent to them, and part of it is that while most of them may be addressed, some can still slip through the cracks.

by ssl232 3y ago

bertil 3y ago

Very true. In general, I think that piece has a bit of a European blind side when starting by “The United States leads the world in airline safety.”

deleted 3y ago
[deleted]

josephg 3y ago

With the number of data breaches we see cropping up, I wonder if a similar law could be written to hold companies liable for the safe handling of personal data.

izacus 3y ago
TBH having software engineer output in general be liable to some minimum safety, correctness and quality standard would be a god send for the world. But of course the developers will revolt against that.
- mnau 3y ago
  How do you define "minimum safety, correctness and quality standard"?
  - deleted 3y ago
    [deleted]
  - pintxo 3y ago
    By having professionals agree on a minimal set, and writing them down?
    - mnau 3y ago
      So you propose no idea of "minimal set". If a liability is proposed, it has to be reasonably binary state: compliant/non-compliant. Just like every time, there is no concrete proposal what constitute "minimal set". That's like "make education better", with no concrete plan. We can agree on goal, but on on the method.
      - josephg 3y ago
        There are several ways we could write a list of best practices. But the simplest would be to simply attach a financial cost to leaking any personal data to the open internet. This is essentially how every other industry already works: If my building falls down, the company which made it is financially liable. If I get sick from food poisoning, I can sue the companies responsible for giving me that food. And so on. We could also write a list of "best practices" - like they do in the construction and aviation industries. Things like: - Never store personal data on insecure devices (eg developer laptops which have FDE disabled or weak passwords) - Install (or at least evaluate) all security updates from your OS vendor and software dependencies - Salt all passwords in the database And so on. If you locked some competent security engineers in a room for a few days, it would be pretty easy to come up with a reasonable list of practices. There would have to be some judgement in how they're applied, just like in the construction industry. But if companies are held liable if customer data was leaked as a result of best practices not being followed, well, I imagine the situation would improve quite quickly. Its boring work. Compliance is always boring. But it might be better than the current situation, and our endless list of data breaches.
- pintxo 3y ago
  I don't think it would be the developers that would revolt. Well some would for sure, but the real opposition will come from the C-level, as this would give their employees way too much power to say no.
- EthicalSimilar 3y ago
  Would just cause more companies to outsource anyway.
  - josephg 3y ago
    It might do the opposite. Imagine if the company was held responsible for the quality of the software regardless of how it was made. I suspect it would be much easier to meet any quality standards if software was written in-house.
    - lmz 3y ago
      It would get outsourced to the first provider who can indemnify the company against any failures in the software. Whether or not any provider would dare to provide such a service however...
      - josephg 3y ago
        However it happens, it still attaches a legal & financial cost to lazy security practices. And makes it actually in companies' best interest to do security auditing. I think that would be a net win for computer security - and consumers everywhere.
- m_fayer 3y ago
  The agile-peddling consultants and managers would certainly be the first to the barricades.
  - izacus 3y ago
    As long as they're first to the stocks too, we're fine with it ;)
- beAbU 3y ago
  If you want to call yourself "engineer" then at a minimum all those standards and minimum requirements should apply, no questions asked. I've heard stories of hauling civil engineers out of retirement and in front of a tribunal after some structure that collapsed before it's time, or it was found that the original design was flawed in some way. An engineer "signing off" on something actually means something, with actual stakes. Of course the "developers" will revolt against this, because they are not engineers. A developer does not get to sign off on anything, and everything they do must be scrutinised by the engineer in charge before it can be deployed. Doing this for your common run-of-the-mill CRUD app or HTML website is obviously overkill. Just like you don't need an engineer to replace a broken window or hang a new door. But when it comes to things that actually matter, like data safety and privacy, you must ensure you bring in the right culture (let alone job title).
  - avgcorrection 3y ago
    > If you want to call yourself "engineer" then at a minimum all those standards and minimum requirements should apply, no questions asked. I don’t call myself that so I’m all good. My general vocation is just “technologist”. Plus: not a protected title in my jurisdiction.
  - renewiltord 3y ago
    All those engineers need to be hauled up because they’re killing people. Software engineers by contrast are high performance: barely any fatalities and so much value created. It’s why it’s not construction firms that are the most valuable companies but software companies. You can count on software engineers to build things that won’t kill, for the most part. Other kinds of engineers, on the other hand, are constantly killing people. We need higher standards for engineers. They could learn from software folks. If you can’t do it safely don’t do it.
    - josephg 3y ago
      I have a friend who works in a hospital. Apparently the hospital software they use constantly freezes for like, 30+ seconds while they're trying to get work done. Meanwhile my friend has had her passport, entire medical history and all sorts of personal information leaked by Medibank then Optus, within a few months of each other. Neither company as far as I can tell has been held in any way to account for the blunder. Meanwhile the Post Office Scandal is rocking the UK - where a software mistake landed a bunch of completely innocent people in jail and led to multiple suicides. And don't even get me started on the impact of social media. We might not kill as many people as engineers. Maybe. But we certainly cause more than our share of harm to society.
- toast0 3y ago
  I'm am engineer by many definitions, although I don't have or wear a stripey hat, so not the most important definition. I'm not licensed though and have no signoff rights or responsibilities. If I were to consider signing off on my work or the work of my colleagues after review, the industry would have to be completely different. I have been in the industry for more than 20 years and I can count the number of times I've had a complete specification for a project I worked on with zero hands. I can't sign off that the work is to spec it the spec is incomplete or nonexistant. Writing, consuming, and verifying specs costs time and money and adds another layer of people into the process. The cost of failure and the cost to remediate failures discovered after release for most software is too low to justify the cost of rigor. There's exceptions: software in systems involved with life safety, avionics, and the like obviously have a high cost for failure, and you can't take a crashed plane, turn it off and on and then transport the passengers. You don't get a civil engineer to sign off on a one-story house in most cases either.
  - dave4420 3y ago
    Any software system that might produce legal evidence should also be an exception… it feels a bit too much of a common case to call it an exception though. (This is in reference to the Horizon scandal, yes.)
- trey-jones 3y ago
  I think you can prevent a lot of revolt by just finding a way of putting this other than "software engineer output" should be held to standards. The whole point of this article is that the individual is not the problem. Rather, the system should prevent the individual from being the problem. Nobody wants software quality to be regulated by people who don't know anything about software. So, probably we should be regulating ourselves. It's not an easy problem though. Can you really state a minimum safety standard for software that is quantifiable? - All tests passing? Not good enough - 100% test coverage? Not good enough - AI approved code? ha
- deleted 3y ago
  [deleted]
- ticviking 3y ago
  Our employers will revolt because we will either cost more or be slower. The price of an developer is already very high compared to other skilled labor
- caskstrength 3y ago
  > But of course the developers will revolt against that. Why would developers revolt? OTOH users would definitely be unhappy when every little app for their iPhone suddenly costs as much as an airplane ticket.
- HeyLaughingBoy 3y ago
  > and quality standard This is my life: https://en.wikipedia.org/wiki/IEC_62304 https://en.wikipedia.org/wiki/IEC_62304
  - alfanick 3y ago
    Oh I also unfortunately know this standard by heart, hi!
- rcxdude 3y ago
  The EU is doing something like this with some proposed legislation. Basically it's a CE mark for software. The current version is very much a bare minimum, though. But it does impose liability on people selling software.
jacquesm 3y ago
In the EU this already happened and slowly but surely it is having an effect. Some US companies are still fighting it tooth and nail though.

kqr 3y ago

Indeed. - An accident is a combination of the system being in a hazardous state and unfavourable environmental conditions that turn the hazard into an accident. - A near miss is when the system was in the hazardous state but the environmental conditions were in our favour so the accident didn't happen. We can't control environmental conditions, we can just make sure to keep the system out of the hazardous state. This means a near miss contains all we need to perform a proper post mortem. The environmental conditions are really immaterial.

varjag 3y ago

(This is a harmonized EU legislation)

m_fayer 3y ago

By this logic, the recent Boeing 737 incident should be considered a gift from the heavens. Which I think is actually a good way to look at it, though we don’t yet know whether this gift will be squandered.

joshfee 3y ago
I think a company taking things seriously would view it as a gift from the heavens, but the fact that Boeing is seeking an exemption for certification doesn't really seem like they're putting safety ahead of profits.
- kqr 3y ago
  Like many large corporations they are no longer in the product development business but the capital management business.
- jacobr1 3y ago
  On the other hand their customers and regulators are taking it seriously, which will enable mitigation.

mathstuf 3y ago

> One of the most important pieces of legislation in the UK that helped improve safety in workplaces was the Health and Safety at Work Act 1974, which placed a duty of care on organisations to look after the wellbeing of their staff. I've not been following closely, but I suspect the Post Office did not follow this. Are there ramifications happening along these lines for the Post Office fiasco? Or is it already a big enough thing that this would just be "oh, and also that too I guess" kind of thing?

HideousKojima 3y ago
Amongst many other complications in the UK Post Office scandal that make it different from what's being discussed here: the victims in the post office scandal were franchisees, not employees.

dr_orpheus 3y ago

Near misses are absolutely like gifts from the heavens. I was part of a post-investigation for a near miss that would've looked like this if things had gone a bit differently: https://en.wikipedia.org/wiki/NOAA-19#Damage_during_manufacture https://en.wikipedia.org/wiki/NOAA-19#Damage_during_manufact...

deleted 3y ago
[deleted]

toss1 3y ago

YEs. Almost everyone has seen at factories the prominently posted signs updated daily: "XX Days Since a Time Lost Accident". Some years ago, I noticed in some more technically advanced shops a change: "XX Days Since a Near Miss", along with an invitation and method to report near-misses. Seems like progress; a near-miss noticed is indeed a real gift, if followed-up.

gav 3y ago

One interesting read on the topic of near misses and High-Reliability Organizations is the paper "Going Solid", which has a great summary here: https://cohost.org/mononcqc/post/888958-paper-going-solid https://cohost.org/mononcqc/post/888958-paper-going-solid

mattrick 3y ago

I've been obsessed with the US Chemical Safety Board videos on YouTube that describe in great detail the events that lead up to industrial accidents. One of the common themes I've seen among them is that there's usually some sort of warning sign or near miss that goes ignored by the people responsible for them since they don't cause any major damage. Then a few days or months later that system fails in an entirely predictable way with catastrophic consequences. A good example of this is the fatal phosgene gas release at a DuPont chemical plant[1]. [1]: https://www.youtube.com/watch?v=ISNGimMXL7M https://www.youtube.com/watch?v=ISNGimMXL7M

nayuki 3y ago
Yeah, shoutouts to the USCSB channel; I've been watching them for years. https://www.youtube.com/@USCSB/videos https://www.youtube.com/@USCSB/videos Back to aviation, I can recommend these channels that explain air accidents: https://www.youtube.com/@MentourPilot/videos https://www.youtube.com/@MentourPilot/videos , https://www.youtube.com/@MiniAirCrashInvestigation/videos https://www.youtube.com/@MiniAirCrashInvestigation/videos , https://www.youtube.com/@AirSafetyInstitute/videos https://www.youtube.com/@AirSafetyInstitute/videos
rcxdude 3y ago
It is worth keeping in mind that you don't see the other side of the equation in these reports: how many warning signs and near misses that didn't result in a major accident. Part of that is just the odds, and why people and organisations can become complacent to them, and part of it is that while most of them may be addressed, some can still slip through the cracks.