7 ms·

Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosage

by dcow 5y ago

Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. These mistakes are grave and should be costly. Pilots aren’t allowed small fuckups when flying planes. The problem is that companies don’t want to invest the time in good security because it’s less profitable. Or, they are old and decrepit. If the company runs critical infrastructure, then peoples’ safety is at risk. If they don’t then surely a new fad can take their place. Why should we tolerate small fuckups and negligence that only serves a company's shareholders’ interests? I do know places that don’t allow random individuals to tail you into a building and have cameras on entryways so you can’t sit there and attempt to pick the lock endlessly. And they have good locks. Just because it’s harder to “see” digital systems doesn't make them any less relevant to secure. People in software more-so on average just don't understand how the internet works so they don’t know where to put the cameras and how to identify the shady individual trying to tail someone through an entryway that requires authorization. This problem is fixable but not when the prime directive is “ship this experimental product as fast as you possibly can or I’ll hire an offshore contractor”. And not when your extent of experience with operating software is running a cute web server on your dev laptop. People are actually doing really dumb shit. Don’t pretend that everyone is writing good software and following best practices and they just happen to get hacked by a script kiddie “oh no”. These are sophisticated actors targeting mature but vulnerable companies that have chosen to selfishly forgo healthy security practices. Again, why do we want these machines participating in society? Why don’t we want them replaced with more secure versions?

ipaddr 5y ago
Doctors are allowed many small fuckups and some major. Docotors have insurance and medical associations with the best lawyers.
sokoloff 5y ago
> Pilots aren’t allowed small fuckups when flying planes. Pilots are allowed small fuckups all the time when flying planes. I've got around 1500 hours and haven't had a perfect flight yet. When I stop flying, I expect to have 3000 hours and still to not have had a perfect flight. Aviation is designed around having redundancies, tolerances, and contingencies for these small fuckups.
- rollcat 5y ago
  > Aviation is designed around having redundancies, tolerances, and contingencies for these small fuckups. Are there lessons for software engineering to be learned from aviation?
  - sokoloff 5y ago
    I think so. I’ve enshrined an adaptation of FAR 91.3 into our technical operations handbook (which is incorporated into our SOX compliance policies). The intention was to make sure that a specific group is clearly, unambiguously, and completely in charge. They are the final authority as to the production operation. (We call this group “Problem Management”, though in the ITIL framework this is more akin to critical incident management) “Problem Management is exercising emergency authority here” clears away any ambiguity and the confidence this gives them means they are free to act to resolve an incident rather than wonder who they need to call to give the blessing. https://www.law.cornell.edu/cfr/text/14/91.3 https://www.law.cornell.edu/cfr/text/14/91.3 When onboarding new members to that group, we go over this, including the origin story as part of helping them understand how (and that) we expect them to use this. In other areas? Probably. Aviation tries not to bet lives on a single piece of equipment working perfectly. Where that’s unavoidable (like the Jesus nut and pin on a helicopter), inspection and maintenance procedures are more stringent.
  - klodolph 5y ago
    SREs I know talk about aviation incidents all the time, spinning it into a “what lessons do we learn?” - Aviation incidents are investigated and the investigation is typically bent on discovering root causes, not assigning blame. Do the same thing with bugs & security breaches, you’ll learn more. - Pilots spend time in simulators practicing unusual situations like water landings or engine failure—do the same thing in software, simulate failure scenarios and intrusion events and see how your engineers and technicians handle those events. You develop new training materials and redesign systems based on what you learned in simulations. - Pilots rely heavily on checklists rather than memory to ensure that all steps have been completed. This translates well to QA, deployment, various tasks people you need to do in production like migrations or whatnot. - Pilots require good UX to do their job and cannot be overloaded with too many tasks. Sometime an incident will be traced to dangerous things that don’t seem dangerous when you look at the UI (like thrust reversers), or traced to systems that assigned too many duties to pilots (who get overloaded). For example, if you have a button that deletes a database, it should require a serious confirmation step, like a molly guard on a plane. - Tasks must be explicitly delegated to individual people (which is what happens on an airplane). For example, in the Gimli Glider incident, the flight engineer role was eliminated and the engineer's responsibilities were not explicitly delegated to the pilot or copilot, but assigned to “both”. Same should be true for tasks in software engineering and incident response. We also talk about how medicine seems uninterested in learning these lessons, for various reasons.
secondaryacct 5y ago
You wrote a long diatribe I admit I didnt read but the first paragraph, but just to say this: this is not profitable because competitors dont do it. If competitors have to do it because no insurance, suddenly the cycle starts and everyone starts doing it. It s a risk assessment, as we pile crap on top of crap in deeper and deeper systems, security holes grow more numerous to a point it will be profitable to cover them better than the competition.
rectang 5y ago
Doctors don't get blamed when someone they treat subsequently gets shot. Civil engineers don't get blamed when a saboteur blows up the bridge they designed.
- dcow 5y ago
  Have you ever driven under the base of a large bridge (you can’t it’s restricted)? I’m sure you’ve seen the road spikes and the anti car bomb barricades put around high profile buildings… when something is valuable and a possible target of attack you protect it and make it less vulnerable.
- ajkjk 5y ago
  This is not a good analogy. Civil engineers would be blamed if a force they were supposed to compensate for, like a storm, destroys a bridge they designed.
  - Spivak 5y ago
    The grey area is if attackers built a weather machine to create exactly the perfect storm tailored to stress a small design defect until it fails. Even more so when the probability of such a storm occurring naturally is 0.
    - Gwarzo 5y ago
      That's a great analogy.
    - mastax 5y ago
      The thing is, our current understanding is that such a weather machine doesn't exist so it isn't necessary to design around it. In the cyber security world such machines absolutely do exist so we should be expected to design around them.
warkdarrior 5y ago
Engineers design for resilience to physical failures, whose types are known and can be estimated, predicted, and modeled. Cybersecurity has to deal with human attackers, which cannot be modeled.
- syshum 5y ago
  >>Cybersecurity has to deal with human attackers, which cannot be modeled. It absolutely can, it is called Zero Trust, as has been a cyber security model for many many years now The problem is most organizations have their entire business workflow around the Ring Fence model, and organizations is extremely prone to "we have always done it that way" when it comes to business processes so they expect IT to buy some software, or some widget to bolt on to the ring fence that will protect them Businesses need to completely rework their internal methodologies, and processes which is something most companies refuse to do
- staticassertion 5y ago
  We model human attackers all the time, it's probably one of the core components of the role.
- throw10920 5y ago
  Human behavior is absolutely modelable - usefully, even ("all models are wrong but some are useful") - and people in security do it all the time. Moreover, engineers also design for resilience to human "attackers" in physical systems all the time. If anything, cybersecurity is easier because you can leverage the work of tens of thousands of other people in your tooling. You wanna re-write ssh or fail2ban yourself?
Apes 5y ago
"Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. These mistakes are grave and should be costly. Pilots aren’t allowed small fuckups when flying planes." What a beautiful dream.
- dcow 5y ago
  I’m not saying it doesn't happen. I’m saying when it does there are grave consequences. Malpractice insurance is a thing and a similar analog to ransomeware insurance.
yibg 5y ago
Yet doctors have insurance
- dcow 5y ago
  Which generally covers civil lawsuits up to the point where it’s determined that the doctor was grossly negligent. Then they aren't allowed to practice anymore.
  - Griffinsauce 5y ago
    You just jumped from "small fuckup" to "grossly negligent", come on now.
deleted 5y ago
[deleted]
charcircuit 5y ago
>Engineers aren’t allowed “one small fuckup” when building bridges and buildings Yes they are. When estimating the forces you need to handle you can round up. This means that if you made a small mistake it like won't matter since you planned to handle much more than necessary. >in the OR or when calculating dosages Dosages don't need to me exact. There are some tolerances and it's easy to imagine a system to eliminate most of human error from this process. >Pilots aren’t allowed small fuckups when flying planes. Sure they are. A mistake doesn't mean your plane will crash or fall out of the sky. Planes can already fly themselves. >Why should we tolerate small fuckups and negligence that only serves a company's shareholders’ interests? Because hindsight is 20/20.
- cuu508 5y ago
  “Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands.” - Unknown
dboreham 5y ago
A bridge is O(10^6) less complex than a F500 IT setup.
mcot2 5y ago
Bad anology because the system is broken all the way down. If what you are saying is true a bridge engineer wouldn’t even be able to build a bridge because their are no materials in the world to build a safe bridge out of.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
unclebucknasty 5y ago
>Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR These are really poor analogies. A better analogy would be engineers who then had to protect those bridges from attack via a number of different modes, including drone and cruise missle strikes; and seemingly legitimate travelers on the bridge who are actually malicious. Likewise, a better analogy would be doctors who had to contend with nurses who are bad actors, substituting saline for needed drugs or deliberately administering higher than prescribed dosages, etc.
indymike 5y ago
> The problem is that companies don’t want to invest the time in good security because it’s less profitable. Or, they are old and decrepit. If the company runs critical infrastructure, then peoples’ safety is at risk. Most of the time, it's that a non-technical decision maker decides to take a badly mis-calculated risk. "Bob in sales can't access the files he needs." "IT give him admin rights! Now! no... I don't even understand or care to understand that access control mumbo-jumbo you are talking about." A week later, "Hey, you know we just got bitlockered. Bob's laptop got virused, and because we gave him admin rights his machine encrypted all the things AND the backup server too."
markus_zhang 5y ago
> Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. I do agree with the rest of the post but this is definitely NOT true. Your house probably has small issues here and there if you are willing to pay for an inspector. Doctors fucking up things is not news too, plus you never know if there is "one small fuckup" as long as you don't feel particularly bad afterwards. The thing is I think you guys regard real world engineering too high. Small fuck ups happen all day.
- nousermane 5y ago
  Exactly. Threat modelling is a thing outside IT secutiry too. Youtube channel "MentourPilot" [0] has some excellent examples of how civil aviation was built to withstand "small fuckups": with "Swiss cheese" model. [1] [0] https://www.youtube.com/c/MentourPilotaviation/videos https://www.youtube.com/c/MentourPilotaviation/videos [1] https://en.wikipedia.org/wiki/Swiss_cheese_model https://en.wikipedia.org/wiki/Swiss_cheese_model
  - dcow 5y ago
    I think most of the quibbling is over the semantics of “one small fuckup”. It’s in quotes for two reasons 1) because the comment I was responding to worded it that way it and 2) I’m deliberatly calling that characterization bullshit: one _truly_ small fuckup obviously doesn't leave you vulnerable to ransomware. Systems need to be able to withstand of small things, have redundancies, engineers build in tolerances, yada yada.. of course! Were talking about *things that leave your entire business in the hands of ransom ware criminals* or *things that kill people* or *things that crash planes* or *things that cause bridges to collapse*. Those things can’t be tolerated by definition because they are fatal failures. That’s my point. We would not tolerate companies that build bridges that collapsed all the time and keep letting them build bridges so why should we tolerate companies that get hacked repetitively by ransomeware groups? Insurers are running because it’s not good business to insure these risks. Sounds like the market is working just fine.
- hulitu 5y ago
seized 5y ago
As someone in the healthcare vendor side of things.... Doctors make a lot of mistakes. There is plenty of software, processes, etc around trying to catch and mitigate those mistakes.
mdaidc 5y ago
What if I told you that an adversary can place explosives at different parts of the bridge and set them off to see if the bridge goes down? and they can try over and over again until they succeed once, and then you are screwed. this is the asymmetric nature of the threat. The governments of the west need to align and go after the offenders or sanction countries that give them refuge. It's impossible for a single company to defend itself against adversaries that can keep trying over and over again with impunity.
MattGaiser 5y ago
Engineers, doctors, and pilots are not dealing with an equally clever human on the other end trying to beat them. They wouldn't do well if someone were casually changing a number in their calculations or swapping bottles or screwing with the instruments.
deleted 5y ago
[deleted]
registeredcorn 5y ago
> Pilots aren’t allowed small f*ups when flying planes. I don't blame you if you don't take a strangers word on this, but I would strongly encourage you to look at this YouTube channel VASAviation (https://www.youtube.com/user/victor981994 https://www.youtube.com/user/victor981994) or watch the excellent movie, "Charlie Victor Romeo", which cover in extreme detail the routine, regular, and on-going mistakes that are made in aviation. Mistakes happen in aviation ALL THE TIME; they just don't cause death or crashes in every event. I can't speak for Engineers and Doctors, but for: pilots, ground crews, flight mechanics, and ATC I have a tiny soapbox. Each of these jobs routinely on a daily basis have mistakes that are made. Sometimes the errors are minor. Other times they are major, and aren't discovered for months or years after the fact, which either led to the loss of millions of dollars worth of equipment, or worse. Sure, CEOs and such are usually held responsible for those mistakes, but the aviation community focuses more on the ideals of, "Every rule, written in blood" mentality. Things are permitted, until they aren't. This is precisely because it caused a disaster. Things are safe(r) now because they were significantly less safe just a few years ago. The aviation community is focused on preventative actions as a result of the ongoing looming threat of failures. There is no way to avoid them entirely. There is an understanding that when you are in the air, there is NEVER certainty. You are NEVER safe. You should constantly be concerned about what can (and will) eventually go wrong. It's only a matter of how bad the failure will be. TLDR Aviation isn't safe because they "don't allow mistakes" but because they have a rigid and foundational system around how to deal with failure.
908B64B197 5y ago
> Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. These mistakes are grave and should be costly. Pilots aren’t allowed small fuckups when flying planes. That's wrong. Engineers aren't allowed to fail. They can have as many "fuckups" along the way, as long as they manage to compensate for it and maintain safety it's not a failure. Doctors... well, this profession has a group mentality of shielding one another from publicly acknowledging and documenting mistakes. Truth is ransomware it utterly useless against a simple back-up and recovery scenario. Any competent engineers can and will deliver one if paid to. It's that companies don't see the point and don't care. As long as the insurance premium is less than what it will cost to implement such a thing they will keep getting ransomed.
dan-robertson 5y ago
I think your first paragraph is bogus for two reasons, which I’ll leave at the end[1] because I’d rather respond to you main point than the silly reactionary thing you wrote at the top. I feel like you aren’t really responding to the claim that having good security is extremely hard and expensive and that it seems bad for every company to have to spend a lot of money on a problem which is likely far from their core competency. The thing you’re actually asking for is for a lot of businesses to go bankrupt trying to be perfect at something that is not their traditional domain and then you make some claim that they have the audacity to try to create value for their shareholders instead? Isn’t it outrageous that companies might try to do literally the thing companies are meant to do instead of being experts in computer security. I suppose you would counter that it is easy or cheap or imperative for them to be secure but, in the world where hacking is relatively easy, cheap, and profitable ($1-10mm sounds like a lot to a person but not to a country or reasonably sized business) and there are myriad ways to make trivial-seeming configuration errors in software that leave you open to attack, I think it is neither easy nor cheap. And frankly I think it is not imperative for businesses either because their purpose is to make money and while cybersecurity is an important risk to worry about, they can only invest in it so long as they are making money from the thing they are actually meant to be good at. [1] regarding the first graf: 1. Mostly small errors by engineers or doctors or pilots are… small errors. Bridges won’t collapse if the concrete was mixed slightly too wet, and bigger cracks are usually noticed and mitigated in good time. Doctors get dosages wrong all the time (which is why there are attempts to train nurses to question everything the doctors say) and while malpractice is definitely a thing, there are lots of cases where things go wrong at a small scale and are either fixed or unnoticed. If you ever ask a doctor about this in a social setting they will probably have plenty of examples. And if you get something wrong in a plane you are unlikely to immediately crash. Even landings can be aborted quite late. Big planes crash very infrequently and I think it is silly to therefore assume that the pilots are generally infallible. So I claim that your direct statement that these are not allowed is false. 2. I also claim the analogy is bad because the examples you give are not adversarial. The patients are generally not trying to trick their doctor into making the wrong prescription and Athena geography isn’t going to shift and change to try to exploit weaknesses in the bridge built across it. The pilot flies high above the ground with separation from other planes and various safety systems. Even in an unlikely event like an engine stall, there are possibilities to recover. In computer security the threats are fast and numerous so any small error can be quickly magnified.
ByteJockey 5y ago
> Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. These mistakes are grave and should be costly. Pilots aren’t allowed small fuckups when flying planes. That's BS and you know it. Any system that relies on humans being perfect fails rather quickly. While major incompetence is generally punished, all of those systems you describe have ways built in to deal with routine human failures.
Griffinsauce 5y ago
> Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. Yes they are. You should talk to an actual doctor before making such claims.
RcouF1uZ4gsC 5y ago
> Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. But engineers aren’t held responsible if a malicious actor drivers a bulldozer into one of the bridge’s columns and brings it down. Same with a doctor. If someone is maliciously messing with the meds, then they are not responsible for over or under dosing. With these security incidences, we are dealing with malicious actors trying to being down the system. I don’t think you can use analogies from civil engineering or medicine for that.

by dcow 5y ago

ipaddr 5y ago

Doctors are allowed many small fuckups and some major. Docotors have insurance and medical associations with the best lawyers.

sokoloff 5y ago

> Pilots aren’t allowed small fuckups when flying planes. Pilots are allowed small fuckups all the time when flying planes. I've got around 1500 hours and haven't had a perfect flight yet. When I stop flying, I expect to have 3000 hours and still to not have had a perfect flight. Aviation is designed around having redundancies, tolerances, and contingencies for these small fuckups.

rollcat 5y ago
> Aviation is designed around having redundancies, tolerances, and contingencies for these small fuckups. Are there lessons for software engineering to be learned from aviation?
- sokoloff 5y ago
  I think so. I’ve enshrined an adaptation of FAR 91.3 into our technical operations handbook (which is incorporated into our SOX compliance policies). The intention was to make sure that a specific group is clearly, unambiguously, and completely in charge. They are the final authority as to the production operation. (We call this group “Problem Management”, though in the ITIL framework this is more akin to critical incident management) “Problem Management is exercising emergency authority here” clears away any ambiguity and the confidence this gives them means they are free to act to resolve an incident rather than wonder who they need to call to give the blessing. https://www.law.cornell.edu/cfr/text/14/91.3 https://www.law.cornell.edu/cfr/text/14/91.3 When onboarding new members to that group, we go over this, including the origin story as part of helping them understand how (and that) we expect them to use this. In other areas? Probably. Aviation tries not to bet lives on a single piece of equipment working perfectly. Where that’s unavoidable (like the Jesus nut and pin on a helicopter), inspection and maintenance procedures are more stringent.
- klodolph 5y ago
  SREs I know talk about aviation incidents all the time, spinning it into a “what lessons do we learn?” - Aviation incidents are investigated and the investigation is typically bent on discovering root causes, not assigning blame. Do the same thing with bugs & security breaches, you’ll learn more. - Pilots spend time in simulators practicing unusual situations like water landings or engine failure—do the same thing in software, simulate failure scenarios and intrusion events and see how your engineers and technicians handle those events. You develop new training materials and redesign systems based on what you learned in simulations. - Pilots rely heavily on checklists rather than memory to ensure that all steps have been completed. This translates well to QA, deployment, various tasks people you need to do in production like migrations or whatnot. - Pilots require good UX to do their job and cannot be overloaded with too many tasks. Sometime an incident will be traced to dangerous things that don’t seem dangerous when you look at the UI (like thrust reversers), or traced to systems that assigned too many duties to pilots (who get overloaded). For example, if you have a button that deletes a database, it should require a serious confirmation step, like a molly guard on a plane. - Tasks must be explicitly delegated to individual people (which is what happens on an airplane). For example, in the Gimli Glider incident, the flight engineer role was eliminated and the engineer's responsibilities were not explicitly delegated to the pilot or copilot, but assigned to “both”. Same should be true for tasks in software engineering and incident response. We also talk about how medicine seems uninterested in learning these lessons, for various reasons.

secondaryacct 5y ago

You wrote a long diatribe I admit I didnt read but the first paragraph, but just to say this: this is not profitable because competitors dont do it. If competitors have to do it because no insurance, suddenly the cycle starts and everyone starts doing it. It s a risk assessment, as we pile crap on top of crap in deeper and deeper systems, security holes grow more numerous to a point it will be profitable to cover them better than the competition.

rectang 5y ago

Doctors don't get blamed when someone they treat subsequently gets shot. Civil engineers don't get blamed when a saboteur blows up the bridge they designed.

dcow 5y ago
Have you ever driven under the base of a large bridge (you can’t it’s restricted)? I’m sure you’ve seen the road spikes and the anti car bomb barricades put around high profile buildings… when something is valuable and a possible target of attack you protect it and make it less vulnerable.
ajkjk 5y ago
This is not a good analogy. Civil engineers would be blamed if a force they were supposed to compensate for, like a storm, destroys a bridge they designed.
- Spivak 5y ago
  The grey area is if attackers built a weather machine to create exactly the perfect storm tailored to stress a small design defect until it fails. Even more so when the probability of such a storm occurring naturally is 0.
  - Gwarzo 5y ago
    That's a great analogy.
  - mastax 5y ago
    The thing is, our current understanding is that such a weather machine doesn't exist so it isn't necessary to design around it. In the cyber security world such machines absolutely do exist so we should be expected to design around them.

warkdarrior 5y ago

Engineers design for resilience to physical failures, whose types are known and can be estimated, predicted, and modeled. Cybersecurity has to deal with human attackers, which cannot be modeled.

syshum 5y ago
>>Cybersecurity has to deal with human attackers, which cannot be modeled. It absolutely can, it is called Zero Trust, as has been a cyber security model for many many years now The problem is most organizations have their entire business workflow around the Ring Fence model, and organizations is extremely prone to "we have always done it that way" when it comes to business processes so they expect IT to buy some software, or some widget to bolt on to the ring fence that will protect them Businesses need to completely rework their internal methodologies, and processes which is something most companies refuse to do
staticassertion 5y ago
We model human attackers all the time, it's probably one of the core components of the role.
throw10920 5y ago
Human behavior is absolutely modelable - usefully, even ("all models are wrong but some are useful") - and people in security do it all the time. Moreover, engineers also design for resilience to human "attackers" in physical systems all the time. If anything, cybersecurity is easier because you can leverage the work of tens of thousands of other people in your tooling. You wanna re-write ssh or fail2ban yourself?

Apes 5y ago

"Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. These mistakes are grave and should be costly. Pilots aren’t allowed small fuckups when flying planes." What a beautiful dream.

dcow 5y ago
I’m not saying it doesn't happen. I’m saying when it does there are grave consequences. Malpractice insurance is a thing and a similar analog to ransomeware insurance.

yibg 5y ago

Yet doctors have insurance

dcow 5y ago
Which generally covers civil lawsuits up to the point where it’s determined that the doctor was grossly negligent. Then they aren't allowed to practice anymore.
- Griffinsauce 5y ago
  You just jumped from "small fuckup" to "grossly negligent", come on now.

deleted 5y ago

[deleted]

charcircuit 5y ago

>Engineers aren’t allowed “one small fuckup” when building bridges and buildings Yes they are. When estimating the forces you need to handle you can round up. This means that if you made a small mistake it like won't matter since you planned to handle much more than necessary. >in the OR or when calculating dosages Dosages don't need to me exact. There are some tolerances and it's easy to imagine a system to eliminate most of human error from this process. >Pilots aren’t allowed small fuckups when flying planes. Sure they are. A mistake doesn't mean your plane will crash or fall out of the sky. Planes can already fly themselves. >Why should we tolerate small fuckups and negligence that only serves a company's shareholders’ interests? Because hindsight is 20/20.

cuu508 5y ago
“Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands.” - Unknown

dboreham 5y ago

A bridge is O(10^6) less complex than a F500 IT setup.

mcot2 5y ago

Bad anology because the system is broken all the way down. If what you are saying is true a bridge engineer wouldn’t even be able to build a bridge because their are no materials in the world to build a safe bridge out of.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

unclebucknasty 5y ago

>Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR These are really poor analogies. A better analogy would be engineers who then had to protect those bridges from attack via a number of different modes, including drone and cruise missle strikes; and seemingly legitimate travelers on the bridge who are actually malicious. Likewise, a better analogy would be doctors who had to contend with nurses who are bad actors, substituting saline for needed drugs or deliberately administering higher than prescribed dosages, etc.

indymike 5y ago

> The problem is that companies don’t want to invest the time in good security because it’s less profitable. Or, they are old and decrepit. If the company runs critical infrastructure, then peoples’ safety is at risk. Most of the time, it's that a non-technical decision maker decides to take a badly mis-calculated risk. "Bob in sales can't access the files he needs." "IT give him admin rights! Now! no... I don't even understand or care to understand that access control mumbo-jumbo you are talking about." A week later, "Hey, you know we just got bitlockered. Bob's laptop got virused, and because we gave him admin rights his machine encrypted all the things AND the backup server too."

markus_zhang 5y ago

> Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. I do agree with the rest of the post but this is definitely NOT true. Your house probably has small issues here and there if you are willing to pay for an inspector. Doctors fucking up things is not news too, plus you never know if there is "one small fuckup" as long as you don't feel particularly bad afterwards. The thing is I think you guys regard real world engineering too high. Small fuck ups happen all day.

nousermane 5y ago
Exactly. Threat modelling is a thing outside IT secutiry too. Youtube channel "MentourPilot" [0] has some excellent examples of how civil aviation was built to withstand "small fuckups": with "Swiss cheese" model. [1] [0] https://www.youtube.com/c/MentourPilotaviation/videos https://www.youtube.com/c/MentourPilotaviation/videos [1] https://en.wikipedia.org/wiki/Swiss_cheese_model https://en.wikipedia.org/wiki/Swiss_cheese_model
- dcow 5y ago
  I think most of the quibbling is over the semantics of “one small fuckup”. It’s in quotes for two reasons 1) because the comment I was responding to worded it that way it and 2) I’m deliberatly calling that characterization bullshit: one _truly_ small fuckup obviously doesn't leave you vulnerable to ransomware. Systems need to be able to withstand of small things, have redundancies, engineers build in tolerances, yada yada.. of course! Were talking about *things that leave your entire business in the hands of ransom ware criminals* or *things that kill people* or *things that crash planes* or *things that cause bridges to collapse*. Those things can’t be tolerated by definition because they are fatal failures. That’s my point. We would not tolerate companies that build bridges that collapsed all the time and keep letting them build bridges so why should we tolerate companies that get hacked repetitively by ransomeware groups? Insurers are running because it’s not good business to insure these risks. Sounds like the market is working just fine.
hulitu 5y ago

seized 5y ago

As someone in the healthcare vendor side of things.... Doctors make a lot of mistakes. There is plenty of software, processes, etc around trying to catch and mitigate those mistakes.

mdaidc 5y ago

What if I told you that an adversary can place explosives at different parts of the bridge and set them off to see if the bridge goes down? and they can try over and over again until they succeed once, and then you are screwed. this is the asymmetric nature of the threat. The governments of the west need to align and go after the offenders or sanction countries that give them refuge. It's impossible for a single company to defend itself against adversaries that can keep trying over and over again with impunity.

MattGaiser 5y ago

Engineers, doctors, and pilots are not dealing with an equally clever human on the other end trying to beat them. They wouldn't do well if someone were casually changing a number in their calculations or swapping bottles or screwing with the instruments.

deleted 5y ago

[deleted]

registeredcorn 5y ago

> Pilots aren’t allowed small f*ups when flying planes. I don't blame you if you don't take a strangers word on this, but I would strongly encourage you to look at this YouTube channel VASAviation (https://www.youtube.com/user/victor981994 https://www.youtube.com/user/victor981994) or watch the excellent movie, "Charlie Victor Romeo", which cover in extreme detail the routine, regular, and on-going mistakes that are made in aviation. Mistakes happen in aviation ALL THE TIME; they just don't cause death or crashes in every event. I can't speak for Engineers and Doctors, but for: pilots, ground crews, flight mechanics, and ATC I have a tiny soapbox. Each of these jobs routinely on a daily basis have mistakes that are made. Sometimes the errors are minor. Other times they are major, and aren't discovered for months or years after the fact, which either led to the loss of millions of dollars worth of equipment, or worse. Sure, CEOs and such are usually held responsible for those mistakes, but the aviation community focuses more on the ideals of, "Every rule, written in blood" mentality. Things are permitted, until they aren't. This is precisely because it caused a disaster. Things are safe(r) now because they were significantly less safe just a few years ago. The aviation community is focused on preventative actions as a result of the ongoing looming threat of failures. There is no way to avoid them entirely. There is an understanding that when you are in the air, there is NEVER certainty. You are NEVER safe. You should constantly be concerned about what can (and will) eventually go wrong. It's only a matter of how bad the failure will be. TLDR Aviation isn't safe because they "don't allow mistakes" but because they have a rigid and foundational system around how to deal with failure.

908B64B197 5y ago

> Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. These mistakes are grave and should be costly. Pilots aren’t allowed small fuckups when flying planes. That's wrong. Engineers aren't allowed to fail. They can have as many "fuckups" along the way, as long as they manage to compensate for it and maintain safety it's not a failure. Doctors... well, this profession has a group mentality of shielding one another from publicly acknowledging and documenting mistakes. Truth is ransomware it utterly useless against a simple back-up and recovery scenario. Any competent engineers can and will deliver one if paid to. It's that companies don't see the point and don't care. As long as the insurance premium is less than what it will cost to implement such a thing they will keep getting ransomed.

dan-robertson 5y ago

I think your first paragraph is bogus for two reasons, which I’ll leave at the end[1] because I’d rather respond to you main point than the silly reactionary thing you wrote at the top. I feel like you aren’t really responding to the claim that having good security is extremely hard and expensive and that it seems bad for every company to have to spend a lot of money on a problem which is likely far from their core competency. The thing you’re actually asking for is for a lot of businesses to go bankrupt trying to be perfect at something that is not their traditional domain and then you make some claim that they have the audacity to try to create value for their shareholders instead? Isn’t it outrageous that companies might try to do literally the thing companies are meant to do instead of being experts in computer security. I suppose you would counter that it is easy or cheap or imperative for them to be secure but, in the world where hacking is relatively easy, cheap, and profitable ($1-10mm sounds like a lot to a person but not to a country or reasonably sized business) and there are myriad ways to make trivial-seeming configuration errors in software that leave you open to attack, I think it is neither easy nor cheap. And frankly I think it is not imperative for businesses either because their purpose is to make money and while cybersecurity is an important risk to worry about, they can only invest in it so long as they are making money from the thing they are actually meant to be good at. [1] regarding the first graf: 1. Mostly small errors by engineers or doctors or pilots are… small errors. Bridges won’t collapse if the concrete was mixed slightly too wet, and bigger cracks are usually noticed and mitigated in good time. Doctors get dosages wrong all the time (which is why there are attempts to train nurses to question everything the doctors say) and while malpractice is definitely a thing, there are lots of cases where things go wrong at a small scale and are either fixed or unnoticed. If you ever ask a doctor about this in a social setting they will probably have plenty of examples. And if you get something wrong in a plane you are unlikely to immediately crash. Even landings can be aborted quite late. Big planes crash very infrequently and I think it is silly to therefore assume that the pilots are generally infallible. So I claim that your direct statement that these are not allowed is false. 2. I also claim the analogy is bad because the examples you give are not adversarial. The patients are generally not trying to trick their doctor into making the wrong prescription and Athena geography isn’t going to shift and change to try to exploit weaknesses in the bridge built across it. The pilot flies high above the ground with separation from other planes and various safety systems. Even in an unlikely event like an engine stall, there are possibilities to recover. In computer security the threats are fast and numerous so any small error can be quickly magnified.

ByteJockey 5y ago

Griffinsauce 5y ago

> Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. Yes they are. You should talk to an actual doctor before making such claims.

RcouF1uZ4gsC 5y ago

> Engineers aren’t allowed “one small fuckup” when building bridges and buildings. Doctors aren’t allowed “one small fuckup” in the OR or when calculating dosages. But engineers aren’t held responsible if a malicious actor drivers a bulldozer into one of the bridge’s columns and brings it down. Same with a doctor. If someone is maliciously messing with the meds, then they are not responsible for over or under dosing. With these security incidences, we are dealing with malicious actors trying to being down the system. I don’t think you can use analogies from civil engineering or medicine for that.