3 ms·

So, that's the shitty thing, right? If you're software is being used in, say, an ICU--guess what? The patient is already in bad shape. There's some weird conc

So, that's the shitty thing, right? If you're software is being used in, say, an ICU--guess what? The patient is already in bad shape. There's some weird conception in the medical industry and public at large that the software has to be perfect the first time, every time. And that just isn't possible or economical. And then when people try to do that, they find out that getting test data or trials done is near impossible. Hell, you can't even get the standards to develop with without shelling out thousands and thousands of dollars. The same institutions that clamor so hard for absurd reliability in software make it almost impossible to test that software safely without a gigantic war chest. And they look the other way when they fail to provide patient care or outright butcher people. The difference between your code and a doctor? You can fix your code after it kills a patient. EDIT: And no, this doesn't mean that I suggest that we all should write bad software until the bodycount suggests a refactor. What I mean is that we should write it as well as we're able, and that we should fix it as data for improvement becomes available. On a related note, this is why decision-making should--for now anyways--rest with a physician. We can give them better advice, but the moral accountability should remain with the human at the bedside.

bsder 12y ago
The issue is that "failure of commission" is visible and punishable while "failure of omission" is not. Not providing the best care isn't directly punishable. Not obeying a heart rate alert is punishable. There is also the moral hazard of "How many false negatives are allowed in return for how much benefit?" or, more colloquially, "That treatment will save 999 of 1000 lives, but reliably kill 1 out of 1000. Is that allowable? What are the numbers when it is allowable?" Finally, people need to get over the sense that anything biological is 100% reliable. Biological systems are NOTORIOUSLY unreliable--it's why we need medicine. However, most people don't see it that way.
HarryHirsch 12y ago
There's some weird conception in the medical industry and public at large that the software has to be perfect the first time, every time. Come again? In the real world the reliability problem has been solved. Nuclear power has been around for 60-odd years now and atomic powerplants do not regularly explode. Software on the other hand, has been around for the same time, and we expect it to be bug-afflicted. What is is that nuclear engineers have but software "engineers" are missing?
- angersock 12y ago
  Two centuries at least of engineering experience to build on, for a start? Nuclear plants are relatively straightforward extensions of existing civil, electrical, mechanical, and chemical engineering principles--it's all just steam engines eventually. Also, notice that there are a lot more software installations and people who work on them than nuke plants. Also also note that this supremely reliable technology you are trying to hold up is (rightly or wrongly) being demonized and phased out across the first world. The point you probably want to make is that software engineering should be a more rigorous profession. And yes, we've got a ways to go there, and yes, we're making progress. What is not reasonable is the hypocritical and self-serving risk-aversion many people have to improving processes through software.
- threatofrain 12y ago
  Nuclear power is, as far as I know, a consistent problem, meaning that it's not like the requirements and shape of the problem changes that much. Medical programming, on the other hand, means programming for anything medical. You cannot really pre-model the problem with a general solution, because you don't know what salient problem is going to capture social attention in the next few years. You don't know what devices are going to be created. People might have programming standards and methods for building secure systems, but I would say it's a lot easier to work around a problem that has remained mostly still. Another thing is cost-efficacy. You can always have better and more safe of anything, not just in programming. Aren't car accidents so tragic? They are also one of the prominent causes of accidental death. Well, why don't we mandate that car manufacturers must include better collision safety? We could. We could mandate until all our fears are gone. Then fewer people would have cars. Ecological and public transportation arguments aside, I don't like a world where the gateway of money requires a much higher bar to pass. Wouldn't it be unfortunate if fewer patients could access the range of medical devices which have been rigorously analyzed by a team of better-than-average engineers, and consequently the public experienced net damage?
- TheCoelacanth 12y ago
  We know how to create reliable software; we just don't want to pay for it. We prefer for software to be cheap than for it to be reliable, so that's what we get. In the rare cases where we do spend the extra money on quality, e.g. embedded software in vehicles, we do get reliable software.

bsder 12y ago

The issue is that "failure of commission" is visible and punishable while "failure of omission" is not. Not providing the best care isn't directly punishable. Not obeying a heart rate alert is punishable. There is also the moral hazard of "How many false negatives are allowed in return for how much benefit?" or, more colloquially, "That treatment will save 999 of 1000 lives, but reliably kill 1 out of 1000. Is that allowable? What are the numbers when it is allowable?" Finally, people need to get over the sense that anything biological is 100% reliable. Biological systems are NOTORIOUSLY unreliable--it's why we need medicine. However, most people don't see it that way.

HarryHirsch 12y ago

There's some weird conception in the medical industry and public at large that the software has to be perfect the first time, every time. Come again? In the real world the reliability problem has been solved. Nuclear power has been around for 60-odd years now and atomic powerplants do not regularly explode. Software on the other hand, has been around for the same time, and we expect it to be bug-afflicted. What is is that nuclear engineers have but software "engineers" are missing?

angersock 12y ago
Two centuries at least of engineering experience to build on, for a start? Nuclear plants are relatively straightforward extensions of existing civil, electrical, mechanical, and chemical engineering principles--it's all just steam engines eventually. Also, notice that there are a lot more software installations and people who work on them than nuke plants. Also also note that this supremely reliable technology you are trying to hold up is (rightly or wrongly) being demonized and phased out across the first world. The point you probably want to make is that software engineering should be a more rigorous profession. And yes, we've got a ways to go there, and yes, we're making progress. What is not reasonable is the hypocritical and self-serving risk-aversion many people have to improving processes through software.
threatofrain 12y ago
Nuclear power is, as far as I know, a consistent problem, meaning that it's not like the requirements and shape of the problem changes that much. Medical programming, on the other hand, means programming for anything medical. You cannot really pre-model the problem with a general solution, because you don't know what salient problem is going to capture social attention in the next few years. You don't know what devices are going to be created. People might have programming standards and methods for building secure systems, but I would say it's a lot easier to work around a problem that has remained mostly still. Another thing is cost-efficacy. You can always have better and more safe of anything, not just in programming. Aren't car accidents so tragic? They are also one of the prominent causes of accidental death. Well, why don't we mandate that car manufacturers must include better collision safety? We could. We could mandate until all our fears are gone. Then fewer people would have cars. Ecological and public transportation arguments aside, I don't like a world where the gateway of money requires a much higher bar to pass. Wouldn't it be unfortunate if fewer patients could access the range of medical devices which have been rigorously analyzed by a team of better-than-average engineers, and consequently the public experienced net damage?
TheCoelacanth 12y ago
We know how to create reliable software; we just don't want to pay for it. We prefer for software to be cheap than for it to be reliable, so that's what we get. In the rare cases where we do spend the extra money on quality, e.g. embedded software in vehicles, we do get reliable software.