5 ms·

Great article. Real life horror stories of life-critical software gore, with some good news at the end. It should be illegal to sell software that someones lif

by mbakke 3y ago

Great article. Real life horror stories of life-critical software gore, with some good news at the end. It should be illegal to sell software that someones life depends upon without giving the user the right to inspect and modify the code.

7e 3y ago
Quality of life critical software should be ensured by FDA certification. Homebrew modifications of that software, even in the name of “freedom”, risks the patient’s life and health and should be illegal if uncertified.
- hobs 3y ago
  To distribute? Sure. To make changes to your out of support cyber-eyeball? Nah.
- Kim_Bruning 3y ago
  I think this might be a cultural thing. In some (western) countries, your body is your personal private property, and you have the freedom and ultimate authority over how to use and abuse it, or anything on or in it. (you are still advised to treat your most precious property wisely, obviously) In other (western) countries/subcommunities people feel that obligations to your community are stronger. People from these different cultures can get into some pretty hefty discussions when it comes to things like abortion, drugs, euthansia, or -here- implants.
- eikenberry 3y ago
  So like suicide, drugs and other and other cases where we are denied dominion over ourselves for our own good? IE. Your life and body are not yours, they belong to society and you only get limited access.
  - chpatrick 3y ago
    Society doesn't have to give you the rope to hang yourself.
    - eikenberry 3y ago
      I disagree or rather yes, it does have the responsibility to provide you a rope. It is up to you whether you hang yourself or not.
      - worik 3y ago
        You are taking the position that an individual "owns" themselves That is not obviously true. I feel I belong to my family and my community.
        Xymist 3y ago
        Your position is not universal, and in fact strongly opposed by many. I believe that I have the absolute right to edit or terminate my own existence, either on purpose or accidentally. To the extent that anyone can own a person, people own themselves exclusively.
        worik 3y ago
        > Your position is not universal True. But neither is the other position
      - chpatrick 3y ago
        I disagree, I think if you walk into a pharmacy and ask for something dangerous without a prescription they shouldn't be obligated to give it to you. It's the same with medical equipment that keeps you alive. If you want to risk your life you can do it but no one should be compelled to help you.
        eikenberry 3y ago
        No one should be compelled. I mean it more in a negative manner, that it has an obligation to not stop people from helping. If someone wanted to offer a nitrogen tank, valve, tube and an easily head fitting bag for sale to people who want to commit suicide in a painless and ensured manner they be able to sell that (and people would). But in fact you cannot, and that is wrong.
- zarzavat 3y ago
  Serious question, what does the FDA know about software quality?
  - alistairSH 3y ago
    Surely not less than the average consumer. And surely they could hire experts to do the job.
    - hn_acker 3y ago
      1. Compared to the average person in the FDA's population of people who are in charge of evaluating the medical devices, the average person in the population of people who would make fixes and helpful modifications might have more expertise in determining the quality of the device's normal software. 2. It's not as if the people who depend on the medical devices have to take the word of the community of people who will mod the devices over the word of the FDA.
- schiffern 3y ago
  > Homebrew modifications of that software, even in the name of “freedom”, risks the patient’s life and health and should be illegal if uncertified. The official modifications of that software — in the name of "profit" — are currently risking the patient’s life and health, and therefore should also be illegal by your logic. Surely you must also support effective (ie harsh/deterrent) prosecution and punishment for these crimes as well, correct?
  - patmcc 3y ago
    >>>should be illegal if uncertified. I think this is the key part of the comment - yes, uncertified changes by anyone could feasibly be illegal. The FDA or similar should probably do code reviews.
    - Kim_Bruning 3y ago
      Looking at corner cases for this: What if you fix a bug in your own pacemaker? Would it be ok to: a) Fine you? b) Jail you? c) Force you to revert the change? (plausibly leading to death in an extreme case) [edit: I do agree that there's a chance that making a 'fix' to your own pacemaker might also make it worse. In which case, who do we trust more? The person on the ground with a stake in the matter (however misinformed), or $government_official with no stake in the matter (however well informed). I think it's tricky! ]
      - Xymist 3y ago
        I don't think that scenario is particularly tricky. If you modify someone else's pacemaker, it's a tricky question, even with their consent. If you modify your own, absolutely nothing should stand in your way beyond a nice big notice saying "danger of death,on your head be it". That is, you should have the same freedom to screw with your own personal medical devices that you have to climb out of your own fourth floor window. People have a right, albeit not enshrined in law, to do stupid things that might kill them - at least as long as they don't then ask someone else to save them.
  - ketzo 3y ago
- leghifla 3y ago
  In EU (and probably elsewhere), there are strict rules for the stability of power wheelchair. One such rule is "On a incline of x% (x chosen by the manufacturer), pushing for max speed from stop should not lift the front wheels" To achieve that, the max acceleration must be quite low (software controlled), and the whole experience is sluggish, like trying to steer a car by pulling on rubber bands attached to the wheel. From the moment I found a way to overcome this, I never went back. I know that I can hurt myself if I do something stupid, but I prefer this hypothetical risk instead of cursing 100 times a day because I cannot move how I want. It has been 10 years and I never got hurt. I understand that such "high" risk device cannot be sold, but forbidding someone to change this is like inflicting a second handicap on him.
  - Buttons840 3y ago
    I suppose we all have, or should have, the right to try stupid things. Sometimes experience and competence are more important than 100% safety. Your comment made me realize how limiting it would be to be physically incapable of taking even the smallest risk.
  - hyeonwho22 3y ago
    That is a very poor regulation. Why enforce wheel lift? What matters is that the chair doesn't tip over - that the center of gravity remains in the center of the four wheels.
- mtlmtlmtlmtl 3y ago
  Surely the patient should have the right to risk their own life?
- KevinGlass 3y ago
  Safetism is a great curse on the world. I cannot disagree with you more.
throwawaysleep 3y ago
So you would prefer it not be developed?
- BobaFloutist 3y ago
  The software is clearly not the primary product. While there might need to be a carve out or a specific licensing scheme developed to protect them from liability in the case of modified software, I doubt these companies would experience serious financial setbacks if they made their software free and open. And don't tell me that SaaS is an integral part of the business model for medical device companies. There's no world in which they can't figure out how to turn a profit without charging a monthly fee to use your tens of thousands of dollars eyeball.
  - oconnor663 3y ago
    > The software is clearly not the primary product. Sure, in this case. But that means that the rule we're considering actually needs a big asterisk next to it, something like "when the software in question isn't the primary product." That sounds like a thorny regulatory question, and any answer to that question other than "I know it when I see it" probably has big loopholes. This might be unnecessary nitpicking on my part if we're just shooting the breeze about companies we don't like, but if we're actually interested in writing laws, this is a common failure mode. Maybe _the_ common failure mode. On the other hand, "so you would prefer it not be developed" is a less-than-entirely-charitable way of making this point. Of course @mbakke would _not_ prefer that, and it might avoid an unnecessary round of back-and-forth to make a reasonable guess about what they would prefer and work from there :)
- hcks 3y ago
  This is being downvoted yet there’s a reason why this types of treatments always starts being developed to serve the US market initially
dheera 3y ago
I have an ICD (implanted cardioverter-defibrillator) to save my life if my heart stops. I was also given a proprietary box that sits at home, reads data from it and sends it to my cardiologist over a cellular network, on demand. As part of periodic remote checkups I'm supposed to sit next to it, press the button, which causes it to read data and send any abnormal heart rhythms it detected (via cellular network), whether it treated it (via a shock, in which case I would have known anyway) or whether the abnormal rhythm resolved itself with no treatment (in which case it's worth it that they check out what it picked up). I have to do this about 2-4 times a year. Every time I hit the button I'm charged $200. Even if there are ZERO events. 90%+ of the time there are zero events. There is NO interface provided to me where I can read the data directly. There is no way for me to read the device on my own, see zero events, and inform my cardiologist that there are no events and that there is nothing new to diagnose. I hate this medical system. The device is great for saving my life but I want access to read its data without being charged.
- subw00f 3y ago
  This is nuts. Who charges you? Is it the company that makes these devices? What if you want a different “provider”?
  - dheera 3y ago
    Stanford Healthcare charges me for "general classification" just for a nurse to open up their computer and see that there are zero events. Boston Scientific, the device maker, does not have an interface for patients, they only send data to hospitals directly. I'm not currently willing to switch to a different ICD because Boston Scientific's ICD has successfully saved my life 3/3 times in out-of-hospital situations and 2/2 times during in-hospital testing where they induced ventricular vibrillation in controlled testing and I'd rather not risk trying something different. Insurance wouldn't pay for an extra surgery deemed unnecessary, anyway. I could switch healthcare providers, but I'm not sure if the others in my area are better at cardiology.
    - tredre3 3y ago

by mbakke 3y ago

7e 3y ago

Quality of life critical software should be ensured by FDA certification. Homebrew modifications of that software, even in the name of “freedom”, risks the patient’s life and health and should be illegal if uncertified.

hobs 3y ago
To distribute? Sure. To make changes to your out of support cyber-eyeball? Nah.
Kim_Bruning 3y ago
I think this might be a cultural thing. In some (western) countries, your body is your personal private property, and you have the freedom and ultimate authority over how to use and abuse it, or anything on or in it. (you are still advised to treat your most precious property wisely, obviously) In other (western) countries/subcommunities people feel that obligations to your community are stronger. People from these different cultures can get into some pretty hefty discussions when it comes to things like abortion, drugs, euthansia, or -here- implants.
eikenberry 3y ago
So like suicide, drugs and other and other cases where we are denied dominion over ourselves for our own good? IE. Your life and body are not yours, they belong to society and you only get limited access.
- chpatrick 3y ago
  Society doesn't have to give you the rope to hang yourself.
  - eikenberry 3y ago
    I disagree or rather yes, it does have the responsibility to provide you a rope. It is up to you whether you hang yourself or not.
    - worik 3y ago
      You are taking the position that an individual "owns" themselves That is not obviously true. I feel I belong to my family and my community.
      - Xymist 3y ago
        Your position is not universal, and in fact strongly opposed by many. I believe that I have the absolute right to edit or terminate my own existence, either on purpose or accidentally. To the extent that anyone can own a person, people own themselves exclusively.
        worik 3y ago
        > Your position is not universal True. But neither is the other position
    - chpatrick 3y ago
      I disagree, I think if you walk into a pharmacy and ask for something dangerous without a prescription they shouldn't be obligated to give it to you. It's the same with medical equipment that keeps you alive. If you want to risk your life you can do it but no one should be compelled to help you.
      - eikenberry 3y ago
        No one should be compelled. I mean it more in a negative manner, that it has an obligation to not stop people from helping. If someone wanted to offer a nitrogen tank, valve, tube and an easily head fitting bag for sale to people who want to commit suicide in a painless and ensured manner they be able to sell that (and people would). But in fact you cannot, and that is wrong.
zarzavat 3y ago
Serious question, what does the FDA know about software quality?
- alistairSH 3y ago
  Surely not less than the average consumer. And surely they could hire experts to do the job.
  - hn_acker 3y ago
    1. Compared to the average person in the FDA's population of people who are in charge of evaluating the medical devices, the average person in the population of people who would make fixes and helpful modifications might have more expertise in determining the quality of the device's normal software. 2. It's not as if the people who depend on the medical devices have to take the word of the community of people who will mod the devices over the word of the FDA.
schiffern 3y ago
> Homebrew modifications of that software, even in the name of “freedom”, risks the patient’s life and health and should be illegal if uncertified. The official modifications of that software — in the name of "profit" — are currently risking the patient’s life and health, and therefore should also be illegal by your logic. Surely you must also support effective (ie harsh/deterrent) prosecution and punishment for these crimes as well, correct?
- patmcc 3y ago
  >>>should be illegal if uncertified. I think this is the key part of the comment - yes, uncertified changes by anyone could feasibly be illegal. The FDA or similar should probably do code reviews.
  - Kim_Bruning 3y ago
    Looking at corner cases for this: What if you fix a bug in your own pacemaker? Would it be ok to: a) Fine you? b) Jail you? c) Force you to revert the change? (plausibly leading to death in an extreme case) [edit: I do agree that there's a chance that making a 'fix' to your own pacemaker might also make it worse. In which case, who do we trust more? The person on the ground with a stake in the matter (however misinformed), or $government_official with no stake in the matter (however well informed). I think it's tricky! ]
    - Xymist 3y ago
      I don't think that scenario is particularly tricky. If you modify someone else's pacemaker, it's a tricky question, even with their consent. If you modify your own, absolutely nothing should stand in your way beyond a nice big notice saying "danger of death,on your head be it". That is, you should have the same freedom to screw with your own personal medical devices that you have to climb out of your own fourth floor window. People have a right, albeit not enshrined in law, to do stupid things that might kill them - at least as long as they don't then ask someone else to save them.
- ketzo 3y ago
leghifla 3y ago
In EU (and probably elsewhere), there are strict rules for the stability of power wheelchair. One such rule is "On a incline of x% (x chosen by the manufacturer), pushing for max speed from stop should not lift the front wheels" To achieve that, the max acceleration must be quite low (software controlled), and the whole experience is sluggish, like trying to steer a car by pulling on rubber bands attached to the wheel. From the moment I found a way to overcome this, I never went back. I know that I can hurt myself if I do something stupid, but I prefer this hypothetical risk instead of cursing 100 times a day because I cannot move how I want. It has been 10 years and I never got hurt. I understand that such "high" risk device cannot be sold, but forbidding someone to change this is like inflicting a second handicap on him.
- Buttons840 3y ago
  I suppose we all have, or should have, the right to try stupid things. Sometimes experience and competence are more important than 100% safety. Your comment made me realize how limiting it would be to be physically incapable of taking even the smallest risk.
- hyeonwho22 3y ago
  That is a very poor regulation. Why enforce wheel lift? What matters is that the chair doesn't tip over - that the center of gravity remains in the center of the four wheels.
mtlmtlmtlmtl 3y ago
Surely the patient should have the right to risk their own life?
KevinGlass 3y ago
Safetism is a great curse on the world. I cannot disagree with you more.

throwawaysleep 3y ago

So you would prefer it not be developed?

BobaFloutist 3y ago
The software is clearly not the primary product. While there might need to be a carve out or a specific licensing scheme developed to protect them from liability in the case of modified software, I doubt these companies would experience serious financial setbacks if they made their software free and open. And don't tell me that SaaS is an integral part of the business model for medical device companies. There's no world in which they can't figure out how to turn a profit without charging a monthly fee to use your tens of thousands of dollars eyeball.
- oconnor663 3y ago
  > The software is clearly not the primary product. Sure, in this case. But that means that the rule we're considering actually needs a big asterisk next to it, something like "when the software in question isn't the primary product." That sounds like a thorny regulatory question, and any answer to that question other than "I know it when I see it" probably has big loopholes. This might be unnecessary nitpicking on my part if we're just shooting the breeze about companies we don't like, but if we're actually interested in writing laws, this is a common failure mode. Maybe _the_ common failure mode. On the other hand, "so you would prefer it not be developed" is a less-than-entirely-charitable way of making this point. Of course @mbakke would _not_ prefer that, and it might avoid an unnecessary round of back-and-forth to make a reasonable guess about what they would prefer and work from there :)
hcks 3y ago
This is being downvoted yet there’s a reason why this types of treatments always starts being developed to serve the US market initially

dheera 3y ago

I have an ICD (implanted cardioverter-defibrillator) to save my life if my heart stops. I was also given a proprietary box that sits at home, reads data from it and sends it to my cardiologist over a cellular network, on demand. As part of periodic remote checkups I'm supposed to sit next to it, press the button, which causes it to read data and send any abnormal heart rhythms it detected (via cellular network), whether it treated it (via a shock, in which case I would have known anyway) or whether the abnormal rhythm resolved itself with no treatment (in which case it's worth it that they check out what it picked up). I have to do this about 2-4 times a year. Every time I hit the button I'm charged $200. Even if there are ZERO events. 90%+ of the time there are zero events. There is NO interface provided to me where I can read the data directly. There is no way for me to read the device on my own, see zero events, and inform my cardiologist that there are no events and that there is nothing new to diagnose. I hate this medical system. The device is great for saving my life but I want access to read its data without being charged.

subw00f 3y ago
This is nuts. Who charges you? Is it the company that makes these devices? What if you want a different “provider”?
- dheera 3y ago
  Stanford Healthcare charges me for "general classification" just for a nurse to open up their computer and see that there are zero events. Boston Scientific, the device maker, does not have an interface for patients, they only send data to hospitals directly. I'm not currently willing to switch to a different ICD because Boston Scientific's ICD has successfully saved my life 3/3 times in out-of-hospital situations and 2/2 times during in-hospital testing where they induced ventricular vibrillation in controlled testing and I'd rather not risk trying something different. Insurance wouldn't pay for an extra surgery deemed unnecessary, anyway. I could switch healthcare providers, but I'm not sure if the others in my area are better at cardiology.
  - tredre3 3y ago