4 ms·

Is an open-source pacemaker safer than closed-source?

27 points by willvarfar 15y ago

mikegerwitz 15y ago
Karen Sandler of the GNOME Foundation discussed an issue similar to this nearly a year ago at OSCON (2011): https://www.youtube.com/watch?v=nFZGpES-St8 https://www.youtube.com/watch?v=nFZGpES-St8 Indeed, the first thing that came to mind when I saw this presentation was RMS' comment, as mentioned in this article.
- willvarfar 15y ago
  yes, the abc story linked in the article touches on it; your video is much clearer source, I'll add the link too :)
- mlinksva 15y ago
  Sandler gave a much longer version of the talk at LCA https://www.youtube.com/watch?v=5XDTQLa3NjE https://www.youtube.com/watch?v=5XDTQLa3NjE The paper both are based on is at https://www.softwarefreedom.org/news/2010/jul/21/software-defects-cardiac-medical-devices-are-life-/ https://www.softwarefreedom.org/news/2010/jul/21/software-de... edit: Also, similar points can be made about other embedded and safety-critical software. security through obscurity in automobiles is scary, some materials on this at http://she-devel.com http://she-devel.com
pella 15y ago
http://news.ycombinator.com/item?id=3958629 http://news.ycombinator.com/item?id=3958629
seanp2k2 15y ago
>"Is an open-source pacemaker safer than closed-source?" Yes, see arguments on "security through obscurity" End of discussion.
ctdonath 15y ago
The article assumes that end users finding problems in the software would prove beneficial to the company. To the contrary, I'd expect the company would soon be sued into oblivion for "allowing" production release of such an "obvious" problem. While certainly the malicious can figure out how to abuse a system without source code, the owner/author doesn't have to hand the means of abuse to the malicious on a silver platter. Yes, end users (me being one) have an interest in identifying flaws in their physically internalized lifespan-affecting software. The company making the product has an interest in limiting access to that source code. There are plenty who would rather use the source code to destroy the latter than to benefit the former. This is why we (in the USA) have the FDA: recognizing that the public has an interest in reviewing a product's engineering for safety purposes, yet respecting the company's need to ward off the malicious, representatives of the public are sent in to review the product for corrective improvement without exposing the company to malicious harm.
sliverstorm 15y ago
Maybe I'm missing something here, but why is security from attackers a concern for pacemakers?
- willvarfar 15y ago
  Because medical devices are pretty shoddy right now? http://www.theregister.co.uk/2011/10/27/fatal_insulin_pump_attack/ http://www.theregister.co.uk/2011/10/27/fatal_insulin_pump_a...
- ctdonath 15y ago
  Because they use wireless interfaces. Because some people find surreptitious conspiratorial killing beneficial, be it involving large amounts of currency or mere amusement. Because an "accidental" bump from a stranger, with suitable planning & equipment, can have "unrelated" terminal results seconds or days later. Because that planning becomes a whole lot easier for someone who has the source code.
ctdonath 15y ago
An under-discussed point is change control on life-support software. No doctor will want to be associated with installation of a life-support device which the patient can reprogram at will. Every change, starting from creation of the first blank main.c file, MUST be documented as considered, deliberated, implemented, reviewed, tested, verified, validated, and documented by certified personnel to the satisfaction of everyone from patient to surgeon to maintenance doctor to board of directors to sovereign jurisdictional government. We're not talking "oops, reboot" or "get the backup" or "revert to the previous version" failures should a mistake be made, we're talking "patient will die in 30 seconds flat if you screw up". Writing a FOSS version of the software keeping me alive right now sounds lofty, but I don't think the surgeon who installed my pacer would have done so if he thought I might actually replace the software outright, and the doc & techs who maintain it certainly wouldn't continue to. The potential positive benefits of an open-sourced pacemaker are obvious and well-expressed. Oft missed are the litigative & metabolic nightmare scenarios. One dangling pointer and I'm dead.
- cullen 15y ago
  An open-source life-support device need not necessarily be reprogrammable by the consumer.
  - ctdonath 15y ago
    RMS: The only way I could justify this is if I began developing a free replacement for that very program. It is ok to use a nonfree program for the purpose of developing its free replacement. Developing a free replacement software for a pacemaker is kinda pointless without the ability for the consumer to reprogram it.
  - TazeTSchnitzel 15y ago
    Whilst Richard Stallman and his GPLv3 would like it to be so, you're correct that this isn't the point. It's not supposed to be reprogrammable by the consumer, but the consumer should be able to know what code is running on it.
robomartin 15y ago
Let's look at it from a different perspective. Would an open-source self-driving car be safer than a closed-source version? Possibly. Why? Because a lot of testing is required in order to make something like that safe. For one, you need to have or build a car with all the required sensors, servos and computers. Of course, this is not impossible. It does limit the number of available hackers to those who can afford to build such a car. Let's say, for the sake of argument, that this car costs US $50K to develop. If the hacker doesn't want to modify his/her own car an additional expense of US $15K to $25K would be added to this. Yes, much code could be developed in a simulator, but as any micro-mouse competitor can tell you, there's a vast difference between a simulator and real-life. Ultimately, to make something better, safer, faster and more intelligent, real-life testing is imperative. Any time you touch the real world things get expensive and slow. That said, a dedicated small group of hackers around the world with at least $50K to burn and lots of free time could work on an open-source self-driving car. And the code may or may not be better than the closed-source. Open-source doesn't automatically mean "better", "safer", "faster" or "more intelligent". Now, switching to the pacemaker question. What does it take to develop an open-source pace-maker that doctors can and will actually implant. I don't know. I am not in that field. My guess is that the cost is in the millions. Sure, one could screw around with code and ideas and pretend to be building a pace-maker. In the end, when all the smoke and bullshit clears out, you have to implant the thing in a living organism and test it. Multiple times. And, you will kill some of your subjects. Or you are likely to. How many hackers can do that? Probably not many. Then there's the issue of regulatory testing and FDA approval process. That is a very expensive process. Who would undertake that? The point is that open-source is great when it is confined to code running on a commodity device. In terms of the financial investment, it costs exactly zero for someone to decide to make another javascript library. Why? Because for that person to even consider doing so it means that they already own a computer and probably know javascript very well. How many non-coding backhoe operators who do not own computers decide, overnight, to develop a javascript library? Exactly. Yes, there are a lot of open-source/open-hardware projects, but they are absolutely trivial in comparison to a pacemaker. Implanting something into a human being is a big deal. Then you have to look at the patient's perspective: Doctor: "Mr. Obama, you need a pacemaker. We have two choices. First is this pacemaker developed by big-bad-established-closed-source company that makes too much money. You also have the ability to use this model, developed by fifty guys all over the world and it is open source" Obama: "Who are these guys?" Doctor: "Hackers who believe that open source is a better way to build a pacemaker. The argument is that the code and design are completely open and, hence, bound to be scrutinized by other hackers to a degree that closed-source can't even begin to approach" Obama: "Yeah, but, what guarantees do they provide? Who is the responsible party? What recourse do I have if something goes wrong and I need serious help?" Doctor: "Well, there's a company that sells them. So, that's your point of contact. No different from the big-bad medical company". Obama: "C?" Doctor: "Nope, C++" Obama: "Preemptive?" Doctor: "Round-robin" Obama: "I don't know. I'm kind of concerned. Let's go with the closed-source version. I don't know if I want to trust my life to a bunch of hackers around the world." Dumb example, but you can't ignore the last issue with these things. RMS might be more than willing to experiment with his own life (I wonder). This is certainly not the case for Joe Average. Given the choice, they'll probably opt for what they perceive as safer and more reputable, rather than the hacker version. As much as folks in the tech community might understand the value of open source, the aforementioned non-coding backhoe operator who doesn't own a computer will have absolutely no intellectual connection to the concept of open source. All he will hear when given the choice is that a bunch of pimple-faced teenagers in their underwear developed this pacemaker in their bedrooms and experimented with their dogs. He'll opt for column A out of fear.
tomp 15y ago
I know of a guy who had a defective pacemaker. While he was having a shower, he got shocked, he assumed it was electricity. After a few more shocks, he quickly realized that it's his pacemakers. He took a number of additional shocks before he was sedated in a hospital. Now, he's still got the pacemaker inside, but it's off. It turns out that the company making the pacemakers knew that a small percentage were defective and that things like this could happen (unfortunately, I don't know what company that is). He's lucky he's still alive. He said that he's never going to have another (active) pacemaker - he'd rather die than have to go through such painful experience again.
- ctdonath 15y ago
  Easy to say if you have the option to go without. I don't.
- fr0sty 15y ago
  Nit-pick: pacemakers don't shock you, internal cardioverter defibrillators do. http://www.nlm.nih.gov/medlineplus/pacemakersandimplantabledefibrillators.html http://www.nlm.nih.gov/medlineplus/pacemakersandimplantabled...
billpatrianakos 15y ago
Do all programmers live in a bubble like the author? I hope I don't. - Don't mess with your own implanted medical device. Even doctors go to other doctors when they think there's a problem - Only someone living in a bubble thinks there are any significant number of doctors who can program From start to finish I hated the whole premise. It's one thing to say you want to screw with open source pace makers and a whole other thing to try to convince us (yourself?) it's safer.
- maxerickson 15y ago
  He's not suggesting that people screw with a pacemaker implanted in their own body. The suggestion is that the consequences of the source being open would be mostly beneficial (to the users of the devices), if not entirely so.
josefonseca 15y ago
How about the microcode on the CPU running the pacemaker's code?
- derleth 15y ago
  Unless the CPU is a one-off pacemaker-only design, the arguments for open-source here are stronger because orders of magnitude more people would have a reason to look at the code and possibly improve it.

27 points by willvarfar 15y ago

Karen Sandler of the GNOME Foundation discussed an issue similar to this nearly a year ago at OSCON (2011): https://www.youtube.com/watch?v=nFZGpES-St8 https://www.youtube.com/watch?v=nFZGpES-St8 Indeed, the first thing that came to mind when I saw this presentation was RMS' comment, as mentioned in this article.

willvarfar 15y ago
yes, the abc story linked in the article touches on it; your video is much clearer source, I'll add the link too :)
mlinksva 15y ago
Sandler gave a much longer version of the talk at LCA https://www.youtube.com/watch?v=5XDTQLa3NjE https://www.youtube.com/watch?v=5XDTQLa3NjE The paper both are based on is at https://www.softwarefreedom.org/news/2010/jul/21/software-defects-cardiac-medical-devices-are-life-/ https://www.softwarefreedom.org/news/2010/jul/21/software-de... edit: Also, similar points can be made about other embedded and safety-critical software. security through obscurity in automobiles is scary, some materials on this at http://she-devel.com http://she-devel.com

pella 15y ago

http://news.ycombinator.com/item?id=3958629 http://news.ycombinator.com/item?id=3958629

seanp2k2 15y ago

>"Is an open-source pacemaker safer than closed-source?" Yes, see arguments on "security through obscurity" End of discussion.

ctdonath 15y ago

The article assumes that end users finding problems in the software would prove beneficial to the company. To the contrary, I'd expect the company would soon be sued into oblivion for "allowing" production release of such an "obvious" problem. While certainly the malicious can figure out how to abuse a system without source code, the owner/author doesn't have to hand the means of abuse to the malicious on a silver platter. Yes, end users (me being one) have an interest in identifying flaws in their physically internalized lifespan-affecting software. The company making the product has an interest in limiting access to that source code. There are plenty who would rather use the source code to destroy the latter than to benefit the former. This is why we (in the USA) have the FDA: recognizing that the public has an interest in reviewing a product's engineering for safety purposes, yet respecting the company's need to ward off the malicious, representatives of the public are sent in to review the product for corrective improvement without exposing the company to malicious harm.

sliverstorm 15y ago

Maybe I'm missing something here, but why is security from attackers a concern for pacemakers?

willvarfar 15y ago
Because medical devices are pretty shoddy right now? http://www.theregister.co.uk/2011/10/27/fatal_insulin_pump_attack/ http://www.theregister.co.uk/2011/10/27/fatal_insulin_pump_a...
ctdonath 15y ago
Because they use wireless interfaces. Because some people find surreptitious conspiratorial killing beneficial, be it involving large amounts of currency or mere amusement. Because an "accidental" bump from a stranger, with suitable planning & equipment, can have "unrelated" terminal results seconds or days later. Because that planning becomes a whole lot easier for someone who has the source code.

ctdonath 15y ago

An under-discussed point is change control on life-support software. No doctor will want to be associated with installation of a life-support device which the patient can reprogram at will. Every change, starting from creation of the first blank main.c file, MUST be documented as considered, deliberated, implemented, reviewed, tested, verified, validated, and documented by certified personnel to the satisfaction of everyone from patient to surgeon to maintenance doctor to board of directors to sovereign jurisdictional government. We're not talking "oops, reboot" or "get the backup" or "revert to the previous version" failures should a mistake be made, we're talking "patient will die in 30 seconds flat if you screw up". Writing a FOSS version of the software keeping me alive right now sounds lofty, but I don't think the surgeon who installed my pacer would have done so if he thought I might actually replace the software outright, and the doc & techs who maintain it certainly wouldn't continue to. The potential positive benefits of an open-sourced pacemaker are obvious and well-expressed. Oft missed are the litigative & metabolic nightmare scenarios. One dangling pointer and I'm dead.

cullen 15y ago
An open-source life-support device need not necessarily be reprogrammable by the consumer.
- ctdonath 15y ago
  RMS: The only way I could justify this is if I began developing a free replacement for that very program. It is ok to use a nonfree program for the purpose of developing its free replacement. Developing a free replacement software for a pacemaker is kinda pointless without the ability for the consumer to reprogram it.
- TazeTSchnitzel 15y ago
  Whilst Richard Stallman and his GPLv3 would like it to be so, you're correct that this isn't the point. It's not supposed to be reprogrammable by the consumer, but the consumer should be able to know what code is running on it.

robomartin 15y ago

Let's look at it from a different perspective. Would an open-source self-driving car be safer than a closed-source version? Possibly. Why? Because a lot of testing is required in order to make something like that safe. For one, you need to have or build a car with all the required sensors, servos and computers. Of course, this is not impossible. It does limit the number of available hackers to those who can afford to build such a car. Let's say, for the sake of argument, that this car costs US $50K to develop. If the hacker doesn't want to modify his/her own car an additional expense of US $15K to $25K would be added to this. Yes, much code could be developed in a simulator, but as any micro-mouse competitor can tell you, there's a vast difference between a simulator and real-life. Ultimately, to make something better, safer, faster and more intelligent, real-life testing is imperative. Any time you touch the real world things get expensive and slow. That said, a dedicated small group of hackers around the world with at least $50K to burn and lots of free time could work on an open-source self-driving car. And the code may or may not be better than the closed-source. Open-source doesn't automatically mean "better", "safer", "faster" or "more intelligent". Now, switching to the pacemaker question. What does it take to develop an open-source pace-maker that doctors can and will actually implant. I don't know. I am not in that field. My guess is that the cost is in the millions. Sure, one could screw around with code and ideas and pretend to be building a pace-maker. In the end, when all the smoke and bullshit clears out, you have to implant the thing in a living organism and test it. Multiple times. And, you will kill some of your subjects. Or you are likely to. How many hackers can do that? Probably not many. Then there's the issue of regulatory testing and FDA approval process. That is a very expensive process. Who would undertake that? The point is that open-source is great when it is confined to code running on a commodity device. In terms of the financial investment, it costs exactly zero for someone to decide to make another javascript library. Why? Because for that person to even consider doing so it means that they already own a computer and probably know javascript very well. How many non-coding backhoe operators who do not own computers decide, overnight, to develop a javascript library? Exactly. Yes, there are a lot of open-source/open-hardware projects, but they are absolutely trivial in comparison to a pacemaker. Implanting something into a human being is a big deal. Then you have to look at the patient's perspective: Doctor: "Mr. Obama, you need a pacemaker. We have two choices. First is this pacemaker developed by big-bad-established-closed-source company that makes too much money. You also have the ability to use this model, developed by fifty guys all over the world and it is open source" Obama: "Who are these guys?" Doctor: "Hackers who believe that open source is a better way to build a pacemaker. The argument is that the code and design are completely open and, hence, bound to be scrutinized by other hackers to a degree that closed-source can't even begin to approach" Obama: "Yeah, but, what guarantees do they provide? Who is the responsible party? What recourse do I have if something goes wrong and I need serious help?" Doctor: "Well, there's a company that sells them. So, that's your point of contact. No different from the big-bad medical company". Obama: "C?" Doctor: "Nope, C++" Obama: "Preemptive?" Doctor: "Round-robin" Obama: "I don't know. I'm kind of concerned. Let's go with the closed-source version. I don't know if I want to trust my life to a bunch of hackers around the world." Dumb example, but you can't ignore the last issue with these things. RMS might be more than willing to experiment with his own life (I wonder). This is certainly not the case for Joe Average. Given the choice, they'll probably opt for what they perceive as safer and more reputable, rather than the hacker version. As much as folks in the tech community might understand the value of open source, the aforementioned non-coding backhoe operator who doesn't own a computer will have absolutely no intellectual connection to the concept of open source. All he will hear when given the choice is that a bunch of pimple-faced teenagers in their underwear developed this pacemaker in their bedrooms and experimented with their dogs. He'll opt for column A out of fear.

tomp 15y ago

I know of a guy who had a defective pacemaker. While he was having a shower, he got shocked, he assumed it was electricity. After a few more shocks, he quickly realized that it's his pacemakers. He took a number of additional shocks before he was sedated in a hospital. Now, he's still got the pacemaker inside, but it's off. It turns out that the company making the pacemakers knew that a small percentage were defective and that things like this could happen (unfortunately, I don't know what company that is). He's lucky he's still alive. He said that he's never going to have another (active) pacemaker - he'd rather die than have to go through such painful experience again.

ctdonath 15y ago
Easy to say if you have the option to go without. I don't.
fr0sty 15y ago
Nit-pick: pacemakers don't shock you, internal cardioverter defibrillators do. http://www.nlm.nih.gov/medlineplus/pacemakersandimplantabledefibrillators.html http://www.nlm.nih.gov/medlineplus/pacemakersandimplantabled...

billpatrianakos 15y ago

Do all programmers live in a bubble like the author? I hope I don't. - Don't mess with your own implanted medical device. Even doctors go to other doctors when they think there's a problem - Only someone living in a bubble thinks there are any significant number of doctors who can program From start to finish I hated the whole premise. It's one thing to say you want to screw with open source pace makers and a whole other thing to try to convince us (yourself?) it's safer.

maxerickson 15y ago
He's not suggesting that people screw with a pacemaker implanted in their own body. The suggestion is that the consequences of the source being open would be mostly beneficial (to the users of the devices), if not entirely so.

josefonseca 15y ago

How about the microcode on the CPU running the pacemaker's code?

derleth 15y ago
Unless the CPU is a one-off pacemaker-only design, the arguments for open-source here are stronger because orders of magnitude more people would have a reason to look at the code and possibly improve it.