3 ms·
Your pacemaker should be running open source software
- Veserv 3y agoActually, your pacemaker should be running code certified by unbiased, competent third party experts to be free of safety defects with the liability to match. That is what we demand of every other physical, electrical, chemical, and biological component of medical devices; seems only fair we hold the software to the same safety standard. Open source is orthogonal to quality at that level. It may be a desirable, additional quality to ensure openness and prevent obsolescence, but is no substitute for actual verification efforts by trained professionals specialized in the field actively tasked with analysis. All we have to do is look to critical software like OpenSSL and the Heartbleed vulnerability to see that “many eyes” is insufficient to achieve the necessary quality in even widely used, critical software. There is no guarantee of “any eyes”, and no guarantee they are competent or have sufficient domain knowledge (think of how long it takes to get up to speed on a new software project) to meaningfully assess the systems. We must have known, active, unbiased, competent third party verification for critical software just like we demand of all other components and fields. And yes, I am aware of the certification process for Class 3 medical device software, and no it is demonstrably insufficient for our current use cases as seen time and time again.
- transpute 3y ago> running code certified by unbiased, competent third party Running reproducible builds of open-source code certified by unbiased, competent third party experts, so that the semantics and runtime behavior of the code can be continuously evaluated by multiple unbiased, competent third party experts, since the business definition of "third party" can change over time. Where binary blobs are needed, the blobs can use reproducible builds and the certified source code can be escrowed so that the binary blobs can be reproducibly created by a third party.
- Veserv 3y agoAre you worried that a medical device manufacturer will intentionally replace certified software with known uncertified software? That would be shockingly stupid. There is exactly zero plausible deniability if they did that. That is “slam-dunk go directly to jail and get fined billions” stupid. There is exactly zero reason to worry about that at all at the device manufacturer level (component suppliers might lie which transitively affects the final product, but that should be handled when identifying the target of evaluation). If they want to be evil, there are plenty of ways that do not require them to put their evil in legally binding writing. If you are concerned about a hack or accidental deployment, that is a real concern, but image integrity, secure boot, and secure update are really part of the certified process/product.
- transpute 3y ago> Are you worried that a medical device manufacturer will intentionally replace certified software with known uncertified software? Only that certified source code and software binaries can be continuously evaluated after production shipment, as original developers and certifiers move onto other projects. See the "Right to Repair" movement. Over timeframes of multiple years, we learn more about stacks of hardware, software and threats, as can be observed in the constant stream of CVEs for CPU microcode, firmware, operating systems and application code.
- Veserv 3y agoThat is just standard compulsory defect remediation. Like the hardware components they should be required to monitor and remediate any and all newly discovered defects in any components and the product. Again, this is something they already have to do for the physical components and which is done in literally every other industry to great effect. It is just not done for the software for some bone-headed reason. Note that failure to remediate known defects in components or covering it up is a very serious offense in basically every regulatory regime. That is basically “the thing” that gets you in trouble. Those standards should probably be made stricter, but that is for another time. Just getting software to “industry standard” is a major win.
- jevoten 3y ago> Open source is orthogonal to quality at that level. "Quality" has many meanings. The article gives the example of Sandler, who was unable to read data from her pacemaker, because the correctly functioning, quality, high-assurance software was kept secret from her, and only the manufacturer could decipher it. Regardless, quality is a red herring - the real issue is sovereignty over our own bodies. Which is impossible if we have devices with secret functionality embedded in us. We have a right to know how they work, the same as we have a right to know what medicine doctors administer to us.
- Veserv 3y agoYou do not receive a full chemical makeup, structure, and formulation of the medicine you are ingesting. You do not receive the entire production process of the medicine you are ingesting. You do not receive the full quality control process needed to verify the structure of the medicine you are ingesting. For that matter, even if you received those only a vanishingly small number of individuals would even be qualified to evaluate that information. No, what we as a society do is rely on a certification, verification, and validation process done by third party experts that guarantees the medicine fits a specification and is fit for purpose. This does not always get it right, but it is pretty good and works in literally every other industry. Competent unbiased third party verification is sufficient for safety-critical deployments. Openness is neither sufficient, nor necessary. A closed device with no safety defects is usable, a open device with safety defects is unacceptable. Openness may constitute other goods, such as freedom, agency, and such, but it does not provide safety, efficacy, and fitness for purpose which are the most important elements of medical devices (i.e it is orthogonal to effective operation).
- transpute 3y ago> You do not receive a full chemical makeup, structure, and formulation of the medicine you are ingesting. Individual humans and their medical advisors choose what to ingest, from a marketplace of substances on a spectrum of transparency in content and consequences.
- 3y ago
- deleted 3y ago[deleted]
- zaroth 3y agoFDA approval should at least require key algorithms be disclosed in code form. I’m thinking specifically about “artificial pancreas” aka “looping” insulin pumps, which automatically adjust how much insulin they give you based on a continuous glucose monitor (CGM) providing blood sugar readings. My kids use these devices, and I’ve watched their devices dose insulin (a potentially lethal drug) erratically when the CGM data itself occasionally goes haywire. I wish I could review the algorithm for myself. There’s no excuse for a piece of software which is automatically injecting drugs into my children to be a black box.
- medicalfuzz 3y agoawesome-safety-critical: https://awesome-safety-critical.readthedocs.io/en/latest/ https://awesome-safety-critical.readthedocs.io/en/latest/ FDA > Medical Devices > Cybersecurity: https://www.fda.gov/medical-devices/digital-health-center-excellence/cybersecurity https://www.fda.gov/medical-devices/digital-health-center-ex... "FDA Fact Sheet: THE FDA'S ROLE IN MEDICAL DEVICE CYBERSECURITY: Dispelling Myths and Understanding" https://www.fda.gov/media/123052/download?attachment https://www.fda.gov/media/123052/download?attachment oss-fuzz: https://github.com/google/oss-fuzz https://github.com/google/oss-fuzz : > We support the libFuzzer, AFL++, and Honggfuzz fuzzing engines in combination with Sanitizers, as well as ClusterFuzz, a distributed fuzzer execution environment and reporting tool. > Currently, OSS-Fuzz supports C/C++, Rust, Go, Python, Java/JVM, and JavaScript code. Other languages supported by LLVM may work too. OSS-Fuzz supports fuzzing x86_64 and i386 builds.
- Perenti 3y agoAs a user of this sort of technology, I'm not so sure that the OSS model is best. My ICD uses proper (certified) implementations of crypto to minimise biohacking. How is the OSS supply chain more secure than propriety, given the regular python/php/etc malware in source repositories? Thing is, get the corner cases wrong and you're shocking the hell out of people every 10 seconds or so. Or worse.