9 ms·

"Merge says the antivirus froze access to crucial data acquired during the heart catheterization. Unable to access real-time data, the app crashed spectacularly

by Avernar 10y ago

"Merge says the antivirus froze access to crucial data acquired during the heart catheterization. Unable to access real-time data, the app crashed spectacularly. The company claims that they included proper instructions in their documentation, advising companies to whitelist Merge Hemo's folders in order to prevent crashes from happening, so it seems that the whole incident was nothing more than an oversight on the medical unit's side." Here's how I read that: The programmers of this piece of software assumed that some I/O operation would never fail and when it does the program shits itself. So instead of hardening their software to withstand loss of telemetry gracefully, which would cost time and money for the company, they just give instructions to disable scans on their folder. Odds are good that somewhere this scan will happen (and it did). Either IT doesn't read the release notes or goofs the configuration or an antivirus update clears the white list. Might not even be the antivirus that interferes with the telemetry briefly. But instead of having resilient software it's "the anitvirus software's fault" or "it's IT's fault" when something goes wrong because of their bad management/engineering decision.

sillysaurus3 10y ago
Software shouldn't necessarily try to account for errors in that manner. Usually, the most graceful thing to do is to exit cleanly. For example, if there is a massive amount of data, it has to be stored on disk. It's too large to keep in memory. And if the point of the program is to transform that data in real time, then it has to have access to the disk. The antivirus basically unplugged the disk. What can it do to recover? There's nothing to be done. It should be able to survive that situation, of course. When the disk is plugged back in, it should be able to restart without any problems. But I think that's a different kind of resiliency than what you're referring to. In this case, the only way to recover would be to copy the frozen data to a new area of the hard drive, assuming it retained read access. But such complexities result in brittle implementations, prone to acquiring bugs. What if the disk space runs out? So you check beforehand whether there's enough space. But what if some other program starts consuming disk space in the middle of your copy operation? And so on. It's an endless spiral of design complexity. The situation in the article seems closer to hardware failure than a design oversight.
- wpietri 10y ago
  I totally disagree. Sure, usually the most graceful thing to do is exit and hope a human fixes it. But that's usual because the usual condition is that sudden failure is NBD and a human is right there to screw with it. That's becoming less common, though. When software was mostly something running on a PC doing some boring office task, reliability didn't matter. But as software is running our airplanes, our cars, our medical devices, and even, as with implanted pacemakers and insulin pumps, our bodies, then reliability gos from NBD to BFD. We see the way forward with things Chaos Monkey [1] and crash-only software [2] and the sort of design for failure you see in things like Agent supervisor hierarchies [3], where the way to reliability is through designing for failure recovery from the beginning and testing thoroughly to make sure it really happens. [1] https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey [2] https://en.wikipedia.org/wiki/Crash-only_software https://en.wikipedia.org/wiki/Crash-only_software [3] http://doc.akka.io/docs/akka/snapshot/scala/fault-tolerance.html http://doc.akka.io/docs/akka/snapshot/scala/fault-tolerance....
  - sillysaurus3 10y ago
    It wasn't a rhetorical question. What could this program possibly do to recover? If the CPU fails, no one would say the program was unreliable. In this case, the disk failed, because the antivirus unplugged it. Was the program unreliable?
    - Avernar 10y ago
      The disk did not fail. An I/O operation failed. The first is a permanent condition, the second can be permanent or transient. Big difference. In Linux a signal can cause an I/O to fail. In Windows it's antivirus and other background tasks can cause I/O to fail. What can it do to recover? Retry the I/O operation! It should keep trying until the operator tells it to stop.
      - sillysaurus3 10y ago
        In that scenario, your surgeon would see the program suddenly freeze. The program likely looks like this: data acquisition -> transformation -> display transformation on monitor. If the transformation step fails, the monitor will end up displaying (a) nothing, (b) random data, or (c) the most recent image. None of these help the surgeon continue surgery. It's the same as a crash. If your environment fails, there's nothing you can do to recover. Planes aren't designed to survive the loss of a wing. Why is this case any different?
sathackr 10y ago
Exactly this. As I was reading the article I hoped to find this exact point in the HN comments. The fault lies in the bad software. It could have been the indexing service, online defrag, automatic updates, or any of the other various background processes windows runs. If it is critical software, it should be designed in a way to not fail when something non-critical malfunctions, and even the critical pieces should be built with redundancy.
- brians 10y ago
  There are lots of faults. The software failed. The process that directed a helpdesk tech to install AV was a failure of some manager. The decision to engineer systems and networks in a way such that AV seemed like a good idea was a failure of an architect.
  - jevinskie 10y ago
    In my opinion, the software failed because the entire system (software, hardware, and humanware) failed to implement, holistically, a safety critical system. You simply cannot ignore the system as a whole. I'd wager we are in violent agreement. :)
  - dTal 10y ago
    >The decision to engineer systems and networks in a way such that AV seemed like a good idea was a failure of an architect. As ever, a relevant xkcd: https://xkcd.com/463/ https://xkcd.com/463/
- scruple 10y ago
  I work for a medical devices company and I just want to say: We, specifically a few of us on the engineering staff, bring this sort of shit up constantly. I go hoarse having the same conversations over and over and over again about robustness in the face of failure, resiliency, redundancy, etc... The truth is that we're beholden to a board and an executive management team that, quite simply, doesn't give a fuck about our problems. I'm not trying to excuse the company in the article or the company that I work for. And I do not work for the company in the article. I just wanted to point out that I do see how this can happen very easily and repeatedly.
mappu 10y ago
Better to crash (and restart quickly into a known state) than to enter a rare, untested code path.
- brians 10y ago
  For software critical to human life, test the rare code paths.
  - CamperBob2 10y ago
    The easiest way to be sure a code path will run properly is to avoid writing it in the first place. This kind of application should be designed to run in a highly linear, predictable fashion on robust, fault-tolerant hardware. Why is nobody questioning the propriety of using an off-the-shelf Windows PC in safety-of-life applications?
    - Avernar 10y ago
      > The easiest way to be sure a code path will run properly is to avoid writing it in the first place. Agreed. So the app shouldn't contain anything extra not related to it's primary function. However, handling error conditions reported by the operating trumps the extraneous code rule. But there are many ways to handle an error, including ignoring it if that's the proper thing to do. Crashing is never the proper thing to do. If the program had simply exited at the very minimum, a restart would have taken a lot less time than a complete reboot of the machine. The software crashed that badly that it required a reboot of the machine. > Why is nobody questioning the propriety of using an off-the-shelf Windows PC in safety-of-life applications? They are, in the other threads. But using a better OS for the task wouldn't prevent the coding error the programmer did. Let's say they chose Linux. A signal goes off or something else happens and their read call fails. Since they expect all their I/O to succeed they crash just like the Windows box. If they bothered to handle the error and check for EINTR they'd know it was interrupted and not a hardware failure. My point is, changing operating systems doesn't protect you from poorly coded applications.
      - 10y ago
Joof 10y ago
Agreed. This is what happens when software fails catastrophically. People die and we lose confidence in software that could save lives if it worked properly.
LeifCarrotson 10y ago
I build software that does the exact same thing. We're running automotive tests, and our management/customers are unwilling to invest in solutions that will work in spite of the fact that Windows is not a real-time OS. We use a National Instruments DAQ card, and need the PC to respond within 50 ms to issue new commands for hours or days. Remarkably, it usually (over hundreds of machines and decades of operation) does. When it doesn't, it's blamed on antivirus or firewall or technicians using the PC for other things while the software runs. National Instruments provides real-time IO systems, but they cost a lot more than the basic systems. You can write driver-layer code that will run in real-time on Windows, but that takes longer. Our customers and management, with varying levels of comprehension of the problem, elect to not spend that money. I hate to say it, but if we didn't make this compromise, there are competitors who would.
- vardump 10y ago
  > We use a National Instruments DAQ card, and need the PC to respond within 50 ms to issue new commands for hours or days. Remarkably, it usually (over hundreds of machines and decades of operation) does. When it doesn't, it's blamed on antivirus or firewall or technicians using the PC for other things while the software runs. It works as long as full code path and data it requires is not paged out. Or some other thread doesn't consume I/O resources, etc. In other words, it's not guaranteed at all. Only way to get Windows to react reliably within 50 ms is in a kernel driver, as response to an IRQ. There's considerable jitter even in IRQ, but usually worst case service times are 200-500 microseconds. Depends a lot on other devices and on your IRQ priority. It's worse for passive level drivers (IRQL == 0). 50 ms guaranteed response time requires the code and data is in non-paged pool.
ryandrake 10y ago
You can't get enough up votes. Crashing because an I/O operation fails? That's sounds like simply a bug in the software. The developer didn't handle an error properly, and QA didn't test the software on an environment with elevated I/O activity. I've done enough code reviews over the years and seen enough ignoring errors from read(), ignoring malloc() returning null, not handling exceptions, etc. Good developers give a shit but many just don't care at all and think crashing or exiting when you're out of disk space is just fine.
dTal 10y ago
You'd think it would take longer to write the instructions than it would to just throw a try-catch block in there.

by Avernar 10y ago

sillysaurus3 10y ago

Software shouldn't necessarily try to account for errors in that manner. Usually, the most graceful thing to do is to exit cleanly. For example, if there is a massive amount of data, it has to be stored on disk. It's too large to keep in memory. And if the point of the program is to transform that data in real time, then it has to have access to the disk. The antivirus basically unplugged the disk. What can it do to recover? There's nothing to be done. It should be able to survive that situation, of course. When the disk is plugged back in, it should be able to restart without any problems. But I think that's a different kind of resiliency than what you're referring to. In this case, the only way to recover would be to copy the frozen data to a new area of the hard drive, assuming it retained read access. But such complexities result in brittle implementations, prone to acquiring bugs. What if the disk space runs out? So you check beforehand whether there's enough space. But what if some other program starts consuming disk space in the middle of your copy operation? And so on. It's an endless spiral of design complexity. The situation in the article seems closer to hardware failure than a design oversight.

wpietri 10y ago
I totally disagree. Sure, usually the most graceful thing to do is exit and hope a human fixes it. But that's usual because the usual condition is that sudden failure is NBD and a human is right there to screw with it. That's becoming less common, though. When software was mostly something running on a PC doing some boring office task, reliability didn't matter. But as software is running our airplanes, our cars, our medical devices, and even, as with implanted pacemakers and insulin pumps, our bodies, then reliability gos from NBD to BFD. We see the way forward with things Chaos Monkey [1] and crash-only software [2] and the sort of design for failure you see in things like Agent supervisor hierarchies [3], where the way to reliability is through designing for failure recovery from the beginning and testing thoroughly to make sure it really happens. [1] https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey [2] https://en.wikipedia.org/wiki/Crash-only_software https://en.wikipedia.org/wiki/Crash-only_software [3] http://doc.akka.io/docs/akka/snapshot/scala/fault-tolerance.html http://doc.akka.io/docs/akka/snapshot/scala/fault-tolerance....
- sillysaurus3 10y ago
  It wasn't a rhetorical question. What could this program possibly do to recover? If the CPU fails, no one would say the program was unreliable. In this case, the disk failed, because the antivirus unplugged it. Was the program unreliable?
  - Avernar 10y ago
    The disk did not fail. An I/O operation failed. The first is a permanent condition, the second can be permanent or transient. Big difference. In Linux a signal can cause an I/O to fail. In Windows it's antivirus and other background tasks can cause I/O to fail. What can it do to recover? Retry the I/O operation! It should keep trying until the operator tells it to stop.
    - sillysaurus3 10y ago
      In that scenario, your surgeon would see the program suddenly freeze. The program likely looks like this: data acquisition -> transformation -> display transformation on monitor. If the transformation step fails, the monitor will end up displaying (a) nothing, (b) random data, or (c) the most recent image. None of these help the surgeon continue surgery. It's the same as a crash. If your environment fails, there's nothing you can do to recover. Planes aren't designed to survive the loss of a wing. Why is this case any different?

sathackr 10y ago

Exactly this. As I was reading the article I hoped to find this exact point in the HN comments. The fault lies in the bad software. It could have been the indexing service, online defrag, automatic updates, or any of the other various background processes windows runs. If it is critical software, it should be designed in a way to not fail when something non-critical malfunctions, and even the critical pieces should be built with redundancy.

brians 10y ago
There are lots of faults. The software failed. The process that directed a helpdesk tech to install AV was a failure of some manager. The decision to engineer systems and networks in a way such that AV seemed like a good idea was a failure of an architect.
- jevinskie 10y ago
  In my opinion, the software failed because the entire system (software, hardware, and humanware) failed to implement, holistically, a safety critical system. You simply cannot ignore the system as a whole. I'd wager we are in violent agreement. :)
- dTal 10y ago
  >The decision to engineer systems and networks in a way such that AV seemed like a good idea was a failure of an architect. As ever, a relevant xkcd: https://xkcd.com/463/ https://xkcd.com/463/
scruple 10y ago
I work for a medical devices company and I just want to say: We, specifically a few of us on the engineering staff, bring this sort of shit up constantly. I go hoarse having the same conversations over and over and over again about robustness in the face of failure, resiliency, redundancy, etc... The truth is that we're beholden to a board and an executive management team that, quite simply, doesn't give a fuck about our problems. I'm not trying to excuse the company in the article or the company that I work for. And I do not work for the company in the article. I just wanted to point out that I do see how this can happen very easily and repeatedly.

mappu 10y ago

Better to crash (and restart quickly into a known state) than to enter a rare, untested code path.

brians 10y ago
For software critical to human life, test the rare code paths.
- CamperBob2 10y ago
  The easiest way to be sure a code path will run properly is to avoid writing it in the first place. This kind of application should be designed to run in a highly linear, predictable fashion on robust, fault-tolerant hardware. Why is nobody questioning the propriety of using an off-the-shelf Windows PC in safety-of-life applications?
  - Avernar 10y ago
    > The easiest way to be sure a code path will run properly is to avoid writing it in the first place. Agreed. So the app shouldn't contain anything extra not related to it's primary function. However, handling error conditions reported by the operating trumps the extraneous code rule. But there are many ways to handle an error, including ignoring it if that's the proper thing to do. Crashing is never the proper thing to do. If the program had simply exited at the very minimum, a restart would have taken a lot less time than a complete reboot of the machine. The software crashed that badly that it required a reboot of the machine. > Why is nobody questioning the propriety of using an off-the-shelf Windows PC in safety-of-life applications? They are, in the other threads. But using a better OS for the task wouldn't prevent the coding error the programmer did. Let's say they chose Linux. A signal goes off or something else happens and their read call fails. Since they expect all their I/O to succeed they crash just like the Windows box. If they bothered to handle the error and check for EINTR they'd know it was interrupted and not a hardware failure. My point is, changing operating systems doesn't protect you from poorly coded applications.
    - 10y ago

Joof 10y ago

Agreed. This is what happens when software fails catastrophically. People die and we lose confidence in software that could save lives if it worked properly.

LeifCarrotson 10y ago

I build software that does the exact same thing. We're running automotive tests, and our management/customers are unwilling to invest in solutions that will work in spite of the fact that Windows is not a real-time OS. We use a National Instruments DAQ card, and need the PC to respond within 50 ms to issue new commands for hours or days. Remarkably, it usually (over hundreds of machines and decades of operation) does. When it doesn't, it's blamed on antivirus or firewall or technicians using the PC for other things while the software runs. National Instruments provides real-time IO systems, but they cost a lot more than the basic systems. You can write driver-layer code that will run in real-time on Windows, but that takes longer. Our customers and management, with varying levels of comprehension of the problem, elect to not spend that money. I hate to say it, but if we didn't make this compromise, there are competitors who would.

vardump 10y ago
> We use a National Instruments DAQ card, and need the PC to respond within 50 ms to issue new commands for hours or days. Remarkably, it usually (over hundreds of machines and decades of operation) does. When it doesn't, it's blamed on antivirus or firewall or technicians using the PC for other things while the software runs. It works as long as full code path and data it requires is not paged out. Or some other thread doesn't consume I/O resources, etc. In other words, it's not guaranteed at all. Only way to get Windows to react reliably within 50 ms is in a kernel driver, as response to an IRQ. There's considerable jitter even in IRQ, but usually worst case service times are 200-500 microseconds. Depends a lot on other devices and on your IRQ priority. It's worse for passive level drivers (IRQL == 0). 50 ms guaranteed response time requires the code and data is in non-paged pool.

ryandrake 10y ago

You can't get enough up votes. Crashing because an I/O operation fails? That's sounds like simply a bug in the software. The developer didn't handle an error properly, and QA didn't test the software on an environment with elevated I/O activity. I've done enough code reviews over the years and seen enough ignoring errors from read(), ignoring malloc() returning null, not handling exceptions, etc. Good developers give a shit but many just don't care at all and think crashing or exiting when you're out of disk space is just fine.

dTal 10y ago

You'd think it would take longer to write the instructions than it would to just throw a try-catch block in there.