7 ms·

Medical device engineer / founder here. I'd love for the Apple Watch to fulfill Steve Blank's aims here, but there are very real clinical problems that don't au

Medical device engineer / founder here. I'd love for the Apple Watch to fulfill Steve Blank's aims here, but there are very real clinical problems that don't automatically go away because Apple is great at industrial and UI design. The false positive risk with their a-fib detection will cause thousands of patients to ask their healthcare providers for further tests, only to find out they were fine. This isn't unlike the issue where iPhones have been inundating 911 call centers with unintentional "butt dials". This doesn't mean that Apple won't be successful in healthcare. It's just that the main challenge isn't in creating a nicer product. The challenge is in showing a net positive impact on patient outcomes.

ghaff 8y ago
>where iPhones have been inundating 911 call centers with unintentional "butt dials". This happened to me just a few months ago. I'm sitting in the passenger's seat of a car and, out of the blue, I get a call from a weird Caller ID which, of course, I hang up on. I get another call, this time from a regular looking number in the local area code. I reluctantly answer it (in spite of my friend driving telling me to hang up) and it's nearby emergency services. I'm of course totally confused and ended up hanging up on them when they start asking my name etc. It turns out there's a setting you can change to make this harder to do by accident but I had no idea until I looked it up.
debt 8y ago
Not to mention the health angle requires Apple lock down the device even further to stay HIPPA compliant; to keep from prying eyes. Apple has pit the FDA against the FBI.
- nradov 8y ago
  Nothing in HIPAA would require Apple to lock down the device in ways that would inhibit the FBI. Those are entirely separate issues. For example, a key escrow scheme (which I don't support) could still be fully HIPAA compliant.
ascagnel_ 8y ago
AFib detection will only be reported if it's detected six times, to reduce the impact of false positives. https://twitter.com/benbajarin/status/1040018930637520896 https://twitter.com/benbajarin/status/1040018930637520896
- mikekij 8y ago
  This is good to know! I'm still genuinely curious how many false positives this leads to.
  - MBCook 8y ago
    It will be interesting to see, but they ran that study for year and a half or more (listed in article) collecting data based on the old watch is probably to try to see how well they could detect afib just from the heart rate sensor. Apple is a big rich company, but I imagine their lawyers wanted to make sure they were REALLY sure they weren’t about to screw themselves over with tens of thousands of lawsuits, above and beyond the FDA requirements.
    - monkmartinez 8y ago
      You sure do put a lot a faith into a $500 watch. I routinely see $40000 cardiac monitors spit out false positives Ie. actute MI or Afib with RvR on healthy patients in the field.
      - MBCook 8y ago
        It’s not supposed to be in the same category. That’s why doesn’t worry me. All they have to do is not generate too many false positives and maybe help a few people out. It certainly not designed to replace a REAL device at a doctors office/hospital for diagnostic purposes.
      - Arn_Thor 8y ago
        And this watch is not supposed to be perfect, nor replace medical devices. If it works well it will be an added benefit to the user, and could alert users about heart issues (and has done so!)
oflannabhra 8y ago
> The question is are they are going to create millions of unnecessary doctors’ visits from unnecessarily concerned users or are they going to save thousands of lives? My bet is both – until traditional healthcare catches up with the fact that in the next decade screening devices will be in everyone’s hands (or wrists.)
- mikekij 8y ago
  True! But a similar debate has played out in the mammography space. While routine mammograms have undoubtedly saved thousands of lives, some researchers have concluded that the complications caused by false positives (e.g. infections after a biopsy) may make the net contribution of routine mammograms negative. In other words, any individual is more likely to be harmed by routine mammograms than helped. I'm not saying that the healthcare features in the new Apple Watch are bad; just that it will be interesting to see what their net contribution to healthcare will be.
  - JProthero 8y ago
    Isn't there a meaningful difference in this case between the consequences of false positives from routine mammograms, or, say PSA testing, compared to detecting Atrial Fibrillation? False positives will always cause undue distress, and that's a factor worth considering, but the consequences of a false positive from a mammogram or PSA test could be an invasive biopsy, as you point out, or even unnecessary treatment. Wouldn't the consequences of a false positive AF detection typically be much less serious though, such as some additional non-invasive tests? There's a financial and psychological cost to that, but is it really comparable to a false positive indication for cancer, which would be much more difficult to follow up, especially weighed against the benefits of detecting undiagnosed conditions? When I saw the product launch I shared your concerns about emergency services being inundated with calls from watches detecting falls, but apparently the chief executive of the US National Emergency Number Association isn't concerned. He said this about it: “These are the real beginnings of exciting innovation. I don’t see, at least initially, an overwhelming number of false positives coming in. But only time will tell.” https://www.bbc.com/news/technology-45518040 https://www.bbc.com/news/technology-45518040
bluGill 8y ago
False positives are not really an issue. Last I heard 60% of chest pain cases ERs see are false positives (constipation or something like that doesn't require ER care). However it is sometimes the only symptom you get of real medical emergencies so you go to the ER anyway. There are also a number of people who die of heart attacks with no symptoms. If Apple can get even 1% of them to the ER that would be a huge win. If the false positive rate is 80% that is still enough real positives that ERs will be get used to telling people "This time it is nothing, but it is good you came in anyway because sometimes this is all the warning you get". The real worry is false negatives - someone who has chest pain and decides not to go because the watch says all is okay. These people will die when a hospital could save them.
- mikekij 8y ago
  Great point. I should have focused on false negatives in my original comment to make my point. Thank you!
- monkmartinez 8y ago
  The exploding ER / EMS system would like to have a word with you! Every niggle or discomfort, it seems, can generate a 911 call and a visit to the ER. Legally, EMS can not diagnose in the field so we send the bulk of people that call 911 into the ER for further evaluation... whether they really need it or not. I can tell you that most don't.
  - BRAlNlAC 8y ago
    counterpoint. The healthcare system pushes people into emergency medicine for non-emergencies. About 10 years ago when I was in high school I got a cut on my chin that required stitches, my mom got an urgent appointment with my PCP later that day–who then told me to go the ER, saying, "we don't even have a suture in this clinic" We were trying to save time, money and not burden the system, and instead shot ourselves in the foot.
    - nradov 8y ago
      The healthcare system, or at least the insurance side of it, actively tries to push patients to urgent care clinics instead of ERs for that type of problem. Most health plans have a phone number where you can call a nurse to ask whether you should go to the ER or not. And the vast majority of primary care clinics do have sutures in stock.
ska 8y ago
Working in the industry you already know this but FP vs TP rates are a fundamental issue in nearly any system like this (i.e. non diagnostic). You typically can’t get rid of them, so the trick is to balance it in such a way for net positive benefit as you note. This may mean some unneeded visits, balanced by more needed interventions that would otherwise be missed. Apple Watch isn’t really any different here, except potentially in scale. While that does mean potential impact is large, there is no a priori reason to assume apple has got it wrong; it’s not that difficult to find the right people for this sort of project. It would be very interesting to see an ROC curve ...
soperj 8y ago
>Apple is great at industrial and UI design This is no longer the case.
walkingolof 8y ago
John Mandrola, a cardiac electrophysiologist have some interesting insights on the Apple Watch http://www.drjohnm.org/2018/09/thoughts-on-the-apple-watch/ http://www.drjohnm.org/2018/09/thoughts-on-the-apple-watch/
deleted 8y ago
[deleted]
netwanderer2 8y ago
Has anyone actually had a chance to try this feature on the Apple Watch 4? All the demo videos so far keep saying the feature is coming but no one actually has used it yet. I'm curious in how frequent it produces false positive results and in what types of situations or activities. Having said that, I love the direction Apple is taking with this product. Smart watches have been on the market for quite a number of years now but they're pretty much restricted to being gimmicky devices only as most people don't find it very useful or have a real need for it. If Apple could achieve the breakthrough in blood glucose monitoring feature then I strongly believe we will see mass adoption in very near future.
visarga 8y ago
> iPhones have been inundating 911 call centers with unintentional "butt dials" Because they fill that lock screen with junk making it a butt dial magnet.
killjoywashere 8y ago
As a pathologist I worry less about false positives and a lot more about false negatives.

ghaff 8y ago

>where iPhones have been inundating 911 call centers with unintentional "butt dials". This happened to me just a few months ago. I'm sitting in the passenger's seat of a car and, out of the blue, I get a call from a weird Caller ID which, of course, I hang up on. I get another call, this time from a regular looking number in the local area code. I reluctantly answer it (in spite of my friend driving telling me to hang up) and it's nearby emergency services. I'm of course totally confused and ended up hanging up on them when they start asking my name etc. It turns out there's a setting you can change to make this harder to do by accident but I had no idea until I looked it up.

debt 8y ago

Not to mention the health angle requires Apple lock down the device even further to stay HIPPA compliant; to keep from prying eyes. Apple has pit the FDA against the FBI.

nradov 8y ago
Nothing in HIPAA would require Apple to lock down the device in ways that would inhibit the FBI. Those are entirely separate issues. For example, a key escrow scheme (which I don't support) could still be fully HIPAA compliant.

ascagnel_ 8y ago

AFib detection will only be reported if it's detected six times, to reduce the impact of false positives. https://twitter.com/benbajarin/status/1040018930637520896 https://twitter.com/benbajarin/status/1040018930637520896

mikekij 8y ago
This is good to know! I'm still genuinely curious how many false positives this leads to.
- MBCook 8y ago
  It will be interesting to see, but they ran that study for year and a half or more (listed in article) collecting data based on the old watch is probably to try to see how well they could detect afib just from the heart rate sensor. Apple is a big rich company, but I imagine their lawyers wanted to make sure they were REALLY sure they weren’t about to screw themselves over with tens of thousands of lawsuits, above and beyond the FDA requirements.
  - monkmartinez 8y ago
    You sure do put a lot a faith into a $500 watch. I routinely see $40000 cardiac monitors spit out false positives Ie. actute MI or Afib with RvR on healthy patients in the field.
    - MBCook 8y ago
      It’s not supposed to be in the same category. That’s why doesn’t worry me. All they have to do is not generate too many false positives and maybe help a few people out. It certainly not designed to replace a REAL device at a doctors office/hospital for diagnostic purposes.
    - Arn_Thor 8y ago
      And this watch is not supposed to be perfect, nor replace medical devices. If it works well it will be an added benefit to the user, and could alert users about heart issues (and has done so!)

oflannabhra 8y ago

> The question is are they are going to create millions of unnecessary doctors’ visits from unnecessarily concerned users or are they going to save thousands of lives? My bet is both – until traditional healthcare catches up with the fact that in the next decade screening devices will be in everyone’s hands (or wrists.)

mikekij 8y ago
True! But a similar debate has played out in the mammography space. While routine mammograms have undoubtedly saved thousands of lives, some researchers have concluded that the complications caused by false positives (e.g. infections after a biopsy) may make the net contribution of routine mammograms negative. In other words, any individual is more likely to be harmed by routine mammograms than helped. I'm not saying that the healthcare features in the new Apple Watch are bad; just that it will be interesting to see what their net contribution to healthcare will be.
- JProthero 8y ago
  Isn't there a meaningful difference in this case between the consequences of false positives from routine mammograms, or, say PSA testing, compared to detecting Atrial Fibrillation? False positives will always cause undue distress, and that's a factor worth considering, but the consequences of a false positive from a mammogram or PSA test could be an invasive biopsy, as you point out, or even unnecessary treatment. Wouldn't the consequences of a false positive AF detection typically be much less serious though, such as some additional non-invasive tests? There's a financial and psychological cost to that, but is it really comparable to a false positive indication for cancer, which would be much more difficult to follow up, especially weighed against the benefits of detecting undiagnosed conditions? When I saw the product launch I shared your concerns about emergency services being inundated with calls from watches detecting falls, but apparently the chief executive of the US National Emergency Number Association isn't concerned. He said this about it: “These are the real beginnings of exciting innovation. I don’t see, at least initially, an overwhelming number of false positives coming in. But only time will tell.” https://www.bbc.com/news/technology-45518040 https://www.bbc.com/news/technology-45518040

bluGill 8y ago

False positives are not really an issue. Last I heard 60% of chest pain cases ERs see are false positives (constipation or something like that doesn't require ER care). However it is sometimes the only symptom you get of real medical emergencies so you go to the ER anyway. There are also a number of people who die of heart attacks with no symptoms. If Apple can get even 1% of them to the ER that would be a huge win. If the false positive rate is 80% that is still enough real positives that ERs will be get used to telling people "This time it is nothing, but it is good you came in anyway because sometimes this is all the warning you get". The real worry is false negatives - someone who has chest pain and decides not to go because the watch says all is okay. These people will die when a hospital could save them.

mikekij 8y ago
Great point. I should have focused on false negatives in my original comment to make my point. Thank you!
monkmartinez 8y ago
The exploding ER / EMS system would like to have a word with you! Every niggle or discomfort, it seems, can generate a 911 call and a visit to the ER. Legally, EMS can not diagnose in the field so we send the bulk of people that call 911 into the ER for further evaluation... whether they really need it or not. I can tell you that most don't.
- BRAlNlAC 8y ago
  counterpoint. The healthcare system pushes people into emergency medicine for non-emergencies. About 10 years ago when I was in high school I got a cut on my chin that required stitches, my mom got an urgent appointment with my PCP later that day–who then told me to go the ER, saying, "we don't even have a suture in this clinic" We were trying to save time, money and not burden the system, and instead shot ourselves in the foot.
  - nradov 8y ago
    The healthcare system, or at least the insurance side of it, actively tries to push patients to urgent care clinics instead of ERs for that type of problem. Most health plans have a phone number where you can call a nurse to ask whether you should go to the ER or not. And the vast majority of primary care clinics do have sutures in stock.

ska 8y ago

Working in the industry you already know this but FP vs TP rates are a fundamental issue in nearly any system like this (i.e. non diagnostic). You typically can’t get rid of them, so the trick is to balance it in such a way for net positive benefit as you note. This may mean some unneeded visits, balanced by more needed interventions that would otherwise be missed. Apple Watch isn’t really any different here, except potentially in scale. While that does mean potential impact is large, there is no a priori reason to assume apple has got it wrong; it’s not that difficult to find the right people for this sort of project. It would be very interesting to see an ROC curve ...

soperj 8y ago

>Apple is great at industrial and UI design This is no longer the case.

walkingolof 8y ago

John Mandrola, a cardiac electrophysiologist have some interesting insights on the Apple Watch http://www.drjohnm.org/2018/09/thoughts-on-the-apple-watch/ http://www.drjohnm.org/2018/09/thoughts-on-the-apple-watch/

deleted 8y ago

[deleted]

netwanderer2 8y ago

Has anyone actually had a chance to try this feature on the Apple Watch 4? All the demo videos so far keep saying the feature is coming but no one actually has used it yet. I'm curious in how frequent it produces false positive results and in what types of situations or activities. Having said that, I love the direction Apple is taking with this product. Smart watches have been on the market for quite a number of years now but they're pretty much restricted to being gimmicky devices only as most people don't find it very useful or have a real need for it. If Apple could achieve the breakthrough in blood glucose monitoring feature then I strongly believe we will see mass adoption in very near future.

visarga 8y ago

> iPhones have been inundating 911 call centers with unintentional "butt dials" Because they fill that lock screen with junk making it a butt dial magnet.

killjoywashere 8y ago

As a pathologist I worry less about false positives and a lot more about false negatives.