9 ms·

The Apple Watch heart monitor sends too many people to the doctor

30 points by Nevaeh 6y ago

reaperducer 6y ago
If the choice is "The Apple Watch heart monitor sends too many people to the doctor" and "The Apple Watch heart monitor sends too few people to the doctor," I'm OK with this.
- ihattendorf 6y ago
  I agree, but obviously it depends on how many people is "too many". From the article it looks like 10% of those who went to the doctor were diagnosed with a cardiac condition, but what if it was 1%? A 90% false positive rate is already fairly high.
  - henriquez 6y ago
    Maybe it’s just more common to have heart “anomalies” than people think. Historically it’s never been common for people having heart monitors strapped to them all the time, so now we’re In uncharted territory. Maybe there’s some scientific learning we still have to do about how the heart works, and what is considered “in spec.”
    - cratermoon 6y ago
      It is common! I know as a heart patient I once wore a Holter monitor (https://www.mayoclinic.org/tests-procedures/holter-monitor/about/pac-20385039 https://www.mayoclinic.org/tests-procedures/holter-monitor/a...) and in the process learned that your heartbeat isn't perfect. Pre-beats are pretty common and completely harmless.
  - closetohome 6y ago
    Another way of looking at it is that 1 in 10 people who were alerted to a potential health issue ended up needing medical assistance. That's actually pretty good for a basic non-medical screening.
    - acdha 6y ago
      That's true with a big caveat: how many of those people get invasive treatments for things which would not have actually become problems? They've backed away from screening as aggressively for certain cancers because the risks of aggressive treatment turned out to be significant and it turned out that more people than expected had tumors which didn't develop into a problem before they died of something else. I don't know how plausible that is with heart issues — maybe aggressive prescription of something like statins with significant side-effects? — but it's very important to consider the cost of false-positives.
      - ebg13 6y ago
        > how many of those people get invasive treatments for things which would not have actually become problems? That's really the doctor's issue to address, not the device's. Sticking your head in the sand isn't the right solution to the problems of dealing with information too aggressively. > They've backed away from screening as aggressively for certain cancers because the risks of aggressive treatment turned out to be significant and it turned out that more people than expected had tumors which didn't develop into a problem before they died of something else. Literally the only way to build viable metrics for which tumors will develop into problems is for _everyone_ get screened, not for no one to get screened. The problem of treating too aggressively is a treatment problem, not a measurement problem. If we avoid measuring, we have no way to learn how to treat better. If we decide to never look because we don't yet know how to handle the information, then we will never know how to handle the information.
        DanBC 6y ago
        > The problem of treating too aggressively is a treatment problem, not a measurement problem. One of the most important questions you can ask your doctor is "what happens if I don't have this test?" and "What happens if I don't have this treatment?" But most people don't ask these questions. And so it really is a measurement problem. Over-testing, over-diagnosis, and over-treatment are well understood mechanisms of harm. https://ebm.bmj.com/content/23/1/1 https://ebm.bmj.com/content/23/1/1 https://www.bmj.com/content/362/bmj.k2820 https://www.bmj.com/content/362/bmj.k2820 The harm caused by inappropriate screening is so severe that we (UK) have a committee to weigh up the harm vs benefit of different screening programmes. https://www.gov.uk/government/groups/uk-national-screening-committee-uk-nsc https://www.gov.uk/government/groups/uk-national-screening-c...
  - deleted 6y ago
    [deleted]
- Clent 6y ago
  It's not clear from the article if it actually sent anyone to the doctor or if they simply brought it up during an normal visit.
elheffe80 6y ago
Sent me to the doctor, had surgery a few weeks later. Too many is OK IMO.
- bobowzki 6y ago
  Really, ablation?
- Grazester 6y ago
  Do you go to your doctor for an annual checkup? Was this something that could have been missed or something that is brought on suddenly?
  - ebg13 6y ago
    > Do you go to your doctor for an annual checkup? Doctors will rarely subject anyone to thorough evaluation for things if they aren't over the common danger age and don't report specific symptoms. Your typical annual checkup for people not 50+ is a fairly generic blood panel and very little else. After 50 they may add, what, a prostate probe for men?
    - crazygringo 6y ago
      Exactly. In my life I've had one doctor who performed an EKG as part of my annual checkup, once. The other ~10 primary care physicians I've had never did -- my annual was mostly just a few questions, some blood work, quick reflex tests and I think they looked in my ears and eyes. If my Apple Watch detected a heart irregularity, I can say with certainty it's not something that would be caught otherwise at my next annual checkup.
      - Grazester 6y ago
        At my last physical my doctor did an EKG. It was not requested by me. I guess it all depends on your doctor.
    - Ancapistani 6y ago
ed25519FUUU 6y ago
I see the “the false positives is okay” sentiment widely here. Just keep in mind that medical mistakes is the 3rd leading cause of death in the US. It’s statistically in your favor to limit your exposure to medical mistakes. https://www.google.com/amp/s/www.cnbc.com/amp/2018/02/22/medical-errors-third-leading-cause-of-death-in-america.html https://www.google.com/amp/s/www.cnbc.com/amp/2018/02/22/med...
- solidblu 6y ago
  So you are suggesting we don't go to the doctor for the leading cause of death because of the 3rd leading risk? Damned if you do, Damned if you don't.
- dahart 6y ago
  It's complicated and IMO misleading to claim medical mistakes are the 3rd leading "cause" of death as if it's a fact and as if there are clear and better alternatives. No question there are mistakes and room for improvement, but you're linking to meta-studies where people decide what "mistake" means, not a recognized system of categorizing causes of death. I feel like the wording is vaguely implying general malfeasance, while the studies you linked to actually include categories of mistakes that are sometimes unavoidable and sometimes have very unclear lines of responsibility relative to the patient. For example, "One example of a lethal error of communication between provider and patient occurred when cardiologists failed to warn their 19-year-old patient not to run. The patient had experienced syncope while running, and 5 days of inpatient, diagnostic testing were inconclusive; however, his cardiologists knew he was not ready to return to running but failed to warn him against this risk. Having not been warned against running, he resumed running and died 3 weeks later while running." https://journals.lww.com/journalpatientsafety/Fulltext/2013/09000/A_New,_Evidence_based_Estimate_of_Patient_Harms.2.aspx https://journals.lww.com/journalpatientsafety/Fulltext/2013/... This categorization assigns 100% of the blame to a cardiologist who was not performing a procedure and who did not know that the patient would die. The patient chose to do something for which they'd recently experienced issues with, without waiting for complete test results or clear instructions. It's an accident, and unfortunate, but my point is it's very problematic to conclude that the cause of death here is a medical mistake, rather than a heart condition. This is the representative example that the Journal of Patient Safety uses to illustrate communication errors, so it seems reasonable to assume there are many such examples. Furthermore, how many people would be dying if there was no care instead? Probably a lot more, and that's not accounted for. To call mistakes the 3rd leading cause of death, I think we need to account for how many people were saved by medical care too... and when you think about it that way I suspect it's complicated if not impossible to tally that number, meaning that it's also complicated if not impossible to rank medical accidents as a cause of death against objectively identifiable categories like heart attack vs car accident.
crazygringo 6y ago
> Only around 10 percent of people who saw a doctor at the Mayo Clinic after noticing an abnormal pulse reading on their watch were eventually diagnosed with a cardiac condition... it may take up doctor and patient time unnecessarily. I... utterly fail to see the problem here. If it were 0.01% that ended up having a problem, then OK I can see how this is not a signal worth paying attention to. But 10%?! 10% is huge. Plus, there's no evidence of any harmful side effects here -- it's not like people are being sent into unnecessary surgery because of what their Apple Watch said. They're simply being given normal tests. Now if Apple can make this more accurate, then great. But if this saves even a few lives, then the fact that 9 out of 10 warnings are false positives still seems extremely worth it, no? The article paints this as a failure, when all I'm seeing is success.
- teruakohatu 6y ago
  We also don't know the doctors' false negitive rate.
- meritganset 6y ago
  Let's take this idea to the extreme for a moment and see if you agree that it's a problem. A new device is created that alerts 50% of the US population that they have a condition that needs to be treated. Only 10% of those cases warranted further action. Do you imagine there would be adverse side effects of hospitals being overrun with patients that didn't need to be there? How can you utterly fail to see a problem with an influx of false positives to an already overloaded healthcare system?
  - unwoundmouse 6y ago
    This is ridiculous, if such a device exists and can detect a new condition that 5% of the US population has with a 80% false positive rate, and the condition was as serious as heart disease this would really be a miracle device and people would be grateful that such a thing exists. I really have distaste for those comments that attempt to mask ideology with a veneer of logic. You attempt to use a limiting case argument that actually proves the original comment's argument and then end with rhetoric about how the healthcare system is being flooded already. That may be true, but that isn't relevant to the utility of the apple watch in this scenario.
SomaticPirate 6y ago
It’s funny that this uses data from Mayo, a hospital know for medical over-utilization. Mayo has the “Medallion” program where high net worth clients can basically have any test they want. Full body MRI? No problem. Want a team of doctors to evaluate every bump on you skin? Medallion members can request that. Arguably this allows Mayo to support lower income people but medical over-utilization is a part of Mayo’s business model.
blakesterz 6y ago
The study found about 260 out of about 767,000 people came in because of something they saw on their watch. Out of the ~260, 41 actually had something wrong. Is it safe to assume those 41 people would NOT have gone in without the watch? Maybe. Here's the numbers from the study: https://academic.oup.com/view-large/figure/207880397/ocaa137f1.tif https://academic.oup.com/view-large/figure/207880397/ocaa137... Full study: https://academic.oup.com/jamia/article/27/9/1359/5911974 https://academic.oup.com/jamia/article/27/9/1359/5911974
- qz2 6y ago
  That's a pretty excellent result when you put those numbers on the table.
InTheArena 6y ago
I've had the utterly opposite experience. My father, and my wife's mother have both had cardiac issues. In both cases, doctors praised them for having a Apple Watch, and in one case, said to use that for monitoring her situation rather then send her home with a long term monitor and all that entails. Another extended family member did have a singificant issue that was first revealed with a watch. I personally got a false alert once. I kept a eye on it, and it was fine - mentioned it my doctor, did a EKG just to be sure at my next checkup. That's the way this should work.
alexggordon 6y ago
I, like the average person here it seems, fail to see the issue here, but that aside, this is most likely not a large enough sample size[0] to be a highly trusted statistic. That said, let's look at this from a different perspective. > The Apple Watch abnormal pulse detection feature was designed to address asymptomatic (so-called silent) atrial fibrillation. Atrial fibrillation is the most common cardiac arrhythmia, affecting over 30 million people worldwide[0]. So, assuming there's 8 billion people in the world, this affects .375% of the population. Assuming this was a statistically significant study, that means that if everyone in the world had an apple watch, this would only let 300 million people know they potentially had an AFIB event. Now, an excess of 270 million might seem excessive--but I would weigh excessive with the burden this event puts on the medical community. The primary means of diagnoses for an AFIB event is an ECG[1], which costs $150 on Amazon. Assuming they don't have to buy an ECG for each individual, I would assume the cost to most health care clinics for diagnosis is around $100. However, this test 1. Does not have to be administered by anyone other than a nurse. 2. Only has to be administered once in a lifetime for a diagnosis. So now 10% doesn't seem that bad given that the burden is distributed to the global healthcare system over time. Keep in mind too, even if every single one of these people visited the ER, then this would be completely in line with the fact that only 10% of ER visits result in an admission[2] to the hospital. [0] https://academic.oup.com/jamia/article/27/9/1359/5911974 https://academic.oup.com/jamia/article/27/9/1359/5911974 [1] https://www.mayoclinic.org/diseases-conditions/atrial-fibrillation/diagnosis-treatment/drc-20350630 https://www.mayoclinic.org/diseases-conditions/atrial-fibril... [2] https://www.cdc.gov/nchs/fastats/emergency-department.htm https://www.cdc.gov/nchs/fastats/emergency-department.htm

30 points by Nevaeh 6y ago

If the choice is "The Apple Watch heart monitor sends too many people to the doctor" and "The Apple Watch heart monitor sends too few people to the doctor," I'm OK with this.

ihattendorf 6y ago
I agree, but obviously it depends on how many people is "too many". From the article it looks like 10% of those who went to the doctor were diagnosed with a cardiac condition, but what if it was 1%? A 90% false positive rate is already fairly high.
- henriquez 6y ago
  Maybe it’s just more common to have heart “anomalies” than people think. Historically it’s never been common for people having heart monitors strapped to them all the time, so now we’re In uncharted territory. Maybe there’s some scientific learning we still have to do about how the heart works, and what is considered “in spec.”
  - cratermoon 6y ago
    It is common! I know as a heart patient I once wore a Holter monitor (https://www.mayoclinic.org/tests-procedures/holter-monitor/about/pac-20385039 https://www.mayoclinic.org/tests-procedures/holter-monitor/a...) and in the process learned that your heartbeat isn't perfect. Pre-beats are pretty common and completely harmless.
- closetohome 6y ago
  Another way of looking at it is that 1 in 10 people who were alerted to a potential health issue ended up needing medical assistance. That's actually pretty good for a basic non-medical screening.
  - acdha 6y ago
    That's true with a big caveat: how many of those people get invasive treatments for things which would not have actually become problems? They've backed away from screening as aggressively for certain cancers because the risks of aggressive treatment turned out to be significant and it turned out that more people than expected had tumors which didn't develop into a problem before they died of something else. I don't know how plausible that is with heart issues — maybe aggressive prescription of something like statins with significant side-effects? — but it's very important to consider the cost of false-positives.
    - ebg13 6y ago
      > how many of those people get invasive treatments for things which would not have actually become problems? That's really the doctor's issue to address, not the device's. Sticking your head in the sand isn't the right solution to the problems of dealing with information too aggressively. > They've backed away from screening as aggressively for certain cancers because the risks of aggressive treatment turned out to be significant and it turned out that more people than expected had tumors which didn't develop into a problem before they died of something else. Literally the only way to build viable metrics for which tumors will develop into problems is for _everyone_ get screened, not for no one to get screened. The problem of treating too aggressively is a treatment problem, not a measurement problem. If we avoid measuring, we have no way to learn how to treat better. If we decide to never look because we don't yet know how to handle the information, then we will never know how to handle the information.
      - DanBC 6y ago
        > The problem of treating too aggressively is a treatment problem, not a measurement problem. One of the most important questions you can ask your doctor is "what happens if I don't have this test?" and "What happens if I don't have this treatment?" But most people don't ask these questions. And so it really is a measurement problem. Over-testing, over-diagnosis, and over-treatment are well understood mechanisms of harm. https://ebm.bmj.com/content/23/1/1 https://ebm.bmj.com/content/23/1/1 https://www.bmj.com/content/362/bmj.k2820 https://www.bmj.com/content/362/bmj.k2820 The harm caused by inappropriate screening is so severe that we (UK) have a committee to weigh up the harm vs benefit of different screening programmes. https://www.gov.uk/government/groups/uk-national-screening-committee-uk-nsc https://www.gov.uk/government/groups/uk-national-screening-c...
- deleted 6y ago
  [deleted]
Clent 6y ago
It's not clear from the article if it actually sent anyone to the doctor or if they simply brought it up during an normal visit.

elheffe80 6y ago

Sent me to the doctor, had surgery a few weeks later. Too many is OK IMO.

bobowzki 6y ago
Really, ablation?
Grazester 6y ago
Do you go to your doctor for an annual checkup? Was this something that could have been missed or something that is brought on suddenly?
- ebg13 6y ago
  > Do you go to your doctor for an annual checkup? Doctors will rarely subject anyone to thorough evaluation for things if they aren't over the common danger age and don't report specific symptoms. Your typical annual checkup for people not 50+ is a fairly generic blood panel and very little else. After 50 they may add, what, a prostate probe for men?
  - crazygringo 6y ago
    Exactly. In my life I've had one doctor who performed an EKG as part of my annual checkup, once. The other ~10 primary care physicians I've had never did -- my annual was mostly just a few questions, some blood work, quick reflex tests and I think they looked in my ears and eyes. If my Apple Watch detected a heart irregularity, I can say with certainty it's not something that would be caught otherwise at my next annual checkup.
    - Grazester 6y ago
      At my last physical my doctor did an EKG. It was not requested by me. I guess it all depends on your doctor.
  - Ancapistani 6y ago

ed25519FUUU 6y ago

I see the “the false positives is okay” sentiment widely here. Just keep in mind that medical mistakes is the 3rd leading cause of death in the US. It’s statistically in your favor to limit your exposure to medical mistakes. https://www.google.com/amp/s/www.cnbc.com/amp/2018/02/22/medical-errors-third-leading-cause-of-death-in-america.html https://www.google.com/amp/s/www.cnbc.com/amp/2018/02/22/med...

solidblu 6y ago
So you are suggesting we don't go to the doctor for the leading cause of death because of the 3rd leading risk? Damned if you do, Damned if you don't.
dahart 6y ago
It's complicated and IMO misleading to claim medical mistakes are the 3rd leading "cause" of death as if it's a fact and as if there are clear and better alternatives. No question there are mistakes and room for improvement, but you're linking to meta-studies where people decide what "mistake" means, not a recognized system of categorizing causes of death. I feel like the wording is vaguely implying general malfeasance, while the studies you linked to actually include categories of mistakes that are sometimes unavoidable and sometimes have very unclear lines of responsibility relative to the patient. For example, "One example of a lethal error of communication between provider and patient occurred when cardiologists failed to warn their 19-year-old patient not to run. The patient had experienced syncope while running, and 5 days of inpatient, diagnostic testing were inconclusive; however, his cardiologists knew he was not ready to return to running but failed to warn him against this risk. Having not been warned against running, he resumed running and died 3 weeks later while running." https://journals.lww.com/journalpatientsafety/Fulltext/2013/09000/A_New,_Evidence_based_Estimate_of_Patient_Harms.2.aspx https://journals.lww.com/journalpatientsafety/Fulltext/2013/... This categorization assigns 100% of the blame to a cardiologist who was not performing a procedure and who did not know that the patient would die. The patient chose to do something for which they'd recently experienced issues with, without waiting for complete test results or clear instructions. It's an accident, and unfortunate, but my point is it's very problematic to conclude that the cause of death here is a medical mistake, rather than a heart condition. This is the representative example that the Journal of Patient Safety uses to illustrate communication errors, so it seems reasonable to assume there are many such examples. Furthermore, how many people would be dying if there was no care instead? Probably a lot more, and that's not accounted for. To call mistakes the 3rd leading cause of death, I think we need to account for how many people were saved by medical care too... and when you think about it that way I suspect it's complicated if not impossible to tally that number, meaning that it's also complicated if not impossible to rank medical accidents as a cause of death against objectively identifiable categories like heart attack vs car accident.

crazygringo 6y ago

> Only around 10 percent of people who saw a doctor at the Mayo Clinic after noticing an abnormal pulse reading on their watch were eventually diagnosed with a cardiac condition... it may take up doctor and patient time unnecessarily. I... utterly fail to see the problem here. If it were 0.01% that ended up having a problem, then OK I can see how this is not a signal worth paying attention to. But 10%?! 10% is huge. Plus, there's no evidence of any harmful side effects here -- it's not like people are being sent into unnecessary surgery because of what their Apple Watch said. They're simply being given normal tests. Now if Apple can make this more accurate, then great. But if this saves even a few lives, then the fact that 9 out of 10 warnings are false positives still seems extremely worth it, no? The article paints this as a failure, when all I'm seeing is success.

teruakohatu 6y ago
We also don't know the doctors' false negitive rate.
meritganset 6y ago
Let's take this idea to the extreme for a moment and see if you agree that it's a problem. A new device is created that alerts 50% of the US population that they have a condition that needs to be treated. Only 10% of those cases warranted further action. Do you imagine there would be adverse side effects of hospitals being overrun with patients that didn't need to be there? How can you utterly fail to see a problem with an influx of false positives to an already overloaded healthcare system?
- unwoundmouse 6y ago
  This is ridiculous, if such a device exists and can detect a new condition that 5% of the US population has with a 80% false positive rate, and the condition was as serious as heart disease this would really be a miracle device and people would be grateful that such a thing exists. I really have distaste for those comments that attempt to mask ideology with a veneer of logic. You attempt to use a limiting case argument that actually proves the original comment's argument and then end with rhetoric about how the healthcare system is being flooded already. That may be true, but that isn't relevant to the utility of the apple watch in this scenario.

SomaticPirate 6y ago

It’s funny that this uses data from Mayo, a hospital know for medical over-utilization. Mayo has the “Medallion” program where high net worth clients can basically have any test they want. Full body MRI? No problem. Want a team of doctors to evaluate every bump on you skin? Medallion members can request that. Arguably this allows Mayo to support lower income people but medical over-utilization is a part of Mayo’s business model.

blakesterz 6y ago

The study found about 260 out of about 767,000 people came in because of something they saw on their watch. Out of the ~260, 41 actually had something wrong. Is it safe to assume those 41 people would NOT have gone in without the watch? Maybe. Here's the numbers from the study: https://academic.oup.com/view-large/figure/207880397/ocaa137f1.tif https://academic.oup.com/view-large/figure/207880397/ocaa137... Full study: https://academic.oup.com/jamia/article/27/9/1359/5911974 https://academic.oup.com/jamia/article/27/9/1359/5911974

qz2 6y ago
That's a pretty excellent result when you put those numbers on the table.

InTheArena 6y ago

I've had the utterly opposite experience. My father, and my wife's mother have both had cardiac issues. In both cases, doctors praised them for having a Apple Watch, and in one case, said to use that for monitoring her situation rather then send her home with a long term monitor and all that entails. Another extended family member did have a singificant issue that was first revealed with a watch. I personally got a false alert once. I kept a eye on it, and it was fine - mentioned it my doctor, did a EKG just to be sure at my next checkup. That's the way this should work.

alexggordon 6y ago

I, like the average person here it seems, fail to see the issue here, but that aside, this is most likely not a large enough sample size[0] to be a highly trusted statistic. That said, let's look at this from a different perspective. > The Apple Watch abnormal pulse detection feature was designed to address asymptomatic (so-called silent) atrial fibrillation. Atrial fibrillation is the most common cardiac arrhythmia, affecting over 30 million people worldwide[0]. So, assuming there's 8 billion people in the world, this affects .375% of the population. Assuming this was a statistically significant study, that means that if everyone in the world had an apple watch, this would only let 300 million people know they potentially had an AFIB event. Now, an excess of 270 million might seem excessive--but I would weigh excessive with the burden this event puts on the medical community. The primary means of diagnoses for an AFIB event is an ECG[1], which costs $150 on Amazon. Assuming they don't have to buy an ECG for each individual, I would assume the cost to most health care clinics for diagnosis is around $100. However, this test 1. Does not have to be administered by anyone other than a nurse. 2. Only has to be administered once in a lifetime for a diagnosis. So now 10% doesn't seem that bad given that the burden is distributed to the global healthcare system over time. Keep in mind too, even if every single one of these people visited the ER, then this would be completely in line with the fact that only 10% of ER visits result in an admission[2] to the hospital. [0] https://academic.oup.com/jamia/article/27/9/1359/5911974 https://academic.oup.com/jamia/article/27/9/1359/5911974 [1] https://www.mayoclinic.org/diseases-conditions/atrial-fibrillation/diagnosis-treatment/drc-20350630 https://www.mayoclinic.org/diseases-conditions/atrial-fibril... [2] https://www.cdc.gov/nchs/fastats/emergency-department.htm https://www.cdc.gov/nchs/fastats/emergency-department.htm