5 ms·

This isn't quite as big of a problem as they're portraying and there is a lot of misinformation in these comments. For context I'm a radiologist at one of the a

This isn't quite as big of a problem as they're portraying and there is a lot of misinformation in these comments. For context I'm a radiologist at one of the academic centres involved in the ongoing TMIST (3D mammography) trial. For starters, false positives are an inherent part of medical diagnosis. Interpretation and diagnosis is probabilistic based on ROC curves balancing different sensitivities and specificities balanced with disease prevalence as well as the significance of missed/delayed diagnosis. Every concern that has been raised in these comments is factored into our reporting system. All radiologists use the BI-RADS reporting system which is internationally validated and expresses probabilities of malignancy to guide further management based on specific imaging. features that have been validated against pathologic data from excised surgical specimens and biopsies. This was done to homogenize reporting practices. For example a BI-RADS 5 lesion, which is based on specific imaging features not a gestalt/expert opinion, denotes a 95% probability of malignancy and would generally result urgent biopsy + surgical consultation. A BI-RADS 3 lesion has a 2% chance of malignancy and would generally be followed with serial imaging for a specified interval. BI-RADS 4 is the middle, this is broken down into A/B/C but essentially anything with 3%+ probability goes for biopsy. That means we expect up to 97% of the lesions we biopsy will have been false positives. This is intentional and desired, this is because the harms of missing breast cancer is horrible, early detection results in treatment with lumpectomy + radiation vs advanced breast cancer needing systemic therapies and radical resections. Second, what happens when there is an abnormality on a screening mammogram is not straight to biopsy. We use additional mammography views and ultrasound to help us find out what's going on before making that decision. We also have MRI to troubleshoot and increasingly for screening, certainly for high risk and extremely dense breasts. Tomosynthesis or 3D mammography is not a cure-all for false positives, it's also not intended for dense breasts. The point of Tomosynthesis is to reduce callbacks for overlapping breast tissue which can sometimes look like a real cancer, and to increase sensitivity for architectural distortion (an occult or infiltrative cancer distorting the fibroglandular architecture) which can be really hard to pick up on 2D. It works really great for that. Patients with extremely dense breasts should really be getting screening MRIs (US as a lesser alternative) in today's age and this is happening with increasing frequently. Further reading: https://en.wikipedia.org/wiki/Receiver_operating_characteristic https://en.wikipedia.org/wiki/Receiver_operating_characteris... https://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/Bi-Rads https://www.acr.org/Clinical-Resources/Reporting-and-Data-Sy... https://www.nejm.org/doi/full/10.1056/NEJMoa1903986 https://www.nejm.org/doi/full/10.1056/NEJMoa1903986 https://www.cancer.gov/about-cancer/treatment/clinical-trials/nci-supported/tmist https://www.cancer.gov/about-cancer/treatment/clinical-trial...

dontreact 5y ago
For starters, false positives are an inherent part of medical diagnosis. Interpretation and diagnosis is probabilistic based on ROC curves balancing different sensitivities and specificities balanced with disease prevalence as well as the significance of missed/delayed diagnosis. I totally get this and it's technical terminology. I think my takeaway from trying to explain why this isn't a big deal on this thread is that calling a tomo or mammo "positive" is just a recipe for confusion. I understand why it's done this way historically, but the idea of a mammo or tomo being "positive for cancer" makes no sense since there will almost always be either an ultrasound a biopsy or something else before a diagnosis is made. It's a test for whether a more invasive/expensive test is worth it, not a test for cancer. Curious what your opinion is on AI for DBT? I think there is great long term potential here, even more so than the potential that we have seen so far in mammo (such as https://www.nature.com/articles/s41586-019-1799-6 https://www.nature.com/articles/s41586-019-1799-6), because it's easier for AI to thoroughly look at things in 3d and spot new patterns that are not obvious to human eyes.
- haldujai 5y ago
  AI in medical imaging has been very limited and ineffective thus far. My institution pays for an expensive lesion detection / mammography CAD software package but it generates so many false positives I don’t use it in my practice. Overall having to review an extra n pseudolesions per scan results in decision fatigue and increases my miss rate, at least in my experience. There are several issues with AI when it comes to radiology that I can’t address with a single comment but given your question my biggest criticism would be that we don’t have a sensitivity problem in medicine. What we have is largely a specificity problem, but that’s inherent with imaging which is not a ground truth representation of a physical entity but rather how this tissue attenuates photons (for mammo). AI, like self driving cars, has to be an order of magnitude “safer” which generally translates into decreased specificity. What I mean by this is that I don’t miss a significant number of breast cancers, in fact it’s very very low. So an AI to improve my detection is pointless to me, what I really want is to recall less patients but AI won’t necessarily help with that because it’s inherent limitations of the imaging modality (scar and cancer look the same). I think you are in the right track that AI looking at information humans are not currently looking at is the future / next step. Not to replace a task already handled by physicians who have the ability to integrate disparate health records (I.e. does this patient have easier access to breast MRI or biopsy given location/insurance/biopsy schedule/MRI schedule/hospital resources and what are her personal goals of care to help me decide what to do with this lesion) that is currently silo’s and inaccessible to AI.

dontreact 5y ago

For starters, false positives are an inherent part of medical diagnosis. Interpretation and diagnosis is probabilistic based on ROC curves balancing different sensitivities and specificities balanced with disease prevalence as well as the significance of missed/delayed diagnosis. I totally get this and it's technical terminology. I think my takeaway from trying to explain why this isn't a big deal on this thread is that calling a tomo or mammo "positive" is just a recipe for confusion. I understand why it's done this way historically, but the idea of a mammo or tomo being "positive for cancer" makes no sense since there will almost always be either an ultrasound a biopsy or something else before a diagnosis is made. It's a test for whether a more invasive/expensive test is worth it, not a test for cancer. Curious what your opinion is on AI for DBT? I think there is great long term potential here, even more so than the potential that we have seen so far in mammo (such as https://www.nature.com/articles/s41586-019-1799-6 https://www.nature.com/articles/s41586-019-1799-6), because it's easier for AI to thoroughly look at things in 3d and spot new patterns that are not obvious to human eyes.

haldujai 5y ago
AI in medical imaging has been very limited and ineffective thus far. My institution pays for an expensive lesion detection / mammography CAD software package but it generates so many false positives I don’t use it in my practice. Overall having to review an extra n pseudolesions per scan results in decision fatigue and increases my miss rate, at least in my experience. There are several issues with AI when it comes to radiology that I can’t address with a single comment but given your question my biggest criticism would be that we don’t have a sensitivity problem in medicine. What we have is largely a specificity problem, but that’s inherent with imaging which is not a ground truth representation of a physical entity but rather how this tissue attenuates photons (for mammo). AI, like self driving cars, has to be an order of magnitude “safer” which generally translates into decreased specificity. What I mean by this is that I don’t miss a significant number of breast cancers, in fact it’s very very low. So an AI to improve my detection is pointless to me, what I really want is to recall less patients but AI won’t necessarily help with that because it’s inherent limitations of the imaging modality (scar and cancer look the same). I think you are in the right track that AI looking at information humans are not currently looking at is the future / next step. Not to replace a task already handled by physicians who have the ability to integrate disparate health records (I.e. does this patient have easier access to breast MRI or biopsy given location/insurance/biopsy schedule/MRI schedule/hospital resources and what are her personal goals of care to help me decide what to do with this lesion) that is currently silo’s and inaccessible to AI.