5 ms·

I'm just finishing interventional radiology training and I moonlight as a diagnostic radiologist (not to mention having an undergrad/master's in computer scienc

I'm just finishing interventional radiology training and I moonlight as a diagnostic radiologist (not to mention having an undergrad/master's in computer science). Almost 90% of the diagnostic studies I read could be pre-drafted by AI. That's where the money is and where AI-in-radiology companies should focus. The money is not in detecting hemorrhage or pulmonary embolism. It's a classic fallacy to think that life-saving means money-saving. Rather, the money in radiology is reads per day. Here's a user story: A private practice radiologist reads 20 abdomen and pelvis CT scans with contrast per day. In each of these studies, he must write a short description of each organ. For example, "Gall bladder: Unremarkable" or "Gallbladder: Cholecystitis without evidence of cholecystitis" or "Gallbladder: Dependent sludge." There are around 15 such organs (liver, gallbladder, pancreas, spleen, adrenal glands, kidneys, etc.). The AI should auto-populate the radiologist report with an appropriate description of each organ system. The job of the radiologist is to confirm what the AI says in each section, and to go into further detail as needed. It's essentially just customizing the existing template to each patient. This type of pre-drafting is exactly what radiology residents do and what companies like vRad do.

haldujai 3y ago
What you’re describing is what a template does, I just dictate “stone” or “sludge” from a pick list and never a normal statement. There is also existing technology that creates a structured report from free dictations. Your math also doesn’t add up because 90%ish of studies are normal so you can basically sign a template without looking at the pictures and be more or less entirely accurate, before adding AI. Also what practice only reads 20 abdo CTs a day? Seriously, let me know I could use a more relaxing job. Lastly, an AI that functions with the competency of a PGY3 resident does not exist at the moment or on the horizon. Especially for cross sectional studies. Source: am a diagnostic radiologist and AI researcher (on the NLP side).
throwaway85858 3y ago
this! I asked a neurologist and she said at least 40% of her time is spent on formulaic discharge letters and rounds documentation.
bee_rider 3y ago
Would it be possible to instead have a checkbox for the status of each organ, and have it default to whatever the average case is?
- deleted 3y ago
  [deleted]
- aabajian 3y ago
  Drop-down lists (called pick-list in PowerScribe speak) are very common. The issue is that they are completely separate from the image viewer and have no knowledge of the current patient's findings. Admittedly, even a simple classifier could likely choose the "most likely" completion phrase...but again there's a disconnect between the viewer software and the dictation software.
NeuroCoder 3y ago
MD PhD student here and this is 100% on the money and what many I work with try to do. The diagnostic classifiers seem to be about pushing papers and getting funding.
la64710 3y ago
So essentially the job of the radiologist is to carefully compare what the AI generated with the report to ensure that the generated text is correct. So in fact the radiologist now has to read both pieces of text (AI generated and the report itself) instead of just reading the report and typing her conclusions. Does it seem like less work or more work? I am not a radiologist but as a programmer trying to work with AI generated BS most of the time I find the work to be increased. We need AI that can reliably produce results (maybe through testing and internal feedback mechanisms).
kurthr 3y ago
What percentage of the diagnostic radiologists won't bother to even review the AI output? 3%, 10%, 30%? It'll increase as the AI improves. Unless the image is sent to multiple radiologists or their metrics (eg spotting AI errors) are updated, we'll get crap results. I know one top tier diagnostic radiologist and he could tell which piece of equipment an image was taken on (or if something is out of calibration) with a fairly quick glance. Regularly on the phone with techs and technologists to get things fixed or reimaged. He retired recently. From his telling most of the work has been outsourced to India for the last 10-15 years. He only gets the really challenging stuff. edit: Interestingly, it seems like breast imaging is still done more in house and there are regular reviews, but that's from a different source.
- gymbeaux 3y ago
  We already have an issue in the US at least, of doctors waxing about how their favorite part of their job is the relationship with their patients, and how they take an “evidence-based approach” and take time to listen to their patients… only for them to do the polar opposite in reality. This is just going to be another case of patients needing to take ownership of their own care. It’s no longer “tell the doc your symptoms and they’ll figure out what’s wrong”, it’s now “tell the doc what you think you have and if they interrupt you and brush it off, go to another doc” until someone agrees with your diagnosis and gets you the referral or test or prescription that you need. For those who know what I’m talking about, I’m sorry. For those who don’t- you’ll learn this lesson someday.
  - jmhmd 3y ago
    To be fair, this is not how doctors want to practice medicine.
  - BiteCode_dev 3y ago
    You really have to double check everything yourself, which sucks if you are weaken. I was at the hospital last week for a pretty bad condition and my IV was badly set. I told the nurse to change it for 4 days, they kept refusing, trying to adjust it. Eventually I just told the doctor what was happening, and he said "oh, that's why your readings were messed up, I was wondering why I had to change your drug dose every day". They changed my IV in the next hours, and my conditions improved overnight. I discussed that with some doctor and nurse friends, and they all told me to never, ever trust a health practitioner you don't know very well. You need to be very proactive in your treatment, because they are exhausted, have many people to treat in a row and are only humans.
jmhmd 3y ago
You are partially correct, but I think the value is not in generating a draft for the radiologist to review, but to take a radiologists “raw” dictation, which pertains only to relevant findings in a case, and transform that into a complete, well formatted report. This is what scribes do in lots of different medical specialties, and save a ton of time. In my experience, reviewing pre drafted reports, as from residents, can often make my job slower rather than faster, as you have to read the whole thing to know what needs editing. Also, at least in my practice, I would double the number of CTs read in an average day :)
abxytg 3y ago
I've seen some awesome demos of exactly this, ingesting priors and all of that.
oars 3y ago
Thank you for sharing your insight.
jacurtis 3y ago
A system like this will lead to an overall increase of mis-readings. Humans are incredibly forgiving and lack focus when confirming opinions, versus generating raw opinions. Imagine a Radiologist that already has to read 20 scans per day. With AI helping, and Radiologists just confirming, they will probably be expected to analyze 40 scans per day. Eventually the Radiologist will quickly get into a habit of confirm, confirm, confirm... 500 times in a day. They will stop treating each case as a real reading and instead they look at the AI's output and think, "looks about right" and will become complacent in actually providing valuable feedback. Furthermore, they will be expected to produce more work, meaning they are giving every case less attention than previous. They will begin to just trust the AI to keep up with their schedule. The better use of AI is to flip it on its head. Let the radiologists continue to work like normal. They will produce reports like they do now, only 20 per day. But the AI will analyze each scan after them, and if it flags the scan as remarkable or disagrees with the radiologist, then it bubbles the scan back to the top for further analysis. This way Radiologists are still expected to provide their expertise as they do currently and we generally rely on their opinion, but we will have an added layer provided by AI to catch anything that might be missed. This overall improves the accuracy of medical imaging instead of compromising it (which the original proposed AI solution certainly would do). Productivity would not be increased in this scenario, but accuracy (and indirectly malpractice instances) would improve. I for one, would not want to get scanned somewhere that relies on AI to analyze my scan. But i would love a place that used AI as a second pair of eyes to confirm a human's findings.
- cicce19 3y ago
  There is a trade off of quality and quantity. Most healthcare systems reimburse based on quantity so which way do you think AI will be implemented?

haldujai 3y ago

What you’re describing is what a template does, I just dictate “stone” or “sludge” from a pick list and never a normal statement. There is also existing technology that creates a structured report from free dictations. Your math also doesn’t add up because 90%ish of studies are normal so you can basically sign a template without looking at the pictures and be more or less entirely accurate, before adding AI. Also what practice only reads 20 abdo CTs a day? Seriously, let me know I could use a more relaxing job. Lastly, an AI that functions with the competency of a PGY3 resident does not exist at the moment or on the horizon. Especially for cross sectional studies. Source: am a diagnostic radiologist and AI researcher (on the NLP side).

throwaway85858 3y ago

this! I asked a neurologist and she said at least 40% of her time is spent on formulaic discharge letters and rounds documentation.

bee_rider 3y ago

Would it be possible to instead have a checkbox for the status of each organ, and have it default to whatever the average case is?

deleted 3y ago
[deleted]
aabajian 3y ago
Drop-down lists (called pick-list in PowerScribe speak) are very common. The issue is that they are completely separate from the image viewer and have no knowledge of the current patient's findings. Admittedly, even a simple classifier could likely choose the "most likely" completion phrase...but again there's a disconnect between the viewer software and the dictation software.

NeuroCoder 3y ago

MD PhD student here and this is 100% on the money and what many I work with try to do. The diagnostic classifiers seem to be about pushing papers and getting funding.

la64710 3y ago

So essentially the job of the radiologist is to carefully compare what the AI generated with the report to ensure that the generated text is correct. So in fact the radiologist now has to read both pieces of text (AI generated and the report itself) instead of just reading the report and typing her conclusions. Does it seem like less work or more work? I am not a radiologist but as a programmer trying to work with AI generated BS most of the time I find the work to be increased. We need AI that can reliably produce results (maybe through testing and internal feedback mechanisms).

kurthr 3y ago

What percentage of the diagnostic radiologists won't bother to even review the AI output? 3%, 10%, 30%? It'll increase as the AI improves. Unless the image is sent to multiple radiologists or their metrics (eg spotting AI errors) are updated, we'll get crap results. I know one top tier diagnostic radiologist and he could tell which piece of equipment an image was taken on (or if something is out of calibration) with a fairly quick glance. Regularly on the phone with techs and technologists to get things fixed or reimaged. He retired recently. From his telling most of the work has been outsourced to India for the last 10-15 years. He only gets the really challenging stuff. edit: Interestingly, it seems like breast imaging is still done more in house and there are regular reviews, but that's from a different source.

gymbeaux 3y ago
We already have an issue in the US at least, of doctors waxing about how their favorite part of their job is the relationship with their patients, and how they take an “evidence-based approach” and take time to listen to their patients… only for them to do the polar opposite in reality. This is just going to be another case of patients needing to take ownership of their own care. It’s no longer “tell the doc your symptoms and they’ll figure out what’s wrong”, it’s now “tell the doc what you think you have and if they interrupt you and brush it off, go to another doc” until someone agrees with your diagnosis and gets you the referral or test or prescription that you need. For those who know what I’m talking about, I’m sorry. For those who don’t- you’ll learn this lesson someday.
- jmhmd 3y ago
  To be fair, this is not how doctors want to practice medicine.
- BiteCode_dev 3y ago
  You really have to double check everything yourself, which sucks if you are weaken. I was at the hospital last week for a pretty bad condition and my IV was badly set. I told the nurse to change it for 4 days, they kept refusing, trying to adjust it. Eventually I just told the doctor what was happening, and he said "oh, that's why your readings were messed up, I was wondering why I had to change your drug dose every day". They changed my IV in the next hours, and my conditions improved overnight. I discussed that with some doctor and nurse friends, and they all told me to never, ever trust a health practitioner you don't know very well. You need to be very proactive in your treatment, because they are exhausted, have many people to treat in a row and are only humans.

jmhmd 3y ago

You are partially correct, but I think the value is not in generating a draft for the radiologist to review, but to take a radiologists “raw” dictation, which pertains only to relevant findings in a case, and transform that into a complete, well formatted report. This is what scribes do in lots of different medical specialties, and save a ton of time. In my experience, reviewing pre drafted reports, as from residents, can often make my job slower rather than faster, as you have to read the whole thing to know what needs editing. Also, at least in my practice, I would double the number of CTs read in an average day :)

abxytg 3y ago

I've seen some awesome demos of exactly this, ingesting priors and all of that.

oars 3y ago

Thank you for sharing your insight.

jacurtis 3y ago

A system like this will lead to an overall increase of mis-readings. Humans are incredibly forgiving and lack focus when confirming opinions, versus generating raw opinions. Imagine a Radiologist that already has to read 20 scans per day. With AI helping, and Radiologists just confirming, they will probably be expected to analyze 40 scans per day. Eventually the Radiologist will quickly get into a habit of confirm, confirm, confirm... 500 times in a day. They will stop treating each case as a real reading and instead they look at the AI's output and think, "looks about right" and will become complacent in actually providing valuable feedback. Furthermore, they will be expected to produce more work, meaning they are giving every case less attention than previous. They will begin to just trust the AI to keep up with their schedule. The better use of AI is to flip it on its head. Let the radiologists continue to work like normal. They will produce reports like they do now, only 20 per day. But the AI will analyze each scan after them, and if it flags the scan as remarkable or disagrees with the radiologist, then it bubbles the scan back to the top for further analysis. This way Radiologists are still expected to provide their expertise as they do currently and we generally rely on their opinion, but we will have an added layer provided by AI to catch anything that might be missed. This overall improves the accuracy of medical imaging instead of compromising it (which the original proposed AI solution certainly would do). Productivity would not be increased in this scenario, but accuracy (and indirectly malpractice instances) would improve. I for one, would not want to get scanned somewhere that relies on AI to analyze my scan. But i would love a place that used AI as a second pair of eyes to confirm a human's findings.

cicce19 3y ago
There is a trade off of quality and quantity. Most healthcare systems reimburse based on quantity so which way do you think AI will be implemented?