6 ms·

woah. woah. woah. hold on a second here... are we comfortable enough with understanding all of the behavior of deep learning models to where we can confidently

by a-dub 5y ago

woah. woah. woah. hold on a second here... are we comfortable enough with understanding all of the behavior of deep learning models to where we can confidently put them in the pipeline for diagnostic clinical imaging? i'm okay with using them for image analysis, but denoising and other image production tasks seems dangerous. how do you know what you're looking at is real as opposed to something that just looks convincing? (like deep neural nets are famous for producing)

phkahler 5y ago
I'm kind of thinking the opposite. Image analysis is where you don't want AI. Noise removal is further upstream (I'm assuming) and if it fails wouldn't it cause significant artifacts (blur for example) in the images? It would be helpful to see results with and without this correction, or even with varying degrees of it.
thfuran 5y ago
What's the worst that could happen? https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...
iancmceachern 5y ago
Yes. This is past tense, other companies are already doing this, in the clinic.
- lostlogin 5y ago
  A link: https://www.siemens-healthineers.com/magnetic-resonance-imaging/technologies-and-innovations/deep-resolve https://www.siemens-healthineers.com/magnetic-resonance-imag...
MauranKilom 5y ago
Agreed, but I believe "using AI for inference from sparse observations" is unfortunately a thing already.
- VikingCoder 5y ago
  That's not "unfortunate." Sparse observations save lives. A quicker MR. Less X-Ray exposure. It's totally valid to worry about validation, but to the degree you can validate image processing algorithms of any kind - AI or otherwise - they absolutely save lives.
  - aidenn0 5y ago
    NMRI use microwaves, not x-rays.
    - VikingCoder 5y ago
      I was talking about MR and CT. Applies to PET, too. Image processing saves lives.
    - lostlogin 5y ago
      Yes, and when a quick MRI is available, it can remove the need for a CT. Fast brain protocols are now less than 5 minutes. This makes things practical that weren't before.
- lostlogin 5y ago
  It's not unfortunate, it improves acquisition times and image quality - I use it daily.
petra 5y ago
It's better than nothing. Let's start with those situations, and slowly build a database proving or disproving this technology.
- jcims 5y ago
  >However, MRI accessibility is low and extremely inhomogeneous around the world. According to the 2020 Organisation for Economic Co-operation and Development (OECD) statistics, there are approximately 65,000 installations of MRI scanners worldwide (~7 per million inhabitants) Given that my little podunk hospital in the midwest seems to have roughly 5x the worldwide average number of MRI machines, totally agree.
  - lostlogin 5y ago
    MRI is also a massive revenue generator. That's a key reason they buy them.
l33tman 5y ago
This project doesn't use AI to improve the image, they use it to estimate the EMI noise from the surroundings. So they're not "filling in the gaps" in the actual resulting 3D voxel volume with fantasy voxels (which I hope will never ever fly in a clinical setting). "To tackle the EMI signals from the external environments and internal low-cost electronics during scanning, we developed a deep learning driven EMI cancellation scheme" So it's kind of using deep learning to improve the SnR in the RF reception. Of course this could theoretically also lead to "fantasy voxels" but due to the nature of MRI decoding, I'm willing to guess that bad predictions of the EMI interference will not show up as unnoticeable alterations of realistic tissue imaging but rather as artefacts all over the volume, like you normally see in clinical MRIs that weren't taken 100% optimally.
- ortusdux 5y ago
  I'm glad that this is the approach that they are taking. There have been plenty of issues with fMRI false positives due to misconfigured software. The most famous would probably be the IG Nobel winning study that detected brain activity in a store-bought salmon: https://blogs.scientificamerican.com/scicurious-brain/ignobel-prize-in-neuroscience-the-dead-salmon-study/ https://blogs.scientificamerican.com/scicurious-brain/ignobe... https://www.discovermagazine.com/mind/fmri-gets-slap-in-the-face-with-a-dead-fish#.UTOm8zdjp9U https://www.discovermagazine.com/mind/fmri-gets-slap-in-the-... Later studies called into question the results of between 10% and 40% of historic fMRI studies: https://blogs.warwick.ac.uk/nichols/entry/bibliometrics_of_cluster/ https://blogs.warwick.ac.uk/nichols/entry/bibliometrics_of_c... https://www.pnas.org/content/113/28/7900 https://www.pnas.org/content/113/28/7900
  - jacquesm 5y ago
    > The most famous would probably be the IG Nobel winning study that detected brain activity in a store-bought salmon: A store-bought dead salmon. I am assuming that most salmons bought in stores are dead but that particular detail is rather relevant here. Also that had me laughing, what a great move.
    - 5y ago
scratcheee 5y ago
Exactly my thoughts too. I'm fine with a simple noise-removal pass, but if the AI is context-aware, what's to stop it saying "hmm, this brain would look more like a normal brain if I remove these tumors". Obviously, they'll test for that, but that only handles common cases they concider, it's always going to be a risk for more unusual sceanrios, and the danger with altering the data is that anyone looking at the results wont have a way to tell how dubious that data is. Reminds me of https://en.wikipedia.org/wiki/Xerox#Character_substitution_bug https://en.wikipedia.org/wiki/Xerox#Character_substitution_b... which was _so much_ worse than the equivilent OCR bug because it occured at the image level, where everyone expects errors to to produce noise, not contextly sensible and sharp _but wrong_ characters. EDIT: based on other comments below, this is thankfully not the case, the AI just understands noise, it doesn't try to "fill in the blanks" based on how brains are supposed to look.
- xattt 5y ago
  Therein lies the dilemma of this technology: would a scanner that might sometimes substitute information be better than no scanner at all?
- robertlagrant 5y ago
  > a simple noise-removal pass Even that is inventing data, no?
  - scratcheee 5y ago
    Yes, but context is key. Denoising can on average improve the result, but sometimes it will be wrong. Spotting when it goes wrong is potentially a difficult task, but generally the difficulty scales pretty clearly with the difficulty of understanding the original image anyway. If you can't spot when a denoising filter has screwed up, chances are you wouldn't have spotted anything interesting in the original image anyway. But once an AI is context-aware things get way more complicated - it will try very hard to produce an image that doesn't _look_ wrong. Even if it goes wrong, it can go wrong and still succeed in managing to make an image that looks correct, it just no longer matches the real brain that was scanned. Perhaps it decided a tumor was just a smudge on the lense, and invented some brain to go behind it. An operator expecting to see brain and seeing brain wont think anything of it. When the patient dies, they may look back and say "wow, that tumor didn't exist at all just 3 days before! that should be impossible!". tldr: Having an ai that might make mistakes is one thing, having an ai that can just invent exactly the data everyone is expecting to see is dangerous.
VikingCoder 5y ago
That's a valid risk. You're asking a cost-benefit question. The cost of an invalid diagnosis is indeed high. The cost of no diagnosis at all is also high. This device will not replace the MR at your local hospital. It will be the first MR device in hospitals that have never had one before.
- lostlogin 5y ago
  Veterinary MRI may be another application.
  - VikingCoder 5y ago
    Good point. Cargo inspection? Luggage inspection? I dunno.
    - lostlogin 5y ago
      There are a lot of applications that are surprising. I’ve scanned for salmon farmers (is that the term?) who want to check they are breeding good fish and are looking at spine alignment. I’ve scanned logs for forestry managers who want to look at something in their trees. I’ve scanned old hearts that have been sitting in formalin for decades. All are probably better at higher field strength but maybe some of that can be compensated for by scanning for longer? A log isn’t going to move, and a dead fish scan is likely only limited by the time it takes for it to rot. I’d be scared of scanning unknown things, it might be a low field MRI, but it’s still a big magnet.
axg11 5y ago
Whether you’re comfortable with it or not - it’s already happened and in production. Look up Subtle Medical and GE AIR Recon.
- lostlogin 5y ago
  Literally just purchased this - Installed tomorrow: https://www.siemens-healthineers.com/magnetic-resonance-imaging/technologies-and-innovations/deep-resolve https://www.siemens-healthineers.com/magnetic-resonance-imag...
mattkrause 5y ago
Amen! I have seen an alarming number of talks where someone proposes to algorithmically add Gado contrast or turn a T1 into a T2 image. In a few very specific contexts, this makes sense (e.g., aligning a T1 taken in one session with a T2 taken in another). Otherwise though, it seems dangerous to mistake a "real" image with the expected image given another one.
- lostlogin 5y ago
  If reducing gadolinium dose is the aim, a more prompt following of the literature, radiologist request (rather than surgeon demand) and weight based dosing would drastically reduce dosage. A moaning radiographer, what a surprise!
fluidcruft 5y ago
Deep learning reconstructions are already marketed/sold in high-end commercial scanners.

by a-dub 5y ago

phkahler 5y ago

I'm kind of thinking the opposite. Image analysis is where you don't want AI. Noise removal is further upstream (I'm assuming) and if it fails wouldn't it cause significant artifacts (blur for example) in the images? It would be helpful to see results with and without this correction, or even with varying degrees of it.

thfuran 5y ago

What's the worst that could happen? https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...

iancmceachern 5y ago

Yes. This is past tense, other companies are already doing this, in the clinic.

lostlogin 5y ago
A link: https://www.siemens-healthineers.com/magnetic-resonance-imaging/technologies-and-innovations/deep-resolve https://www.siemens-healthineers.com/magnetic-resonance-imag...

MauranKilom 5y ago

Agreed, but I believe "using AI for inference from sparse observations" is unfortunately a thing already.

VikingCoder 5y ago
That's not "unfortunate." Sparse observations save lives. A quicker MR. Less X-Ray exposure. It's totally valid to worry about validation, but to the degree you can validate image processing algorithms of any kind - AI or otherwise - they absolutely save lives.
- aidenn0 5y ago
  NMRI use microwaves, not x-rays.
  - VikingCoder 5y ago
    I was talking about MR and CT. Applies to PET, too. Image processing saves lives.
  - lostlogin 5y ago
    Yes, and when a quick MRI is available, it can remove the need for a CT. Fast brain protocols are now less than 5 minutes. This makes things practical that weren't before.
lostlogin 5y ago
It's not unfortunate, it improves acquisition times and image quality - I use it daily.

petra 5y ago

It's better than nothing. Let's start with those situations, and slowly build a database proving or disproving this technology.

jcims 5y ago
>However, MRI accessibility is low and extremely inhomogeneous around the world. According to the 2020 Organisation for Economic Co-operation and Development (OECD) statistics, there are approximately 65,000 installations of MRI scanners worldwide (~7 per million inhabitants) Given that my little podunk hospital in the midwest seems to have roughly 5x the worldwide average number of MRI machines, totally agree.
- lostlogin 5y ago
  MRI is also a massive revenue generator. That's a key reason they buy them.

l33tman 5y ago

This project doesn't use AI to improve the image, they use it to estimate the EMI noise from the surroundings. So they're not "filling in the gaps" in the actual resulting 3D voxel volume with fantasy voxels (which I hope will never ever fly in a clinical setting). "To tackle the EMI signals from the external environments and internal low-cost electronics during scanning, we developed a deep learning driven EMI cancellation scheme" So it's kind of using deep learning to improve the SnR in the RF reception. Of course this could theoretically also lead to "fantasy voxels" but due to the nature of MRI decoding, I'm willing to guess that bad predictions of the EMI interference will not show up as unnoticeable alterations of realistic tissue imaging but rather as artefacts all over the volume, like you normally see in clinical MRIs that weren't taken 100% optimally.

ortusdux 5y ago
I'm glad that this is the approach that they are taking. There have been plenty of issues with fMRI false positives due to misconfigured software. The most famous would probably be the IG Nobel winning study that detected brain activity in a store-bought salmon: https://blogs.scientificamerican.com/scicurious-brain/ignobel-prize-in-neuroscience-the-dead-salmon-study/ https://blogs.scientificamerican.com/scicurious-brain/ignobe... https://www.discovermagazine.com/mind/fmri-gets-slap-in-the-face-with-a-dead-fish#.UTOm8zdjp9U https://www.discovermagazine.com/mind/fmri-gets-slap-in-the-... Later studies called into question the results of between 10% and 40% of historic fMRI studies: https://blogs.warwick.ac.uk/nichols/entry/bibliometrics_of_cluster/ https://blogs.warwick.ac.uk/nichols/entry/bibliometrics_of_c... https://www.pnas.org/content/113/28/7900 https://www.pnas.org/content/113/28/7900
- jacquesm 5y ago
  > The most famous would probably be the IG Nobel winning study that detected brain activity in a store-bought salmon: A store-bought dead salmon. I am assuming that most salmons bought in stores are dead but that particular detail is rather relevant here. Also that had me laughing, what a great move.
  - 5y ago

scratcheee 5y ago

Exactly my thoughts too. I'm fine with a simple noise-removal pass, but if the AI is context-aware, what's to stop it saying "hmm, this brain would look more like a normal brain if I remove these tumors". Obviously, they'll test for that, but that only handles common cases they concider, it's always going to be a risk for more unusual sceanrios, and the danger with altering the data is that anyone looking at the results wont have a way to tell how dubious that data is. Reminds me of https://en.wikipedia.org/wiki/Xerox#Character_substitution_bug https://en.wikipedia.org/wiki/Xerox#Character_substitution_b... which was _so much_ worse than the equivilent OCR bug because it occured at the image level, where everyone expects errors to to produce noise, not contextly sensible and sharp _but wrong_ characters. EDIT: based on other comments below, this is thankfully not the case, the AI just understands noise, it doesn't try to "fill in the blanks" based on how brains are supposed to look.

xattt 5y ago
Therein lies the dilemma of this technology: would a scanner that might sometimes substitute information be better than no scanner at all?
robertlagrant 5y ago
> a simple noise-removal pass Even that is inventing data, no?
- scratcheee 5y ago
  Yes, but context is key. Denoising can on average improve the result, but sometimes it will be wrong. Spotting when it goes wrong is potentially a difficult task, but generally the difficulty scales pretty clearly with the difficulty of understanding the original image anyway. If you can't spot when a denoising filter has screwed up, chances are you wouldn't have spotted anything interesting in the original image anyway. But once an AI is context-aware things get way more complicated - it will try very hard to produce an image that doesn't _look_ wrong. Even if it goes wrong, it can go wrong and still succeed in managing to make an image that looks correct, it just no longer matches the real brain that was scanned. Perhaps it decided a tumor was just a smudge on the lense, and invented some brain to go behind it. An operator expecting to see brain and seeing brain wont think anything of it. When the patient dies, they may look back and say "wow, that tumor didn't exist at all just 3 days before! that should be impossible!". tldr: Having an ai that might make mistakes is one thing, having an ai that can just invent exactly the data everyone is expecting to see is dangerous.

VikingCoder 5y ago

That's a valid risk. You're asking a cost-benefit question. The cost of an invalid diagnosis is indeed high. The cost of no diagnosis at all is also high. This device will not replace the MR at your local hospital. It will be the first MR device in hospitals that have never had one before.

lostlogin 5y ago
Veterinary MRI may be another application.
- VikingCoder 5y ago
  Good point. Cargo inspection? Luggage inspection? I dunno.
  - lostlogin 5y ago
    There are a lot of applications that are surprising. I’ve scanned for salmon farmers (is that the term?) who want to check they are breeding good fish and are looking at spine alignment. I’ve scanned logs for forestry managers who want to look at something in their trees. I’ve scanned old hearts that have been sitting in formalin for decades. All are probably better at higher field strength but maybe some of that can be compensated for by scanning for longer? A log isn’t going to move, and a dead fish scan is likely only limited by the time it takes for it to rot. I’d be scared of scanning unknown things, it might be a low field MRI, but it’s still a big magnet.

axg11 5y ago

Whether you’re comfortable with it or not - it’s already happened and in production. Look up Subtle Medical and GE AIR Recon.

lostlogin 5y ago
Literally just purchased this - Installed tomorrow: https://www.siemens-healthineers.com/magnetic-resonance-imaging/technologies-and-innovations/deep-resolve https://www.siemens-healthineers.com/magnetic-resonance-imag...

mattkrause 5y ago

Amen! I have seen an alarming number of talks where someone proposes to algorithmically add Gado contrast or turn a T1 into a T2 image. In a few very specific contexts, this makes sense (e.g., aligning a T1 taken in one session with a T2 taken in another). Otherwise though, it seems dangerous to mistake a "real" image with the expected image given another one.

lostlogin 5y ago
If reducing gadolinium dose is the aim, a more prompt following of the literature, radiologist request (rather than surgeon demand) and weight based dosing would drastically reduce dosage. A moaning radiographer, what a surprise!

fluidcruft 5y ago

Deep learning reconstructions are already marketed/sold in high-end commercial scanners.