7 ms·

Machine Learning for MRI Image Reconstruction

79 points by raffihotter 5y ago

olliej 5y ago
Things like MRIs are the last thing you want to be using ML to invent detail in. This proposal basically says using ML we can quarter the number of frequencies we sample and still get good looking scans. But the full resolution is made by inventing details based on statistics from a biased input (most MRIs are taken due to something being wrong). Again, as with super resolution, ML cannot add detail that isn’t there, anything it creates is simply based on the statistical model it formed from the training set.
- davidhyde 5y ago
  Playing devil's advocate here but can't machine learning be used to remove noise rather than add detail? Removing noise would reveal detail hidden in the data kind of like the result you get after applying a spectral filter to a fourier transformed image. For example: https://www.youtube.com/watch?v=s2K1JfNR7Sc https://www.youtube.com/watch?v=s2K1JfNR7Sc
  - halpert 5y ago
    "Removing noise" is equivalent to adding detail.
  - olliej 5y ago
    The problem would be blurring or denoising meaningful information, but I don't know enough to say they don't do any denoising. I can imagine the data being noisy, but perhaps it isn't? shrug :D
    - notarandomer 5y ago
      Yes, accelerating MRI acquisition increases noise in the images as well as introducing aliasing artifacts. I think the issue is that some modern reconstruction methods (e.g. compressed sensing that was mentioned in the article) produce predictable biases, e.g. adding a risk of smoothing out details, but for ML we don't always know in what way it will bias the reconstructed image (add details, remove important information...), and I think that is what people often worry about.
  - aix1 5y ago
    Personally, I am not even sure that "ML vs non-ML" is a useful dichotomy. If method A can be rigorously demonstrated[*] to produce superior results to method B, does it even matter whether mathematically it is constructed out of fast Fourier transforms, Metz filters, layers of convolutions or whatever else? [*] For example, by measuring the quality of reconstruction of a known image (e.g. a real or digital phantom) or, in the ideal world, by evaluating clinical outcomes.
- deleted 5y ago
  [deleted]
tqi 5y ago
> This makes it hard to predict when and how deep learning methods will fail (there are no theoretical guarantees that deep learning will work). I actually think we know fairly well how deep learning methods work (and what the shortcomings are), we just have no way to interpret the models it produces. Wouldn't ML techniques to reduce scan times fail at the most critical moments, ie when patients had unusual or unexpected ailments? Using ML in on downsampled MRI images feels akin to having an artist with a lot of familiarity of human anatomy touch up a scan.
- csee 5y ago
  I speculate (and hope) that ML diagnostics will help give medical care access to really poor people in really poor countries. There's not enough cheap doctors to help all of them, and if ML can speed things up and reduce marginal costs to zero, even if it degrades quality of care, a lot of lives could be saved. 80% ML + 20% human is better than no medical care at all.
  - jstx1 5y ago
    Those countries are lacking basic healthcare standards and infrastructure. I doubt that lack of diagnosis due to understaffing is the bottleneck.
    - csee 5y ago
      Countries with a GDP per capita of $5,000-$10,000 typically do have good medical care in private, but most of the population is excluded because of cost. If we give the doctors ML tools to increase bandwidth, then that should help the situation by increasing supply. Suppose we could 10x the bandwidth for routine scans. The cost should go down in private, public health capacity will go up, and overall more people should be able to access it.
      - exdsq 5y ago
        The issue is training people to use the machines and keeping them running, not even the doctors themselves.
01acheru 5y ago
Can’t wait to do an MRI and hear the doc say “You’re all set, good to go!”, only to discover that I actually had a tumor but that really clever ML algorithm thought that it was noise and should’ve been smoothed out… I don’t want to be part of it, thanks
- ryan93 5y ago
  Why would the ML algorithm necessarily change the scan. The radiologist could still look at the unadulterated MRI.
  - lostlogin 5y ago
    MRI is completely adulterated at every stage. Algorithms and filters make the final result palatable. The raw data is a k-space data file. It’s not really human readable (though you can spot noise spikes etc).
  - 01acheru 5y ago
    There is no unadulterated result, you are doing less sampling and relying on ML to fill the gaps. So you either have the ML reconstructed result or a subsampled MRI. Healthcare is an area where we need good and clean data as much as possible, let’s use ML reconstruction somewhere else.
    - ryan93 5y ago
      Interesting. Would love to see an example of a tumor so small a radiologist could see it but that a ML algorithm would smooth out
      - 01acheru 5y ago
        It is not about big or small, I don’t think you understand how ML work. And by the way, tumors can be really small.
        exdsq 5y ago
vitorsr 5y ago
> Though compressed sensing can improve the image quality relative to a vanilla inverse Fourier transform, it still suffers from artifacts. Odd remark. FDA approves compressed sensing products (e.g., [1], [2], [3], [4]) precisely because it is possible (and provably so) to quantify and/or characterize such “artifacts” up to substantial equivalence. [1] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K162722 https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn... [2] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K163312 https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn... [3] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K173079 https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn... [4] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K173617 https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn...
- bick_nyers 5y ago
  This is actually quite interesting and currently relevant to me, thanks for sharing
  - vitorsr 5y ago
    The following query might also interest you then: "510(k)" "deep learning" site:accessdata.fda.gov/cdrh_docs Or alternatively replace "deep learning" by either another distinctive theory/methodology employed or trade/device name. I do remind you to cross-check what material product is being reviewed.
nik282000 5y ago
Need training data? Offer scans at half price in America if they agree to hand over their images!
londons_explore 5y ago
Could this be taken one step further... Use the ML in-the-loop during an MRI scan, to look at the data collected so far, then decide which frequency should be measured next to most improve the quality of the result? This can also all be simulated offline without an MRI machine to test on with just access to a few full scans... So could be a good weekend project for someone here on HN, and your technique might even be in use by the time you need an MRI scan and will mean your doctor can get results slightly quicker and you get better healthcare, together with hundreds of millions of other people!

79 points by raffihotter 5y ago

Things like MRIs are the last thing you want to be using ML to invent detail in. This proposal basically says using ML we can quarter the number of frequencies we sample and still get good looking scans. But the full resolution is made by inventing details based on statistics from a biased input (most MRIs are taken due to something being wrong). Again, as with super resolution, ML cannot add detail that isn’t there, anything it creates is simply based on the statistical model it formed from the training set.

davidhyde 5y ago
Playing devil's advocate here but can't machine learning be used to remove noise rather than add detail? Removing noise would reveal detail hidden in the data kind of like the result you get after applying a spectral filter to a fourier transformed image. For example: https://www.youtube.com/watch?v=s2K1JfNR7Sc https://www.youtube.com/watch?v=s2K1JfNR7Sc
- halpert 5y ago
  "Removing noise" is equivalent to adding detail.
- olliej 5y ago
  The problem would be blurring or denoising meaningful information, but I don't know enough to say they don't do any denoising. I can imagine the data being noisy, but perhaps it isn't? shrug :D
  - notarandomer 5y ago
    Yes, accelerating MRI acquisition increases noise in the images as well as introducing aliasing artifacts. I think the issue is that some modern reconstruction methods (e.g. compressed sensing that was mentioned in the article) produce predictable biases, e.g. adding a risk of smoothing out details, but for ML we don't always know in what way it will bias the reconstructed image (add details, remove important information...), and I think that is what people often worry about.
- aix1 5y ago
  Personally, I am not even sure that "ML vs non-ML" is a useful dichotomy. If method A can be rigorously demonstrated[*] to produce superior results to method B, does it even matter whether mathematically it is constructed out of fast Fourier transforms, Metz filters, layers of convolutions or whatever else? [*] For example, by measuring the quality of reconstruction of a known image (e.g. a real or digital phantom) or, in the ideal world, by evaluating clinical outcomes.
deleted 5y ago
[deleted]

tqi 5y ago

> This makes it hard to predict when and how deep learning methods will fail (there are no theoretical guarantees that deep learning will work). I actually think we know fairly well how deep learning methods work (and what the shortcomings are), we just have no way to interpret the models it produces. Wouldn't ML techniques to reduce scan times fail at the most critical moments, ie when patients had unusual or unexpected ailments? Using ML in on downsampled MRI images feels akin to having an artist with a lot of familiarity of human anatomy touch up a scan.

csee 5y ago
I speculate (and hope) that ML diagnostics will help give medical care access to really poor people in really poor countries. There's not enough cheap doctors to help all of them, and if ML can speed things up and reduce marginal costs to zero, even if it degrades quality of care, a lot of lives could be saved. 80% ML + 20% human is better than no medical care at all.
- jstx1 5y ago
  Those countries are lacking basic healthcare standards and infrastructure. I doubt that lack of diagnosis due to understaffing is the bottleneck.
  - csee 5y ago
    Countries with a GDP per capita of $5,000-$10,000 typically do have good medical care in private, but most of the population is excluded because of cost. If we give the doctors ML tools to increase bandwidth, then that should help the situation by increasing supply. Suppose we could 10x the bandwidth for routine scans. The cost should go down in private, public health capacity will go up, and overall more people should be able to access it.
    - exdsq 5y ago
      The issue is training people to use the machines and keeping them running, not even the doctors themselves.

01acheru 5y ago

Can’t wait to do an MRI and hear the doc say “You’re all set, good to go!”, only to discover that I actually had a tumor but that really clever ML algorithm thought that it was noise and should’ve been smoothed out… I don’t want to be part of it, thanks

ryan93 5y ago
Why would the ML algorithm necessarily change the scan. The radiologist could still look at the unadulterated MRI.
- lostlogin 5y ago
  MRI is completely adulterated at every stage. Algorithms and filters make the final result palatable. The raw data is a k-space data file. It’s not really human readable (though you can spot noise spikes etc).
- 01acheru 5y ago
  There is no unadulterated result, you are doing less sampling and relying on ML to fill the gaps. So you either have the ML reconstructed result or a subsampled MRI. Healthcare is an area where we need good and clean data as much as possible, let’s use ML reconstruction somewhere else.
  - ryan93 5y ago
    Interesting. Would love to see an example of a tumor so small a radiologist could see it but that a ML algorithm would smooth out
    - 01acheru 5y ago
      It is not about big or small, I don’t think you understand how ML work. And by the way, tumors can be really small.
      - exdsq 5y ago

vitorsr 5y ago

> Though compressed sensing can improve the image quality relative to a vanilla inverse Fourier transform, it still suffers from artifacts. Odd remark. FDA approves compressed sensing products (e.g., [1], [2], [3], [4]) precisely because it is possible (and provably so) to quantify and/or characterize such “artifacts” up to substantial equivalence. [1] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K162722 https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn... [2] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K163312 https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn... [3] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K173079 https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn... [4] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K173617 https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn...

bick_nyers 5y ago
This is actually quite interesting and currently relevant to me, thanks for sharing
- vitorsr 5y ago
  The following query might also interest you then: "510(k)" "deep learning" site:accessdata.fda.gov/cdrh_docs Or alternatively replace "deep learning" by either another distinctive theory/methodology employed or trade/device name. I do remind you to cross-check what material product is being reviewed.

nik282000 5y ago

Need training data? Offer scans at half price in America if they agree to hand over their images!

londons_explore 5y ago

Could this be taken one step further... Use the ML in-the-loop during an MRI scan, to look at the data collected so far, then decide which frequency should be measured next to most improve the quality of the result? This can also all be simulated offline without an MRI machine to test on with just access to a few full scans... So could be a good weekend project for someone here on HN, and your technique might even be in use by the time you need an MRI scan and will mean your doctor can get results slightly quicker and you get better healthcare, together with hundreds of millions of other people!