8 ms·
FastMRI leverages adversarial training to remove image artifacts
- lokimedes 7y agoCould be useful for SAR and SAS imagery as well perhaps.
- throwlaplace 7y agoIsn't this basically SRGAN? Edit: sorry I guess since there's an explicit rotation module it's closer to SRGAN+deformable convolutions.
- zn473 7y agoThe adversary in this work never sees non-reconstructed images, so it looks like it's completely unrelated to SRGAN.
- mustachionut 7y agoEven without anything fancy, is there a speed vs clarity parameter(s) when doing an MRI? It seems an easy improvement would be to spend more time getting a clear picture of the specific area of interest, vs now where the whole scan seems to be done at full clarity.
- throwaway4220 7y agoYes, definitely true for many artifacts! Although due to Nyquist, ghosting artifacts sometimes require you to increase the field of view. What bothers me here is when the artifacts hide underlying pathology, and these algorithms "learn" what a normal knee mri looks like and just show you that. IMO it is a medical liability that must be addressed.
- viraptor 7y agoYeah, I'm worried how any automatic correction which is not completely specified can be used in medical imaging. We sometimes fail to even compress images correctly (remember the scanners changing numbers due to compression?), so trying to automatically remove artefacts sounds dangerous. We already teach doctors about the artefacts and how to handle them. The image doesn't need to be pretty - just functional.
- lostlogin 7y agoThis is mostly handled by MR techs and it is their job to sort this out. Many of the automated tasks are pretty good, and those that aren’t get rejected fast. We don’t tend to get a new sequence/tool/parameter and just run with it, it’s used with the old one until a degree of trust and understanding is established. I’m an MR tech shirking off.
- throwaway4220 7y agoYeah, I would trust an MR tech's tried and tested parameters way before trusting any fancy algorithm or even a new sequence.
- zn473 7y agoYes, although currently scans are typically done at "full clarity" following a "standard" clinical protocol that is the same for everyone. It's generally thought that in the future the field will move towards using scans are are more tailored to each particular patient.
- lostlogin 7y agoAgreed. However the cost of getting to the scanner, getting on and off it, administration and reporting etc needs to be factored in. If you can remove a potential patient recall or repeat scan by doing an extra 3 minute sequence, it’s worth doing.
- ska 7y agoWorse, there is a whole family of parameters. It's worth thinking of an MRI as a programmable machine for doing certain types of physics experiments. Sometimes you have an area of interest, sometimes you don't. A lot of the practical (i.e. clinical level, not research work) on specific areas of interest is still in coil design, since body coils often don't do well. There are all sorts of things that make it difficult (e.g. imaging is in frequency domain, localizing things with gradients can be time consuming in ways not entirely directly related to clarity, etc.) This sort of thing is addressing issues that come up with acceleration techniques that rely on redundancy in the sampled space to "cheat" and not capture everything. The obvious concern with a ML approach here is that it may replace something interesting with something more normal. I'd hate to be the one tasked with V&V for this, honestly.
- lostlogin 7y ago> It seems an easy improvement would be to spend more time getting a clear picture of the specific area of interest, vs now where the whole scan seems to be done at full clarity. This is exactly what is done already. Every method of one can name for reducing scan times is used, and some we can’t name are used too. Speed nearly always comes at the expense of quality, although some acceleration techniques and tech developments have lead to improvements that are pretty much without time penalty. These include signal digitisation at the coil and other methods of getting more for for less (note that this equation doesn’t include money!).
- est31 7y agoI'm no fan of this. What if it treats a tumor as an artifact? This reminds me of the xerox scandal about broken OCR that erroneously deduplicated parts of images that had different contents. This module might work well, but the modules by cheap competitors might have such behaviour, and it's extremely hard to test that an implementation is bug free.
- yboris 7y agoWhat if doctors get both, the untouched originals and the images with the artifacts removed? Seems like it solves the problem you're concerned with?
- est31 7y agoThe xerox scanners had a setting to disable compression as well. People are lazy and don't enable the compressions. Although they are highly skilled, radiologists don't have time to inspect each image, so why bother looking at the raw originals? The question is rather: does this feature improve diagnoses? Sure, the images look nicer now. But that's not why they are being created. MRI images are made for inspection by trained radiologists who are already filtering out artifacts. So is this tool better at this job, or does it actually worsen the ability of the radiologists to read the images like those xerox scans? Maybe I'm a bit paranoid, idk. After all, diffusion MRI is already being used for surgical planning even though it has several shortcomings. But in that instance there are probably no good alternatives, while here the alternative is the trained eye of a radiologist.
- ska 7y agoIt gets even worse than that sometimes. For example, I remember a study from back when digital xray was getting going, where radiologist were asked to say which processing they liked better (since none of them looked quite like the very non-linear film versions) and scored on performance. They didn't perform best on the types they liked best. This wasn't a great study in terms of power, but it was interesting. I've met plenty of rad-oncs and radiologists who are convinced they can "read through the noise" just fine, and want consistent imaging more than artifact reduction. I'm not sure how empirically this has ever been tested.
- Gatsky 7y agoIs anyone else concerned that facebook has an interest in MRI?
- deepnotderp 7y agoThis is a bad idea, neural nets upscale by "hallucinating" in the details. That's fine for videos for entertainment, not for medical imaging. And this is distinctly different from compressed sensing which uses a high frequency and mathematical basis.
- zn473 7y agoI went to a medical imaging workshop recently, and the consensus was that deep learning approaches will completely replace classical compressed sensing. They are using the same principles of acquiring randomized samples, so it's still compressed sensing, they just produce dramatically better results than classical CS techniques.
- deepnotderp 7y ago> They are using the same principles of acquiring randomized samples, so it's still compressed sensing, See the second part of my comment. This is only in principle. In practice compressed sensing uses a higher frequency basis and more importantly, this basis is generally not learned, preventing common case bias. Ie a rare condition won't be ignored because it isn't statistically common enough for the NN model to learn.
- lvs 7y agoNo thanks. If it can remove artifacts, it can also introduce them. Nobody should be using this on patients. This is a straightforward misapplication of AI.
- voicedYoda 7y agoFacebook has absolutely no reason to be doing work with healthcare. Sure they have great computing power and top engineering talent to figure out how to sell more ads, but the trade-off for any educational facility to freely hand over medical data (de-identified or not) is wreckless.
- nradov 7y agoWhat exactly is the concern with de-identified medical data? This is common practice in medical research and explicitly allowed under federal law.
- pmiller2 7y agoLet's start with how it's very difficult to properly de-identify medical data: https://www.careersinfosecurity.com/patient-data-be-truly-de-identified-for-research-a-12708 https://www.careersinfosecurity.com/patient-data-be-truly-de...
- mikeortman 7y agoWhat till you find out about the Google / Ascension partnership. I trust seasoned talent being paid hundreds of thousands of dollars a year in partnership with equally well paid healthcare professionals over PhD students scraping by on grant dollars, keeping their code and datasets in a private GitHub repo that will never see the light of day except for a citation in research papers of other scholars. Not trying to be mean, but if Facebook is trying to fix their moral compass with dollars, go for it.
- 2ndwind 7y agoDoes anyone know how this differs from what Subtle Medical is doing? https://subtlemedical.com/ https://subtlemedical.com/
- mikeortman 7y agoPlease don’t make sweeping, generalizing opinions on the implications of the work. It’s a subjective problem to solve, so if are not a radiologist who has first-hand experience with this issue, stop. Here are the results from the paper: The radiologists ranked our adversarial approach as better than the standard and dithering approaches with an aver- age rank of 2.83 out of a possible 3. This result is statisti- cally significantly better than either alternative with p-values 1.09 × 10−11 and 2.18 × 10−11 respectively, and the adver- sarial approach was ranked as the best or tied for best in 85.8% of 120 total evaluations (95% CI: 0.78-0.91). The dithering approach is also statistically significantly better than the standard approach. We also asked radiologists if banding was present (in any form) in the reconstructions in each case. This evaluation is highly subjective, as “banding” is hard to define in a pre- cise enough way to ensure consistency between evaluators. Considering each radiologist’s evaluation independently, on average banding is still reported to be present in 72.5% (95% CI: 0.62-0.82) of cases even with the adversarial learn- ing penalty. The radiologists were not consistent in their rankings; the overall percentages reported by the six radiol- ogists were 20%, 75%, 75%, 80%, 85%, and 100% for the adversarial reconstructions. In contrast, for the baseline and dithered reconstructions, only one radiologist reported less than 100% presence of banding for each method (80% and 85% presence respectively, from different radiologists). We believe these numbers could be improved if more tuning went into the model; however, it’s also possible that features of the sub-sampled reconstructions generally may be con- fused with banding, and so any method using sub-sampling might be considered by radiologists as having banding. Sub- sampled reconstructions generally have cleaner regional boundaries and lower noise levels than the corresponding ground-truth.
- p1necone 7y agoIntuitively I don't see that there's much value in asking radiologists to subjectively "rank" the images. Surely the thing that needs to be tested here is patient outcomes?
- vanderZwan 7y agoThat needs to be tested eventually - there's a reason we go from petridish testing to animal testing to human testing with medicine, it stands to reason that medical tools should follow similar stages.
- ebg13 7y agoA lot of people here are rightly concerned about the dangers of falsely marking something as an artifact, but let me present additional data that will hopefully sway you a little bit... If you need an MRI or a CT of an area adjacent to orthopedic implants, you are currently 100% SOL because distortion or reflection artifacts from the metal completely destroy the imagery across a medically significant distance. There are computational filtering techniques for reducing these artifacts, but, respectfully, they are still really terrible, and close to the implants you can't see shit. All advancements in this area short of inventing new imaging physics will most likely be purely computational corrections. Consider that.
- vardump 7y agoI think I'd prefer radiologists use both computationally filtered and this. Computational filtering has also advanced over the years.
- ebg13 7y agoThis _is_ computational filtering. It's not philosophically any different. Every filtering method algorithmically guesses what's important or what's real and what's not.
- Reelin 7y agoI agree with your earlier point, but disputing the usage of the term computational filtering here is truly pedantic. Yes, by definition machine learning approaches are a subset of computational approaches, but there are clear differences in terms of (at least) failure modes between machine learning and other techniques. In context, "non-machine-learning based filtering methods" is what was being referred to. Importantly, the internals of non-machine-learning based approaches are more readily understandable and their output is much more predictable.
- creato 7y ago> It's not philosophically any different. I disagree. I think using techniques that work by attempting to model physical processes that we understand are philosophically different from ML approaches that are learning arbitrary functions.
- DataDrivenMD 7y agoA physician's $0.02 - The clinical relevance of FB's work is clearly stated in the blog post: "While state-of-the-art facilities today use 3 Tesla MRI machines, scanners with lower-strength magnets (1.5 Tesla, for example) are still commonly used around the world." Considering that a 1.5T MRI machine costs about $1M less than a comparable 3T model (+/- the cost of warranty, support, and installation), FB's work in this area has the potential to make a BIG positive impact on the lives of millions of patients. Which is why I will be cheering them on. If they reproduce their results in other clinical settings, the immediate impact on patient care includes: 1) accelerating diagnosis (and treatment) for patients with traumatic brain injuries (by effectively up-scaling lower resolution scans) 2) healthcare providers in developing countries will effectively get a low-cost "upgrade" to their existing equipment 3) cancer patients in rural America could be monitored for treatment response in a setting that is closer to home (because rural communities tend to be resource-poor in terms of medical technology). If we consider that a logical extension of their work could be to develop a compression algorithm for MRI data, then it's easy to see an even broader impact that includes: 1) connecting rural patients with high-quality radiologist services (i.e. remote MRI interpretations), and 2) decrease the cost of long-term storage, access, and retrieval for MRI data. On the topic of FB's issues with privacy: I agree that FB has a long way to earn my trust as a doctor and a patient. That being said, it's important to give credit where credit is due. It seems that FB gained access to the imaging data by working collaboratively with NYU on this specific project. By comparison, it's an open secret among those of us in the biomedical informatics community that over the course of many years Google Cloud has quietly gained access to the personal health information of millions of Americans. So, when it comes to privacy concerns, it's important to avoid being myopic - the concern is valid, but the primary threat may not be as obvious as it first seems.
- orr721 7y ago> 2) healthcare providers in developing countries will effectively get a low-cost "upgrade" to their existing equipment I am VERY pessimistic about this. I don't know how well you know medical equipment providers but this will never be sold as a low-cost "upgrade" to existing machines. It will be sold with new equipment only and with a hefty surcharge as an option enabling higher patient throughput. There is no real money in upgrades. Most equipment lasts only 8-10 years anyway.
- heyitsguay 7y agoI just gave a talk on similar work being done in microscopy: https://leapmanlab.github.io/nihai/jan20/ https://leapmanlab.github.io/nihai/jan20/ . The tl;dr (in microscopy but apparently also in mri) is AI imaging can evidently enable new concrete solutions to intractable imaging problems, but the failure modes are really treacherous. The example on slide 39, taken from another excellent review paper, does a great job illustrating the problem. I think these methods will get more trustworthy, but i wouldn't stake my life (or my paper's prestigious research results) on them at the moment.