5 ms·

I have talked at length with several people in the FastMRI project and in my opinion it is actually very dangerous. There is no feasible way to validate that su

I have talked at length with several people in the FastMRI project and in my opinion it is actually very dangerous. There is no feasible way to validate that such a model will not hallucinate normal tissue in the presence of a rare abnormality. The argument often used is that advanced reconstruction techniques such as compressed sensing have not required validating against very rare abnormalities, however when deep neural networks get involved you have a nearly universal function approximator whose behavior is nowhere near as bounded. The FDA will probably set the low bar of validating that the reconstruction algorithm fares well in the face of a few abnormalities and call it a day, instead of the proper (admittedly infeasible) validation of testing against all abnormalities that the reconstruction algorithm will encounter. Facebook will probably happily jump over this low bar, and patients will get hurt.

dzhiurgis 8y ago
From what I understand there are tons of MRI applications. If you are looking for something exotic of course you will try to get a human, probably best in the field. But if you are trying to bring this to masses one can start with with something "simple" like bones or some some relatively trivial organ. Not that MRI prices themselves are dropping much - gotta wait for room temperature superconductors.
skwb 8y ago
To be honest I am highly skeptical that FB would be able to have the sort of institutional capacity to launch any sort of medical device health product. It takes a non trivial amount of effort and time to fully productize it (in terms of system integration/ product testing/ FDA submission). Unless there is involvement with one of the major MRI manufacturers, I don't foresee this going far. Many more players in the game that actually have a decent shot of getting something like this to market.
- duado 8y ago
  The thing is that because the highest-ROI application of machine learning is online advertising, unquestionably the two deepest pools of machine learning talent are at Google and Facebook. So this may be able to overcome the obvious culture mismatch between "move fast and break things" and the FDA.
  - skwb 8y ago
    Google has a long long history of failing in the medical space. They tend to overengineer their products in a way that makes sense if you’re a computer engineer, but not if you understand anything about the healthcare system. Facebook has no legitimate history developing these products either. It’s more involved to develop health products than it is to sell ads. And when it comes to deep learning, optimizing your ml for 2D rbg is not the same beast as volumetric 3D data. And besides, let’s say tomorrow they have a method to do it tomorrow: how do they prospectively scam patients? How do they deploy the algorithm in a clinical setting? For any of this as a product to work, it would have to be integrated into a mri controller. Unless I’m eating my words at RSNA this year and deep mind is presenting their work, I’ll remain highly skeptical that this is going anywhere beyond a PR story that’s been sold to the media.
throwawaw434234 8y ago
Deepmind has projects related to 'formal verification' and conservative bounding, presumably for these very reasons.
tntn 8y ago
I don't have strong feelings about this, but a few thoughts come to mind regarding exploring accelerated MRIs via neural networks. 1. If an MRI can be done 10x faster with the same results except in exceptionally rare cases, might that not still be a win? Order of magnitude reduction in time may translate to substantial reduction in cost and increase opportunity for applications. It seems like it is worth considering whether these benefits might be worth compromising the accuracy of the imaging. 2. How accurate are radiologists at diagnosing / detecting these rare abnormalities that validation might miss? If radiologists are actually pretty mediocre at this, might it be OK to make the scan slightly imperfect if the next stage (the human) is already very imperfect?
- dontreact 8y ago
  1. Maybe, but that should be a tradeoff that is made consciously with some analysis and care, instead of just jumping over a low FDA bar, which is what everyone in this space seems to be doing. I think there are enough abnormalities that can be fatal which a net would just fill in (e.g. aortic dissection?) 2. There are thousands of different abnormalities. From what I understand about the FDA validation process for this sort of thing there would be only 10s of abnormalities. There are likely, many, many of them that are quite obvious to radiologists. And once again, this would be a question that should be studied carefully when people's lives are at risk, instead of just assuming that it will be fine then going ahead to "move fast and break things"
  - bartimus 8y ago
    It's my understanding it's sometimes exactly those obvious abnormalities that are missed by radiologists.
    - nitrogen 8y ago
      Adding another lossy step will only make even more anomalies get missed, or worse, misdiagnosed as something else (I'm thinking about the photocopiers that sometimes change numbers in documents). The argument that "our system is already bad so we should just merge this new bad component because it doesn't make it worse" is bad in software, and unacceptable in medicine.
LeanderK 8y ago
I never understood these arguments. Isn't it like claiming self-driving cars via neural networks would never be possible because you can't test that the neural network would take the correct decision in every situation? I view the whole issue stochastically, with the immediate aim being to make (significantly) fewer errors than the current approach, which is having a human decide. I don't claim that I can design an experiment which could selve as an indication whether we are improving upon human judgments, but I think this should be the goal. Reflecting upon my view, i think it comes from the experience of training ml-algorithms. You are always minimizing errors, but you goal is almost never to make 0 errors, because often your data is noisy and you are probably overfitting. I know the medical enviroments are more sensitive, but I can't really wrap my head around how we could design a learning algorithms that does not make any error and works on all abnormalies. I think it will always missclassify. Rephrasing my argument: I think the approval should be given if an significant expected improvement over the distribution of real-life abnormalies can be detected and not over the uniform-dsitribution over all abnormalies. EDIT: detecting out-of-distribution samples is hard and I don't think this is a solution and leads to a false sense of security.
- PavlikPaja 8y ago
  It isn't. It obviously is possible to drive a car. The problem here is you're trying to reconstruct image from less data than what is fundamentally possible. Compressed sensing is already as low as you can get, unless you prove it is possible to reconstruct the image accurately from even less data, then it's probably pointless to try use neural networks to decode it, and it's actually a bad idea anyway. The problem with neural networks is that they can reconstruct something that looks "normal", not necessarily something that is accurate. The more abnormal the scan, the more likely it will get reconstructed as something that looks perfectly fine even when it's not.
  - LeanderK 8y ago
    I really don't know anything about MRI and their usage, i was mainly criticising that his arguments sounded like a lot of the arguments I've heard against other usages of ML-Algorithms. What I meant: There's probably a medical reason why you want such a product and if a more readily available MRI saves (really significantly) more lives than the chance that it might miss some abnormalies which could lead to death, then I think we should allow it. That's what I meant with a stochastic view. If we, for example, only have a few scans per hospital available because the chance might exist that we missclassify something and lots of people get worse or delayed treatment because they are not high enough on the priority list to get access to a super-resulution MRI with a fidelity they don't really need (again, i don't know anything, just to illustrate my point), then I think something is wrong. His argument just sounded dismissive without giving a, to my uninformed point of view, valid reason.
dekhn 8y ago
nobody expects these systems to be perfect, which is why they have less validation against extremely rarely observed problems- the total impact is small enough that we don't prevent good systems from being approved. Rather than saying "very dangerous", it would be safe to say that some individuals woudl not be well-served, but ideally, the overall health of the population would increase for a reasonable expenditure.

dzhiurgis 8y ago

From what I understand there are tons of MRI applications. If you are looking for something exotic of course you will try to get a human, probably best in the field. But if you are trying to bring this to masses one can start with with something "simple" like bones or some some relatively trivial organ. Not that MRI prices themselves are dropping much - gotta wait for room temperature superconductors.

skwb 8y ago

To be honest I am highly skeptical that FB would be able to have the sort of institutional capacity to launch any sort of medical device health product. It takes a non trivial amount of effort and time to fully productize it (in terms of system integration/ product testing/ FDA submission). Unless there is involvement with one of the major MRI manufacturers, I don't foresee this going far. Many more players in the game that actually have a decent shot of getting something like this to market.

duado 8y ago
The thing is that because the highest-ROI application of machine learning is online advertising, unquestionably the two deepest pools of machine learning talent are at Google and Facebook. So this may be able to overcome the obvious culture mismatch between "move fast and break things" and the FDA.
- skwb 8y ago
  Google has a long long history of failing in the medical space. They tend to overengineer their products in a way that makes sense if you’re a computer engineer, but not if you understand anything about the healthcare system. Facebook has no legitimate history developing these products either. It’s more involved to develop health products than it is to sell ads. And when it comes to deep learning, optimizing your ml for 2D rbg is not the same beast as volumetric 3D data. And besides, let’s say tomorrow they have a method to do it tomorrow: how do they prospectively scam patients? How do they deploy the algorithm in a clinical setting? For any of this as a product to work, it would have to be integrated into a mri controller. Unless I’m eating my words at RSNA this year and deep mind is presenting their work, I’ll remain highly skeptical that this is going anywhere beyond a PR story that’s been sold to the media.

throwawaw434234 8y ago

Deepmind has projects related to 'formal verification' and conservative bounding, presumably for these very reasons.

tntn 8y ago

I don't have strong feelings about this, but a few thoughts come to mind regarding exploring accelerated MRIs via neural networks. 1. If an MRI can be done 10x faster with the same results except in exceptionally rare cases, might that not still be a win? Order of magnitude reduction in time may translate to substantial reduction in cost and increase opportunity for applications. It seems like it is worth considering whether these benefits might be worth compromising the accuracy of the imaging. 2. How accurate are radiologists at diagnosing / detecting these rare abnormalities that validation might miss? If radiologists are actually pretty mediocre at this, might it be OK to make the scan slightly imperfect if the next stage (the human) is already very imperfect?

dontreact 8y ago
1. Maybe, but that should be a tradeoff that is made consciously with some analysis and care, instead of just jumping over a low FDA bar, which is what everyone in this space seems to be doing. I think there are enough abnormalities that can be fatal which a net would just fill in (e.g. aortic dissection?) 2. There are thousands of different abnormalities. From what I understand about the FDA validation process for this sort of thing there would be only 10s of abnormalities. There are likely, many, many of them that are quite obvious to radiologists. And once again, this would be a question that should be studied carefully when people's lives are at risk, instead of just assuming that it will be fine then going ahead to "move fast and break things"
- bartimus 8y ago
  It's my understanding it's sometimes exactly those obvious abnormalities that are missed by radiologists.
  - nitrogen 8y ago
    Adding another lossy step will only make even more anomalies get missed, or worse, misdiagnosed as something else (I'm thinking about the photocopiers that sometimes change numbers in documents). The argument that "our system is already bad so we should just merge this new bad component because it doesn't make it worse" is bad in software, and unacceptable in medicine.

LeanderK 8y ago

I never understood these arguments. Isn't it like claiming self-driving cars via neural networks would never be possible because you can't test that the neural network would take the correct decision in every situation? I view the whole issue stochastically, with the immediate aim being to make (significantly) fewer errors than the current approach, which is having a human decide. I don't claim that I can design an experiment which could selve as an indication whether we are improving upon human judgments, but I think this should be the goal. Reflecting upon my view, i think it comes from the experience of training ml-algorithms. You are always minimizing errors, but you goal is almost never to make 0 errors, because often your data is noisy and you are probably overfitting. I know the medical enviroments are more sensitive, but I can't really wrap my head around how we could design a learning algorithms that does not make any error and works on all abnormalies. I think it will always missclassify. Rephrasing my argument: I think the approval should be given if an significant expected improvement over the distribution of real-life abnormalies can be detected and not over the uniform-dsitribution over all abnormalies. EDIT: detecting out-of-distribution samples is hard and I don't think this is a solution and leads to a false sense of security.

PavlikPaja 8y ago
It isn't. It obviously is possible to drive a car. The problem here is you're trying to reconstruct image from less data than what is fundamentally possible. Compressed sensing is already as low as you can get, unless you prove it is possible to reconstruct the image accurately from even less data, then it's probably pointless to try use neural networks to decode it, and it's actually a bad idea anyway. The problem with neural networks is that they can reconstruct something that looks "normal", not necessarily something that is accurate. The more abnormal the scan, the more likely it will get reconstructed as something that looks perfectly fine even when it's not.
- LeanderK 8y ago
  I really don't know anything about MRI and their usage, i was mainly criticising that his arguments sounded like a lot of the arguments I've heard against other usages of ML-Algorithms. What I meant: There's probably a medical reason why you want such a product and if a more readily available MRI saves (really significantly) more lives than the chance that it might miss some abnormalies which could lead to death, then I think we should allow it. That's what I meant with a stochastic view. If we, for example, only have a few scans per hospital available because the chance might exist that we missclassify something and lots of people get worse or delayed treatment because they are not high enough on the priority list to get access to a super-resulution MRI with a fidelity they don't really need (again, i don't know anything, just to illustrate my point), then I think something is wrong. His argument just sounded dismissive without giving a, to my uninformed point of view, valid reason.

dekhn 8y ago

nobody expects these systems to be perfect, which is why they have less validation against extremely rarely observed problems- the total impact is small enough that we don't prevent good systems from being approved. Rather than saying "very dangerous", it would be safe to say that some individuals woudl not be well-served, but ideally, the overall health of the population would increase for a reasonable expenditure.