28 ms·

“AI promised to revolutionize radiology but so far its failing”

• newyankee 5y ago
  The reason a clean alternative to radiologists in the form of AI is not available because of the inertia of the medical system. Due to its innate conservative nature, a beta testing in a third country with successful results will only be the pathway for it to be adopted by richer countries with stricter medical systems. I feel AI in medicine is a boon for developing countries if used properly. Especially diagnostics.
  • sambe 5y ago
    As I said above, it seems fairly clear if you go to the original article that there is a real problem with the existing systems adapting poorly to different setups - they are trained on one system/hospital and then don't generalise well.
  • dailybagel 5y ago
    Why should medical "beta testing" happen in a third-world country? Is there some reason the higher risk of an experimental procedure is more acceptable there than (say) Boston or Dallas?
    • gbear605 5y ago
      Unfortunately doctors are a lot less prevalent in a lot of developing countries, especially in subsaharan Africa. If an AI can do a third of the things that a doctor can do, that means that many more people can be treated. In the US, it just means that the appointments can be cheaper. So the developing countries have a lot more to gain from things like AI. The US and other developed countries should be doing more to help the situation, including training and paying doctors to work in those countries, but AI can potentially save a lot of lives there in the meantime. Of course, AI can only save lives if it works reliably, which doesn’t seems to be the case yet, but that hopefully can be overcome.
      • querulous 5y ago
        you think doctors are scarce in subsaharan africa but mri machines and xrays and ultrasounds are plentiful?
        • Google234 5y ago
          It’s must easier and faster to buy a machine than train a doctor
          • nradov 5y ago
            It really isn't. Some countries can produce a trained physician for less than the cost of a new MRI machine. And beyond the capital expense those machines are expensive to operate due to technicians, maintenance, consumable supplies, power, etc.
          • Fomite 5y ago
            This is in contradiction to my experience working on medicine in Africa, where there are often very well trained people coping with a water system that doesn't always work.
        • gbear605 5y ago
          I know that they aren’t, but that doesn’t mean that AI that doesn’t use those tools is impossible.
    • newyankee 5y ago
      AI assisted virtual Doctor > No Doctor
    • tomp 5y ago
      90% reliable doctor in Switzerland > 10% reliable AI > 0% reliable no doctor in Africa
  • qayxc 5y ago
    > Especially diagnostics. I don't think so. Maybe I'm just too old, but I remember vividly that the same was said about expert systems back in the late 90s and early 2000s. 20 years later and no one is even considering expert systems for automated diagnosis anymore. The problem with current machine learning models is their blackbox character. You cannot query the system for why a diagnosis was made and verify it's "reasoning". Tests rely on using the systems as oracles instead and in medical diagnosis, a patient's medical history is just as important as the latest lab results. No amount of ML (in its current form) will be able to manage to interview patients accordingly. It might work as a tool for assisting professionals, but it's nowhere near in a state that warrant's its use for automated diagnosing of patients.
    • Der_Einzige 5y ago
      Wtf - many ML models today are either full on white boxes or are directly interpretable in various ways (e.g. LIME algorithm). Even neural networks have good interpretability tools (e.g. captum). ML is not the black box nightmare that I see it described as on here. You can figure out feature contributions and can quite easily (and accurately) verify it's reasoning. If you really need these kind of models, look into various kinds of tree based ML models like random forests or boosted trees...
      • qayxc 5y ago
        > look into various kinds of tree based ML models like random forests or boosted trees... Those are the expert systems that have fallen out of favour over a decade ago, so thanks, but no thanks. > many ML models today are either full on white boxes or are directly interpretable in various ways Sources please, and remember to use relevant sources only, i.e. interpretation of medical image analysis like here [1]. Notably activation maps told researchers precisely nothing about what the neural net was actually basing its conclusions on: > For other predictions, such as SBP and BMI, the attention masks were non-specific, such as uniform ‘attention’ or highlighting the circular border of the image, suggesting that the signals for those predictions may be distributed more diffusely throughout the image. So much for "fully white boxes" and "direct interpretability"... [1] https://storage.googleapis.com/pub-tools-public-publication-data/pdf/a79654ba4743b72dda1d64045518be1bc127a9dd.pdf https://storage.googleapis.com/pub-tools-public-publication-...
• carbocation 5y ago
  I think that the role of radiologists in the medical system is misunderstood. Radiologists are consultants. Yes, in some cases - many cases, even - you just want a result to an answer to a specific, common question from an imaging study. And in those cases, I am sure that deep learning-based readings will do a fine job. But for more diffuse inquiries, or for times when there is disagreement or uncertainty over a reading, radiologists are wonderful colleagues to engage in discussion. I'm not super interested in predicting employment trends, but it's hard to imagine a world where the radiologist-as-consultant disappears.
  • newyankee 5y ago
    I really respect the intensity of training all medical practitioners have and the responsibility society puts on them. However i think there is an urgent need of reforming medical systems to leverage all the new trends in a responsible manner. Augmenting the capabilities of Doctors is one way, but better frictionless anonymised data sharing can also be very useful. However the only thing that prevents a success of new approach other than incumbents is that it is difficult to determine winners & losers of new approaches and it is likely that larger players have better chances of being more successful instead of many smaller players.
    • potatoman22 5y ago
      That, and trying to standardize a data format for something as complex as a medical record
  • gowld 5y ago
    > Yes, in some cases - many cases, even - you just want a result to an answer to a specific, common question from an imaging study And these questions are already outsourced to India.
  • koheripbal 5y ago
    It's an oversimplification to say AI will replace xyz job. It seems more likely that AI will simply sift through the data more thuroughly and look holistically to catch things a radiologist might miss. A radiologist, for example, might miss spotting a small tumor in an xray taking for an unrelated hip injury. AI has a lot of complimentary value.
• new299 5y ago
  There was some interesting work recently published in nature on augmenting therapy selection: https://www.nature.com/articles/s41591-021-01359-w https://www.nature.com/articles/s41591-021-01359-w “Overall, 89% of ML-generated RT plans were considered clinically acceptable and 72% were selected over human-generated RT plans in head-to-head comparisons.” This seems like it could be a way forward. Where AI is used to propose alternative and improve patient outcomes.
  • boleary-gl 5y ago
    That's the way it is used today - for instance in mammography there is Computer aided detection (CAD): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1665219/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1665219/ That's been in use for some time. But like many parts of radiology it really only can be a second look tool that as you mentioned proposes alternatives or suggests things. The false positive rate for CAD is substantially higher than for humans because of the human ability to see symmetry and patterns in very diverse tissue sets like one sees in screening mammography. And the nature of screening tests like mammography means that actually percentages like "89%" isn't really good enough. You have to be more specific and sensitive than that to have a successful program, and I'm not sure that ML will ever be able to get there...there's a lot of experience and human intuition involved at some point that would be hard to replicate...and I know that because people have been trying to do that for decades.
    • ska 5y ago
      The comparisons are pretty tricky to do right, especially with systems that have been trained with the assumption that they are operating as "a second check". For what it's worth, that language was popularized by the first such system approved to market by FDA, in the mid-late 90s. It had, amongst other things, a NN stage. Even at that time, such systems were better than some radiologists at most tasks, and most radiologists at some tasks - but breadth was the problem, as was generalization over a lot of different set ups. I think this is more a data problem than an algorithmic one. With something as narrow as screening mammo CAD (very different than diagnostic), it's quite plausible that it could become a more effective "1st pass" tool than humans on average, but to get there would involve unprecedented data sharing and access (that 1st system was trained on a few thousand 2d sets, nothing like enough to capture sample variability)
• Inhibit 5y ago
  Am I missing something or does this look like poor uptake for (possibly) reasons other than performance? The article cites a lack of use as justification for the assumption that the model doesn't work. It might just take time. Especially in medical. A century (+) ago hand-washing took time to adopt unless I'm mis-remembering.
  • sambe 5y ago
    Both that and "people refuse to use technology that makes them redundant" indeed seem to be hinted at. However, one of the quotes says that poor performance is the reason and if you click through to the original article, it seems fairly clear that there is a problem with the existing systems adapting poorly to different setups.
    • kspacewalk2 5y ago
      It's not about "poor performance", in the same way as poor IDE performance isn't really the root cause of my laptop's stubborn inability to write good Python code. Radiology is about using a generalist medical education to diagnose (or be instrumental in diagnosing) patients. Pattern matching or statistical information are a rather modest subset of that skillset.
  • username_my1 5y ago
    but the article is right in saying that AI advocates said no need for any new radiologists. I know this in the field of vehicle damages, AI is good but not good enough to take over... and that has been the case for a long while yet every now and then a new company / product comes saying that the future is AI and no need for human effort.
    • Inhibit 5y ago
      True. I was internalizing "no new need" as hyperbole because it didn't make sense (given the reality of medical) but that's my mistake. On re-reading, a highly specialized and entrenched workforce having a 30% uptake on a new technology in only a few years seems phenomenal.
• throwthere 5y ago
  The original article referred to by the blog post is here from last week-- https://qz.com/2016153/ai-promised-to-revolutionize-radiology-but-so-far-its-failing/ https://qz.com/2016153/ai-promised-to-revolutionize-radiolog... . The conclusion is that AI will revolutionize radiology. It's just that nobody knows when. And it's not like there's some socioeconomic or whatever barrier preventing AI from being used (as an aside, there are barriers of course)-- it's simply that AI isn't good enough yet. It's not a surprise to anyone who relies on radiologists and has reviewed the current AI state of the art. Yes, with machine X on patients meeting criteria Y, you can rule out specific disease Z. But the algorithms don't generalize very well. It's like declaring you'll have self-driving cars in 5 years because you can drive one straight on a highway in sunny Arizona that only occasionally causes fatal crashes.
• gowld 5y ago
  Are ignored research papers a source of startup ideas? Should entrepeneurs build products based on this research, to sell or get aquired by incumbents?
• jofer 5y ago
  There are many parallels with seismic interpretation here. Many companies/etc keep promising to "revolutionize" interpretation and remove the need for the "tedious" work of a geologist/geophysicist. This is very appealing to management for a wide variety of reasons, so it gets a lot of funding. What folks miss is that an interpreter _isn't_ just drawing lines / picking reflectors. That's less than 1% of the time spent, if you're doing it right. Instead, the interpreter's role is to incorporate all of the information from _outside_ the image. E.g. "sure we see this here, but it can't be X because we see Y in this other area", or "there must be a fault in this unimaged area because we see a fold 10 km away". By definition, you're in a data poor environment. The features you're interested in are almost never what's clearly imaged -- instead, you're predicting what's in that unimaged, "mushy" area over there through fundamental laws of physics like conservation of mass and understanding of the larger regional context. Those are deeply difficult to incorporate in machine learning in practice. Put a different way, the role is not to come up with a reasonable realization from an image or detect feature X in an image. It's to outline the entire space of physically valid solutions and, most importantly, reject the non-physically valid solutions.
  • ludamad 5y ago
    I wonder how often these projects truly need someone with on the ground experience guiding it, as the textbook tasks as you say are easy for even the humans
  • boleary-gl 5y ago
    This is a really good point and example. I spent 10 years in mammography software, and I saw first hand how many outside factors can impact a physician's decision to biopsy or not a given artifact on an image. Things like family history, patient's history, cycle timing, age, weight, other risk factors all play a role in a smart radiologist making the right decision for that patient. And the pattern recognition on top of that is really hard - it's not just about the pattern you see at a particular spot in an image, it's the whole image in the context of what that looks like. Could ML get better over time with this? Sure...but they've been using CAD in mammography for decades and it still hasn't replaced radiologists at all. Could a model be made to include those over variables? Sure...but again the complexity of that kind of decision making is something that requires a lot more "intelligence" than any AI or ML system exhibits today and in my mind in the foreseeable future. Just collecting that data in a structured, consistent way is more challenging than people realize.
• paxys 5y ago
  Has AI revolutionized anything other than driving up user engagement/addiction on shitty websites?
  • mustafa_pasi 5y ago
    Speech and language transcription and translation has gone a very long way. Still not perfect but almost at human level in some instances.
  • The_rationalist 5y ago
    Asking such a question is only proof of your own ignorance. I invite you to discover the state of the art and its scopes https://paperswithcode.com/sota https://paperswithcode.com/sota
    • Der_Einzige 5y ago
      While this comment is a good start, we should remember that for some scores, SOTA is only loosely correlated with improvements in downstream performance. This is true in things like summarization with ROUGE scores (which suck and everyone hates them)
  • pantulis 5y ago
    Have you ever heard about Alexa?
    • kilnr 5y ago
      I don't know, the NSA has been capable of that level of mass surveillance for a long time before Amazon was.
• ChicagoBoy11 5y ago
  The metric of radiology jobs as a sign of the lack of AI revolution in the field seems poor to me. Sadly, much of our medical infrastructure (and the jobs it creates) only have a very tenuous relationship to the actual care and the quality that it delivers. Rather, most of the infrastructure tries to optimize for the billing and the legislation that surrounds it. One of the ways to immediately see this is for us, a technical crowd, to puzzle on why it seems that Moore's law doesn't seem to affect medical technology... AT ALL. Some of the same procedures using the same machines from decades ago today cost A LOT more than they used to, for instance. This isn't to say that this AI revolution in radiology hasn't been underwhelming; I just think that using this job metric is a poor indicator of the technology's capability.
  • azalemeth 5y ago
    The role of the radiologist isn't just mapping image[range_x, range_y, range_z] to disease -- it's also including a vast amount of Bayesian priors on the rest of the patient's notes, and their referring colleagues' hints of an indication. For example, often the question isn't just "does this person have mitral valve regurgitation yes/no", it's more along the lines of "is there evidence from this cardiac MRI scan that their mitral valve regurgitation is significant and able to explain the symptoms that we have -- and if so, is it amenable to treatment". That's a totally different question -- are the symptoms beyond what would be expected for the patient; and is there a plausible mechanism are all second-level radiological questions, well beyond the level of "please classify this stack of images into either healthy, cyst, or tumour". Another random example would be the little old lady who comes in with breathlessness at rest: she may well have a large, slow-growing lung cancer (that any AI algorithm would easily diagnose) that she may well die with or of, but the acute dysponea could be down to an opportunistic LRTI that remains treatable with a course of antibiotics (and visible on a plain film chest x-ray). Capturing that sort of information is a lot, lot harder. You're also forgetting that the cost of an expensive imaging modality like MRI or CT is amortised over 10 years, and that -- by far -- the biggest cost of running the service is the staff. The doctors do more than push buttons. In many services, actually, they don't acquire the scans or interact with patients at all.
    • 5y ago
• PaulHoule 5y ago
  "Expert Systems" that could diagnose and treat disease were technically successful in the 1970, see https://en.wikipedia.org/wiki/Mycin https://en.wikipedia.org/wiki/Mycin This technology never made it to market because of various barriers; at that time you didn't have computer terminals in a hospital or medical practice. Docs want to keep their feeling of autonomy despite much medical knowledge being rote memorization and rule-based. The vanguard of medicine is "patient centered" and tries to feed back statistics to help in decisions like "what pill do I prescribe this patient for high blood pressure?" -- the kind of 'reasoning with uncertainty' that an A.I. can do better than you. As for radiology the problem is that images are limited in what they can resolve. Tumors can hide in the twisty passages of the abdomen and imaging by MRI is frequently inconclusive in common sorts of pain such as back pain, knee pain, shoulder pain, neck pain and ass pain.
  • jquaint 5y ago
    > The vanguard of medicine is "patient centered" and tries to feed back statistics to help in decisions like "what pill do I prescribe this patient for high blood pressure?" -- the kind of 'reasoning with uncertainty' that an A.I. can do better than you. I think this illustrates why AI in medicine is a hard problem. I'm not actually sure this is a clear cut AI/Statistics problem. Mainly because "what pill do I prescribe this patient for high blood pressure?" has lots of hidden questions. AI solves "what pill will statistically leads to a higher survival rate", but that is not the only consideration. Often doctors have to balance side effects and other treatments. What is easier for the patent: A lifestyle change to reduce blood pressure? or Enduring the side effects of the pill? This type of question is quite difficult for our AI's to answer at the moment Most drugs have side effects that are hard to objectively measure the impact of.
    • nradov 5y ago
      There are also coverage rules to consider. Payers often require providers to try less expensive treatments first and will only authorize more expensive pills if the patient doesn't respond well.
  • deleted 5y ago
    [deleted]
• apercu 5y ago
  The biggest reason corporations want AI to succeed is so they no longer have to share even a meagre percentage of revenue with you any longer.
  • gpm 5y ago
    The biggest reason humans want AI to succeed however is so that they can stop spending their time serving other humans in repetitive, mundane, and boring work. Wealth distribution is a problem that can be solved better or worse with or without AI. Better with AI looks like things along the lines of low working hours with good pay, UBI, or so on, things we simply can't afford without automating the work. Worse without AI looks like things like slavery, something we don't even have incentive to resort to if we do automate the work. Let's not confuse our current political issues with how we distribute wealth with issues with AI. They're issues with our institutions, some of those issues are exacerbated by better technology, but they can and should be solved.
    • apercu 5y ago
      Do you not feel it naive to assume that those that currently hold the vast majority of wealth and power won't continue to do so and exploit the rest of us? I mean, they will own all the AI's, right?
      • gpm 5y ago
        Yes and no. I don't think it's likely that inequality will go away, a minority will likely continue to hold the vast majority of wealth. But, as technology gets better the incentive to exploit the bottom end of the wealth and power spectrum goes down, technology does the job better than manual labor. Moreover people don't actually like to see other people suffer as a rule of thumb, so given the absence of other incentives the QOL for the bottom end of the spectrum likely goes up. UBI is a much bigger ask when it's 30% of the GDP than 3% (made up numbers). Especially given that we have reasonably functional democracies in many countries today, I think it's pretty likely that the social problem works out well. It's also not like I expect AIs/useful programs to be a rare resource, the best one might be, but like all computer programs they're infinitely clonable, there are only artificial sources of scarcity, and scarcity of (more or less fungible) computational power. This isn't a setup that lends itself well to the hording of resources. Moreover maintaining the technology will probably consume some humans in frankly not that exciting jobs like "maintenance programming"... and those humans probably won't come from the high end up the wealth/power spectrum, but will be high leverage. That points to a society that is likely to have reasonably good class mobility. The cards are stacked in favor of democracy working reasonably well.
• bagswatchesus 5y ago
  I am sure that deep learning-based readings will do a fine job. But for more diffuse inquiries, or for times when there is disagreement or uncertainty over a reading https://www.bagsshoesfr.com/hermes/accessories/blankets.html https://www.bagsshoesfr.com/hermes/accessories/blankets.html
• scythe 5y ago
  Medical physics student here. I work for a hospital that pays $silly per annum to use a type of expensive treatment planning software for radiation oncology. The software comes with a built-in automatic contouring based on "AI". One of our units covered contouring and the role of the medical physicist vis a vis contouring, which is generally to act as a double check layer behind the radiologist. We received about an hour of instruction on how to contour. After that, the instructor and the class unanimously agreed that every single student had learned to beat the software at recognizing the parotid gland. And not by a small margin. Why is it so bad? Security is a big reason. The software that can be installed on hospital computers is tightly controlled. Our research group is currently hamstrung by IT after they got mad at us for using PowerShell scripts to rename files. This was itself a workaround for the limitations of above-mentioned software. In turn, we tend to end up with a few exorbitantly priced omnibus programs rather than a lot of nice Unixy utilities that do one thing and do it well, because it lowers the IT approval overhead and the market has gone that way. Even though my personal situation is frustrating, I obviously recognize that you can't simply allow hospital faculty to install whatever executables they please in the age of ransomware. Commenters hoping for a quick fix are wrong. Almost all meme alternatives have downsides that won't be obvious at first. (But I still wish every day that Windows would go away.)
• kn1ght 5y ago
  I worked in a small research company that had a method (segmentation + CNNs, etc) a few years back. We had some exciting stuff also on masking effects, but as soon as we got into SaMD and the main revenue stream (grants) dried up, engineering closed down.
• kevinalexbrown 5y ago
  The QZ article is narrowly correct but widely misleading. It almost willfully ignores the momentum and direction. In reality, radiologists will not be summarily replaced one day. They will get more and more productive as tools extend their reach. This can occur even as the number of radiologists increases. Here's a recent example where Hinton was right in concept: recent AI work for lung cancer detection made radiologists perform better in an FDA 510k clearance. 20 readers reviewed all of 232 cases using both a second-reader as well as a concurrent first reader workflows. Following the read according to both workflows, five expert radiologists reviewed all consolidated marks. The reference standard was based on reader majority (three out of five) followed by expert adjudication, as needed. As a result of the study’s truthing process, 143 cases were identified as including at least one true nodule and 89 with no true nodules. All endpoints of the analyses were satisfactorily met. These analyses demonstrated that all readers showed a significant improvement for the detection of pulmonary nodules (solid, part-solid and ground glass) with both reading workflows. https://www.accessdata.fda.gov/cdrh_docs/pdf20/K203258.pdf https://www.accessdata.fda.gov/cdrh_docs/pdf20/K203258.pdf (I am proud to have worked with others on versions of the above, but do not speak for them or the approval, etc) The AI revolution in medicine is here. That is not in dispute by most clinicians in training now, nor, from all signs, by the FDA. Not everyone is making use of it yet, and not all of it is perfect (as with radiologists - just try to get a clean training set). But the idea that machine learning/ai is overpromising is like criticizing Steve Jobs in 2008 for overpromising the iphone by saying it hasn't totally changed your life yet. Ok.
  • robk 5y ago
    There are indeed areas where it's being used to complement radiologists as a second review and reduce the recall rate https://www.kheironmed.com/news/press-release-new-results-show-miatm-kheirons-breast-screening-ai-could-help-solve-the-breast-cancer-screening-workforce-crisis-and-covid-backlog https://www.kheironmed.com/news/press-release-new-results-sh...
    • ramraj07 5y ago
      This is how it needs to be approached. AI systems and rule based systems that work together with the clinicians to enhance their decision making ability instead of replacing them.
• btilly 5y ago
  I believe that the problem is psychological. Many years ago, back in the 1990s, wavelet based algorithms were able to outperform humans on detecting tumors. The thing is that the algorithms were better on the easy parts of the mammogram, and worse on the hard parts. So researchers thought that humans plus software should do better still, because the humans would focus on the hard parts that they did better and the software would catch the rest. Unfortunately according to a talk that I was at, it didn't work that way. It turns out that radiologists already spend most of their time on the hard parts. So they quickly dismissed the mistakes of theirs that they found as careless errors, and focused on the mistakes of the algorithm in the hard part as evidence that the algorithm didn't work well. And the result was that the radiologists were so resistant to working with the software that it never got deployed. For the same psychological reason I expect radiologists to never voluntarily adopt AI. And they will resist until we reach a point that the decision is taken out of their hands because hospitals face malpractice suits for not having used superior AI.
• mcguire 5y ago
  "What happened? The inert AI revolution in radiology is yet another example of how AI has overpromised and under delivered. . . ." Isn't this how all of the previous AI Winters started?
• screye 5y ago
  "ML promises to revolutionize 'X' because of the explosion of data in the modern era." Outside of some singularity whack-jobs, that's always been the promise. The explosion of data in the field is a necessary requirement. Healthcare fields make it nigh impossible to access data in way that will allow for fast prototyping or detailed experimentation. This isn't just about privacy either. Each Hospital treats even anonymized samples as a prospective source of income and a competitive advantage. I understand why they do it from a profit motive perspective, but it is certainly being traded off against prospective decreases in healthcare prices and significantly improved diagnostics. ML revolutionized Vision because of Imagnenet and Coco. ML revolutionized Language when Google scraped the entire internet for BERT. Graph neural networks have started working now that they're being run on internet sized knowledge graphs. Even self-driving companies know that the key to autonomous diving mecca lies in the data and not the models. (Karpathy goes into intricate detail here during his talks) If a field wishes to claim that ML has failed to revolutionize it, I would ask it to first meet the one requirement ML needs satisfied: Large-scale publically-ish available labelled data. The sad thing is that Healthcare is not incapable of providing this. It's just that the individual players do not want to cooperate to make it happen.
• sebastianvoelkl 5y ago
  I work in a startup that is building a software for radiologists that uses AI. From what I experienced so far is that the software is definitely not the problem. Our software already is better at detecting lesions, aneurisms and we are close to tumors too. Our goal is not to replace the radiologist but rather to decrease the error rate. But there is definitely a difference between training a model at home with perfectly preprocessed data and working with the raw 'real-life' data + monetizing it + making the UX/Ui for it etc.
• rdudekul 5y ago
  In my 'biased' view, AI has already revolutionized more fields than most people will 'ever' recognize. However unfounded fears and insecurities around jobs are keeping its real potential at bay. My bet is the actual impact will first be realized in poorest countries (India included) and then will spread to more advanced countries (US/G7).
• deleted 5y ago
  [deleted]
• bjornsing 5y ago
  Or, as they say: ”It is difficult to get a man to understand something when his salary depends upon his not understanding it.”
• jl2718 5y ago
  Radiologists are not even trying. They treat their methods like FDA-approved medical devices. Even basic image segmentation to help with 3D structure recognition is off-limits. The benefits of neural networks in diagnostic radiology will not occur in one shot, and I don’t think it will happen at all in the United States until people start sending their data elsewhere. But good luck getting it. I just got my CT scan from a total mis-diagnosis that resulted in an unnecessary surgery. It came on a CD that won’t read in the only drive I have access to. And even that is just images. It’s not possible to get the actual data. This is not a failure of DNN. This is active AMA hostility toward technology, and not just in radiology. Just watch, people are going to start going elsewhere for medical care. They will do everything they can to get insurance requirements, subsidies, and laws against it, but they will lose. They are a dishonest Luddite cartel, and they’re hurting people.
  • brnt 5y ago
    I left radiotherapy for this reason: they haven't the faintest idea what they are doing, and are constrained by whatever manufacturers managed to get the FDA to approve, which is 20 year old tech at best. Not that radiotherapists/logists know how to deal with modern tech... A small hint: our software repo was migrated TO svn in 2014. When people do 'AI' in radiology, they mean making little with scripts in Tensorflow. Sure, it's a beginning, but I've seen entire institutes being unable to get to grips and move past that stage. You wouldn't be able to tell from their slides, of course.
    • TeMPOraL 5y ago
      Not sure how modern AI is going to help here. > they haven't the faintest idea what they are doing, and are constrained by whatever manufacturers managed to get the FDA to approve (...) If you take this layer of proprietary magic nobody understands, and add DNNs to mix, you'll get... two layers of proprietary magic which nobody understands (and possibly owned by two different parties).
      • brnt 5y ago
        If only there were two layers of proprietary magic ;)
• ErikVandeWater 5y ago
  I don't think it's Luddites holding AI back as some comments have suggested. In the medical field, indemnity is the name of the game. An expert will always have to sign off on whatever the AI suggests.
• jonplackett 5y ago
  ai promised to revolutionize ________ but so far its failing
• MattGaiser 5y ago
  I would be happy with it being a 2nd opinion clinic. Not replacing radiologists, but "hey doc, have you considered X, Y, and Z that make the model think it is actually A instead of B?"
  • ska 5y ago
    That's traditionally how most ML has been used in radiology systems (where it is).
• helsinkiandrew 5y ago
  > Geoffrey Hinton … in 2016, “We should stop training radiologists now, it’s just completely obvious within five years deep learning is going to do better than radiologists.” >… Indeed, there is now a shortage of radiologists that is predicted to increase over the next decade. I hope those two aren’t related. It’s ok to bet your wealth on some technology or business existing or not in 5 years time, but people’s lives and health are more important
• ska 5y ago
  This discussion is a lot older than "deep" models. While statements like Hinton's quoted one are obviously silly (often the case with naive takes from deep learning maximalists) there is clearly a lot of room for more impact from algorithms, but I think it's mostly limited by data quality and access. This is not an easy problem to solve for a range of reasons: privacy, logistics, incentives.
• jp57 5y ago
  I think this article points out the fact that the creation of a technology in the lab, and its effective operational deployment are two different problems, both difficult, requiring different skills and resources. An imperfect-but-instructive analogy would be between vaccine development and vaccine delivery. Once a vaccine has been developed and shown to be safe and effective, the hard work is just beginning. In the case of COVID, billions of doses must be produced, and then delivered to people, the delivery requires not just shipping of the doses, but matching the doses with an equal number (billions) of syringes, hypodermic needles, cotton swabs, band-aids, alcohol swabs, etc. People have to be recruited to deliver the doses, systems must be created to manage the demand and the queues, etc. The operational problem of delivering the vaccine to the world is arguably harder than its creation and testing. Likewise, the successful operational rollout of an AI-mediated automated or semi-automated decisioning problem is a complex problem requiring a totally different skillset than that of ML researchers in the lab. Computer systems and human procedures have to be created to manage the automation of the decision; decision results must be tracked and errors fed back to the lab to update models. Radiologists (including new radiologists) will of course be needed to understand the errors and provide correct labels, etc. Trust and mindshare in the medical community has to be built up. These things are not easy.
• mchusma 5y ago
  These systems are basically banned from working as actual replacements for radiologists, so it's no surprise they aren't yet. We have repeatedly proved the superiority of expert systems (in the 90s) and AI and select medical tasks. However, there is a legal monopoly (in the US) that requires most medical tasks to be performed by expensive doctors. If people were able to use these tools directly, we could see dramatically better results because we would be giving people decent healthcare at basically zero cost. Cost is by far the biggest problem in healthcare today. Low cost would change behavior in a large number of medical tasks, and early detection of cancer is the most obvious. If you could get a mediocre readout for free, you would probably do so more often. Cancer in particular is almost entirely an early detection problem. Using AI to assist radiologists is probably never going to be a huge thing. Just like AI assisted truck driving is never going to be huge (because it doesn't solve the core problem).
  • ceejayoz 5y ago
    > We have repeatedly proved the superiority of expert systems (in the 90s) and AI and select medical tasks. Part of the problem here, though, is a human radiologist may look at a "is the bone broken? x-ray and go "yeah, obviously, but what's this subtle spot here?" and find an early state bone cancer or something along those lines. There's a value to that. The AI might give you the right answer, too, but miss the more subtle issue the AI isn't equipped to spot.
• tomrod 5y ago
  AI and data science dev here, in the trenches. My rule of thumb is that the overlap between jobs AI can replace and RPA can replace seems to be almost 100% of AI capabilities. The successful AI projects I've encountered usually either build something totally new or augment the existing workforce. It can be hard for us technologists to appreciate, but the inefficiencies of technology, policy, and people configurations can't always be resolved by technology alone.
  • ___luigi 5y ago
    > .. but the inefficiencies of technology .. can't always be resolved by technology alone Technology will always resolve inefficiencies of technology. Check your mobile in your pocket, and mobile in the 90's.
    • tomrod 5y ago
      I can appreciate your confusion and apologize for my lack of clarity. Note that my comment was on the interplay of the three domains, not the technology domain in a vacuum.
• nairoz 5y ago
  Surprised to read this. Having worked in the field, I see a growing interest in AI from the radiology community attested by the RSNA new AI journal. It's not about replacing radiologists but helping them in their daily work, as a safety net (double check) or as a prioritization tool.
• TaupeRanger 5y ago
  How could it? We find most things we're capable of finding. Medicine needs more treatments, cures, and prevention techniques, not more diagnosis.
• tlear 5y ago
  Few years back I contracted for an AI startup, long story short we ran a simple test of one anotator radiologist with 15years of experience to another(similar amount of experience) over 50 or so CT scans. They agreed on about 60% of the time lol and I mean easily spot table size “things” that were annotated as potentially malignant nodules or “not at all because they were “scars”. That’s when I knew we did not wtf we were doing.
  • ska 5y ago
    Reader variability is one of the many things that make this stuff a lot harder than it looks at from the outside.
    • tlear 5y ago
      My take on it was that we are not so far removed from witchdoctors when it comes to medicine then we would like to believe. Hard data on efficiency and performance of human doctors is very far from hard and their ability to diagnose things that are not blatantly obvious and are bout to kill you dead unless you get lucky and surgery works or chemo works(or does not and we have no clue why) is very limited. There is a huge amount of we want to believe.. and survivor bias going on in cancer treatment. Lot's of it is very harmful as well.
• boringg 5y ago
  Does anyone know the collective time and energy put into deep learning models versus the social benefit? I recognize that this is near impossible to calculate and the benefits will hopefully be for many years to come. It does feel like the hype around deep learning has been large though and significant progress has been not as sticky as hoped.
• mikesabbagh 5y ago
  >Many companies/etc keep promising to "revolutionize" I think the technology is already here, but society does not allow technology to fail at a similar rate as a human. Also the second question, who is to blame when it fails? Doctors have malpractice insurance. A radiologist has to sign every report(and get paid). When a tesla auto-pilot has an accident, it hits the news. This is while humanity is having thousands of accidents a day. Mammography is the most difficult radiography to interpret. Cant we start with the regular chest xrays? how about bone fractures and trauma x-rays? Those are easier, and i am sure the cost of such xray will be very low. So I think the problem is with politics and legal. Do u know that 80% of doctors visits are for simple complaints like headache, back pain or prescription refill? Do u really think AI cant solve this? It is all about the money baby
• andrewtbham 5y ago
  There seems to be a lot of startups in this space.... from a google search: https://www.medicalstartups.org/top/radiology/ https://www.medicalstartups.org/top/radiology/ I know of one local startup personally... https://www.aimetrics.com/ https://www.aimetrics.com/ Does anyone know if any of them are getting any traction?
  • rcpt 5y ago
    It's not a bad idea tbh. Currently nurse practitioners (think: nursing degree then two years of online college) are winning the right to run their own independent medical practices all over the place. You can get a Xanax prescription after your first 15min zoom call in much of the US right now. The political consensus is that doctors are overeducated and overpriced so I think an AI replacement could still win licensing even if it doesn't match their accuracy.
    • Workaccount2 5y ago
      What gets me about doctors, and maybe I'm just unlucky/haven't seen enough doctors, is that I never get that "expert" vibe from them. You know when you're talking to someone who does, say, database management. And they have been at it for 15 years, have a bunch accreditation and are well compensated for their work. You just get the impression that you can pull the most esoteric question about databases out, and they'll go on for 45 minutes about all the nuances of it. No matter how hard you try, you with a mild database understanding, would never be able to pin them. I just have never gotten that vibe from a doctor. I always felt like I was only a question or two away from them shrugging, me googling, and me finding the answer.
      • phobosanomaly 5y ago
        I think a lot of it has to do with the fact that the database guy actually implements things in an environment that can be manipulated at will. Doctors don't implement things in an environment that they control. Patients come to them with a chief complaint, and the doctor tries to resolve or manage it to the best of their ability with a minimal intervention according to a set of guidelines someone else wrote down. A doctor can't sit there and play with the diagnostic/treatment process in the same way a database guy can go play with the database software. At best the doctor can sit there with a textbook or medical journal and try to memorize more facts, or take notes, but it's not the same as pulling apart code, running it in different ways, and seeing how it behaves. Medical school is a continuous process of memorizing shit off of flash cards culled from a textbook. You don't actually build anything, or implement anything, or do anything in a real-world sense that would make you an expert in the same way as someone who was working with a system that they were able to take apart and play with and manipulate. There's no real way to develop the kind of deep knowledge you're talking about in that environment. A diesel mechanic can pull apart and engine. Hold every part in their hand. Drive a diesel-engine vehicle. Observe all the things that go wrong. Simulate, and innovate. An individual doctor can't really do any of that. Medical schools are even axing dissections, so med students are lucky if they get to see what the hell peritoneum actually looks like.
• AngeloAnolin 5y ago
  The intrinsic uniqueness of human physiology and the differing assessments made by health practitioners make this area of medicine quite challenging. This is compounded by the fact that different device manufacturers in the field of radiology each has their own proprietary technology that delivers different medical imaging analysis. While there has been a lot of headway performed in terms of data interchange, the race by the multitude of players in this area of medicine is staggering that one will always try to proclaim as more revolutionary and innovative.
• spoonjim 5y ago
  You gotta give it a minute! This isn’t like Facebook shipping their latest startup clone where they just slap something together in 10 weeks and call it a day. This will be a multi decade process.
• toolslive 5y ago
  I've done a project in this AI domain a decade ago: look at different scans and then decide if a pixel is cancer tissue or not. Project succeeded and I'm pretty sure some radiologist is enjoying his 4h work week.
• williesleg 5y ago
  Thanks H1B Visa lottery winners.
• fsloth 5y ago
  These guys - MVision - seem to promise automated segmentation using AI. Is this failing or more into the correct direction? https://www.mvision.ai/ https://www.mvision.ai/
• wheresvic4 5y ago
  It's very interesting to see this on HN because we're actively working in this space, albeit on building a training platform but the long-term goal is to generate models that can outperform the current ones that require a lot of expert input. Shameless plug: https://www.rapmed.net https://www.rapmed.net
• jtdev 5y ago
  ML/AI is such an irresistible siren song for so many… the possibilities are seemingly endless. But the “sales and marketing” people are getting over their skis selling AI/ML to the point of smothering the tech. The next AI winter is going to be long and cold…
• microdrum 5y ago
  It can be true both that AI beats radiologists AND the number of radiologist jobs in the U.S. is going up.
• justbjarne 5y ago
  The relevant question is not why radiologists aren't widely using AI software, but how accurate is the AI software relative to their human counterparts. Previous studies on the subject indicate that the accuracy of radiologists and AI software is comparable. Radiologists are among the highest paid medical specialists and have little incentive to use AI software. It would be against their own interest - their compensation would go down, and their skills would be commoditized. Never mind that if they provided diagnosis feedback to the AI software to further strengthen the ANN models it would accelerate the decline of their profession. HMOs and governments ultimately set the pay scale for this service. Some hospitals are already outsourcing radiology work to India. It's just a matter of time before AI is used more widely in the field due to cost constraints.
• throwitaway1235 5y ago
  "only about 11% of radiologists used AI for image interpretation in a clinical practice. Of those not using AI, 72% have no plans to do so" Highly intelligent people, of which radiologists would fall under, are not going to adopt technology clearly aimed at replacing them.
• mrfusion 5y ago
  Isn’t this simply a case of over regulation?
• deleted 5y ago
  [deleted]
• justbored123 5y ago
  "only about 11% of radiologists used AI for image interpretation in a clinical practice. Of those not using AI, 72% have no plans to do so while approximately" I hope the author is joking. Does he really expect the technology to be pushed forward by the people that is going to replace??? They are your main obstacle after the technical issues, don't use their adoption as a metric. It is just crazy to say to people "hey I'm going to make your profession obsolete and take your job an status in society away by implementing this new tech that is just way better that you, help me do it".
• yurlungur 5y ago
  At the end of the day it's the human's duty to provide a diagnosis. The lack of a complete product solution sure isn't helping, but even if there's a company that provides that, most people still wouldn't trust that if it's purely driven by AI. The way AI should go into these fields is to first provide tools for the specialists already in the field, to increase their productivity.
• blackvelvet 5y ago
  Radiologist here with an interest in this topic. I think the problem with most AI applications in radiology thus far is that they simply don't add enough value to the system to gain widespread use. If something truly revolutionary comes along, and it causes a clinical benefit, healthcare systems will shift to adapt this in a few years. AI just hasn't lived up to it's promise, and I agree it's because most of the people involved don't get that the job of a radiologist is way more complex than they think it is. Everytime I open a journal, I see more examples of either downright AI nonsense ('We used AI to detect COVID by the sounds of a cough') or stuff that's just cooked up in a lab somewhere for a publication ('Our algorithm can detect pathology X with an accuracy of 95%, here's our AUC'). Hyperbolic headlines - Geoff Hinton saying in 2016 that it's time to stop training radiologists springs to mind - don't help the over promise of AI, and then they shoot themselves in the foot when they underdeliver. Earlier discussions about radiologists being self interested in sabotaging AI is tinfoil hat stuff - if I had an AI algorithm in the morning that could sort out the 20 lung nodules in a scan, or tell me which MS plaque is new in a field of 40, I'd be able to report twice as many scans and make twice as much money. Companies come along every month promising their AI pixie dust is going to improve your life. It probably will, but 10 years from now, not today. The AI Rad companies are caught in an endless hype cycle of overpromising and under delivering.
  • ska 5y ago
    > self interested in sabotaging AI is tinfoil hat stuff Agree this in nonsense. Not a radiologist but have worked with many. The big barriers to AI impact in radiology are a) translation is a lot harder than people think, b) access to enough high quality data with good cohort characteristics c) good labeling (most of the interesting problems aren't really amenable to unsupervised) a d) generalization, as always. It doesn't help that for the most part medical device companies aren't good at algorithms and algorithms companies aren't good at devices, lots of rookie mistakes made on both sides.
    • blackvelvet 5y ago
      Also PACS isn't designed to implement algorithms. PACS is legacy software that is, by and large, terrible.
• chsasank 5y ago
  Let me start with declaring conflict of interest: I work in one of the aforementioned AI startups, qure.ai. Bear with my long comment. AI is starting to revolutionise radiology and imaging, just not in the ways we think. You would imagine radiologists getting replaced by some automatic algorithm and we stop training radiologists thereafter. This is not gonna happen anytime soon. Besides, there's not much to gain by doing that. If there are already trained radiologists in a hospital, it's pretty dumb to replace them with AI IMO. AI instead is revolutionising imaging in a different way. Whenever we imagine AI for radiology, you probably imagine dark room, scanners and films. I appeal you to imagine patient instead. And point of care. Imaging is one of the best diagnostics out there: non invasive and you can actually see what is happening inside the body without opening it up. Are we training enough radiologists to support this diagnostic panacea? In other words, is imaging limited by the growth of radiologists? Data does suggest lack of radiologists. Especially in the lower and medical income countries.[1] Most of the world's population lives in these countries. In these countries, hospitals can afford CT or X-Ray scanners (at least the pre-owned ones) but can't afford having a radiologist on premise. In India, there are roughly 10 radiologists per million.[2] (For comparison, US has ~ 10x more radiologists.) Are enough imaging exams being ordered by these 10 radiologists? What point is there to 'enhance' or 'replace' these 10 radiologists? So, coming to my point: AI will create new care pathways and will revolutionize imaging by allowing more scans to be ordered. And this is happening as we speak. In March 2021, WHO released guidelines saying that AI can be used as an alternative to human readers for X-Rays in the tuberculosis (TB) screening [3]. It turns out AI is both more sensitive and specific than human reader (see table 4 in [3]). Because TB is not a 'rich country disease', nobody noticed this, author included likely. Does this directive hurt radiologists? Nope, because there are none to be hurt: Most of the TB cases are in rural areas and no radiologist will travel to random nowhere village in Vietnam. This means more X-rays can be ordered, more patients treated, all without taking on the burden of training ultra-specialist for 10 years. References: 1. https://twitter.com/mattlungrenMD/status/1382355232601079811 https://twitter.com/mattlungrenMD/status/1382355232601079811 2. https://health.economictimes.indiatimes.com/news/industry/there-is-a-surfeit-of-indian-doctors-globally-but-we-have-a-shortage-of-doctors-within-india-dr-sunita-maheshwari/67349998 https://health.economictimes.indiatimes.com/news/industry/th... 3. https://apps.who.int/iris/bitstream/handle/10665/340255/9789240022676-eng.pdf https://apps.who.int/iris/bitstream/handle/10665/340255/9789...
• mpreda 5y ago
  Spelling: "it's failing" instead of "its failing", in the title no less.
• fny 5y ago
  Title should read "Peak of Inflated Expectations Reached". I've worked on many health AI/ML projects. The last decade has produced tremendously powerful prototypes which lay clear paths forward for productization. Sure, assay software that no one cares enough about hasn't been updated, but you better believe the medical apparatus as a whole will welcome any tool that increases throughput and increases margins. Automating radiology or facilitating radiology does just that. Sure radiologists might not like it, but radiologist do not operate health systems, MBAs do. For some perspective, medical devices take 3-7 years for approval. Most are not game-changing technologies. The ImageNet moment came in 2012. How can we reasonably expect to have functioning automated radiology only a decade from when we realized deep nets could classify cats and dogs?
  • austinjp 5y ago
    > radiologist do not operate health systems, MBAs do. This is highly dependent on your geographical location.
• dm319 5y ago
  I've just been in an MDT meeting that was meant to have a radiologist in it, but due to annual leave we didn't have anyone. I think people in tech don't have much of an idea of what radiologists do - the conclusion from a scan depends very much on the clinical context. In an MDT setting there is significant discussion about the relevance and importance of particular findings.
• deleted 5y ago
  [deleted]
• SneakyTornado29 5y ago
  AI isn't here to replace radiologists. It is here to augment them.
• Ensorceled 5y ago
  With digital lightboxes, 3d imaging, automated segmentation and other workflow improvements, a lot of the "low hanging fruit" has already been removed from the process. When my doctor thought I might have a stress fracture, I went for x-rays. I with my stint 10 year previous in medical imaging could tell at a glance I didn't have a stress fracture. The radiologists report was a brief "does not present with stress fracture nor any other visible issue". AI is not going to eliminate this; a radiologist is still going to need to spend 30 seconds "sign" the diagnosis, it's not going to take much out of the system. The real work in radiology is the hard cases, ones that require diagnosis and consulting with other specialists. If AI can help a orthopaedic surgeon plan a surgery for a car accident victim; then it will start replacing radiologists.
  • Ice_cream_suit 5y ago
    If: 1. I was really concerned about a stress fracture 2. and the history and examination findings strongly suggested one 3. and if it would make a difference to managing the case, then I would have escalated to MRI / nuclear scan +- SPECT fusion ( choice between the two depending on time frame amd affordability to the patient.)
• varelse 5y ago
  If you take career advice from the brainfarts of thought leaders in any other field besides the one you're intent to join, you're going to have a bad time. And even then, thought leaders rarely build, but they sure love the sound of their own impotent voices and the disproportionate influence platforms like TED provide them to virtue signal and other buzzwords the dystopic tech hivemind conjured into existence to stay relevant. Caveat Emptor...
• rscho 5y ago
  If you want to replace radiologists, then start by understanding what radiologists do. If your only answer to that is 'they describe what they see' then you'll have to think a lot harder than that.
• ipspam 5y ago
  That guy, Jovan Pulitzer, of election audit fame, claims to have patents on this. Not saying anything.... Just it's a popular field, and seems like lots of people piling in...... Without expected results....
• incrudible 5y ago
  If 95% is good enough for you, machine learning will probably get you there rather easily. With many of the really valuable use-cases, it's just not good enough. If 100% of the time you need an expert to tell if a sample falls within the 95% of successes or 5% failures, you're not adding any value. Even if you're bulk processing stuff that would've otherwise been ignored, somebody will have to deal with those signals. The net effect is more work, not less. In other words, would-be radiologists ought to stay in school.
• ziofill 5y ago
  and so is grammar
• Ice_cream_suit 5y ago
  A lot of imaging has significant, non-trivial consequences. I am sitting here waiting for a MRI of a rare and interesting soft tissue tumour to come back in the next hour or two. https://rarediseases.org/rare-diseases/tenosynovial-giant-cell-tumor/ https://rarediseases.org/rare-diseases/tenosynovial-giant-ce... It was picked up by a musculoskeletal radiologist who noticed some slightly atypical features on an ultrasound and reccomended that I send the patient for a MRI. I arranged the ultrasound since the clinical course was atypical and I was reminded of a case of PVNS (a diffuse form of this ) that I had seem a few years ago. I have arranged to have one of the leading soft tissue sarcoma experts in the country to see the person if the MRI confirms the suspicion. Hopefully, the fact that it has been detected early will minimise the likelihood of consequences like limb amputation and early funeral expenses.
• Daub 5y ago
  In my experience, a lot or radiology is closer to fortune telling than image analysis. I worked with image processing people who were researching ways to automate or semi-automate the work done in medical imaging. They quickly learned that the best way to learn from someone in the medical profession was not to sit down and talk with them, but to to follow them around all day. The ground-truth lay in the gestalt.
• mrkramer 5y ago
  This attitude will lead to the "AI Winter" again don't do it.
• gdiephuis 5y ago
  I have always maintained that seismic interpretation entails the derivation of a plausible geological model, obviously limited to what a seismic image can provide. On the other hand, insight in the local geology and knowledge about the physical properties of ambient rocks and fluids are key in narrowing down to "plausible" solutions. Similarities with interpretation of radiology seem obvious. In the last years, a fair number of seismic interpretation articles have appeared, reporting on the use of AI-like techniques, more focused on 2 & 3D pattern recognition rather than on interpretation in low signal-to-noise environments, as described below. Evidently, only success stories are reported, but it is safe to conclude that AI-like techniques definitely are an addition tool in the evaluation of the subsurface. Hence I would presume that something similar eventually will take place in the development of interpretation of radiology data. After all, geophysical techniques and physical methods for medical purposes have both benefited from each other since many decades
• tpoacher 5y ago
  AI, and tech in general has revolutionised Radiology. It just hasn't revolutionised "Radiologists". Nor was it ever intended to. Not many people deeply realise this, but Radiology is a specialisation, just like Oncology or Cardiology. A radiologist is not just someone who "interprets images on a computer": a diagnosis is sought based on the radiological findings, patient history, and other examination/investigation findings.
• exikyut 5y ago
  Nobody'll see this, but totally offtopic: see the weird random horizontal line in the margin off to the right? It's a bit of injected spam. The site seems to have been slightly hacked.