4 ms·

As a radiologist I have found Claude and ChatGPT to be absolutely terrible at MRI and I would not trust it one bit. It has its merits if you need to research st

As a radiologist I have found Claude and ChatGPT to be absolutely terrible at MRI and I would not trust it one bit. It has its merits if you need to research stuff that is more text based, but radiological images is just something that they cannot interpret good enough (yet)

lostlogin 3mo ago
AI makes up for its poor reporting by enhancing the images. Current Siemens MR software ‘Deep Resolve’ makes up the signal (adding about 50%), then makes up every second pixel, and then, for 3D sequences, makes up every second slice. It’s locking about 59% of the time off each sequences. And it’s really really good. I’m an MR tech.
- microgpt 3mo ago
  Sorry? You use AI to hallucinate medical images and that's good?
  - uecker 3mo ago
    It is not really the same as LLMs. I wouldn't call it AI. And I wouldn't say "makes up". I work in this field and this is certainly based also in part on my research.
    - lostlogin 3mo ago
      ‘Makes up’ is inaccurate for sure. But it’s not strictly true to call it acquired data either. After years of collecting artifacts and errors, I have more and more respect for the tool. But it’s jarring. I open a sequence, decrease the acquired resolution, add the AI and get a scan that’s quicker and higher resolution. It’s an amazing time to be an MR tech.
      - uecker 3mo ago
        It is amazing. It is the result of two decades of research in image reconstruction algorithms. The machine learning is part of it, but that it is sold as "AI" has probably more to do with marketing.
        lostlogin 3mo ago
        It certainly has a lot of marketing behind it. https://marketing.webassets.siemens-healthineers.com/2861d15b73d6b450/717822286611/MR-Deep-Resolve-Basics-Infographic-USA-2022-HOOD05162003301480.PDF https://marketing.webassets.siemens-healthineers.com/2861d15...
        fluidcruft 3mo ago
        I haven't seen it marketed as "AI" by GE, Siemens or Philips. They usually gesture at "deep learning" or "compressed sensing". No radiologist is buying "AI" scanners. Radiologists are probably among the most jaded of an audience about the word "AI" due to decades of undelivered promises. AI is synonymous with "worthless trash" to them, not to mention everyone says "AI" is going to put them out of work. lol
    - microgpt 3mo ago
      Super-resolution is certainly distinct from hallucinating - it just rearranged data that was already there to make it easier for the human eye to see - but should be used with care. I can easily imagine that an upscaling algorithm makes it so a certain defect is clearly not present, when the source image is ambiguous (which the radiologist would have noticed), and in reality the defect is present.
      - shiandow 3mo ago
        I would definitely be wary using the more advanced super resolution schemes. It took some work preventing it from drawing faces everywhere. MRI is already a form of compressed sensing, I would much prefer statistical forms of super resolution to ones based on training data. Even if it is only trained on MRIs it will see some noise and plausibly expand it into whatever disease fits.
  - gavinray 3mo ago
    It's just DLSS/Frame Generation for MRI's.
  - sota_pop 3mo ago
    Most upscaling and super-resolution techniques I’ve seen use various implementations of interpolation; typically nearest-neighbor approaches. Although I don’t work in the medical field and haven’t checked in on the research at least since ViTs overtook CNNs for other areas of computer vision.
- rasmus1610 3mo ago
  but those are two different things. Of course something like Deep Resolve is great, as are modern model based reconstruction algorithms for CTs, but here we are talking about LLMs and their ability to interpret medical images, which has nothing to do with what you said.
- throwawayffffas 3mo ago
  Sure but claude and ChatGPT are not Siemens 'Deep resolve'.
pickleRick243 3mo ago
It's like people who expect ChatGPT to be really good at chess because chess engines with super-human performance have been around for decades, so obviously the latest frontier LLM that took billions to train should find the task trivial. Actually, I'm curious what ChatGPT 5.5's ELO is- I wouldn't be too surprised if it's 2000+ just from its basic understanding of chess principles from all the content it has digested.
- nicksergeant 3mo ago
  Interestingly LLMs are extremely bad at chess position _images_. I have to imagine if you give it positions in text it'd be pretty great but when I was learning chess and pasting images of positions in for analysis I couldn't believe how wrong it was. I actually thought it was looking at the board in reverse but even when pointing out problems it seemed completely incapable of understanding what it was missing (of course... it doesn't really "understand" anything). LLMs truly are marvels with text but anything spatial seems to really mess it up, somehow.
  - unholiness 3mo ago
    > I have to imagine if you give it positions in text it'd be pretty great Not at all? LLMs are a terrible match for the kind of analysis a chess engine does (scaled deep search, deeply trained position evaluations). It's just not that kind of tool.
    - nicksergeant 3mo ago
      I suppose that's also a good point!
- simonreiff 3mo ago
  ChatGPT is completely unplayable at chess on its own. It's unable to keep track of the state of the chess position and therefore will make an illegal move within about 10-12 moves. I would put GPT-5.5's rating at 400, since it can't even make legal moves reliably. I've tried to pay chess with GPT-5.5, even played it again tonight, allowing it to use `python-chess` to keep track of the state of the position and to get a list of legal moves at each turn, so that it was fair. I also gave it blindfold odds, again to make it a fair fight, but it was not even close. GPT still isn't better than maybe 1000 Elo, maybe 1200 tops. Even with what amounts to being able to see the position and also being unable to make an illegal move, GPT-5.5 hangs material left and right, doesn't make a plan, and got smoked even when I gave it blindfold odds, to the point it's boring for me to play even under those conditions. I'm not sure it's better than whatever the GPT model was that was out about 8 months ago. I also thought it might be somewhat better than a beginner due to reading chess books, but no, it's complete garbage at playing chess, not even average-level skill.

lostlogin 3mo ago

AI makes up for its poor reporting by enhancing the images. Current Siemens MR software ‘Deep Resolve’ makes up the signal (adding about 50%), then makes up every second pixel, and then, for 3D sequences, makes up every second slice. It’s locking about 59% of the time off each sequences. And it’s really really good. I’m an MR tech.

microgpt 3mo ago
Sorry? You use AI to hallucinate medical images and that's good?
- uecker 3mo ago
  It is not really the same as LLMs. I wouldn't call it AI. And I wouldn't say "makes up". I work in this field and this is certainly based also in part on my research.
  - lostlogin 3mo ago
    ‘Makes up’ is inaccurate for sure. But it’s not strictly true to call it acquired data either. After years of collecting artifacts and errors, I have more and more respect for the tool. But it’s jarring. I open a sequence, decrease the acquired resolution, add the AI and get a scan that’s quicker and higher resolution. It’s an amazing time to be an MR tech.
    - uecker 3mo ago
      It is amazing. It is the result of two decades of research in image reconstruction algorithms. The machine learning is part of it, but that it is sold as "AI" has probably more to do with marketing.
      - lostlogin 3mo ago
        It certainly has a lot of marketing behind it. https://marketing.webassets.siemens-healthineers.com/2861d15b73d6b450/717822286611/MR-Deep-Resolve-Basics-Infographic-USA-2022-HOOD05162003301480.PDF https://marketing.webassets.siemens-healthineers.com/2861d15...
      - fluidcruft 3mo ago
        I haven't seen it marketed as "AI" by GE, Siemens or Philips. They usually gesture at "deep learning" or "compressed sensing". No radiologist is buying "AI" scanners. Radiologists are probably among the most jaded of an audience about the word "AI" due to decades of undelivered promises. AI is synonymous with "worthless trash" to them, not to mention everyone says "AI" is going to put them out of work. lol
  - microgpt 3mo ago
    Super-resolution is certainly distinct from hallucinating - it just rearranged data that was already there to make it easier for the human eye to see - but should be used with care. I can easily imagine that an upscaling algorithm makes it so a certain defect is clearly not present, when the source image is ambiguous (which the radiologist would have noticed), and in reality the defect is present.
    - shiandow 3mo ago
      I would definitely be wary using the more advanced super resolution schemes. It took some work preventing it from drawing faces everywhere. MRI is already a form of compressed sensing, I would much prefer statistical forms of super resolution to ones based on training data. Even if it is only trained on MRIs it will see some noise and plausibly expand it into whatever disease fits.
- gavinray 3mo ago
  It's just DLSS/Frame Generation for MRI's.
- sota_pop 3mo ago
  Most upscaling and super-resolution techniques I’ve seen use various implementations of interpolation; typically nearest-neighbor approaches. Although I don’t work in the medical field and haven’t checked in on the research at least since ViTs overtook CNNs for other areas of computer vision.
rasmus1610 3mo ago
but those are two different things. Of course something like Deep Resolve is great, as are modern model based reconstruction algorithms for CTs, but here we are talking about LLMs and their ability to interpret medical images, which has nothing to do with what you said.
throwawayffffas 3mo ago
Sure but claude and ChatGPT are not Siemens 'Deep resolve'.

pickleRick243 3mo ago

It's like people who expect ChatGPT to be really good at chess because chess engines with super-human performance have been around for decades, so obviously the latest frontier LLM that took billions to train should find the task trivial. Actually, I'm curious what ChatGPT 5.5's ELO is- I wouldn't be too surprised if it's 2000+ just from its basic understanding of chess principles from all the content it has digested.

nicksergeant 3mo ago
Interestingly LLMs are extremely bad at chess position _images_. I have to imagine if you give it positions in text it'd be pretty great but when I was learning chess and pasting images of positions in for analysis I couldn't believe how wrong it was. I actually thought it was looking at the board in reverse but even when pointing out problems it seemed completely incapable of understanding what it was missing (of course... it doesn't really "understand" anything). LLMs truly are marvels with text but anything spatial seems to really mess it up, somehow.
- unholiness 3mo ago
  > I have to imagine if you give it positions in text it'd be pretty great Not at all? LLMs are a terrible match for the kind of analysis a chess engine does (scaled deep search, deeply trained position evaluations). It's just not that kind of tool.
  - nicksergeant 3mo ago
    I suppose that's also a good point!
simonreiff 3mo ago
ChatGPT is completely unplayable at chess on its own. It's unable to keep track of the state of the chess position and therefore will make an illegal move within about 10-12 moves. I would put GPT-5.5's rating at 400, since it can't even make legal moves reliably. I've tried to pay chess with GPT-5.5, even played it again tonight, allowing it to use `python-chess` to keep track of the state of the position and to get a list of legal moves at each turn, so that it was fair. I also gave it blindfold odds, again to make it a fair fight, but it was not even close. GPT still isn't better than maybe 1000 Elo, maybe 1200 tops. Even with what amounts to being able to see the position and also being unable to make an illegal move, GPT-5.5 hangs material left and right, doesn't make a plan, and got smoked even when I gave it blindfold odds, to the point it's boring for me to play even under those conditions. I'm not sure it's better than whatever the GPT model was that was out about 8 months ago. I also thought it might be somewhat better than a beginner due to reading chess books, but no, it's complete garbage at playing chess, not even average-level skill.