3 ms·
Last week, a tweet went viral of a Claude user fighting with radiologists claiming the LLM found a tumor when the radiologists did not (most of the replies were
by minimaxir 3y ago
Last week, a tweet went viral of a Claude user fighting with radiologists claiming the LLM found a tumor when the radiologists did not (most of the replies were rightfully dunking on it):
> A friend sent me MRI brain scan results and I put it through Claude.
> No other AI would provide a diagnosis, Claude did.
> Claude found an aggressive tumour.
> The radiologist report came back clean.
> I annoyed the radiologists until they re-checked. They did so with 3 radiologists and their own AI. Came back clean, so looks like Claude was wrong.
> But looks how convincing Claude sounds! We're still early...
https://twitter.com/misha_saul/status/1771019329737462232 https://twitter.com/misha_saul/status/1771019329737462232
- binarymax 3y agoThis is super dangerous. It's WebMD all over again but much worse. It's hard enough to diagnose but now you have to fight against some phantom model that the patient thinks is smarter than their doctor.
- latentsea 3y agoWebLLMD
- mateo1 3y agoAfter reading the diagnosis I caught myself wanting to examine the MRI to see if the bright area really exists, which means I fell for this too. Imagine being the person who received this diagnosis. Of course you're going to be concerned, even if it is LLM garbage.
- y04nn 3y agoThere is probably a disproportionate amount of texts describing the presence of a tumor on a scan than texts describing what a normal scan would look like. So this is expected. But I think that is not the purpose of a LLM. There should be a classification algorithm detecting the presence or not of a tumor and then a a LLM can be used to generate a description of it or why the scan is normal. And I don't know any, but there is probably already tools to help radiologists, what they call their "own AI".