6 ms·
Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine
- rvnx 2y agoGPT4 is quite weak for medical diagnosis and radiology. Send a picture and a description of the symptoms and it will often give you bad results. When I mean bad, I mean that it cannot identify properly if the bone is human or animal, or which bone it is, and describes something completely different. The benchmarks are too optimistic in such cases.
- deleted 2y ago[deleted]
- pona-a 2y agoThat's been my experience too. Feeding it some of my X-rays, which a radiologist could interpret in a matter of minutes, had it throwing guesses at a wall, before deferring to a professional. And given the recent story about state-of-the-art vision models failing to tell if two lines intersect [0], makes me even more doubtful about VLLMs use in radiology for the forceable future. [0] https://news.ycombinator.com/item?id=40926734 https://news.ycombinator.com/item?id=40926734
- sigmoid10 2y agoOne med student and nine physicians aren't the greatest study dataset for comparing human performance, but the results are still pretty wild. GPT4-V can reliably outperform or match doctors on all but the hardest questions in a closed book setting. If you take their results at face value and apply them to modern medical care environments, it means that you are probably better off with GPT than with a real doctor if you don't have something super peculiar that would take even true experts a lot of close attention and research to figure out. It will certainly beat an uninterested or overworked clinician. I'm really looking forward to studies that look at performance comparisons in realistic environments. I believe there is a potential revolution brewing for PCPs. Having someone who actually listens and has real in-depth knowledge that goes beyond normal med students could be a game changer when compared to the usual conveyor belt care that gives everyone the same diagnosis based on a quick glance.
- rscho 2y ago> better off with GPT than with a real doctor if you don't have something super peculiar ... or if you've got anything that requires know-how/practical experience, which coincidentally is what makes most of medicine.
- sigmoid10 2y agoThat is not what this study suggests.
- rscho 2y agoThis was not tested in the study.
- sigmoid10 2y agoIt was. Look at the dataset. This is stuff that takes tons of experience to work with and it also explains why GPT was so much better than the med student. Although my original N=1 criticism here still applies.
- rscho 2y agoWe're not speaking of the same thing when we say 'know-how', then. No, it was not tested.
- sigmoid10 2y agoIf you're referring to physical examinations (and I don't know what else you could be talking about since you didn't specify it), then the point still stands. You could theoretically make an argument that the sensation of performing the examination yourself vs. just hearing a first-hand account could make a difference, and I would agree. But in a clinical setting you won't have the expert chief physician re-do the physical either, so the argument is kind of pointless.
- notRobot 2y agoI would like to share my relevant experience here. I am dealing with serious post COVID complications (Long COVID). Doctors in most places don't seem to be taking this condition seriously and believe it to be a made up thing, but COVID causing damage to bodies and brains is a very real thing. Thus, for a lot of long COVID patients, treatment has to largely be figured out by oneself, often through reading upcoming studies and experimental treatment reports from online forums. ChatGPT has been really helpful to me and other patients I know to help make sense of and get context around medication, supplements or other treatments we hear about. I'm not usually asking it what to do, but it really helps me understand what a given medication (say sulbutiamine or whatever) does and how it works and what potential interactions it has with other medicines etc. The other day it helped me figure out which amino acid complex to take, I put in the nutritional information of different products and had a long conversation with it to figure out which one is a better fit for me, and then confirmed this with external web searches. I've also used it to help figure out what bloodwork to get and what diets or exercise routines are feasible for someone in my position (with limited mobility and strength and other resources). I can confidently say that I would be much in a much worse place than where I currently am with my health if I didn't have ChatGPT to consult with. It gives me important context and information to help me make informed decisions about my health. These are conversations I'd ideally have with a doctor but at least ChatGPT fucking listens and knows some stuff. I don't want to have to rely on a damn chatbot for my health, but high quality doctors aren't accessible to me due to inequalities of this world. So while y'all work on that, I'mma do what I can to survive.
- bheadmaster 2y ago> Doctors in most places don't seem to be taking this condition seriously and believe it to be a made up thing, but COVID causing damage to bodies and brains is a very real thing. What happened with "trust the experts"?
- llamaimperative 2y agoI don’t think someone should use GPT this way but this snark isn’t productive. It was always lazy shorthand for “we don’t fully understand what’s going on but in general you’re probably better off by trusting the current scientific consensus,” which is a consistently good approach across time and space even if the consensus turns out to be wrong. And for cases where there isn’t broad consensus, like the etiology of Long COVID, then obviously “trust the experts” is an almost semantically meaningless statement anyway. This is not analogous to climate change or vaccines or other areas where you hear this (again, lazy) phrase a lot.
- mjburgess 2y agoReminder once again that "accuracy" is irrelevant in the real-world. It's only a benchmark metric of the ML research community little concerned with any real-world impact. Comparing the "accuracy" of an LLM to 9 med students is just rank pseudoscience. The practice of medicine does not produce cute little cue-card prompts with 4 options. Diagnosis means trading the risk of various interventions against their diagnostic value, including a wait-and-see strategy. It means consulting with the patient on impact, as well as conferring with other specialists. It's of less-than-zero value to have an 80% machine accuracy and 75% clinician accuracy if the impact of the machine's mistakes are high risk, unconcerned with the impact of the intervention on the patient, or provide little therauptic upside. Likewise, on these decision benchmarks *Machine decisions are NEVER iterated* -- this makes any claim to "performance" not even borderline pseudoscience, but imv, malpractice. In the realworld practitioner decisions are iterated: doctors do not make high-risk mistake on every sequential decision, given more diagnostic information. It is highly highly unlikely that an inaccurate high-risk judgement call is compounded on each intervention. Whereas machine decisions are never tested in this sequential manner -- if they were, their benchmark performance would drop off a cliff. The production of non-iterated decision benchmarks which measure accuracy and not risk-adjusted real-world impact constitutes, imv, basic malpractice in the ML community. This should be urgently called out before non-experts think that LLMs can give credible medical advice.
- EForEndeavour 2y ago> Reminder once again that "accuracy" is irrelevant in the real-world. Accuracy and other quantitative performance metrics are imperfect for sure, but how else do you propose testing before real-world deployment? How do you propose scalable and feasible testing of human students? > The practice of medicine does not produce cute little cue-card prompts with 4 options. Ah, but the New England Journal of Medicine's Image Challenges (designed to test the knowledge and diagnostic capabilities of medical professionals) does. > It's of less-than-zero value to have an 80% machine accuracy and 75% clinician accuracy if the impact of the machine's mistakes are high risk, unconcerned with the impact of the intervention on the patient, or provide little therauptic upside But this paper does not study the practice of medicine. It intentionally focuses on performance on one specific, well-known medical imaging diagnostic challenge.
- 0655837610 2y ago[flagged]
- deleted 2y ago[deleted]
- simonw 2y agoHere's the main prompt they used: {image} {question} {choices} Please first describe the image in a section named “Image comprehension”. Then, recall relevant medical knowledge that is useful for answering the question but is not explicitly mentioned in a section named “Recall of medical knowledge”. Finally, based on the first two sections, provide your step-by-step reasoning and answer the question in a section named “Step-by-step reasoning”. Please be concise