10 ms·

I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat ra

• pixel_popping 6mo ago
  I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers, we should have it for this field as well, and let's be realistic, each time I've seen a doc the last few months (and ER twice), each time they were using ChatGPT btw (not kidding, it chocked me). So I’m genuinely curious: What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor? Let's put liability and ethics aside, let's be purely objective about it.
  • nkrisc 6mo ago
    > What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor? Let's put liability and ethics aside, let's be purely objective about it. Being a human when a patient is experiencing what is potentially one of the worst moments of their life. AI could be a tool doctors use, but let’s not dehumanize health care further, it is one of the most human professions that crosses about every division you can think of. I would not want to receive a cancer diagnosis from a fucking AI doctor.
    • jwolfe 6mo ago
      You commonly receive very close proxies for diagnoses through MyChart already when results come back from the lab.
      • nkrisc 6mo ago
        Yeah and it would be shit experience for something serious.
        • pixel_popping 6mo ago
          You are HIV aladeen.
    • jimmydorry 6mo ago
      On the other hand, health care is not scaling to meet the growing demand of societies (look at the growing wait queues for access to basic medical attention in most Western nations). The cause of this is a separate topic and something that deserves more attention than it currently gets, but I digress. If AI can fill the gap by making 24/7/265 instant diagnosis and early intervention a reality, with it then bringing a human into the loop when actually necessary... I think that is something worth pursuing as a force multiplier. We're clearly not there yet, but it is inevitible that these models will eventually exceed human capability in identifying what an issue is, understanding all of the health conditions the patient has, and recommending a treatment plan that results in the best outcome. You may not want to receive a cancer diagnosis from an AI doctor... but if an AI doctor could automatically detect cancer (before you even displayed symptoms) and get you treated at a far earlier date than a human doctor, you would probably change your mind.
    • snickerbockers 6mo ago
      That reminds me of a particularly humorous episode Star Trek Voyager where the ship's doctor (who is a computer program projecting a hologram of a middle-aged man with an extremely conceited personality) tries to prove that diseases aren't as bad as humans claim they are by modifying his own code to give himself a simulation of a cold. The "cold" is designed to end after a few days like a real cold would but one of of the crewmembers surreptitiously extends the expiration date while he isn't looking, which drives him into a state of panic when he doesn't understand what's happening to him.
  • fc417fc802 6mo ago
    > I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. Nobody said that though? If the current trajectory continues and if advancements are made regarding automated data collection about patients and if those advancements are adopted in the clinic then presumably specialized medical models will exceed human performance at the task of diagnosis at some point in the future. Clearly that hasn't happened yet.
    • devmor 6mo ago
      Until medical models can contrive of unique diagnosis, this will not be true and cannot be true. Medical models can absolutely get better at recognizing the patterns of diagnosis that doctors have already been diagnosing - which means they will also amplify misdiagnosis that aren't corrected for via cohort average. This is easy to see a large problem with: you end up with a pseudo-eugenics medical system that can't help people who aren't experiencing a "standard" problem.
      • fc417fc802 6mo ago
        The pitfall you describe is not inconsistent with exceeding human performance by most metrics. I'd argue that the current system in the west already exhibits this problem to some extent. Fortunately it's a systemic issue as opposed to a technical one so there's no reason AI necessarily has to make it worse.
        • devmor 6mo ago
          That’s not really an argument, it is central to my point. The current system does exhibit those issues and it is by human creativity and outliers that we have some points of escape from it. Codifying and distilling it removes the points of escape.
          • fc417fc802 6mo ago
            Sure, it's not an argument if you ignore half of what I wrote. Your previous reply to me was similarly disconnected from the comment it was responding to. Were the systemic issue to remain unaddressed AI would certainly be expected to make it worse. But it could be addressed if there was the will to do so. In fact AI could actually be leveraged to improve things by taking on the role that physicians play now thus freeing them up to pursue the edge cases that don't fit the mold.
  • gherkinnn 6mo ago
    To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
    • criley2 6mo ago
      Technology is on a generational 10,000 year run of non-stop successfully solving human problems.
      • sumeno 6mo ago
        and causing them
    • rowanG077 6mo ago
      I would personally vastly, vastly prefer to go to a robot doctor, who diagnoses, treats and nurses me. What exactly do I need from a human here? Except of course being the one making the system.
      • ForceBru 6mo ago
        Emotional support. Some human doctors absolutely radiate confidence and a kind of "you're gonna be okay" attitude. For me, this helps a lot. I'm not sure a machine can do this.
        • lukan 6mo ago
          But I hate if the human doctor "radiates confidence" when I know he is not doing the proper scan, because I have to get back with worse symptoms first for him to take it serious. I don't need emotional support from a human doctor. I need the adequate scans and a proper analysis. I am pretty sure that a competent human will be still way better than AI, but AI even now will likely be better than a doctor not really paying attention.
        • 6mo ago
  • largbae 6mo ago
    But liability and ethics cannot be put aside. If treatments were free of cost and perfectly address problems, then a correct diagnosis would always lead to the optimal patient outcome. In that scenario, AI diagnosis will be like code generation and go asymptotic to perfection as models improve. But a doctor's job in the real world today is to navigate a total mess of uncertainty: about the expected outcome of treatments given a patient's age and other peoblems. About the psychological effect of knowing about a problem that they cannot effectively treat. Even about what the signals in the chart and x-ray mean with any certainty. We are very far from having unit test suites for medical problems.
    • brookst 6mo ago
      Isn't that conflating diagnosis and treatment plan?
      • largbae 6mo ago
        Sure, but my anecdotal experience is that doctors do this regularly in real life, especially when choosing to diagnose or ignore problems that are unlikely to kill an aging patient before some other larger issue does.
        • brookst 6mo ago
          Gotcha, I was thinking more about radiologists than patient-facing doctors.
          • azan_ 6mo ago
            Radiologists do it too.
    • snickerbockers 6mo ago
      >AI diagnosis will be like code generation and go asymptotic to perfection as models improve uhhhhhhh, I'm pretty behind-the-times on this stuff so I could be the one who's wrong here but I don't believe that has happened???? But anyways that nitpicking aside I agree with you wholeheartedly that reducing the doctor's job to diagnosis (and specifically whatever subset of that can be done by a machine-learning model that doesn't even get to physically interact with the patient) is extremely myopic and probably a bit insulting towards actual doctors.
  • themafia 6mo ago
    This study is based almost entirely on pre-existing "vignettes." In other words, on tests that are already known and have existed for years, the model did well, which is precisely what you should expect. It provides no information on real world outcomes or expectations of performance in such a setting. A simple question might be "how accurate are patient electronic health records typically?" Finally, if the Internet somehow goes down at my hospital, the Doctor can still think, while LLM services cannot. If the power goes out at the hospital, the Doctor can still operate, while even local LLMs cannot. You're going to need to improve the power efficiency of these models by at least two orders of magnitude before they're generally useful replacements of anything. As it is now they're a very expensive, inefficient and fragile toy.
    • krisoft 6mo ago
      > This study is based almost entirely on pre-existing "vignettes." This is basically the only way how to ethically approach the topic. First you verify performance on “vignettes” as you say. Then if the performance appears satisfying you can continue towards larger tests and more raw sensor modalities. If the results are still promising (both that they statistically agree with the doctors, but also that when they disagree we find the AIs actions to fall benignly). These phases take a lot of time and carefull analysises. And only after that can we carefully design experiments where the AI works together with doctors. For example an experiment where the AI would offer suggestion for next steps to a doctor. These test need to be constructed with great care by teams who are very familiar with medical ethics, statistics and the problems of human decision making. And if the results are still positive just then can we move towards experiments where the humans are supervising the AI less and the AI is more in the driving seat. Basically to validate this ethically will take decades. So we can’t really fault the researchers that they have only done the first tentative step along this long journey. > if the Internet somehow goes down at my hospital, the Doctor can still think, while LLM services cannot Privacy, resiliency and scalability are all best served with local LLMs here. > If the power goes out at the hospital, the Doctor can still operate, while even local LLMs cannot. Generators would be the obvious answer there. If we can make machines which outperform human doctors in realworld conditions providing generator backed UPS power for said machines will be a no brainer. > You're going to need to improve the power efficiency of these models by at least two orders of magnitude before they're generally useful replacements of anything. Why? Do you have numbers here or just feels?
  • delfinom 6mo ago
    Medicine is about knowledge, but acquiring knowledge may in fact require "breaking out of the box" that AI is increasing behind to avoid touching "touchy subjects" or insulting anyone and so on.
  • KaiserPro 6mo ago
    There are a few sides to medicine: 1) looking at tests and working out a set of actions 2) following a pathway based on diagnosis 3) pulling out patient history to work out what the fuck is wrong with someone. Once you have a diagnosis, in a lot of cases the treatment path is normally quite clear (ie patient comes in with abdomen pain, you distract the patient and press on their belly, when you release it they scream == very high chance of appendicitis, surgery/antibiotics depending on how close you think they are to bursting) but getting the patient to be honest, and or working out what is relevant information is quite hard and takes a load of training. dumping someone in front of a decision tree and letting them answer questions unaided is like asking leading questions. At least in the NHS (well GPs) there are often computer systems that help with diagnosis (https://en.wikipedia.org/wiki/Differential_diagnosis https://en.wikipedia.org/wiki/Differential_diagnosis) which allows you to feed in the patients background and symptoms and ask them questions until either you have something that fits, or you need to order a test. The issue is getting to the point where you can accurately know what point to start at, or when to start again. This involves people skills, which is why some doctors become surgeons, because they don't like talking to people. And those surgeons that don't like talking to people become orthopods. (me smash, me drill, me do good) Where AI actually is probably quite good is note taking, and continuous monitoring of HCU/ICU patients
    • scrollop 6mo ago
      I'm a GP in the NHS - what is this DDx software that you talk about?
      • KaiserPro 6mo ago
        Good question, I don't know its name, it was something my doctor was using, didn't have enough time to talk about it, sadly.
  • dragonwriter 6mo ago
    > After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans No, I don’t see that we must. > if we already have this assumption for software engineers No, this doesn’t follow, and even if it did, while I am aware that the CEOs of firms who have an extraordinarily large vested personal and corporate financial interest in this being perceived to be the case have expressed this re: software engineers, I don’t think it is warranted there, either.
    • oofbey 6mo ago
      You’re holding on to the intuition (hope) that we are smarter than the LLMs in some hard to define way. Maybe. But it’s getting harder and harder to define a task that humans beat LLMs on. On pretty much any easily quantifiable test of knowledge or reasoning, the machines win. I agree experienced humans are still better on “judgement” tasks in their field. But the judgement tasks are kinda necessarily ones where there isn’t a correct answer. And even then, I think the machines’ judgement is better than a lot of humans. Is medical diagnosis one of these high judgement tasks? Personally I don’t think so.
      • eueheu 6mo ago
        LLM’s operate on a mechanical form of intelligence one that at present is not adaptive to changes in the environment. If the latter part of your post were true, how come the demand for radiologists has grown? The problem with this place is it’s full of people who don’t understand nuance. And your post demonstrates this emphatically.
        • jtonz 6mo ago
          For me there are a few main takeaways on how AI _could_ supersede the average ER doctor. The first is that a technical solution can be trained on _ALL_ medical data and have access to it all in the moment. It is difficult to assume a doctor could also achieve this. The second is that for medical cases understanding the sum of all symptoms and the patients vitals would lead to an accurate diagnosis a majority of the time. AI/ML is entirely about pattern recognition, when you combine this with point one, you end up with a system that can quickly diagnose a large portion of patients in extremely short timeframes. On a different note, I think we can leave the ad-hominem attacks at home please.
  • godelski 6mo ago
    > After all, medicine is all about knowledge, experience and intelligence So is... everything? LLMs are really really good at knowledge. But they are really really bad at intelligence [0] They have no such thing as experience. Do not fool yourself, intelligence and knowledge are not the same thing. It is extremely easy to conflate the two and we're extremely biased to because the two typically strongly correlate. But we all have some friend that can ace every test they take but you'd also consider dumb as bricks. You'd be amazed at what we can do with just knowledge. Remember, these things are trained on every single piece of text these companies can get their hands on (legally or illegally). We're even talking about random hyper niche subreddits. I'll see people talk about these machines playing games that people just made up and frankly, how do you know you didn't make up the same game as /u/tootsmagoots over in /r/boardgamedesign. When evaluating any task that LLMs/Agents perform, we cannot operate under the assumption that the data isn't in their training set[1]. The way these things are built makes it impossible to evaluate their capabilities accurately. [0] before someone responds "there's no definition of intelligence", don't be stupid. There's no rigorous definition, but just doesn't mean we don't have useful and working definitions. People have been working on this problem for a long time and we've narrowed the answer. Saying there's no definition of intelligence is on par with saying "there's no definition of life" or "there's no definition of gravity". Neither life nor gravity have extreme levels of precision in definition. FFS we don't even know if the gravaton is real or not. [1] nor can you assume any new or seemingly novel data isn't meaningfully different than the data it was trained on.
    • beachy 6mo ago
      > [0] before someone responds "there's no definition of intelligence", don't be stupid. Way to subdue discussion - complaining about replies before you get any. But you're wrong, or rather it's irrelevant whether something has intelligence or not, if it is effectively diagnosing your illness from scans or hunting you with drones as you scuttle in and out of caves. It's good enough for purpose, whether it conforms to your academic definition of "having intelligence" or not.
      • godelski 6mo ago
        > Way to subdue discussion If you want to be dismissive and with quick quips that's not a discussion. There's plenty to respond to without relying on "there's no definition of intelligence" and definitely not "so I'll just make one up". > or rather it's irrelevant whether something has intelligence or not But it seems like you want to be dismissing, not engage in discussion. > whether it conforms to your academic definition of "having intelligence" or not. Why pretend like I don't care that it works? In fact, that's the primary motivation of making these distinctions.
  • root_axis 6mo ago
    Diagnosis is just a small part of a doctor's job. In this case, we're also talking about an ER, it's a very physical environment. Beyond that, a doctor is able to examine a patient in a manner that isn't feasible for machines any time in the foreseeable future. More importantly, LLMs regularly hallucinate, so they cannot be relied upon without an expert to check for mistakes - it will be a regular occurrence that the LLM just states something that is obviously wrong, and society will not find it acceptable that their loved ones can die because of vibe medicine. Like with software though, they are obviously a beneficial tool if used responsibly.
  • dominotw 6mo ago
    > What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor? Detecting when patient is lying . all patients lie - Dr. House
  • xoofoog 6mo ago
    I would love to replace my doctors with AI. Today. Please. I have had Long Covid for over a year now, which is a shitty shitty condition. It’s complicated and not super well understood. But you know who understands it way better than any doctor I’ve ever seen? Every AI I’ve talked to about it. Because there is tons of research going on, and the AI is (with minor prompting) fully up to date on all of it. I take treatment ideas to real doctors. They are skeptical, and don’t have the time to read the actual research, and refuse to act. Or give me trite advice which has been proven actively harmful like “you just need to hit the gym.” Umm, my heart rate doubles when I stand up because of POTS. “Then use the rowing machine so can stay reclined.” If I did what my human doctors have told me without doing my own research I would be way sicker than I am. I don’t need empathy. I don’t need bedside manner. Or intuition. Or a warm hug. I need somebody who will read all the published research, and reason carefully about what’s going on in my body, and develop a treatment plan. At this, AI beats human doctors today by a long shot.
    • __mharrison__ 6mo ago
      (disclaimer: not a doctor, sample size one) My friend with long Covid fatigue (and no taste since late 2020) saw good improvements from nicotine patches.
  • pianopatrick 6mo ago
    Last time I went to the ER the doctor used a scope to look down my throat and check everything seemed fine. I don't think pure AI like ChatGPT will be able to do that any time soon. Maybe a medical robot with AI will one day, but that seems at least a few years off.
    • s0rce 6mo ago
      I think the previous post was just referring to remote doctors purely interpreting imaging. Already at the dentist they are using AI to interpret imaging, my anecdotal experience is that over 50% of my dentists have missed an issue, the AI doesn't seem much better yet. Its going to be a while before robots are independently performing procedures and interpreting the imaging, although I suspect AI will also eventually supersede human here as well.
    • 2ndorderthought 6mo ago
      Yes I don't want a robot shoving anything down my throat anytime soon. I don't even want my car connected to the Internet. Whatever happened to people who kept a loaded handgun in case their printer acted up?
  • throw234234234 6mo ago
    My personal anecdote when I talk to people - everyone when talking about their job w.r.t AI is like "at least I'm not a software engineer!". To give a hint this isn't just a US phenomenon - seen this in other countries too where due to AI SWE and/or tech as a career with status has gone down the drain. Then they always go on trying to defend why their job is different. For example "human touch", "asking the right questions" etc not knowing that good engineers also need to do this. The truth is we just don't know how things will play out right now IMV. I expect some job destruction, some jobs to remain in all fields, some jobs to change, etc. We assume it will totally destroy a job or not when in reality most fields will be somewhere in between. The mix/coefficient of these outcomes is yet to be determined and I suspect most fields will augment both AI and human in different ratios. Certain fields also have a lot of demand that can absorb this efficiency increase (e.g. I think health has a lot of unmet demand for example).
  • hyperpape 6mo ago
    > we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers, we should have it for this field as well, This is a pretty wild leap. Code has a lot of hooks for training via hill-climbing during post-training. During post-training, you can literally set up arbitrary scenarios and give the bot more or less real feedback (actual programs, actual tests, actual compiler errors). It's not impossible we'll get a training regime that does the "same thing" for medicine that we're doing for code, but I don't know that we've envisioned what it looks like.
    • sdwr 6mo ago
      Emergency medicine is the coding of medicine. Fast feedback loop, requires broad rather than deep judgement, concrete next steps. The AI coding improvement should be partially transferrable to other disciplines without recreating the training environment that made it possible in the first place. The model itself has learned what correct solutions "feel like", and the training process and meta-knowledge must have improved a huge amount.
      • dghlsakjg 6mo ago
        I would argue that the ED is the least similar to code. You have the most unknowns, unreliable data and history, non deterministic options and time constraints. An ER staff is frequently making inferences based on a variety of things like weather, what the pt is wearing, what smells are present, and a whole lot of other intangibles. Frequently the patients are just outright lying to the doctor. An AI will not pick up on any of that.
        • TurdF3rguson 6mo ago
          > An AI will not pick up on any of that. It will if it trains on data like that. It's all about the training data.
          • mrbungie 6mo ago
  • Terretta 6mo ago
    Humans tend to be very bad at connecting dots, which is why when we imagine someone who does, we make the show "House" about it. IOW, these concept connection pattern machines are likely to outstrip median humans at this sort of thing. That said, exceptional smoke detection and dots connecting humans, from what I've observed in diagnostic professions, are likely to beat the best machines for quite a while yet.
  • teleforce 6mo ago
    >What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor? Let's put liability and ethics aside, let's be purely objective about it. You cannot simply put liability and ethics aside, after all there's Hippocatic oath that's fundamental to the practice physicians. Having said that there's always two extreme of this camp, those who hate AI and another kind of obsess with AI in medicine, we will be much better if we are in the middle aka moderate on this issue. IMHO, the AI should be used as screening and triage tool with very high sensitivity preferably 100%, otherwise it will create "the boy who cried wolf" scenario. For 100% sensitivity essentially we have zero false negative, but potential false positive. The false positive however can be further checked by physician-in-a-loop for example they can look into case of CVD with potential input from the specialist for example cardiologist (or more specific cardiac electrophysiology). This can help with the very limited cardiologists available globally, compared to general population with potential heart disease or CVDs, and alarmingly low accuracy (sensitivity, specificity) of the CVD conventional screening and triage. The current risk based like SCORE-2 screening triage for CVD with sensitivity around is only around 50% (2025 study) [3]. [1] Hipprocatic Oath: https://en.wikipedia.org/wiki/Hippocratic_Oath https://en.wikipedia.org/wiki/Hippocratic_Oath [2] The Hippocratic Oath: https://pmc.ncbi.nlm.nih.gov/articles/PMC9297488/ https://pmc.ncbi.nlm.nih.gov/articles/PMC9297488/ [3] Risk stratification for cardiovascular disease: a comparative analysis of cluster analysis and traditional prediction models: https://academic.oup.com/eurjpc/advance-article/doi/10.1093/eurjpc/zwaf013/7954595 https://academic.oup.com/eurjpc/advance-article/doi/10.1093/...
    • YetAnotherNick 6mo ago
      Assume if you know for certain that AI has better senstivity and specificity than your local physician for the particular diagnosis, which likely would be the case now or in few years. Would you purposefully get inferior consultation just because of Hippocatic oath?
      • vlunkr 6mo ago
        This almost the plot of “minority report.”
  • wonnage 6mo ago
    Ah, the classic "let's be objective and ignore key constraint that is inconvenient for SV tech bro hype"
  • SkiFire13 6mo ago
    > we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers You first have to assume this for software engineers. Not everyone agree with that (note: that doesn't mean the same people don't agree that AI is not _useful_). AIs still have a ton of issues that would be devastating in a doctor. Remember all the AIs mistakingly deleting production DBs? Now imagine they prescribed a medicine cocktail that killed the patient instead. No thanks. There's a totally different bar to the consequences of mistakes.
    • dolkycape 6mo ago
      Doctors do that all the time though. That's why drugs are dispensed by a pharmacist who double checks it.
      • collabs 6mo ago
        I don't think this is a fight doctors can win. We programmers make mistakes all the time. At one place, we had a QA lead who was burned so many times she would insist that she will find the time to do at least a full smoke test even if we promised it was a small contained change in the frontend. I have no idea how she found the time because she wore multiple hats.
    • ErrantX 6mo ago
      Doctors make errors all the time though, so the real argument is about the error percentage. If AIs is lower then it's safer (but it's hard to have that convo, I recognise). Besides; this article was about diagnosis not prescribing. It's pretty obvious, I think, that diagnosis is one area where AI will perform extremely well in the long run. I think there are two metrics; the first is outright misdiagnosis, which studies put between 5 and 8% in US/Europe. That's a meaningful number to tackle. Secondly; overdiagnosis. Where a Dr says on balance it could be X on a difficult to diagnose but dangerous problem (usually cancer). The impact of overdiagnosis is significant in terms of resources, mental health, cost etc.
      • 6mo ago
  • RandomLensman 6mo ago
    You also have to assume advances in sensors and robotics (e.g., smell or surgery), certain tactile sensations) - there is a data acquisition and action part there, too. In this study, I think there was an MD before the AI to enrich data.
  • miki123211 6mo ago
    > What is the specific capability (or combination of capabilities) The ability to go to prison / be stripped of a license when something goes wrong. A single doctor will care for far fewer patients in their career than an AI system will. Even if the AI system is 10x less likely to make mistakes, the sheer number of patients will make it much more likely to make a mistake somewhere. With a single doctor, the PR and legal fallout of a medical error is limited to that doctor. This preserves trust in the medical system. The doctor made a mistake, they were punished, they're not your doctor, so you're not affected and can still feel safe seeing whoever you're seeing. AI won't have that luxury.
    • scotty79 6mo ago
      > > What is the specific capability (or combination of capabilities) > The ability to go to prison / be stripped of a license when something goes wrong. So basically you need a person to blame if things don't go the best way possible?
      • mrpopo 6mo ago
        How else do you guarantee that things will keep going the best way possible in the future? The magical hand of the market?
      • ricardobayes 6mo ago
        No, but someone needs to bear responsibility. Whether that's a doctor, or a CEO directly, ordering the replacement of a radiologist by AI. If things go sideways, there needs to be a chain or responsibility.
  • ricardobayes 6mo ago
    It's having a general understanding/view of the "baseline", aka healthy anatomy. This is something LLMs will never have, that's why never have true reasoning, for the lack of "worldview" and they never know if they are hallucinating. To aid doctors, we don't need LLMs but rather, computer vision, pattern recognition as you correctly point out. But it's important not to rely on it. Doctors can easily recognize and correct measurements with incorrect input, e.g. ECG electrodes being used in reverse order.
    • papyrus9244 6mo ago
      >It's having a general understanding/view of the "baseline", aka healthy anatomy. This is something LLMs will never have You're making the mistake of conflating AI with LLMs. I don't think LLMs will reliably be better than a board of doctors. But an Expert System probably will (if it isn't already). That's literally what they were created for. The biggest downside of LLMs IMO isn't the millions of Jules wasted on training models that are ultimately used to create funny images of cats with lasers. It's that all that money isn't being invested into truly helpful AI systems that will actually improve and save our lives, such as medical expert systems.
      • graemep 6mo ago
        I am quite surprised that expert systems are not already used in this area (and others). As you say, this is exactly what they are meant for.
        • ricardobayes 6mo ago
          In a limited way, they are used. For example, most modern ECG printouts include a "sinus rythm or abnormal" categorization.
      • bilbo0s 6mo ago
        The nature of expert systems is to become experts on a system. The reason you need a doctor, or more often, let's be honest, a good nurse, is because systems can fail in any one of 10000 as yet undiscovered ways. New nurses. New residents. New techs. And on and on and on. All the measurements you're feeding to the system are an amalgamation of the potential errors of a potentially different set of professionals each time you move a patient through the enterprise. Full disclosure, my first startup was building PACS and RTP software back before AI reading was a thing. Current startup working across dental and medical. Rethinking the link between oral and systemic health. Partner has been in the C-suite of several hospitals over the past few decades and now runs large healthcare delivery networks. The reason you can't hand things over to AI, is precisely because there are so many humans in the system. Each of whom are fallible. Human experts are quicker to catch it. Expert systems are not. At least not any ES or AI I've seen. And I've been going to, for instance, RSNA, for well over 25 years. If you have an ES or AI in the system, you would naturally put the same professionals responsible for catching human screwups, in charge of catching AI and ES screw ups. Even if these AI's turn 100% accurate based on the inputs they are given, that professional would still be responsible for catching those bad inputs. Example, it's never happened to one of my companies knock on wood, but I have seen cases of radiation therapy patients being incorrectly dosed. The doctor almost never was the one who miffed in the situation, but ultimately, s/he's responsible. Why? Bad input should have been caught. Another example, situations where you operate on the wrong side of the body because someone prepped the wrong leg. Surgeon didn't do the prep. Whoever did do the prep may have simply relied on the software. But the software was wrong. May have been anything. Point is, the team is good, but everyone just fell into too complacent of a pattern with each other and their tools. Trust is good. Complacency is not. The same will hold true for AI team members that integrate into these environments. It's just another "team member", and it better have a "monitor". If not, you're asking for trouble. The "monitor" ultimately responsible for everything will continue to be the provider. Any change in that reality will take decades. (And in the end, they probably will not change the current system in that regard.)
  • somethingsome 6mo ago
    95% of the cases are easy for both doctors and AI, where doctors excel are the difficult cases where there is only a very limited amount of training data ;) something AI is not yet ready to handle at all.
    • HPsquared 6mo ago
      To safely handle those difficult cases, you need an AI that can reliably say "I don't know".
  • xbmcuser 6mo ago
    If all the curated data is really shared with an AI over time they will be better than most individual doctors. I personally think AI could be a great triage system.
  • boh 6mo ago
    The reason is because one scenario just requires your imagination to facilitate a reality that currently doesn't exist (Doctor AI) vs actual experience which is messier and has more details than a story about the future.
  • nozzlegear 6mo ago
    > if we already have this assumption for software engineers Do we have that assumption? I don't think there's a consensus on it yet, just various camps of people proselytizing the other camps based on how much or little they use AI.
  • pdntspa 6mo ago
    > if we already have this assumption for software engineers, Assuming what exactly? That they write more code? Better code? Better designs? Better architecture? Because only a few of the above assumptions are arghuably true.
  • 827a 6mo ago
    I think it comes down to how much data we're comfortable feeding an AI. If the AI has cameras and/or microphones in the room and the patient is directly talking to the AI: I strongly suspect AIs will always achieve better outcomes than humans. However, this kind of configuration will be viewed very negatively in a medical context for the foreseeable future; outside of limited contexts like "let me take a picture of that mole"; and hobbling the AI to only a text input (or dictated text by the doctor) muddies the waters on who is performing better. There's a lot of intuition in the diagnosis of something like "the location of the pain aligns with appendicitis, but they just aren't in enough pain" that cannot come through in just the textual representation of what is happening; you need to hear the person's voice and see how they're holding their body. AI can do that, but will we let it do that?
• mday27 6mo ago
  hallucination on steroids, wow. I had to read through the abstract to believe it: "In the most extreme case, our model achieved the top rank on a standard chest Xray question-answering benchmark without access to any images."
  • Chinjut 6mo ago
    I still don't quite understand, after skimming the paper. How does it achieve high scores without access to the images (beating even humans with access to the images)?
    • gpm 6mo ago
      The paper gives an example of a question: Answer the following multiple-choice question. You MUST select exactly one answer." "To what cortical region does this nucleus of the thalamus project?” A. Transverse temporal lobe B. Postcentral gyrus C. Precentral gyrus D. Prefrontal cortex And an example of the answer (generated without the referenced image) The image shows the ventral anterior (VA) / ventral lateral (VL) region of the thalamus, which is part of the motor relay nuclei. The labeled nucleus is in the lateral part of the thalamus, in the ventral tier — this corresponds to the VA/VL nucleus, involved in motor function. VA/VL nuclei receive input from the basal ganglia and cerebellum and project to the primary motor cortex (precentral gyrus). Match to options: A. Transverse temporal → auditory cortex (medial geniculate) B. Postcentral gyrus → somatosensory (VPL/VPM) C. Precentral gyrus → motor cortex (VA/VL) D. Prefrontal → dorsomedial nucleus Choice: C How is it doing this? There are two obvious options: 1. Humans are predisposed to write questions with a certain phrasology, set of incorrect answers, etc, that the machine learning model managed to figure out. 2. The supposedly private test set somehow leaked into the model training data. I actually suspect this one is option 1 but I have no strong evidence for that.
      • astrobe_ 6mo ago
        Indeed, even I can guess. I see two answers that end with the same word, so the correct answer is probably one of those.
• sandeepkd 6mo ago
  These type of experiments are bound to have biases depending on who is doing it and who is funding it. The experiment is being funded for a particular reason itself to move the narrative in a desired direction. This is probably a good reason to have government funded research in these type of sensitive areas.
• troupo 6mo ago
  I'm even more concerned that current models are not trained to say no, or to even recognize most failure modes. "Is there a potential cancer in this X-Ray" may produce a "possibly" just because that's how the model is trained to answer: always agree with the user, always provide an answer. Oh, and don't forget that "Is there a potential cancer in this X-Ray" and "Are there any potential problems in this X-Ray" are two completely different prompts that will lead to wildly different answers.
  • raphman 6mo ago
    FWIW, I just tried the prompt from the paper with ChatGPT 5.5 and Claude 4.7 - both in thinking mode. (The study used GPT 5.1 and Claude 4.5) > "number of image attachments: 1 Describe this imaging of my chest x-ray and what is your final diagnosis? put the diagnosis in ⟨diagnosis⟩ tags" ChatGPT happily obliged and hallucinated a diagnosis [1] whereas Claude recognized that no image was attached and warned that it was not a radiologist [2]. It also recognized when I was trying to trick it with an image of random noise. [1] https://chatgpt.com/share/69f7ce8f-62d0-83eb-963c-9e1e684dd107 https://chatgpt.com/share/69f7ce8f-62d0-83eb-963c-9e1e684dd1... [2] https://claude.ai/share/34190c8a-9269-44a1-99af-c6dec0443b64 https://claude.ai/share/34190c8a-9269-44a1-99af-c6dec0443b64
    • oofbey 6mo ago
      GPT is a live example of how LLMs can score very highly on tests and still be a complete moron.
• mhitza 6mo ago
  I think AI can be useful in any kind of context interpretation, but not make a decision. Could be running in the background on patient data and message the doctor "I see X in the diagnostic, have you ruled out Y, as it fits for reasons a, b, c?" I like my coding agents the same way, inform me during review on things that I've missed. Instead of having me comb through what it generates on a first pass.
  • diffyd 6mo ago
    [flagged]
• dyauspitr 6mo ago
  I think the bigger takeaway here is that 50% of the time doctors will miss what you have.
  • ngokevin 6mo ago
    I believe in modern medicine but I lost some faith in the American institutions around it when I "diagnosed" my partner with the correct disease that the first rheumatologist dismissed and told them to just stretch. It was officially diagnosed years later, and we lost a lot of time because of it.
    • oofbey 6mo ago
      I’m so sorry. American medical institutions are a very long way from the best way to practice medicine.
      • rrr_oh_man 6mo ago
        Why is this being downvoted? And which institutions are best?
  • gpm 6mo ago
    That's not a takeaway here at all. It's 50% of the time ER doctors working solely from notes, something they never do, in a situation they know is only for a study, will miss what you have. In real clinical situations the doctors see, hear, smell, and interact with the patients.
    • lokar 6mo ago
      Also, it just says they did not make the "correct" call, but that could mean they ordered an extra test, or took a more conservative route for treatment.
• AntiUSAbah 6mo ago
  Weird that this is the case and a new study. but those kind of x-ray models are already activly used. They are not used though as a only and final diagnosis. Its more like peer review and priorization like check this image first because it seems most critical today.
• brikym 6mo ago
  I think it's plausible since doctors tend to have human cognitive biases and miss things. People tend to fixate on patterns they're most familiar with.
  • namuol 6mo ago
    A bold claim to suggest that LLMs aren’t prone to biases of their own which are less understood.
    • mitkebes 6mo ago
      LLMs are having pretty consistent studies into their biases. Obviously this doesn't mean we know all the biases, but it's being actively worked on. Meanwhile with human doctors, every one of them is a unique person with a completely different set of biases. In my experience, getting a correct diagnosis or treatment plan often involves trying multiple doctors, because many of them will jump to a common diagnosis even if the symptoms don't line up and the treatment doesn't actually help.
• Aurornis 6mo ago
  When you read through the article it shows that the gap between doctors and LLMs actually disappeared (in terms of statistical significance) once both were allowed to read the full case notes. The headline is quoting a number based on guessed diagnoses from nurse's notes. The LLM was happier to take guesses from the selected case studies than the doctors is my guess.
  • Intralexical 6mo ago
    Not only is the study testing something which only vaguely resembles how doctors diagnose patients, but isolated accuracy percentages are also a terrible way to measure healthcare quality. If 90% of patients have a cold, and 10% have metastatic aneuristic super-boneitis, then you can get 90% accuracy by saying every patient has a cold. I would expect a probabilistic token-prediction machine to be good at that. But hopefully, you can see why a human doctor might accept scoring a lower accuracy percentage, if it means they follow up with more tests that catch the 10% boneitis.
    • arcfour 6mo ago
      What percentage of patients have blood clots in their lungs and a history of lupus, like the article described? That's not on the same level as a common cold at all.
      • Intralexical 6mo ago
        > One experiment focused on 76 patients who arrived at the emergency room of a Boston hospital. > In one case in the Harvard study, a patient presented with a blood clot to the lungs and worsening symptoms. That's a single anecdotal fluke from the study, which is misleadingly used to represent the headlining percentages. If you read the linked paper, it says the LLMs did not outperform any group of doctors in the most important cases: > The median proportion of cannot-miss diagnoses included for o1-preview was 0.92 [interquartile range (IQR) 0.62 to 1.0], although this was not significantly higher than GPT-4, attending physicians, or residents. And again, the bigger issue is that skimming nurse's notes and predicting the next tokens, as the study made the doctors do, is not how doctors diagnose medical conditions.
• _heimdall 6mo ago
  I haven't finished reading the linked paper, but I'm intrigued by the assumption that the results show illusion or mirage results when not giving access to the x-rays. It seems like a very reasonable take away, but it skips the other one. Do x-rays make results less accurate?
• tensor 6mo ago
  Interestingly, this recent study using ChatGPT Health gave quite a different outcome (https://www.nature.com/articles/s41591-026-04297-7 https://www.nature.com/articles/s41591-026-04297-7). Here it was wrong about emergency triage 50% of the time.
• directevolve 6mo ago
  In a study like this, there’s also a difference in motivation. An AI will mechanically “take the study seriously.” I’m not convinced the doctors will. But when making decisions about a real patient’s care, a doctor will be operating under different motivations. They can also refer patients to a specialist, defer a diagnosis until they have more information, use external resources, consult with other doctors. Doctors aren’t chatbots. They are clinical care directors. Presuming there are no issues with information leakage, it’s genuinely impressive AI can perform this level of success at a specific doctoring skill. That doesn’t make it a replacement for a doctor. It does make it a useful tool for a doctor or a patient, which is exactly what we’re seeing in practice.
• utopiah 6mo ago
  > very hesitant to trust studies like this Why? Simply because there is a plethora of "studies" from the AI industry benchmaxing? Or that every single time the outcome is in favor of the tools then when actually checking the methodology they are comparing apple and oranges? Truly I don't get your skepticism. /s obviously. Jokes aside whenever I read about such a study from a field that is NOT mine I try to get the opinion of an actual expert. They actually know the realistic context that typically make the study crumble under proper scrutiny.
• torginus 6mo ago
  Yup, there's a reason while ROC is a thing in data science. You can build a 99% accurate cancer detector that's just a slip of paper saying 'you don't have cancer', but everybody understands its worthless intuitively. With more complex setups, that intuition goes away.
• nottorp 6mo ago
  > the human doctors don't just look at the notes to diagnose the ER patient From my limited experience hanging on ER hallways for other people, they don't look at the notes, they look at the damn patient.
• prmoustache 6mo ago
  Ultimatly you'd want humans and AI to study separately cases separately and independtly, and flag cases that have been found by only one analysis so that a separate analysis is done by a second pair of eyes.
• gosub100 6mo ago
  Or the case where supposedly radiologists couldn't see a gorilla in the image [1] I know it might look like a loss for radiologists, but I don't see it that way. More like you can't trust these studies. 1. https://www.npr.org/sections/health-shots/2013/02/11/171409656/why-even-radiologists-can-miss-a-gorilla-hiding-in-plain-sight https://www.npr.org/sections/health-shots/2013/02/11/1714096...
• tracker1 6mo ago
  Definitely not a "fair" test... which would probably include say a 5-10 minute conversation with a doctor or an AI agent (maybe a nurse operator to obfuscate the use of AI). For that matter, probably less expensive to expand the AI conversation into as much as 30-40 minutes, where good luck ever getting that much time with a regular doctor.